Method, device and computer equipment for constructing model for identifying abnormal weight waybills

Abstract

The present application relates to a method, apparatus, computer equipment, and storage medium for constructing a model for identifying waybills with abnormal weights. The method comprises: obtaining waybill samples, labeling the waybill samples as positive or negative samples based on whether the waybill samples have abnormal weights, classifying and summarizing the waybill samples based on preset dimension combinations and feature identifier sets corresponding to the dimensions to obtain a feature matrix table, screening the feature matrix tables based on the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table, extracting feature data corresponding to the target dimension combination from the waybill samples based on the target dimension combination corresponding to the target feature matrix table, obtaining fused feature data composed of the feature data, using the combined results of the fused feature data as the sample feature data of the waybill samples, training the initial model, and obtaining a model for identifying waybills with abnormal weights. The method can quickly and accurately obtain the results of identifying waybills with abnormal weights.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for constructing a model for identifying waybills with abnormal weight. Background Art

[0002] In the logistics industry, irregularities in weight measurement are common. Therefore, identifying under-measured shipments in advance is crucial. This not only helps mitigate losses but also helps standardize billing weight practices in the long run.

[0003] To address this issue, the current method mainly relies on random weighing on electronic scales and empirical rules to judge waybills. Due to the strong subjectivity in the judgment process and the small number of waybills identified, the existing method for identifying waybills with abnormal weights is not only slow but also has low accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for constructing a weight abnormal waybill recognition model that can improve recognition speed and accuracy in response to the above technical problems.

[0005] A method for constructing a weight-abnormal waybill recognition model, the method comprising:

[0006] Obtain waybill samples and mark them as positive or negative samples based on whether they have weight anomalies.

[0007] According to the preset dimension combination and the feature identification set corresponding to the dimension, the waybill samples are classified and summarized to obtain a feature matrix table;

[0008] According to the positive and negative sample distribution data in each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table;

[0009] Based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of the feature data;

[0010] The combination results of each fusion feature data are used as the sample feature data of the waybill sample, and the initial model is trained to obtain the weight abnormality waybill recognition model.

[0011] In one embodiment, the waybill samples are classified and summarized according to the preset dimension combination and the feature identification set corresponding to the dimension, and the feature matrix table obtained includes:

[0012] Based on the waybill information collection dimensions and the categories to which the dimensions belong, the dimensions belonging to different categories are combined in pairs to obtain a preset dimension combination;

[0013] According to the feature identification set corresponding to each dimension of the preset dimension combination, a target waybill sample that matches the feature identification set is extracted from the waybill sample;

[0014] The target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0015] In one embodiment, according to the feature identification set corresponding to each dimension of the preset dimension combination, extracting the target waybill sample that matches the feature identification set from the waybill sample includes:

[0016] Combine feature identifiers from different feature identifier sets in pairs and determine the target waybill matching conditions corresponding to each combination;

[0017] According to the target waybill matching conditions, the target waybill sample is matched from the waybill samples.

[0018] In one embodiment, the target waybill samples are classified and summarized according to positive and negative samples, and the feature matrix table corresponding to the dimension combination is obtained, including:

[0019] Divide the time range into at least two time periods based on the time range corresponding to the waybill sample;

[0020] According to the time period to which the target waybill sample belongs and the positive and negative sample labels it carries, the target waybill sample is classified into positive and negative samples by time period to obtain a feature matrix table corresponding to the dimension combination.

[0021] In one embodiment, the feature matrix table includes at least two sets of data;

[0022] According to the positive and negative sample distribution data in each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table including:

[0023] According to the number of positive and negative samples contained in each group of data in the feature matrix table, determine the iv value corresponding to each group of data;

[0024] According to the iv value corresponding to each set of data, the iv value corresponding to the feature matrix table is obtained;

[0025] According to the IV value corresponding to each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table.

[0026] In one embodiment, the combination result of each fusion feature data is used as the sample feature data of the waybill sample to train the initial model, and the obtained weight abnormal waybill recognition model includes:

[0027] The combination result of each fusion feature data is used as the sample feature data of the waybill sample to obtain the training sample;

[0028] Divide the training samples into training set and test set according to the time sequence of the training samples;

[0029] The initial model is trained based on the training set and the test set to obtain a weight-abnormal waybill recognition model.

[0030] In one embodiment, after the initial model is trained based on the training set and the test set to obtain a weight abnormality waybill recognition model, the method further includes:

[0031] For the same number of duplicate waybills, obtain the first-category recognition accuracy without using the weight-abnormal waybill recognition model, and the second-category recognition accuracy with the weight-abnormal waybill recognition model under different threshold conditions.

[0032] According to the first-category recognition accuracy and the second-category recognition accuracy, the target threshold of the weight-abnormal waybill recognition model is determined.

[0033] A device for constructing a model for identifying waybills with abnormal weight, the device comprising:

[0034] The sample marking module is used to obtain waybill samples and mark them as positive or negative samples based on whether the waybill samples have weight anomalies;

[0035] The feature matrix table construction module is used to classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table;

[0036] A feature matrix table screening module is used to screen the feature matrix tables according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table;

[0037] A fusion feature data generation module is used to extract feature data of dimensions corresponding to the target dimension combination from the waybill sample based on the target dimension combination corresponding to the target feature matrix table, and obtain fusion feature data composed of the feature data;

[0038] The model training module is used to use the combination results of each fusion feature data as the sample feature data of the waybill sample to train the initial model and obtain a weight abnormality waybill recognition model.

[0039] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0040] Obtain waybill samples and mark them as positive or negative samples based on whether they have weight anomalies.

[0041] According to the preset dimension combination and the feature identification set corresponding to the dimension, the waybill samples are classified and summarized to obtain a feature matrix table;

[0042] According to the positive and negative sample distribution data in each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table;

[0043] Based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of the feature data;

[0044] The combination results of each fusion feature data are used as the sample feature data of the waybill sample, and the initial model is trained to obtain the weight abnormality waybill recognition model.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0046] Obtain waybill samples and mark them as positive or negative samples based on whether they have weight anomalies.

[0047] According to the preset dimension combination and the feature identification set corresponding to the dimension, the waybill samples are classified and summarized to obtain a feature matrix table;

[0048] According to the positive and negative sample distribution data in each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table;

[0049] Based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of the feature data;

[0050] The combination results of each fusion feature data are used as the sample feature data of the waybill sample, and the initial model is trained to obtain the weight abnormality waybill recognition model.

[0051] The above-mentioned method, device, computer equipment and storage medium for constructing a weight-abnormal waybill recognition model obtains waybill samples, marks the waybill samples as positive and negative samples according to whether the waybill samples have weight abnormalities, classifies and summarizes the waybill samples according to preset dimension combinations and feature identification sets corresponding to the dimensions, and obtains a feature matrix table, thereby reducing the feature dimension and avoiding dimensional disasters. The feature matrix table is screened according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table. Based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimensions corresponding to the target dimension combination are extracted from the waybill samples to obtain fused feature data composed of feature data, thereby realizing the screening of feature dimensions and eliminating variables of less important dimensions. Finally, the combined results of each fused feature data are used as the sample feature data of the waybill sample to train the initial model to obtain a weight-abnormal waybill recognition model. Based on the screening and fusion of feature dimensions, the operating efficiency of the model can be improved, and in the subsequent model application process, the weight-abnormal waybill recognition results can be obtained quickly and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a diagram of the application environment of a method for building a model for identifying waybills with abnormal weights in one embodiment;

[0053] Figure 2 Schematic diagram of a flow chart of a method for constructing a model for identifying waybills with abnormal weight in one embodiment;

[0054] Figure 3 Schematic diagram of a flow chart of a method for constructing a model for identifying waybills with abnormal weight in another embodiment;

[0055] Figure 4 Schematic diagram of a flow chart of a method for constructing a model for identifying waybills with abnormal weight in another embodiment;

[0056] Figure 5 1 is a flow chart of a method for constructing a model for identifying waybills with abnormal weight in another embodiment;

[0057] Figure 6 Schematic diagram of a flow chart of a method for constructing a model for identifying waybills with abnormal weight according to another embodiment;

[0058] Figure 7 This is a structural block diagram of a device for building a model for identifying waybills with abnormal weight in one embodiment;

[0059] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] The method for constructing a weight abnormal waybill recognition model provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The server 104 obtains the waybill sample uploaded by the terminal 102. The waybill sample can be sent to the server 104 by the terminal 102 based on the data acquisition request of the server 104, or can be actively uploaded to the server 104 by the terminal 102 after recording the waybill data. The server 104 stores the uploaded waybill sample and obtains the waybill sample from the stored data when needed. The waybill sample is marked as a positive or negative sample based on whether the weight of the waybill sample is abnormal. The waybill sample is classified and summarized according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table. The feature matrix table is screened according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table. Based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of feature data. The combination result of each fused feature data is used as the sample feature data of the waybill sample to train the initial model to obtain a weight abnormality waybill recognition model. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0062] In one embodiment, Figure 2 As shown in the figure, a method for building a weight abnormal waybill recognition model is provided, and this method is applied to Figure 1 Taking the server in as an example, the method includes the following steps 202 to 210.

[0063] Step 202: Obtain a waybill sample and mark the waybill sample as a positive or negative sample based on whether the waybill sample has weight anomalies.

[0064] A waybill sample refers to a waybill whose weight data has been systematically recorded and weight verification has been completed. Verification refers to the process of retesting the actual weight of the waybill and handling the error between the system-recorded weight and the actual weight accordingly. Specifically, if the error between the system-recorded weight and the actual weight exceeds the preset threshold, it is considered a weight anomaly and a red check is processed. A red check is when the recorded weight data is marked as invalid, which is equivalent to invalidating the original weight data and making it no longer valid. For example, assuming the preset threshold is 0.2kg, if the actual weight of a waybill is 10kg, but the weight is marked as 7kg when entering the system list, the error is 3kg, which is greater than 0.2kg. Therefore, a red check is required, and the waybill is an understated waybill with a red check. If the error between the system-recorded weight and the actual weight does not exceed the preset threshold, the weight is normal and no red check is processed.

[0065] In this embodiment, waybill samples with weight anomalies are marked as positive samples, while waybill samples without weight anomalies are marked as negative samples. Since the ultimate goal of the model is to identify waybill samples that are undercounted and have red-checked, the positive samples can be set as waybill samples with undercounted and red-checked after re-weighing, and the negative samples can be set as waybill samples without red-checked after re-weighing. It is understood that in other embodiments, the marking of positive and negative samples can be reversed, that is, waybill samples with weight anomalies are marked as negative samples, and waybill samples without weight anomalies are marked as positive samples.

[0066] Step 204 : Classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table.

[0067] The dimension combination includes at least two dimensions in the waybill information. For each waybill, there are multiple dimensions of information. Specifically, the information of each dimension of the waybill is divided into three categories, namely customer information, employee information, and waybill record information. Among them, the waybill record information category includes dimensions such as shipping time, shipping week, billing weight, billing amount, product type, time limit type, consignment category, whether it is a parent-child package, freight payment method, whether it is insured, and shipping area; the employee information category includes dimensions such as employee length of service, gender, age, and education level; the customer information category includes dimensions such as customer type, whether it is a monthly settlement customer, and whether it is a trustworthy weight-based monthly settlement customer. Through the combination of dimensions, the feature fusion of waybill information can be achieved to avoid dimensional disasters.

[0068] The number of dimensions included in the dimension combination can be adjusted according to actual needs, for example, the number of dimension combinations can be set to 2, 3, or other numbers. For ease of description, the following embodiments are described with the dimension combination being two dimensions.

[0069] A feature identifier set refers to a set of all value results or value ranges in a certain dimension. For example, employee gender includes male and female, so the feature identifier set corresponding to the employee gender dimension includes the two feature identifiers male and female. For another example, billing weight (unit: kg) includes (0, 1], (1, 5], (5, 10], (10, 20], (20, 50], (50, 300], (300, n), n>300, then the feature identifier set corresponding to the billing weight dimension includes the seven feature identifiers (0, 1], (1, 5], (5, 10], (10, 20], (20, 50], (50, 300], (300, n).

[0070] In this embodiment, for feature identifier sets of categorical variables, one-hot encoding can be used to represent different feature identifiers using feature data 1 and 0 to simplify the representation of feature identifiers. For example, employee gender can be either male or female. Through one-hot encoding, male is represented by 1, and female is represented by 0. For another example, whether a waybill is insured can be either yes or no. Insured is represented by 1, and uninsured is represented by 0.

[0071] Classification and aggregation refers to the process of classifying waybills by dimension combinations and the feature identifier sets corresponding to each dimension in the dimension combination, and then aggregating the classified waybills. Waybill aggregation refers to the summary of the number of positive and negative samples in each category.

[0072] A feature matrix is ​​a categorized summary of dimension combinations. Each dimension combination corresponds to a feature matrix. For example, feature matrices might include: a customer type-billable weight feature matrix, an employee length of service-billable weight matrix, a customer type-employee length of service matrix, a customer type-region feature matrix, and an employee length of service-region feature matrix.

[0073] Step 206 : Screen the feature matrix tables according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table.

[0074] The positive and negative sample distribution data in the feature matrix table refers to the distribution of positive and negative samples in each group of the feature matrix table. For example, the total number of positive and negative samples, the number of positive samples, and the proportion of positive samples in each group.

[0075] Based on the positive and negative sample distribution data in the feature matrix table, the feature importance corresponding to the feature data table is calculated. In an embodiment, the feature importance can be determined by calculating the IV value corresponding to the feature data table. For example, the IV value corresponding to each feature data table is compared with a preset IV threshold, and the feature data tables corresponding to the results greater than the preset IV threshold are screened out as the target feature matrix table.

[0076] In step 208 , based on the target dimension combination corresponding to the target feature matrix table, feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of the feature data.

[0077] The feature matrix table corresponds to the dimension combination one-to-one. After determining the target feature matrix table, the corresponding target dimension combination can be determined. Taking the target dimension combination including the first dimension and the second dimension as an example, the first feature data of the first dimension and the second feature data of the second dimension are extracted from the waybill sample, and the combination result of the first feature data and the second feature data is the fused feature data. It should be noted that although the fused feature data contains data of multiple dimensions, in the data processing process, the fused feature data is treated as a whole for processing. In an embodiment, the first feature data and the second feature data can be combined into a feature vector, and the fused feature data is represented by the feature vector.

[0078] In step 210 , the combination result of each fused feature data is used as the sample feature data of the waybill sample to train the initial model and obtain a weight abnormality waybill recognition model.

[0079] Each target feature matrix table corresponds to a fused feature data. The fused feature data are combined as the sample feature data of the waybill sample. The sample feature data carrying positive and negative sample labels are input into the initial model. By training the initial model and adjusting the parameters, a weight-abnormal waybill recognition model is obtained.

[0080] In an embodiment, a combination method of each fused feature data may be predefined to combine each fused feature data. For example, the combination method includes a combination form, a combination order, and the like.

[0081] The above-mentioned method for constructing a weight-abnormal waybill recognition model obtains waybill samples, marks the waybill samples as positive and negative samples according to whether the waybill samples have weight abnormalities, classifies and summarizes the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension, and obtains a feature matrix table, which reduces the feature dimension and avoids the dimension disaster. The feature matrix table is screened according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table. Based on the target dimension combination corresponding to the target feature matrix table, the feature data of the dimension corresponding to the target dimension combination is extracted from the waybill sample to obtain fused feature data composed of feature data, which realizes the screening of feature dimensions and can eliminate variables with less important dimensions. Finally, the combined results of each fused feature data are used as the sample feature data of the waybill sample to train the initial model to obtain a weight-abnormal waybill recognition model. Based on the screening and fusion of feature dimensions, the operating efficiency of the model can be improved. In the subsequent model application process, the weight-abnormal waybill recognition result can be obtained quickly and accurately.

[0082] In one embodiment, Figure 3 As shown, according to the preset dimension combination and the feature identification set corresponding to the dimension, the waybill samples are classified and summarized to obtain a feature matrix table, namely step 204, including steps 302 to 306.

[0083] In step 302 , based on the waybill information collection dimensions and the categories to which the dimensions belong, dimensions belonging to different categories are combined in pairs to obtain a preset dimension combination.

[0084] Step 304 : According to the feature identification set corresponding to each dimension of the preset dimension combination, a target waybill sample matching the feature identification set is extracted from the waybill sample.

[0085] In step 306, the target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0086] Waybill information collection dimensions refer to dimensions that collect corresponding information for each waybill. The dimensions are categorized into customer information, employee information, and waybill record information. The waybill record information category includes dimensions such as shipping time, shipping week, billing weight, billing amount, product type, timeliness type, consignment type, whether it is a parent-child package, freight payment method, whether it is insured, and shipping area. The employee information category includes dimensions such as employee length of service, gender, age, and education level. The customer information category includes dimensions such as customer type, whether it is a monthly settlement customer, and whether it is a trustworthy weight monthly settlement customer.

[0087] Dimensions belonging to different categories are combined in pairs to obtain a preset dimension combination. Two categories can be selected from the three categories above, and a dimension is selected from each of the two selected categories to be combined. The result is the preset dimension combination. In an embodiment, dimensions belonging to different categories can be combined in pairs by permutation and combination.

[0088] The feature identifier sets corresponding to each dimension can be preconfigured data. A preset dimension combination includes two dimensions. Based on the feature identifier sets corresponding to these two dimensions, target waybill samples that match any feature identifier in both feature identifier sets are extracted from the waybill samples. Finally, the target waybill samples are classified and aggregated according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0089] In an embodiment, when a dimension in a dimension combination meets a preset data grouping condition, a feature identifier set consisting of the grouping results corresponding to the dimension is obtained. The preset data grouping condition means that the data within the entire value range of the dimension can be divided into multiple groups by dividing the data interval. Specifically, the preset data grouping conditions include shipping time period, billing weight, billing amount, etc. In an embodiment, the dimensions that meet the preset data grouping conditions can be directly configured in the configuration data.

[0090] By combining dimensions in pairs according to categories, dimension fusion can be achieved based on different categories, which can avoid dimensional disasters caused by numerous dimensions and improve the effectiveness of data.

[0091] In one embodiment, according to the feature identification set corresponding to each dimension of the preset dimension combination, a target waybill sample matching the feature identification set is extracted from the waybill sample, that is, step 304, including steps 402 to 404.

[0092] Step 402: Combine the feature identifiers in different feature identifier sets in pairs, and determine the target waybill matching condition corresponding to each combination.

[0093] Step 404: According to the target waybill matching conditions, a target waybill sample is matched from the waybill samples.

[0094] The feature identifiers in different feature identifier sets are combined in pairs to obtain multiple combinations. The two feature identifiers corresponding to each combination are the target waybill matching conditions corresponding to the combination. The target waybill sample is obtained by matching from the waybill sample with the goal of meeting the target waybill matching conditions.

[0095] By combining feature identifiers in different feature identifier sets in pairs, the grouping in the same dimension combination is refined, which helps to obtain accurate screening results in the subsequent screening process of the feature matrix table.

[0096] In one embodiment, Figure 5 As shown, the target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination, that is, step 306 includes steps 502 to 504.

[0097] Step 502: Divide the time range into at least two time periods according to the time range corresponding to the waybill sample.

[0098] In step 504, the target waybill samples are classified into positive and negative samples according to the time period to which they belong and the positive and negative sample labels they carry, and a feature matrix table corresponding to the dimension combination is obtained.

[0099] In the embodiment, the time period can be divided by day or by week, which can be determined specifically based on the time range involved in the sample data. Dividing the target waybill samples according to the time period based on the time range can further refine the target waybill samples that meet the target waybill matching conditions into more groups. The refined grouping helps to obtain accurate screening results in the subsequent screening process of the feature matrix table, as shown in Table 1, which is a customer type-billing weight feature matrix table.

[0100] Table 1: Customer Type-Billing Weight Feature Matrix

[0101]

[0102] Among them, when the target waybill matching condition is 0011_0-1 (for customers of the type corresponding to 0011, the billing weight is within 1 kg), by dividing the time period into 3 segments by week, the waybills matching the target waybill matching condition 0011_0-1 are divided from one group into three groups.

[0103] In one embodiment, the feature matrix table includes at least two sets of data. Figure 6 As shown, according to the positive and negative sample distribution data in each feature matrix table, the feature matrix table is screened to obtain a target feature matrix table, including steps 602 to 606.

[0104] Step 602: Determine the iv value corresponding to each group of data according to the number of positive and negative samples contained in each group of data in the feature matrix table.

[0105] Step 604: Obtain the iv value corresponding to the feature matrix table according to the iv value corresponding to each set of data.

[0106] Step 606: Screen the feature matrix tables according to the iv values ​​corresponding to the feature matrix tables to obtain a target feature matrix table.

[0107] In binary classification problems in machine learning, the IV value (information value) is primarily used to assess the predictive power of the input variable. The magnitude of the IV value of a feature variable indicates the strength of that variable's predictive power. Before calculating the IV value, the Woe value must be calculated. Each group in each feature matrix table corresponds to a Woe value; each feature matrix table corresponds to only one IV value, which is the sum of the IV values ​​for each group.

[0108] The specific calculation process of woe and iv values ​​is as follows

[0109] woe:

[0110]

[0111] iv:

[0112]

[0113] Among them: woe i : woe value of each group in each feature matrix table;

[0114] py i : The ratio of the number of positive samples in each feature matrix table to the total number of positive samples;

[0115] pni: the ratio of the number of negative samples in each feature matrix table to the total number of negative samples;

[0116] #y i : The number of positive samples in each feature matrix table;

[0117] #y T : The number of positive samples in all samples;

[0118] #n i : The number of negative samples in each feature matrix table;

[0119] #n T : The number of negative samples in all samples.

[0120] Through the above calculations, we can obtain the IV value corresponding to each feature matrix table. We compare the IV value corresponding to each feature matrix table with the preset IV threshold, filter the feature matrix table, and obtain the target feature matrix table. Taking the preset IV threshold of 0.1 as an example, we filter out feature matrices with IV values ​​greater than 0.1 and eliminate less important dimension combinations to improve model operation efficiency.

[0121] In one embodiment, the combination result of each fusion feature data is used as the sample feature data of the waybill sample to train the initial model, and the obtained weight abnormal waybill recognition model includes:

[0122] The combined results of each fused feature data set are used as the sample feature data for the waybill sample to obtain the training sample. The training sample is divided into a training set and a test set according to the chronological order of the training samples. The initial model is trained based on the training set and the test set to obtain the weight-abnormal waybill recognition model.

[0123] In an embodiment, the xgboost algorithm can be used to model the waybill samples. First, the training set and test set are divided in chronological order (such as using waybill samples in the time range of 2020-05-01 to 2020-05-31 for training, and waybill samples in the time range of 2020-06-01 to 2020-06-07 for testing). Then, the training set is introduced into the algorithm, and a weight abnormality waybill recognition model is obtained after parameter adjustment and training. By determining the training set and test set in chronological order, training and testing can be performed in chronological order, which can obtain more accurate test results compared to random allocation.

[0124] In one embodiment, after training an initial model based on a training set and a test set to obtain a weight-exceeding waybill recognition model, the method further includes: for the same number of duplicate waybills, obtaining the first-category recognition accuracy without using the weight-exceeding waybill recognition model, and the second-category recognition accuracy of the weight-exceeding waybill recognition model under different threshold conditions. A target threshold for the weight-exceeding waybill recognition model is determined based on the first-category recognition accuracy and the second-category recognition accuracy.

[0125] In the embodiment, since the basic evaluation indicators of the previous model are AUC value (the larger the value, the higher the model judgment accuracy), KS value (the larger the value, the better the model prediction accuracy), etc., but for a data set with an imbalance of positive and negative samples, it is too simplistic to evaluate the effect of the model only by these indicators, so this solution adds the calculation of precision and recall rate under different thresholds and the accuracy comparison analysis with the current non-adopted model in the evaluation of model results, and determines the target threshold of the weight abnormal waybill recognition model (compared with the number of duplicate waybills). This can not only better evaluate the model performance, but also, for actual applications, it can also adopt different mandatory strategies by hierarchical division of different thresholds. In the embodiment, by adding different threshold visualization analysis, it is more in line with the actual application scenario.

[0126] It should be understood that, although the various steps in the various flow charts that the above-described embodiments relate to are shown in sequence according to the indications of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless clearly stated herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the various flow charts that the above-described embodiments relate to can include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0127] In one embodiment, Figure 7 As shown, a device for constructing a model for identifying abnormal weight waybills is provided, comprising: a sample marking module 702, a feature matrix table construction module 704, a feature matrix table screening module 706, a fusion feature data generation module 708, and a model training module 710, wherein:

[0128] The sample marking module 702 is used to obtain waybill samples and mark the waybill samples as positive or negative samples based on whether the waybill samples have weight anomalies;

[0129] The feature matrix table construction module 704 is used to classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table;

[0130] A feature matrix table screening module 706 is used to screen the feature matrix tables according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table;

[0131] The fused feature data generating module 708 is configured to extract feature data of the dimensions corresponding to the target dimension combination from the waybill sample based on the target dimension combination corresponding to the target feature matrix table, thereby obtaining fused feature data composed of the feature data;

[0132] The model training module 710 is used to use the combination results of the fused feature data as the sample feature data of the waybill sample to train the initial model and obtain a weight abnormality waybill recognition model.

[0133] In one of the embodiments, the feature matrix table construction module is also used to combine dimensions belonging to different categories in pairs according to the waybill information collection dimensions and the categories to which the dimensions belong, to obtain a preset dimension combination; according to the feature identification set corresponding to each dimension of the preset dimension combination, extract the target waybill samples that match the feature identification set from the waybill samples; classify and summarize the target waybill samples according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0134] In one embodiment, the feature matrix table construction module is also used to combine feature identifiers in different feature identifier sets in pairs to determine the target waybill matching conditions corresponding to each combination; based on the target waybill matching conditions, the target waybill sample is matched from the waybill sample.

[0135] In one of the embodiments, the feature matrix table construction module is also used to divide the time range into at least two time periods according to the time range corresponding to the waybill sample; based on the time period to which the target waybill sample belongs and the positive and negative sample labels carried, the target waybill sample is classified into positive and negative samples according to the time period to obtain a feature matrix table corresponding to the dimension combination.

[0136] In one embodiment, the feature matrix table includes at least two groups of data; the feature matrix table screening module is further used to determine the IV value corresponding to each group of data based on the number of positive and negative samples contained in each group of data in the feature matrix table; based on the IV value corresponding to each group of data, the IV value corresponding to the feature matrix table is obtained; based on the IV value corresponding to each feature matrix table, the feature matrix table is screened to obtain a target feature matrix table.

[0137] In one embodiment, the model training module uses the combination results of each fused feature data as the sample feature data of the waybill sample to obtain a training sample; divides the training sample into a training set and a test set according to the time sequence of the training samples; and trains the initial model based on the training set and the test set to obtain a weight-abnormal waybill recognition model.

[0138] In one embodiment, the device for constructing a weight-abnormal waybill recognition model also includes a model threshold determination module, which is used to obtain, for the same number of duplicate waybills, the first type of recognition accuracy without using the weight-abnormal waybill recognition model and the second type of recognition accuracy of the weight-abnormal waybill recognition model under different threshold conditions; and determine the target threshold of the weight-abnormal waybill recognition model based on the first type of recognition accuracy and the second type of recognition accuracy.

[0139] The above-mentioned device for constructing a weight-abnormal waybill recognition model obtains waybill samples, marks the waybill samples as positive and negative samples according to whether the waybill samples have weight abnormalities, classifies and summarizes the waybill samples according to preset dimension combinations and feature identification sets corresponding to the dimensions, and obtains a feature matrix table, thereby reducing the feature dimension and avoiding dimensional disasters. The feature matrix table is screened according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table, and based on the target dimension combination corresponding to the target feature matrix table, feature data of dimensions corresponding to the target dimension combination are extracted from the waybill samples to obtain fused feature data composed of feature data, thereby realizing the screening of feature dimensions and eliminating variables of less important dimensions. Finally, the combined results of each fused feature data are used as the sample feature data of the waybill sample to train the initial model to obtain a weight-abnormal waybill recognition model. Based on the screening and fusion of feature dimensions, the operating efficiency of the model can be improved, and in the subsequent model application process, the weight-abnormal waybill recognition result can be obtained quickly and accurately.

[0140] Regarding the specific limitations of the device for building a model for identifying waybills with abnormal weight, please refer to the limitations of the method for building a model for identifying waybills with abnormal weight above, which will not be repeated here. The various modules in the above-mentioned device for building a model for identifying waybills with abnormal weight can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0141] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data for constructing a model for identifying weight-abnormal waybills. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for constructing a model for identifying weight-abnormal waybills is implemented.

[0142] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0143] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0144] Obtain waybill samples, and mark them as positive or negative samples based on whether they have weight anomalies; classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table; filter the feature matrix table according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table; based on the target dimension combination corresponding to the target feature matrix table, extract the feature data of the dimensions corresponding to the target dimension combination from the waybill samples to obtain fused feature data composed of the feature data; use the combined results of the fused feature data as the sample feature data of the waybill samples, train the initial model, and obtain a weight anomaly waybill recognition model.

[0145] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0146] According to the dimensions of waybill information collection and the categories to which the dimensions belong, the dimensions belonging to different categories are combined in pairs to obtain a preset dimension combination; according to the feature identification set corresponding to each dimension of the preset dimension combination, the target waybill samples that match the feature identification set are extracted from the waybill samples; the target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0147] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0148] The feature identifiers in different feature identifier sets are combined in pairs to determine the target waybill matching conditions corresponding to each combination; according to the target waybill matching conditions, the target waybill sample is matched from the waybill sample.

[0149] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0150] According to the time range corresponding to the waybill sample, the time range is divided into at least two time periods; according to the time period to which the target waybill sample belongs and the positive and negative sample labels it carries, the target waybill sample is classified into positive and negative samples according to the time period to obtain a feature matrix table corresponding to the dimension combination.

[0151] In one embodiment, the feature matrix table includes at least two sets of data; when the processor executes the computer program, it further implements the following steps:

[0152] According to the number of positive and negative samples contained in each group of data in the feature matrix table, the IV value corresponding to each group of data is determined; according to the IV value corresponding to each group of data, the IV value corresponding to the feature matrix table is obtained; according to the IV value corresponding to each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table.

[0153] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0154] The combination results of each fusion feature data are used as the sample feature data of the waybill sample to obtain the training sample; the training samples are divided into a training set and a test set according to the chronological order of the training samples; the initial model is trained based on the training set and the test set to obtain a weight-abnormal waybill recognition model.

[0155] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0156] For the same number of duplicate waybills, the first-category recognition accuracy without using the abnormal weight waybill recognition model and the second-category recognition accuracy of the abnormal weight waybill recognition model under different threshold conditions are obtained respectively; based on the first-category recognition accuracy and the second-category recognition accuracy, the target threshold of the abnormal weight waybill recognition model is determined.

[0157] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0158] Obtain waybill samples, and mark them as positive or negative samples based on whether they have weight anomalies; classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table; filter the feature matrix table according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table; based on the target dimension combination corresponding to the target feature matrix table, extract the feature data of the dimensions corresponding to the target dimension combination from the waybill samples to obtain fused feature data composed of the feature data; use the combined results of the fused feature data as the sample feature data of the waybill samples, train the initial model, and obtain a weight anomaly waybill recognition model.

[0159] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0160] According to the dimensions of waybill information collection and the categories to which the dimensions belong, the dimensions belonging to different categories are combined in pairs to obtain a preset dimension combination; according to the feature identification set corresponding to each dimension of the preset dimension combination, the target waybill samples that match the feature identification set are extracted from the waybill samples; the target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

[0161] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0162] The feature identifiers in different feature identifier sets are combined in pairs to determine the target waybill matching conditions corresponding to each combination; according to the target waybill matching conditions, the target waybill sample is matched from the waybill sample.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0164] According to the time range corresponding to the waybill sample, the time range is divided into at least two time periods; according to the time period to which the target waybill sample belongs and the positive and negative sample labels it carries, the target waybill sample is classified into positive and negative samples according to the time period to obtain a feature matrix table corresponding to the dimension combination.

[0165] In one embodiment, the feature matrix table includes at least two sets of data; when the computer program is executed by the processor, the following steps are further implemented:

[0166] According to the number of positive and negative samples contained in each group of data in the feature matrix table, the IV value corresponding to each group of data is determined; according to the IV value corresponding to each group of data, the IV value corresponding to the feature matrix table is obtained; according to the IV value corresponding to each feature matrix table, the feature matrix table is screened to obtain the target feature matrix table.

[0167] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0168] The combination results of each fusion feature data are used as the sample feature data of the waybill sample to obtain the training sample; the training samples are divided into a training set and a test set according to the chronological order of the training samples; the initial model is trained based on the training set and the test set to obtain a weight-abnormal waybill recognition model.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0170] For the same number of duplicate waybills, the first-category recognition accuracy without using the abnormal weight waybill recognition model and the second-category recognition accuracy of the abnormal weight waybill recognition model under different threshold conditions are obtained respectively; based on the first-category recognition accuracy and the second-category recognition accuracy, the target threshold of the abnormal weight waybill recognition model is determined.

[0171] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0172] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for constructing a model for identifying waybills with abnormal weight, characterized in that: The method comprises: Obtain a waybill sample, and mark the waybill sample as a positive or negative sample based on whether the waybill sample has an abnormal weight; Classify and summarize the waybill samples according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table; Screening the feature matrix tables according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table; Based on the target dimension combination corresponding to the target feature matrix table, extracting feature data of the dimension corresponding to the target dimension combination from the waybill sample to obtain fused feature data composed of the feature data; The combination result of each fusion feature data is used as the sample feature data of the waybill sample, and the initial model is trained to obtain a weight abnormal waybill recognition model, wherein the waybill samples are classified and summarized according to the preset dimension combination and the feature identification set corresponding to the dimension to obtain a feature matrix table including: According to the dimensions of waybill information collection and the categories to which the dimensions belong, the dimensions belonging to different categories are combined in pairs to obtain the preset dimension combination; according to the feature identification set corresponding to each dimension of the preset dimension combination, the target waybill samples matching the feature identification set are extracted from the waybill samples; the target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

2. The method according to claim 1, characterized in that The step of extracting a target waybill sample that matches the feature identification set from the waybill sample according to the feature identification set corresponding to each dimension of the preset dimension combination includes: Combine feature identifiers from different feature identifier sets in pairs and determine the target waybill matching conditions corresponding to each combination; According to the target waybill matching condition, a target waybill sample is obtained by matching from the waybill samples.

3. The method according to claim 1 or 2, characterized in that The target waybill samples are classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination, including: Divide the time range into at least two time periods according to the time range corresponding to the waybill sample; According to the time period to which the target waybill sample belongs and the positive and negative sample labels carried, the target waybill sample is classified into positive and negative samples according to the time period to obtain a feature matrix table corresponding to the dimension combination.

4. The method according to claim 1, wherein The feature matrix table includes at least two groups of data; The feature matrix table is screened according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table, including: Determine the iv value corresponding to each group of data according to the number of positive and negative samples contained in each group of data in the feature matrix table; According to the iv value corresponding to each set of data, the iv value corresponding to the feature matrix table is obtained; According to the iv value corresponding to each of the feature matrix tables, the feature matrix tables are screened to obtain a target feature matrix table.

5. The method according to claim 1, wherein The combination result of each fusion feature data is used as the sample feature data of the waybill sample to train the initial model to obtain the weight abnormal waybill recognition model, which includes: The combined result of each fusion feature data is used as the sample feature data of the waybill sample to obtain a training sample; Dividing the training samples into a training set and a test set according to the time sequence of the training samples; The initial model is trained based on the training set and the test set to obtain a weight-abnormal waybill recognition model.

6. The method according to claim 5, characterized in that After the initial model is trained based on the training set and the test set to obtain a weight abnormality waybill recognition model, the method further includes: For the same number of duplicate waybills, obtain the first-category recognition accuracy rate without using the abnormal weight waybill recognition model, and the second-category recognition accuracy rate of the abnormal weight waybill recognition model under different threshold conditions; The target threshold of the abnormal weight waybill recognition model is determined based on the first category recognition accuracy and the second category recognition accuracy.

7. A device for constructing a model for identifying abnormal weight waybills, characterized in that: The device comprises: A sample marking module is used to obtain a waybill sample and mark the waybill sample as a positive or negative sample based on whether the waybill sample has an abnormal weight; A feature matrix table construction module is used to classify and summarize the waybill samples according to a preset dimension combination and a feature identifier set corresponding to the dimension to obtain a feature matrix table; A feature matrix table screening module is used to screen the feature matrix tables according to the positive and negative sample distribution data in each feature matrix table to obtain a target feature matrix table; a fused feature data generating module, configured to extract feature data of dimensions corresponding to the target dimension combination from the waybill sample based on the target dimension combination corresponding to the target feature matrix table, and obtain fused feature data composed of the feature data; The model training module is used to use the combination results of each of the fused feature data as the sample feature data of the waybill sample to train the initial model and obtain a weight-abnormal waybill recognition model, wherein the feature matrix table construction module is also used to combine the dimensions belonging to different categories in pairs according to the waybill information collection dimensions and the categories to which the dimensions belong, to obtain the preset dimension combination; according to the feature identification set corresponding to each dimension of the preset dimension combination, the target waybill sample that matches the feature identification set is extracted from the waybill sample; the target waybill sample is classified and summarized according to positive and negative samples to obtain a feature matrix table corresponding to the dimension combination.

8. The device according to claim 7, characterized in that The feature matrix table construction module is also used to combine feature identifiers in different feature identifier sets in pairs to determine the target waybill matching conditions corresponding to each combination; and according to the target waybill matching conditions, a target waybill sample is matched from the waybill sample.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images