Customs declaration form prediction model training method and device based on unbalanced ensemble learning

Abstract

The present invention discloses a method and device for training a customs declaration form prediction model based on unbalanced ensemble learning. The method comprises the following steps: collecting historical customs declaration form data to obtain sample data, segmenting the sample data using stratified random sampling to obtain a training set and a test set; constructing a customs declaration form prediction model, which is an Xgboost model using a weighted cross entropy loss function. The weighted cross entropy loss function includes one of a multiplier weighted cross entropy loss function, a focus loss cross entropy loss function, and a gradient balanced cross entropy loss function; training the customs declaration form prediction model based on the training set, adjusting the sample weight in the customs declaration form prediction model during the training process, wherein the sample weight is the product of the initial weight and the time value weight; testing the trained customs declaration form prediction model based on the test set, and determining the adjusted network model parameters and the weighted cross entropy loss function. The method significantly enhances the model's ability to learn about customs declaration form risks and effectively improves inspection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of customs declaration form prediction, and in particular to a customs declaration form prediction model training method and device based on unbalanced ensemble learning. Background Art

[0002] Risk assessment for customs declarations is a crucial component of risk control for imports in non-trade customs operations. Existing control methods primarily consist of expert-based and manual control. Expert-based control involves business experts analyzing historical customs declaration data to identify rules and conditions and then applying control to newly submitted declarations that meet the requirements. Manual control involves risk control personnel applying control to newly submitted declarations based on a specific volume ratio, taking into account current events and hot topics. Currently, the inspection efficiency of existing passenger baggage control methods ranges from 1% to 3%, indicating low inspection efficiency. Inspection efficiency is defined as the number of risks detected divided by the number of items inspected.

[0003] Customs declaration risks are primarily categorized into tax-related risks and security access risks. For each risk, a binary classification approach is used to determine whether a declaration carries tax-related or security access risks. For binary classification problems, the cross-entropy loss function is commonly used. The proper use of the cross-entropy loss function presupposes an even distribution of the two class labels in the training set. However, for historical customs declaration data on non-trade items, this distribution is extremely unbalanced. Currently, historical declarations in relevant areas are categorized into three types: blacklist (risks identified after inspection), whitelist (no risks identified after inspection), and graylist (no inspection). To ensure smooth passenger entry, the overall inspection rate is low. For the existing passenger luggage security access risk data, the graylist:whitelist:blacklist ratio is 800:70:1. For tax-related risk data, the graylist:whitelist:blacklist ratio is 220:50:1. A blacklist / whitelist ratio exceeding 4:1 will affect the model's final classification performance. In passenger baggage risk detection, graylists have unknown inspection results. However, based on inspection efficiency and the proportion of blacklisted items in the overall data, it can be determined that the actual proportion of whitelisted items within the graylist is also very large. Therefore, graylists are often used as whitelists in data citations, which makes the blacklist and whitelist ratio even more extreme. Common methods for handling imbalanced binary classification problems include negative / oversampling, expanding the dataset, modifying the evaluation method, and adjusting the threshold. However, these methods are currently difficult to adapt to extremely unbalanced data. This extreme imbalance in class data increases the difficulty of model training, which also affects model performance. Summary of the Invention

[0004] In response to the aforementioned problems of extremely unbalanced category data leading to difficulty in model training, poor training results, and low efficiency in customs declaration form inspection, the embodiment of this application aims to propose a customs declaration form prediction model training method and device based on unbalanced ensemble learning to solve the technical problems mentioned in the above background technology section.

[0005] In a first aspect, an embodiment of the present application provides a method for training a customs declaration form prediction model based on unbalanced ensemble learning, comprising the following steps:

[0006] S1: Collect historical customs declaration data and mark sample categories as labels. Perform feature engineering on the historical customs declaration data to obtain feature data. Perform feature encoding on the feature data to obtain sample data. The sample data is split using stratified random sampling to obtain training and test sets.

[0007] S2, building a customs declaration form prediction model, which is an Xgboost model using a weighted cross entropy loss function. The weighted cross entropy loss function includes one of a multiplicative weighted cross entropy loss function, a focused loss cross entropy loss function, and a gradient balanced cross entropy loss function;

[0008] S3: Train the customs declaration prediction model based on the training set. During the training process, adjust the sample weights in the customs declaration prediction model. The sample weight is the product of the initial weight and the time value weight. The initial weight is set according to the sample category, and the time value weight is set according to the distance between the time when the historical customs declaration data occurred and the training time.

[0009] S4, testing the trained customs declaration form prediction model according to the test set, and determining the adjusted network model parameters and weighted cross entropy loss function based on the fact that the trained customs declaration form prediction model meets the preset requirements.

[0010] In some embodiments, in step S1, feature engineering is performed on historical customs declaration data to obtain feature data, specifically including:

[0011] Perform feature processing based on the data type of historical customs declaration data to obtain the features of each data type;

[0012] Feature screening is performed based on the information gain of each feature to obtain feature data.

[0013] In some embodiments, the feature encoding method in step S1 includes category encoding and binning.

[0014] In some embodiments, in step S1, the sample data is segmented using a stratified random sampling method, specifically including: dividing the sample data into a training set and a test set in proportion using stratified sampling, and training the customs declaration prediction model using a k-fold cross-validation method on the training set.

[0015] In some embodiments, the multiplicative weighted cross entropy loss function in step S2 is expressed as:

[0016]

[0017] Among them, y i Indicates the category of sample data i, y i =1 means the sample is blacklisted, y i =0 means the sample is a whitelist, m is the total number of sample data, It represents the predicted probability value of sample data i being on the blacklist, and α represents the multiplication coefficient;

[0018] The focus loss cross entropy loss function is expressed as:

[0019]

[0020] Where γ represents the exponential coefficient;

[0021] The gradient balanced cross entropy loss function is expressed as:

[0022]

[0023]

[0024] Among them, p i It represents the predicted probability value of sample data i being blacklisted. β is obtained by the gradient density of each sample data i. The calculation formula of β is as follows:

[0025]

[0026] Among them, GD(gi) is the gradient density, which is calculated by the predicted probability value p i With the actual value p i * Absolute value of the difference|p i -p i * |Calculated.

[0027] In some embodiments, parameter tuning is performed before training the customs declaration prediction model. The parameters include learning rate, maximum step size, minimum number of leaf nodes, number of training rounds, L1 regularization term coefficient, random feature selection ratio, multiplication coefficient α of the weighted cross entropy loss function, and exponential coefficient γ. A grid search method is used to determine the parameter combination that performs best in the test set.

[0028] In a second aspect, an embodiment of the present application provides a customs declaration form prediction method based on unbalanced ensemble learning, comprising the following steps:

[0029] T1, obtain customs declaration data, perform feature engineering processing and feature encoding on the customs declaration data, and obtain the encoded customs declaration data;

[0030] T2, the customs declaration prediction model trained by the above-mentioned customs declaration prediction model training method based on unbalanced ensemble learning is used to predict the encoded customs declaration data, and the sample category and prediction probability of the customs declaration data are predicted.

[0031] In a third aspect, an embodiment of the present application provides a customs declaration form prediction model training device based on unbalanced ensemble learning, comprising:

[0032] The data acquisition module is configured to collect historical customs declaration data, mark sample categories as labels, perform feature engineering on the historical customs declaration data to obtain feature data, perform feature encoding on the feature data to obtain sample data, and segment the sample data using stratified random sampling to obtain training and test sets;

[0033] a model building module configured to build a customs declaration form prediction model, the customs declaration form prediction model being an Xgboost model that uses a weighted cross entropy loss function, wherein the weighted cross entropy loss function includes one of a multiplicative weighted cross entropy loss function, a focused loss cross entropy loss function, and a gradient balanced cross entropy loss function;

[0034] The model training module is configured to train the customs declaration prediction model based on the training set and adjust the sample weights in the customs declaration prediction model during the training process. The sample weight is the product of the initial weight and the time value weight. The initial weight is set according to the sample category, and the time value weight is set according to the distance between the time when the historical customs declaration data occurred and the training time.

[0035] The model testing module is configured to test the trained customs declaration prediction model based on the test set, and determine the adjusted network model parameters and weighted cross entropy loss function based on whether the trained customs declaration prediction model meets the preset requirements.

[0036] In a fourth aspect, an embodiment of the present application provides a customs declaration form prediction device based on unbalanced ensemble learning, comprising:

[0037] a data processing module configured to obtain customs declaration form data, perform feature engineering processing and feature encoding on the customs declaration form data, and obtain encoded customs declaration form data;

[0038] The prediction module is configured to use the customs declaration prediction model trained by the customs declaration prediction model training device based on unbalanced ensemble learning to predict the encoded customs declaration data, and predict the sample category and prediction probability of the customs declaration data.

[0039] In a fifth aspect, an embodiment of the present application provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0040] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The customs declaration form prediction model training method based on unbalanced ensemble learning of the present invention adds three types of weighted cross entropy loss functions to the Xgboost model, and sets the sample weights and stratified random sampling method of the training data.

[0043] (2) Compared with the existing technology, the customs declaration form prediction model training method based on unbalanced ensemble learning of the present invention significantly enhances the model's ability to learn about customs declaration form risks.

[0044] (3) The present invention's customs declaration form prediction method based on unbalanced ensemble learning increases the overall inspection efficiency to 8.9%, a significant improvement over the 1%-3% inspection efficiency of existing prediction methods. Furthermore, the model predicts the service interface response time within 1 second, significantly shortening the risk prediction time. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a diagram of an exemplary device architecture to which an embodiment of the present application may be applied;

[0047] Figure 2 Schematic diagram of the flow of a customs declaration form prediction model training method based on unbalanced ensemble learning according to an embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the training process of the customs declaration form prediction model training method based on unbalanced ensemble learning according to an embodiment of the present invention;

[0049] Figure 4Schematic diagram of the flow of a customs declaration form prediction method based on unbalanced ensemble learning according to an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the model prediction process of the customs declaration form prediction method based on unbalanced ensemble learning according to an embodiment of the present invention;

[0051] Figure 6 Schematic diagram of a customs declaration form prediction model training device based on unbalanced ensemble learning according to an embodiment of the present invention;

[0052] Figure 7 Schematic diagram of a customs declaration form prediction device based on unbalanced ensemble learning according to an embodiment of the present invention;

[0053] Figure 8 It is a structural diagram of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0055] Figure 1 An exemplary device architecture 100 is shown to which the customs declaration form prediction model training method based on unbalanced ensemble learning or the customs declaration form prediction model training device based on unbalanced ensemble learning according to an embodiment of the present application can be applied.

[0056] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0057] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, 103.

[0058] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.

[0059] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0060] It should be noted that the customs declaration form prediction model training method based on unbalanced ensemble learning provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. Accordingly, the customs declaration form prediction model training device based on unbalanced ensemble learning can be set in the server 105 or in the terminal devices 101, 102, and 103.

[0061] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is merely illustrative. Any number of terminal devices, networks, and servers may be provided as needed. If the processed data does not need to be acquired remotely, the above-described apparatus architecture may not include a network, but only require servers or terminal devices.

[0062] Figure 2 The embodiment of the present application provides a method for training a customs declaration form prediction model based on unbalanced ensemble learning, which includes the following steps:

[0063] S1, collect historical customs declaration data and mark the sample categories as labels, perform feature engineering on the historical customs declaration data to obtain feature data, perform feature encoding on the feature data to obtain sample data, and split the sample data using stratified random sampling to obtain training set and test set.

[0064] In a specific embodiment, in step S1, feature engineering is performed on historical customs declaration data to obtain feature data, specifically including:

[0065] Perform feature processing based on the data type of historical customs declaration data to obtain the features of each data type;

[0066] Feature screening is performed based on the information gain of each feature to obtain feature data.

[0067] Specifically, during feature engineering, the characteristics of customs declaration data are categorized into text, numerical, and itinerary data. Different types of features are processed differently. For example, data derivation of document-related features can yield additional data, such as age and birthplace. Feature engineering aims to identify valuable features. Because the decision tree-based model algorithm is used, the features generated through feature engineering can be further filtered based on the information gain of each feature during model training.

[0068] In a specific embodiment, the feature encoding method in step S1 includes category encoding and binning.

[0069] Specifically, the main function of feature encoding is to digitize different types of feature data, discretize continuous features, and encode feature data into data suitable for Xgboost model training. The embodiments of this application mainly use category encoding (label encoding) and binning (bin). Because the Xgboost model has no effect on the numerical value represented by the feature classification, there is no need to consider the encoding order problem during category encoding.

[0070] In a specific embodiment, in step S1, the sample data is segmented using a stratified random sampling method, specifically including: dividing the sample data into a training set and a test set in proportion using stratified sampling, and training the customs declaration form prediction model using a k-fold cross-validation method on the training set.

[0071] Specifically, the sample data was split into a training set and a test set in a 7:3 ratio through stratified random sampling. During the training of the customs declaration prediction model, the training set was divided into 10 equal parts for cross-validation. During each training session, 1 / 10 of the training set data was randomly reserved as the validation set. This was repeated 10 times, and the results were averaged for validation. The test set was not used in the training iterations; only the performance of the trained model was tested to ensure that there was no data overlap between the training and test sets.

[0072] Stratified sampling is primarily used during the splitting phase between the training and test sets. Due to data imbalance, random sampling can cause fluctuations in the class ratios of training set samples, and in extreme cases, may even result in class loss. The idea behind stratified sampling is to randomly sample samples of different classes using the same proportions, ensuring consistent class ratios across the training and test sets. The validation dataset uses a complete collection of recent customs declarations.

[0073] S2, build a customs declaration form prediction model. The customs declaration form prediction model is an Xgboost model that uses a weighted cross entropy loss function. The weighted cross entropy loss function includes one of the multiplier weighted cross entropy loss function, the focus loss cross entropy loss function, and the gradient balanced cross entropy loss function.

[0074] Specifically, the Xgboost model is an open-source machine learning project that improves upon the Gradient Boosted Decision Tree (GBDT) model. The Xgboost model is an ensemble learning algorithm that uses boosting to improve model accuracy. During training, after each iteration, the algorithm calculates the difference between the current model's predicted value and the actual sample value, and training continues based on this difference in the next iteration. Parallel computation is facilitated by replacing the Taylor second-order expansion of the overall objective function with each additional training round. To achieve this, the model loss function must be second-order differentiable.

[0075] In the method for solving the unbalanced risk classification problem, the embodiments of the present application propose a weighted cross entropy loss function, using three different weighting methods, corresponding to three different model training situations, namely, multiplier weighted cross entropy loss function (WeightCrossEntropy), focus loss cross entropy loss function (FocalLoss-CrossEntropy), and gradient harmonizing mechanism cross entropy loss function (GradientHarmonizingMechanism-CrossEntropy), and combining sample weights and stratified sampling data segmentation to systematically solve the customs declaration risk prediction problem through an unbalanced ensemble learning algorithm.

[0076] The original cross entropy loss function formula is as follows:

[0077]

[0078] Among them, y i Indicates the category of sample data i, y i =1 means the sample is blacklisted, y i =0 means the sample is whitelisted, p i Indicates the predicted probability value of sample data i being blacklisted.

[0079] In a specific embodiment, the multiplier weighted cross entropy loss function in step S2 is expressed as:

[0080]

[0081] Among them, y i Indicates the category of sample data i, y i =1 means the sample is blacklisted, y i=0 means the sample is a whitelist, m is the total number of sample data, represents the predicted probability that sample data i is blacklisted, and α represents the multiplier coefficient. When α increases, the loss function of blacklisted data samples will be amplified, thereby biasing the model weight updates toward learning actual blacklisted samples. When α decreases, the loss function of whitelisted data samples will be amplified, thereby biasing the model weight updates toward learning actual whitelisted samples.

[0082] The focus loss cross entropy loss function is expressed as:

[0083]

[0084] Where γ represents the exponential coefficient; when the γ coefficient is determined, for y i =1, but For samples with smaller values ​​(indicating that they are predicted to be blacklisted, but the probability of blacklisting is small), the loss function will be amplified; for y i =0, but The loss function will also be amplified for samples with larger predictions (indicating a whitelist prediction but a high blacklist probability). Through the exponential coefficient, the model's weight updates tend to learn samples with opposite predictions and large probability gaps.

[0085] The gradient balanced cross entropy loss function is expressed as:

[0086]

[0087] Among them, β is obtained by the gradient density of each sample data i, and the calculation formula of β is as follows:

[0088]

[0089] Among them, GD(g i ) is the gradient density, which is calculated by the predicted probability value p i With the actual value p i * Absolute value of the difference|p i -p i * Finally, the gradient-balanced cross-entropy loss function is calculated. The gradient-balanced cross-entropy loss function assumes that the gradient density distribution is high on the edges and low in the middle, and that the density of valuable, difficult-to-learn samples is low. Therefore, a large coefficient β amplifies the corresponding loss function, thereby biasing the model's weight updates toward learning valuable, difficult-to-learn samples.

[0090] The three weighting methods of the weighted cross entropy loss function are independent and suitable for different scenarios. One method is selected during each training, and each method will perform differently in different data situations. The multiplicative weighted cross entropy loss function is mainly suitable for simple data imbalances, the focused loss cross entropy loss function is mainly suitable for situations with large deviations in prediction probabilities, and the gradient-balanced cross entropy loss function is mainly adjusted based on the gradient density distribution. It is suitable for cases where the calculation results of the key data model are relatively discrete, and does not require additional settings for α or γ. Generally, they are trained separately, and the optimal selection is selected based on different test results. This process is carried out simultaneously with the parameter tuning process.

[0091] In a specific embodiment, the training process of the customs declaration form prediction model based on unbalanced ensemble learning in the embodiment of the present application is as follows: Figure 3 As shown. Parameters are tuned before the customs declaration form prediction model is trained. The parameters include learning rate, maximum step size, minimum number of leaf nodes, number of training rounds, L1 regularization term coefficient, random feature selection ratio, multiplier coefficient α of weighted cross entropy loss function, exponential coefficient γ, and a grid search method is used to determine the parameter combination that performs best in the test set. Parameters that need to be specified before model training are tuned, such as learning rate (learning_rate), maximum step size (max_depth), minimum number of leaf nodes (min_child_weight), number of training rounds (n_estimators), L1 regularization term coefficient (reg_alpha), random feature selection ratio (colsample_bytree), weighted loss function coefficients (α, γ), etc. The embodiment of the present application adopts a grid search method in parameter tuning. By defining the selection range of each parameter, the search space of the parameter combination is determined, and the parameter combination that performs best in the test set is determined. In order to shorten the calculation time of the grid search, when defining the parameter selection range, a large range is first defined, and then the range is further narrowed. Finally, a determined parameter combination is used to train the final model in the complete training data.

[0092] S3, train the customs declaration prediction model based on the training set, and adjust the sample weight in the customs declaration prediction model during the training process. The sample weight is the product of the initial weight and the time value weight. The initial weight is set according to the sample category, and the time value weight is set according to the distance between the occurrence time of the historical customs declaration data and the training time.

[0093] Specifically, the calculation formula for sample weight is: sample weight = initial weight * time value weight;

[0094] The initial weight is assigned based on the category label. The initial weight of the blacklist and whitelist is 2, and the initial weight of the graylist is 1. The time value weight is converted based on the distance between the customs declaration data and the current time. The closer the distance, the higher the time value weight.

[0095] Sample weight can be understood as the importance of the sample. The more important the sample, the more important it will be in model training. The main change is the sample count, which changes the amount of information of the entire sample. In the embodiment of the present application, the sample weight attached to each sample data is different, and the sample weight is calculated by the sample label and the sample occurrence time. If the sample data belongs to the black or white list (with on-site inspection), the initial weight is set to 2, and the gray list (not inspected) is set to 1. The time value weight is calculated according to the distance between the sample data occurrence time and the model training date (training time). The closer the time distance, the higher the time value weight, ensuring that the risk of the most recent customs declaration has a greater impact on the model. Finally, the sample weight is obtained by multiplying the initial weight and the time value weight.

[0096] For the training set data, cross-validation is used to build the model, and a weighted loss function is referenced. During the model iteration process, the first-order and second-order derivatives of the loss function are used to quickly calculate the loss function value of the current round. For samples with large loss function values, they will be given special attention in the next round of iteration.

[0097] S4, testing the trained customs declaration form prediction model according to the test set, and determining the adjusted network model parameters and weighted cross entropy loss function based on the fact that the trained customs declaration form prediction model meets the preset requirements.

[0098] Specifically, after the test set is tested and meets the requirements, the trained customs declaration prediction model will be saved. The saved content includes the model file and feature encoder, which are saved in the form of files to facilitate model calling.

[0099] Figure 4 The embodiment of the present application provides a customs declaration form prediction method based on unbalanced ensemble learning, which includes the following steps:

[0100] T1, obtain the customs declaration data, perform feature engineering processing and feature encoding on the customs declaration data, and obtain the encoded customs declaration data.

[0101] Specifically, the training process of the customs declaration form prediction model based on unbalanced ensemble learning in the embodiment of the present application is as follows: Figure 5As shown in the figure, during the model application process, the prediction service interface receives a request for risk prediction of a customs declaration, obtains the customs declaration data using the request key, and performs a preliminary assessment of whether the declaration data meets the prediction requirements. Through the service interface, the model prediction process can be processed concurrently to avoid request congestion. The obtained customs declaration data is then subjected to feature engineering and encoding. Feature engineering uses the same processing methods as model training to ensure that the model prediction data and the training set have consistent feature fields. The feature encoding step first reads the encoder file saved upon model training completion and uses the same encoder for feature encoding. This ensures that the encoded customs declaration data has consistent feature content with the training set. The encoded customs declaration data serves as the predicted data.

[0102] T2, the customs declaration prediction model trained by the above-mentioned customs declaration prediction model training method based on unbalanced ensemble learning is used to predict the encoded customs declaration data, and the sample category and prediction probability of the customs declaration data are predicted.

[0103] Specifically, the model file is read, and predictions are made on the encoded customs declaration data. Based on the prediction results, the predicted sample category and predicted probability of the customs declaration data are returned. The model prediction results are returned according to the process requirements, and the relevant data is saved at the same time.

[0104] The embodiments of this application adopt a customs declaration form prediction model training method based on unbalanced ensemble learning. After adding a weighted loss function, training data sample weights, and stratified random sampling, the model's learning ability for customs declaration form risks is significantly enhanced compared to existing technologies. Based on the experimental results from March 2021 to October 2021, the overall inspection efficiency of the model reached 8.9%, which is a significant improvement compared to the 1%-3% inspection efficiency of existing prediction methods. At the same time, the response time of the model prediction service interface is within 1 second, which greatly shortens the risk prediction time.

[0105] While traditional ensemble learning methods achieve only 3%-4% validation efficiency on a test set when all other parameters are consistent, this method achieves 12% validation efficiency on the same test set, a significant improvement compared to the test set. This demonstrates that the method proposed in the embodiments of this application is more effective in extremely unbalanced data, such as customs declaration risk prediction.

[0106] Further references Figure 6 As an implementation of the methods shown in the above figures, this application provides an embodiment of a customs declaration form prediction model training device based on unbalanced ensemble learning. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0107] The embodiment of the present application provides a customs declaration form prediction model training device based on unbalanced ensemble learning, which is characterized by comprising:

[0108] Data acquisition module 1 is configured to collect historical customs declaration data, mark sample categories as labels, perform feature engineering on the historical customs declaration data to obtain feature data, perform feature encoding on the feature data to obtain sample data, and segment the sample data using stratified random sampling to obtain training and test sets;

[0109] Model building module 2 is configured to build a customs declaration form prediction model, the customs declaration form prediction model is an Xgboost model using a weighted cross entropy loss function, the weighted cross entropy loss function includes one of a multiplier weighted cross entropy loss function, a focus loss cross entropy loss function, and a gradient balanced cross entropy loss function;

[0110] Model training module 3 is configured to train the customs declaration prediction model based on the training set, and adjust the sample weights in the customs declaration prediction model during the training process. The sample weight is the product of the initial weight and the time value weight. The initial weight is set according to the sample category, and the time value weight is set according to the distance between the time when the historical customs declaration data occurred and the training time.

[0111] The model testing module 4 is configured to test the trained customs declaration form prediction model according to the test set, and determine the adjusted network model parameters and weighted cross entropy loss function based on whether the trained customs declaration form prediction model meets the preset requirements.

[0112] Further references Figure 7 As an implementation of the methods shown in the above figures, this application provides an embodiment of a customs declaration form prediction model training device based on unbalanced ensemble learning. Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] The embodiment of the present application provides a customs declaration form prediction device based on unbalanced ensemble learning, comprising:

[0114] The data processing module 5 is configured to obtain customs declaration form data, perform feature engineering processing and feature encoding on the customs declaration form data, and obtain encoded customs declaration form data;

[0115] The prediction module 6 is configured to use the customs declaration prediction model trained by the customs declaration prediction model training device based on unbalanced ensemble learning to predict the encoded customs declaration data, and predict the sample category and prediction probability of the customs declaration data.

[0116] Reference below Figure 8, which shows an electronic device (eg Figure 1 A schematic structural diagram of a computer device 600 (a server or terminal device as shown). Figure 6 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0117] like Figure 6 As shown, the computer device 600 includes a central processing unit (CPU) 601 and a graphics processing unit (GPU) 602, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 603 or the program loaded from the storage part 609 to the random access memory (RAM) 604. Various programs and data required for the operation of the device 600 are also stored in the RAM 604. The CPU 601, GPU 602, ROM 603 and RAM 604 are connected to each other via a bus 605. An input / output (I / O) interface 606 is also connected to the bus 605.

[0118] The following components are connected to the I / O interface 606: an input section 607 including a keyboard, a mouse, and the like; an output section 608 including a display such as a liquid crystal display (LCD), a speaker, and the like; a storage section 609 including a hard disk and the like; and a communication section 610 including a network interface card such as a LAN card or a modem. The communication section 610 performs communication processing via a network such as the Internet. A drive 611 may also be connected to the I / O interface 606 as needed. A removable medium 612, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 611 as needed, so that a computer program read therefrom can be installed into the storage section 609 as needed.

[0119] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 610, and / or installed from a removable medium 612. When the computer program is executed by the central processing unit (CPU) 601 and the graphics processing unit (GPU) 602, the above-mentioned functions defined in the method of the present application are executed.

[0120] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable medium, or any combination of the two. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0121] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The modules involved in the embodiments described in this application may be implemented in software or hardware, and may also be set in a processor.

[0124] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device: collects historical customs declaration data, and marks the sample categories as labels, performs feature engineering on the historical customs declaration data to obtain feature data, performs feature encoding on the feature data to obtain sample data, and divides the sample data by stratified random sampling to obtain a training set and a test set; constructs a customs declaration prediction model, which is an Xgboost model using a weighted cross entropy loss function, and the weighted cross entropy loss function includes a multiplier weighted cross One of the entropy loss function, focus loss cross entropy loss function and gradient balanced cross entropy loss function; train the customs declaration prediction model according to the training set, adjust the sample weights in the customs declaration prediction model during the training process, the sample weight is the product of the initial weight and the time value weight, the initial weight is set according to the sample category, and the time value weight is set according to the distance between the occurrence time of the historical customs declaration data and the training time; test the trained customs declaration prediction model according to the test set, and determine the adjusted network model parameters and weighted cross entropy loss function based on the fact that the trained customs declaration prediction model meets the preset requirements.

[0125] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A customs declaration prediction model training method based on unbalanced ensemble learning, characterized by: The following steps are involved: S1, collect historical customs declaration data and mark sample categories as labels, perform feature engineering on the historical customs declaration data to obtain feature data, perform feature encoding on the feature data to obtain sample data, and split the sample data using stratified random sampling to obtain training and test sets; S2, constructing a customs declaration form prediction model, which is an Xgboost model using a weighted cross entropy loss function. The weighted cross entropy loss function includes one of a multiplier weighted cross entropy loss function, a focus loss cross entropy loss function, and a gradient balanced cross entropy loss function; the multiplier weighted cross entropy loss function is expressed as: Among them, y i Indicates the category of sample data i, y i =1 means the sample is blacklisted, y i =0 means the sample is a whitelist, m is the total number of sample data, It represents the predicted probability value of sample data i being on the blacklist, and α represents the multiplication coefficient; The focus loss cross entropy loss function is expressed as: Where γ represents the exponential coefficient; The gradient balanced cross entropy loss function is expressed as: Among them, p i It represents the predicted probability value of sample data i being blacklisted. β is obtained by the gradient density of each sample data i. The calculation formula of β is as follows: Among them, GD(g i ) is the gradient density, which is calculated by the predicted probability value p i With the actual value p i * Absolute value of the difference|p i -p i * |Calculated; S3, training the customs declaration form prediction model based on the training set, and adjusting the sample weights in the customs declaration form prediction model during the training process, wherein the sample weight is the product of the initial weight and the time value weight, wherein the initial weight is set according to the sample category, and the time value weight is set according to the distance between the time when the historical customs declaration form data was generated and the training time; S4, testing the trained customs declaration form prediction model according to the test set, and determining adjusted network model parameters and weighted cross entropy loss function based on whether the trained customs declaration form prediction model meets preset requirements.

2. The customs declaration form prediction model training method based on unbalanced ensemble learning according to claim 1 is characterized in that: In step S1, the historical customs declaration data is subjected to feature engineering processing to obtain feature data, specifically including: Perform feature processing based on the data type of the historical customs declaration data to obtain features under each data type; Feature screening is performed according to the information gain of each feature to obtain the feature data.

3. The customs declaration form prediction model training method based on unbalanced ensemble learning according to claim 1 is characterized in that: The feature encoding method in step S1 includes category encoding and binning.

4. The customs declaration form prediction model training method based on unbalanced ensemble learning according to claim 1 is characterized in that: In step S1, the sample data is segmented by stratified random sampling, specifically comprising: segmenting the sample data into a training set and a test set in proportion by stratified sampling, and training the customs declaration form prediction model by using a k-fold cross-validation method on the training set.

5. The customs declaration form prediction model training method based on unbalanced ensemble learning according to claim 1 is characterized in that: Parameters are tuned before training the customs declaration prediction model. The parameters include learning rate, maximum step size, minimum number of leaf nodes, number of training rounds, L1 regularization term coefficient, random feature selection ratio, multiplication coefficient α and exponential coefficient γ of the weighted cross entropy loss function, and a grid search method is used to determine the parameter combination that performs best in the test set.

6. A customs declaration form prediction method based on unbalanced ensemble learning, characterized in that: The following steps are involved: T1, obtaining customs declaration data, and performing feature engineering processing and feature encoding on the customs declaration data to obtain encoded customs declaration data; T2. Use the customs declaration prediction model trained by the customs declaration prediction model training method based on unbalanced ensemble learning according to any one of claims 1 to 5 to predict the encoded customs declaration data, and predict the sample category and prediction probability of the customs declaration data.

7. A customs declaration form prediction model training device based on unbalanced ensemble learning, characterized in that: include: a data acquisition module configured to collect historical customs declaration data, mark sample categories as labels, perform feature engineering on the historical customs declaration data to obtain feature data, perform feature encoding on the feature data to obtain sample data, and segment the sample data using stratified random sampling to obtain a training set and a test set; The model building module is configured to build a customs declaration form prediction model, wherein the customs declaration form prediction model is an Xgboost model using a weighted cross entropy loss function, wherein the weighted cross entropy loss function includes one of a multiplier weighted cross entropy loss function, a focus loss cross entropy loss function, and a gradient balanced cross entropy loss function; the multiplier weighted cross entropy loss function is expressed as: Among them, y i Indicates the category of sample data i, y i =1 means the sample is blacklisted, y i =0 means the sample is a whitelist, m is the total number of sample data, It represents the predicted probability value of sample data i being on the blacklist, and α represents the multiplication coefficient; The focus loss cross entropy loss function is expressed as: Where γ represents the exponential coefficient; The gradient balanced cross entropy loss function is expressed as: Among them, p i It represents the predicted probability value of sample data i being blacklisted. β is obtained by the gradient density of each sample data i. The calculation formula of β is as follows: Among them, GD(g i ) is the gradient density, which is calculated by the predicted probability value p i With the actual value p i * Absolute value of the difference|p i -p i * |Calculated; a model training module configured to train the customs declaration form prediction model based on the training set, and adjust the sample weights in the customs declaration form prediction model during the training process, wherein the sample weight is the product of an initial weight and a time value weight, wherein the initial weight is set according to the sample category, and the time value weight is set according to the distance between the time when the historical customs declaration form data was generated and the training time; The model testing module is configured to test the trained customs declaration form prediction model according to the test set, and determine the adjusted network model parameters and weighted cross entropy loss function based on whether the trained customs declaration form prediction model meets the preset requirements.

8. A customs declaration form prediction device based on unbalanced ensemble learning, characterized in that: include: a data processing module configured to obtain customs declaration form data, and perform feature engineering processing and feature encoding on the customs declaration form data to obtain encoded customs declaration form data; The prediction module is configured to use the customs declaration prediction model trained by the customs declaration prediction model training device based on unbalanced ensemble learning according to claim 7 to predict the encoded customs declaration data, and predict the sample category and prediction probability of the customs declaration data.

9. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images