Customs commodity classification method and system based on multiple features of table

Abstract

The invention discloses a customs commodity classification method and system based on multiple features of a table, and relates to the technical field of customs commodity classification. Comprising the steps that tabular customs commodity declaration data are acquired, and the data comprise numerical features, category features and text features; piecewise linear coding PLE is adopted for numerical value features, integer mapping is adopted for category features to be combined with an embedded layer, and RoBERTa pre-training model is adopted for text features to be combined with a linear transformation layer to be processed; stacking the three types of feature embedding vectors into an embedding matrix, and inputting the embedding matrix into a model based on a Transform architecture to complete multi-feature deep fusion; generating an adversarial sample, constructing a positive and negative sample pair, and combining cross entropy loss and contrast loss to train a model; and inputting to-be-classified tabular customs commodity declaration data into the trained model, and outputting a coordination system code corresponding to the commodity. According to the invention, the time cost and labor cost of manual auditing can be reduced, the error rate of manual classification can be effectively reduced, and the fairness and rationality of tax collection are ensured.

Description

Technical Field

[0001] This invention relates to the field of customs commodity classification technology, specifically to a customs commodity classification method and system based on multiple features of a table. Background Technology

[0002] With the rapid development of international trade, China's total import and export trade value continued to grow from 2020 to 2024, and the variety of commodities increased daily. Customs faced enormous regulatory pressure in ensuring the rapid clearance and tax rate review of import and export commodities. Customs commodity classification is the basis for determining import tariffs and export tax rebates. Different categories of commodities are subject to different tax rates. Accurate classification not only ensures fair and reasonable tariff collection but also serves as an important basis for the country to formulate trade policies and conduct economic analysis.

[0003] Customs commodity classification essentially involves determining the Harmonized System (HS) code to which a commodity belongs based on the commodity declaration text. However, customs commodity declaration data is presented in tabular form, containing multiple features such as numerical values, categories, and text. These features are complex and often suffer from spelling errors, missing key information, and inconsistent data quality. Traditional classification methods have significant shortcomings: 1. Although machine learning-based text classification methods are computationally efficient, they perform poorly in modeling high-order interactions between features and in handling high-dimensional sparse features, making them difficult to handle the complexity of customs form data. 2. While deep learning-based text classification methods can automatically learn feature representations, they lack an effective mechanism to integrate numerical, category, and textual features from customs data. Furthermore, they are sensitive to data noise and are prone to a decrease in classification accuracy due to noise interference. Summary of the Invention

[0004] The purpose of this invention is to propose a customs commodity classification method and system based on multiple features of a table, which can effectively integrate multiple types of features and resist data noise, thereby improving the accuracy and robustness of customs commodity classification.

[0005] According to a first aspect of the embodiments of this disclosure, a customs commodity classification method based on tabular multiple features is provided, comprising the following steps: Obtain tabular customs commodity declaration data, which includes numerical features, category features, and text features; Piecewise linear encoding (PLE) is used for numerical features, integer mapping combined with an embedding layer is used for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer is used for text features to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. The three types of feature embedding vectors are stacked into an embedding matrix, which is then input into a Transformer-based model. The interaction relationship between features is explored through a self-attention mechanism to achieve deep fusion of multiple features. Generate adversarial examples and construct positive and negative sample pairs, and train the model by combining cross-entropy loss and contrastive loss; The tabular customs commodity declaration data to be classified is input into the trained model, which outputs the corresponding Harmonized System (HS) code for the commodity, thereby realizing the automatic classification of customs commodities.

[0006] In one embodiment, the numerical features are processed using piecewise linear encoding (PLE), which divides the feature value range based on the number of class labels, constructs a decision tree, and uses the leaf node region as the division interval; and incorporates the feature index order information into the encoding through a linear layer, specifically:

[0007] In the formula: For input numerical feature values, To divide the total number of intervals, This is the index of the current interval. Let be the upper boundary value of the t-th interval. Let be the upper boundary value of the (t-1)th interval. Let be the activation weight of the t-th interval; Introduce a linear layer to add sequence information to the encoding:

[0008] In the formula, v t It is for each interval B t Assigned trainable embedding vectors, v 0 is a bias vector, in the PLE vector e t Used as weights for the corresponding embedding vectors; Linear(.) denotes a linear layer.

[0009] In one embodiment, the category features are processed using integer mapping combined with an embedding layer. First, each value of the category feature is mapped to a unique integer, then one-hot encoding is performed to obtain a sparse binary vector, which is finally processed through the embedding layer to obtain:

[0010] In the formula: The encoded vector, This is the weight matrix. This is a category bias term.

[0011] In one embodiment, text standardization, word segmentation, and length unification preprocessing are performed before text feature embedding; after the RoBERTa pre-trained model generates context-aware word embedding vectors, the dimensions are adjusted through a linear transformation layer.

[0012] In the formula: Let represent the embedding vector of the j-th text feature obtained after processing. This indicates that the original text is being used. After being input into the RoBERTa pre-trained model, the model generates context-aware word embedding vectors for the text; () represents a linear transformation layer. Indicates the text bias term.

[0013] In one embodiment, the model based on the Transformer architecture is the HScode-Former model, which performs feature fusion on the embedding matrix through the Transformer layer to fully explore the interaction relationship between numerical, category, and text features.

[0014] In one embodiment, the adversarial example is generated by obtaining the gradient of the loss function with respect to the word embedding matrix of the RoBERTa model and applying a perturbation to the word embedding matrix according to the following formula;

[0015] In the formula: The disturbance coefficient is... () is a sign function. The gradient of the loss function. For the model to be trained, HS-coded labels for samples; The rules for constructing positive and negative sample pairs are as follows: the original sample and the corresponding adversarial sample are positive sample pairs, and the original sample and other random samples are negative sample pairs. The cross-entropy loss is:

[0016] In the formula: For the sample size, For the number of HS code categories, For sample category labels, To predict probabilities, Label the hidden states for the RoBERTa model [CLS].

[0017] In one embodiment, the contrastive loss constrains the sample feature space distribution through cosine similarity. First, a nonlinear mapping is performed on the [CLS]-labeled hidden states of the original sample and the adversarial sample, and then the contrastive loss is calculated based on the mapping result. ; The final loss formula is:

[0018] In the formula: These are the weighting coefficients. The cross-entropy classification loss is the value corresponding to the original sample V. To use the cross-entropy classification loss corresponding to the adversarial samples.

[0019] According to a second aspect of the present disclosure, a customs commodity classification system based on tabular multiple features is provided, comprising: The data acquisition module acquires tabular customs commodity declaration data, which includes numerical features, category features, and text features. The multi-feature embedding module uses piecewise linear encoding (PLE) for numerical features, integer mapping combined with an embedding layer for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer for text features, to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. The feature fusion module stacks the embedding vectors of the three types of features into an embedding matrix, inputs it into the model based on the Transformer architecture, and mines the interaction relationship between features through the self-attention mechanism to complete the deep fusion of multiple features. The model training module generates adversarial examples and constructs positive and negative sample pairs, and trains the model by combining cross-entropy loss and contrastive loss. The commodity classification module takes the tabular customs commodity declaration data to be classified, inputs it into the trained model, and outputs the corresponding Harmonized System (HS) code for the commodity, thus realizing the automatic classification of customs commodities.

[0020] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the memory, wherein the processor executes the program to implement the aforementioned customs commodity classification method based on tabular multiple features.

[0021] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the aforementioned customs commodity classification method based on tabular multiple features.

[0022] The advantages of the above technical solutions adopted in this invention compared with the prior art are as follows: 1. Addressing the core challenges of nonlinear numerical features, high-dimensional sparsity of categorical features, textual noise interference, and difficulty in fusing heterogeneous features in customs commodity tabular declaration data, this invention achieves a precise breakthrough through differentiated and adaptable feature processing schemes. Specifically, adaptive piecewise linear encoding divides numerical features into intervals with consistent label distribution, and combines this with linear interpolation to generate embedding vectors, effectively capturing the nonlinear correlations of numerical features; categorical features, after one-hot encoding, are transformed into dense vectors through embedding mapping while fully preserving discrete semantic information; the RoBERTa pre-trained model can deeply extract the contextual semantics of commodity description text, accurately identifying key information; and the three types of features are then deeply fused through a Transformer self-attention mechanism, fully exploring the complex interactions between features and completely solving the problem of heterogeneous feature fusion.

[0023] 2. This invention generates adversarial examples along the gradient direction in the RoBERTa embedding space to simulate text noise in real-world scenarios. It also constructs positive sample pairs between the original sample and the adversarial example, and negative sample pairs between the original sample and other samples. By jointly minimizing the cross-entropy loss and contrastive loss, it effectively constrains the model's representation space. This mechanism enables the model to accurately distinguish different product features while effectively resisting text noise interference. It ensures stable classification performance even when data contains spelling errors, non-standard formats, or missing key information, significantly reducing the impact of noise on classification results.

[0024] 3. Left and right boundaries are set for the encoding process of each feature—numerical, categorical, and textual—to fully cover the feature value range and avoid information loss due to feature values ​​exceeding the encoding interval. This ensures that features of newly added data are fully learned during incremental learning, guaranteeing the model's adaptability to new product categories. Simultaneously, the synergistic effect of multiple rounds of incremental training and contrastive learning significantly reduces the model's convergence time and improves training efficiency. Furthermore, the optimization of feature representation through contrastive learning further enhances the model's adaptability to different scenarios and types of customs product data, greatly improving the model's generalization performance and achieving high-precision HS coding automatic classification in complex dynamic environments.

[0025] 4. The automated and intelligent customs commodity classification method implemented in this invention can replace the traditional manual classification mode, assisting customs departments to significantly improve commodity review efficiency and reduce the time and labor costs of manual review; at the same time, the high-precision classification capability of the model can effectively reduce the error rate of manual classification and ensure the fairness and rationality of tariff collection. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0027] Figure 1 Example image of customs commodity declaration data; Figure 2 To create a tabular flowchart for customs commodity classification; Figure 3 This is a schematic diagram of the HScode-Former structure. Figure 4 Target-aware PLE flowchart; Figure 5 A visual illustration of PLE; Figure 6 For comparison, see the learning framework diagram; Figure 7 This is a tabular example of customs commodity classification. Detailed Implementation

[0028] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0029] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0030] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0031] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0032] Example 1: This embodiment provides a customs commodity classification method based on multiple tabular features, including the following steps: S1. Obtain tabular customs commodity declaration data, which includes numerical features, category features, and text features; like Figure 1 As shown, tabular customs commodity declaration data typically includes a large number of numerical, categorical, and textual features. These features have complex interrelationships that directly affect the accurate classification of customs commodities. Numerical features include customs duty-paid value; categorical features include country of origin and destination; and textual features include commodity specifications and models.

[0033] like Figure 2 As shown, tabular customs commodity classification falls under the category of supervised learning problems with structured data. For this problem, the dataset... D Represented as ,in Indicates the first j The data is labeled (i.e., HS code). Indicates the first j The first data point is The dataset consists of numerical features, categorical features, and textual features. It is divided into three disjoint subsets: ,in For training, Used for early stopping and hyperparameter tuning Used for final evaluation.

[0034] S2. Piecewise linear encoding (PLE) is used for numerical features, integer mapping combined with an embedding layer is used for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer is used for text features to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. like Figure 3 As shown, input features x Convert to embedded representation T :

[0035] In the formula: T represents the characteristic. For a linear layer, PLE() is piecewise linear encoding, x is the input numerical feature, and num is the number of input numerical features.

[0036] Customs commodity declaration data contains several important numerical features, such as customs duty-paid value and the first statutory quantity. Differences between these features can directly affect the final classification of commodities. To fully utilize these numerical features, piecewise linear encoding (PLE) is employed for feature extraction. Unlike directly using raw numerical values ​​as model input, PLE divides the value range of numerical features into multiple intervals and performs linear interpolation within each interval, thereby generating a more expressive vector representation. For the first... i Numerical features x i Divide its value range into T i A set of non-overlapping intervals, denoted as . Each interval Defined as ,in Indicates the boundary of the interval.

[0037] To more effectively explore the relationship between numerical features and labels, a target-aware interval partitioning method is employed. This method divides the feature value range based on the number of category labels to maximize the target consistency of the data within each interval, i.e., to keep the label distribution of each interval as uniform as possible. A decision tree is constructed, and the regions corresponding to its leaf nodes are used as the partitioning intervals for PLE (Target-Aware Leakage). T Taking =4 as an example, the specific process is as follows: Figure 4 As shown.

[0038] After defining the interval boundaries, encode each value xi using the following formula, and visualize the process as follows: Figure 5 As shown;

[0039] In the formula: For input numerical feature values, To divide the total number of intervals, This is the index of the current interval. Let be the upper boundary value of the t-th interval. Let be the upper boundary value of the (t-1)th interval. Let be the activation weight of the t-th interval.

[0040] To enable the model to perceive the order of input embeddings, additional measures must be taken to embed feature index information into the obtained encoding. Since the model is fixed and insensitive to input order, a linear layer is introduced after applying object-aware piecewise linear encoding to add order information to the encoding, thus allowing the model to capture the sequential relationships of input features. The specific formula is as follows:

[0041] in, v t It is for each interval B t Assigned trainable embedding vectors, v 0 is a bias vector, in the PLE vector e t These are used as weights for the corresponding embedding vectors. The purpose of this linear layer (Linear(.)) is to incorporate feature index information into the model's input, enabling the model to effectively utilize the numerical feature information encoded by PLE. By introducing the order information of the features, the model can better understand the sequential relationships between feature values, thereby improving its ability to model numerical features.

[0042] In customs commodity declaration data, categorical features such as country of origin and destination, and unit of measurement are key inputs in tabular customs commodity classification, providing important information about the characteristics of the commodities. To effectively utilize these categorical features, HScode-Former converts them into fixed-length embedding vectors to capture the latent semantic relationships between categories and provide effective input for subsequent Transformer layers.

[0043] For each categorical feature, first map all its possible values ​​to a unique integer code. Assume the categorical features... C The set of values ​​is Each value c i Mapped to a unique integer i Next, one-hot encoding is used to represent each category value as a sparse binary vector, ensuring that different category values ​​are mapped to a high-dimensional sparse space.

[0044] However, sparse vectors encoded with one-hot encoding cannot be directly input into deep learning models. Therefore, HScode-Former converts sparse vectors into dense, continuous vector representations through an embedding layer. The encoded vectors... With weight matrix Multiply and add the bias term. The details are as follows:

[0045] Through the embedding layer, categorical features are effectively transformed into a continuous vector representation, enabling the model to better capture the potential relationships between categories and improve the performance of subsequent models.

[0046] In customs commodity declaration data, text such as commodity names and specifications contain rich semantic information, serving as crucial input features for tabular customs commodity classification tasks. To fully leverage the semantic representation of these textual features, HScode-Former employs a RoBERTa pre-trained model to embed words into the textual data, transforming textual features into context-aware vector representations, thus providing key input for subsequent model processing.

[0047] Before inputting text into the RoBERTa model, it needs to be preprocessed. First, the text is standardized by removing redundant spaces, symbols, and special characters. Then, the RoBERTa model's tokenizer splits the text into sub-word units, assigning a unique vocabulary index to each sub-word. Next, all segmented text sequences are standardized to a fixed length. If the text is too short, it is padded; if the text is too long, it is truncated. The preprocessed text sequence is then fed into the RoBERTa model. RoBERTa utilizes its self-attention mechanism and context modeling capabilities to generate context-aware embedding vectors for each word. These word embedding vectors capture the semantic information of words in different contexts, thus better representing the meaning of the text.

[0048] To ensure consistency in how the model handles different types of features, HScode-Former uses a linear transformation layer to adjust the dimensions of the word embedding vectors generated by RoBERTa. This linear layer maps the dimensions of the generated word embedding vectors to a pre-defined fixed dimension, thus ensuring that the embedding dimensions of text features are consistent with those of numerical and categorical features, thereby ensuring effective feature fusion; as follows:

[0049] S3. Stack the three types of feature embedding vectors into an embedding matrix, input it into the Transformer-based model, and use the self-attention mechanism to explore the interaction relationship between features to complete the deep fusion of multiple features; Specifically, the embedding matrix is

[0050] In the formula: For feature representation, For multi-feature stacking, For numerical features, For category features, For text features.

[0051] To effectively address the issue of non-standard formatting in customs declaration texts, this invention designs a customs text contrastive learning framework. Adversarial examples are generated by adding minor perturbations to the original samples, simulating text noise in real-world scenarios. Then, a contrastive learning strategy is used to optimize the distribution of samples in the feature space, ensuring that semantically similar samples are close together in the embedding space, while samples with significant semantic differences are kept apart. During training, original samples and their adversarial examples form positive sample pairs, while other random samples form negative sample pairs. This training method not only enables the model to effectively distinguish between positive and negative sample pairs but also improves the model's tolerance to input perturbations through adversarial training, thereby enhancing the model's stability and generalization performance. Figure 6 The framework structure for comparative learning of customs texts was demonstrated.

[0052] S4. Generate adversarial examples and construct positive and negative sample pairs, and train the model using a combination of cross-entropy loss and contrastive loss; Adversarial examples, given a sequence of tokens The RoBERTa model outputs a series of context token representations. ,in =RoBERTa([CLS], The standard practice for fine-tuning is to add a softmax classifier to the sentence-level representation of the model, such as in RoberTa. CLS The final hidden state of ] h [CLS] :

[0053] in, c is the number of categories.

[0054] The model is trained by minimizing the cross-entropy loss:

[0055] in, N It is the sample size; Adversarial examples are data input into a model after being slightly perturbed; this perturbation can cause the model to misclassify. Joint training using original samples and adversarial examples is a method to improve the robustness of the model, given a loss function. ,in It is a neural network parameterized by θ. x i It is the input sample, under perturbation r To maximize the loss function, apply a maximum norm constraint:

[0056] Using a first-order approximation, the loss function can be expressed as:

[0057] Solving the above two equations, we obtain the perturbation. r :

[0058] Here, sign() is the sign function, which returns the sign of the gradient, indicating whether the input value is increased or decreased along the gradient direction: Unlike traditional word embedding perturbation methods, HScode-Former directly perturbs the word embedding matrix V of the Transformer encoder to generate adversarial examples. The model first performs forward propagation using the original samples, and then... The gradient of the loss function with respect to the word embedding matrix V is calculated to identify the parts of the word embedding matrix that need adjustment, thereby generating adversarial examples that increase the difficulty of model prediction. Once the gradient is calculated, it is then... A perturbation is applied to the word embedding matrix V. These perturbations, through minute adjustments to the embedding vectors, result in slight variations in the generated adversarial examples, but are sufficient to simulate irregularities or noise in the text. Unlike the traditional approach of directly perturbing the word embedding vectors, this method perturbs the entire embedding matrix, allowing the model to consider more dimensions of variation when generating adversarial examples.

[0059] For a pair of original samples and adversarial samples, the training objective is to make their sentence-level representations as similar as possible, thereby enabling the model to maintain strong robustness in the face of noise and non-standardized data. The model learns to distinguish subtle differences between original samples and adversarial samples while maintaining their consistency in the representation space. Furthermore, to further improve the model's generalization ability, samples from different sources should be spaced further apart in the representation space, making irrelevant samples as far apart as possible in the feature space. To effectively model this relationship of similarity and dissimilarity, this invention introduces contrastive learning as a regularization method. Through contrastive learning, the model is not only constrained by positive sample pairs during training but also further optimized by the effect of negative sample pairs.

[0060] The final hidden state given the [CLS] labels of the original sample The final hidden state of the [CLS] tag corresponding to the adversarial example. ,Will Treat them as a pair of positive examples, and then add a non-linear mapping layer to them:

[0061]

[0062] For a batch ofN The original samples and their corresponding adversarial samples, each pair of positive examples has 2(N−1) For negative examples, that is, all other samples in the batch are negative examples. The goal of contrastive learning is to identify positive examples by training the model to minimize the loss, making positive examples closer to the model and negative examples further away from it.

[0063] in, This represents the cosine similarity between two vectors.

[0064] Finally, a weighted average of the two classification losses and the contrastive loss is taken to obtain the final loss value:

[0065] S5. Input the tabular customs commodity declaration data to be classified into the trained model, and output the corresponding Harmonized System Code (HS Code) for the commodity to achieve automatic classification of customs commodities.

[0066] To verify the performance of the HScode-Former model in actual customs declaration scenarios, this invention selects typical commodity cases and compares and analyzes the model's prediction results with actual HS codes to further observe the model's performance.

[0067] like Figure 7 As shown, the product name "Portable Computer" directly indicates the product category, and the specifications and model include keywords such as "laptop," "CPU 2.9G," "Memory 4G," "Hard Drive 1128G," and "Lenovo," which fully describe the hardware configuration and brand information. The RoBERTa pre-trained model accurately captures technical parameters such as "CPU," "Hard Drive," and "Memory" and the "Lenovo" brand characteristics through context-aware embedding, strengthening the semantic association of the device as a core component of a computer.

[0068] The duty-paid price and the first legal quantity are non-linearly encoded using the PLE module. The unit price calculation (60019 / 18≈3334 yuan) is mapped to an appropriate numerical range. PLE uses a target-aware range division strategy to avoid misclassification as a server or accessory category, ensuring that the model accurately matches the reasonable price range of portable computers.

[0069] The first legal unit of measurement, "001" ("Taiwan"), is transformed into a fixed-dimensional semantic vector through an embedding layer, clearly defining the unit of commodity transaction. The encoded embedding of the country of origin and destination, "142" ("China"), provides the model with information on regional policy connections, assisting in judging tariff rules and commodity circulation characteristics. The model generates samples containing noise perturbations through adversarial training. The contrastive learning framework forces the model to bring the representations of the original samples and adversarial samples closer together in the feature space, while pushing away the embeddings of irrelevant commodities. This mechanism enables the model to focus on the core function of "laptop" even under the interference of the "Lenovo" brand, avoiding misjudgment as consumer electronics accessories due to textual noise.

[0070] In the aforementioned simulation application, the model successfully output the HS code "8471309000", which perfectly matched the actual code. This demonstrates that the multi-feature fusion strategy successfully integrated textual description, numerical range, and category information, improving the model's classification accuracy. Simultaneously, the contrastive learning strategy enhanced the model's robustness to complex declaration data and improved its ability to handle noisy and non-standardized data.

[0071] Based on a real customs commodity declaration, this technical solution was designed for experimentation, and its beneficial effects were analyzed.

[0072] In this invention, the method uses Customs Dataset 1 and Customs Dataset 2 to train and validate the model, respectively. Both customs datasets consist of real text of customs import and export commodity declarations, and are divided into training, validation, and test sets in an 8:1:1 ratio. Statistical data from the customs commodity datasets are shown in Table 1.

[0073] Table 1. Statistics of Customs Commodity Dataset

[0074] Dataset 1 contains 3,958,510 records, with numerical features including customs value and first legal quantity, categorical features including country of origin and country of origin, first legal unit of measurement, and text features including product name and specifications. Dataset 2 contains 6,632,490 records, with numerical features including customs value, first legal quantity, ad valorem tariff rate, ad valorem VAT rate, ad valorem consumption tax rate, and specific consumption tax rate, categorical features including country of origin and country of origin, first legal unit of measurement, and text features including product name and specifications. The experimental parameters are shown in Table 2. During training, the text embeddings were mapped to a 1024-dimensional vector space, and the maximum input text length was set to 32. The model training rate was 1e-4, the dropout rate was 0.1, the batch size was 128, and the training period was 20 epochs. The patience parameter for the early stopping strategy was 5. The AdamW optimizer was used to handle the weight decay problem, and the weight decay rate was set to 1e-5. For the RoBERTa model, the pre-trained model of HuggingFace was used, and the word embeddings were initialized with the model parameters recommended in the original paper. All models were implemented using PyTorch 2.2.1, and the experiments were conducted on an NVIDIA GTX 4070Ti GPU.

[0075] Table 2 Experimental Parameter Settings

[0076] Example 2: This embodiment provides a customs commodity classification system based on tabular multiple features, including: The data acquisition module acquires tabular customs commodity declaration data, which includes numerical features, category features, and text features. The multi-feature embedding module uses piecewise linear encoding (PLE) for numerical features, integer mapping combined with an embedding layer for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer for text features, to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. The feature fusion module stacks the embedding vectors of the three types of features into an embedding matrix, inputs it into the model based on the Transformer architecture, and mines the interaction relationship between features through the self-attention mechanism to complete the deep fusion of multiple features. The model training module generates adversarial examples and constructs positive and negative sample pairs, and trains the model by combining cross-entropy loss and contrastive loss. The commodity classification module takes the tabular customs commodity declaration data to be classified, inputs it into the trained model, and outputs the corresponding Harmonized System (HS) code for the commodity, thus realizing the automatic classification of customs commodities.

[0077] Example 3: An electronic device includes a memory, a processor, and a computer program stored in the memory and running thereon. When the processor executes the program, it implements the aforementioned customs commodity classification method based on multiple tabular features, comprising: Obtain tabular customs commodity declaration data, which includes numerical features, category features, and text features; Piecewise linear encoding (PLE) is used for numerical features, integer mapping combined with an embedding layer is used for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer is used for text features to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. The three types of feature embedding vectors are stacked into an embedding matrix, which is then input into a Transformer-based model. The interaction relationship between features is explored through a self-attention mechanism to achieve deep fusion of multiple features. Generate adversarial examples and construct positive and negative sample pairs, and train the model by combining cross-entropy loss and contrastive loss; The tabular customs commodity declaration data to be classified is input into the trained model, which outputs the corresponding Harmonized System (HS) code for the commodity, thereby realizing the automatic classification of customs commodities.

[0078] Example 4: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned customs commodity classification method based on tabular multiple features, comprising: Obtain tabular customs commodity declaration data, which includes numerical features, category features, and text features; Piecewise linear encoding (PLE) is used for numerical features, integer mapping combined with an embedding layer is used for categorical features, and RoBERTa pre-trained model combined with a linear transformation layer is used for text features to obtain embedded vectors of numerical, categorical, and text features of a unified dimension. The three types of feature embedding vectors are stacked into an embedding matrix, which is then input into a Transformer-based model. The interaction relationship between features is explored through a self-attention mechanism to achieve deep fusion of multiple features. Generate adversarial examples and construct positive and negative sample pairs, and train the model by combining cross-entropy loss and contrastive loss; The tabular customs commodity declaration data to be classified is input into the trained model, which outputs the corresponding Harmonized System (HS) code for the commodity, thereby realizing the automatic classification of customs commodities.

[0079] Those skilled in the art will understand that the modules or steps described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, which can then be stored in a storage device for execution by a computer device. Alternatively, they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. This disclosure is not limited to any particular combination of hardware and software.

[0080] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0081] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A method for customs commodity classification based on table multi-features, characterized in that, The method comprises the following steps: obtaining tabular customs commodity declaration data, the data comprising numerical features, category features and text features; processing the numerical features by piecewise linear encoding (PLE), the category features by integer mapping combined with an embedding layer, and the text features by a RoBERTa pre-training model combined with a linear transformation layer to obtain numerical, category and text feature embedding vectors of a unified dimension; stacking the three types of feature embedding vectors into an embedding matrix, inputting the embedding matrix into a model based on a Transformer architecture, mining the interaction relationship between the features through a self-attention mechanism, and completing deep fusion of the multiple features; generating adversarial samples and constructing positive and negative sample pairs, training the model by combining cross-entropy loss and contrastive loss; inputting tabular customs commodity declaration data to be classified into the trained model, and outputting the coordination system code corresponding to the commodities to realize automatic classification of the customs commodities.

2. The method of claim 1, wherein, The numerical features are processed by piecewise linear encoding (PLE), which divides the feature value domain based on the number of category labels, constructs a decision tree, and uses the leaf node region as the division interval; and the feature index order information is integrated into the encoding through a linear layer, specifically as follows: ； In the formula: is an input numerical characteristic value, is the total number of partition intervals, is a current interval index, is an upper boundary value of the tth interval, is an upper boundary value of the (t-1)th interval, is an activation weight of the tth interval; a linear layer is introduced to add order information to the encoding: ； wherein v t is a trainable embedding vector assigned to each interval B t , v 0 is a bias vector, and e t is used as a weight for the respective embedding vector; Linear(.) denotes a linear layer.

3. The method of claim 1, wherein the method further comprises: the category features are processed by integer mapping combined with an embedding layer, which maps each value of the category features to a unique integer, then one-hot encoding is performed to obtain a sparse binary vector, and finally an embedding layer is used to obtain: ； In the formula: is the encoded vector, is the weight matrix, is the category bias term.

4. The method of claim 1, wherein, The text features are preprocessed by text standardization, segmentation and length unification before embedding processing; after the RoBERTa pre-training model generates context-aware word embedding vectors, the dimensions are adjusted through a linear transformation layer: ； In the formula: represents the embedding vector of the jth text feature after processing, represents the original text The context-aware word embedding vector generated by the model for the text after inputting the text into the RoBERTa pre-training model; () represents a linear transformation layer, represents a text bias term.

5. The method of claim 1, wherein, the model based on the Transformer architecture is an HScode-Former model, which performs feature fusion on the embedding matrix through a Transformer layer to fully mine the interaction relationship between the numerical, category and text features.

6. The method of claim 1, wherein, The adversarial sample generation method is to obtain the gradient of the loss function with respect to the RoBERTa model word embedding matrix, and to apply perturbation to the word embedding matrix according to the following formula: ； In the formula: is a disturbance coefficient, () is a sign function, is a loss function gradient, is a model to be trained, is a sample HS coding label; the positive and negative sample pair construction rule is that the original sample and the corresponding adversarial sample are a positive sample pair, and the original sample and other random samples are negative sample pairs; the cross-entropy loss is: ； In the formula: is the number of samples, is the number of HS coding categories, is the sample category label, is the prediction probability, is the hidden state of the RoBERTa model [CLS] mark.

7. The method of claim 6, wherein the table-based multi-feature customs commodity classification method is characterized by, The contrast loss constrains the sample feature space distribution through a cosine similarity, and first performs nonlinear mapping on the [CLS] mark hidden state of the original sample and the adversarial sample, and then calculates the contrast loss based on the mapping result ; the final loss formula is: ； In the formula: is a weight coefficient, is a cross-entropy classification loss corresponding to the original sample V, is a cross-entropy classification loss corresponding to the adversarial sample.

8. A table multi-feature based customs commodity classification system, characterized in that, It comprises: a data acquisition module for acquiring tabular customs commodity declaration data, the data comprising numerical features, category features and text features; a multi-feature embedding module for processing the numerical features by piecewise linear encoding (PLE), the category features by integer mapping combined with an embedding layer, and the text features by a RoBERTa pre-training model combined with a linear transformation layer to obtain numerical, category and text feature embedding vectors of a unified dimension; a feature fusion module for stacking the three types of feature embedding vectors into an embedding matrix, inputting the embedding matrix into a model based on a Transformer architecture, mining the interaction relationship between the features through a self-attention mechanism, and completing deep fusion of the multiple features; a model training module for generating adversarial samples and constructing positive and negative sample pairs, training the model by combining cross-entropy loss and contrastive loss; The commodity classification module inputs the tabular customs commodity declaration data to be classified into the trained model, and outputs the corresponding coordination system code of the commodity, thereby realizing automatic classification of the customs commodity.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory to run on the processor, characterized in that, The processor implements the table multi-feature-based customs commodity classification method of any one of claims 1-7 when executing the program. 10.A computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the table multi-feature-based customs commodity classification method of any one of claims 1-7.

Images