Big data-based customs commodity classification method

Abstract

The invention discloses a customs commodity classification method based on big data. The method comprises the following specific steps: collecting various data of historical commodities; collecting commodities needing to be classified and related data thereof; carrying out centralized processing on the collected commodities and the related data thereof, and optimizing the commodities and the related data thereof; key features of the commodity data are extracted and selected, so that subsequent classification of commodities is facilitated; constructing a classification model based on the big data, and training the classification model; classifying and adjusting the commodities by using the constructed classification model; the classified commodity data are verified, and whether classification is reasonable or not is judged; and the commodity classification module is used for coding and generating the verified commodity classification data, and feeding back the result and process data of the current commodity classification to the big data acquisition module and the classification model training module. According to the customs commodity classification method based on the big data, massive commodity information can be fully utilized, and the defects of a traditional classification method are effectively overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of commodity classification, and in particular to a customs commodity classification method based on big data. Background Art

[0002] With technological advancements, the deep integration of global industrial chains, and the continuous emergence of new materials, technologies, and designs, the variety and complexity of commodities have exploded. These new commodities often possess diverse attributes and functions, making them difficult to accurately categorize using traditional classification methods. Traditional customs commodity classification relies primarily on the experience of customs brokers and manual consultation of complex customs code manuals. However, the HS Code Manual is complex and frequently updated, requiring customs brokers to devote considerable time and effort to learning and mastering it. Furthermore, manual operations can easily lead to classification errors due to subjective judgment errors, misunderstandings of commodities, or misinterpretation of coding rules. Furthermore, customs in different countries and regions may have varying understandings and classification standards for certain commodities, further complicating commodity classification. Such inaccurate classification can lead to revenue loss, distorted trade statistics, and delays in customs clearance, seriously disrupting the normal order and efficiency of international trade and increasing operational costs and trade risks for businesses. Consequently, new customs commodity classification methods have emerged on the market.

[0003] For example, Chinese patent publication number CN109559029B discloses a method and apparatus for intelligent customs commodity classification. The method includes obtaining historical enterprise declaration data to establish an enterprise declaration data model; obtaining customs commodity classification data to establish a customs commodity classification data model; searching for target classification data in both the enterprise declaration data model and the customs commodity classification data model based on commodity information provided by the user, wherein the target classification data is the target classification data that matches the commodity information in both the enterprise declaration data model and the customs commodity classification data model; and outputting the target classification data. This intelligent customs commodity classification method reduces the reliance on user experience in commodity classification, reduces manual errors, and improves work efficiency.

[0004] The aforementioned patent provides an intelligent customs commodity classification method that establishes a database model and searches and matches the commodity information provided by the user within the database model to obtain the customs code that best matches the user's information. This method automatically classifies the commodity according to the customs code, requiring only the user to provide the commodity information. This reduces the reliance on user experience, reduces manual errors, and improves work efficiency. However, existing intelligent customs commodity classification methods are unable to verify the classified commodities and are unable to readjust the classified commodities when anomalies occur. Summary of the Invention

[0005] The purpose of the present invention is to provide a customs commodity classification method based on big data, aiming to improve the problem that the existing customs commodity intelligent classification method cannot verify the classified commodities and cannot readjust the classified commodities when anomalies occur.

[0006] The present invention is achieved in that: A customs commodity classification method based on big data, the specific steps of the method are as follows: S100, collecting various data of historical commodities; S200: Collect commodities to be classified and their related data; S300: Centrally process the collected products and their related data to optimize the products and their related data; S400: Extract and select key features of product data to facilitate subsequent classification of products; S500: Build a classification model based on big data and train the classification model; S600: categorize and adjust the products using the constructed classification model; S700: Verify the classified commodity data to determine whether the classification is reasonable; S800: Encode and generate the verified commodity classification data. At the same time, feed back the results and process data of this commodity classification to the big data acquisition module and the classification model training module.

[0007] Preferably, in step S100, the specific steps of collecting various data of historical products are as follows: S110. Obtain historical declaration data from the customs declaration system, including commodity name, brand, model, specification, quantity, trading country, trade method, declaration code, and corresponding declaring company information; S120: Collect commodity transaction data from e-commerce platforms, which covers a wealth of commodity information and market trends; S130. Integrate the product data of the manufacturing enterprise, including product design documents, production processes, raw material lists, etc.

[0008] Preferably, in step S200, the specific steps for collecting commodity data that needs to be classified are as follows: S210. Use data crawler technology to collect product information from public e-commerce platforms and corporate official websites; S220. Establish data interfaces with the customs internal system and the enterprise's information management system to achieve real-time or periodic data collection through secure data transmission protocols.

[0009] Preferably, in step S300, the specific steps of optimizing the product data are as follows: S310: Deduplication of data by comparing key identifiers (such as product name, model, customs declaration code, etc.) in the data records to identify and delete duplicate customs declaration records or product information; S320: Process missing values. For product data that lacks key information, fill in the missing values ​​based on the data's distribution patterns and correlations. S330, correcting erroneous data, discovering and correcting errors in data through data validation rules and logical checks; S340, unified data format, converting similar types of information in data from different sources into a unified format; S350. Standardize text data and apply lexical analysis and part-of-speech tagging methods in natural language processing technology.

[0010] Preferably, in step S400, the specific steps of extracting product data features are as follows: S410: Extract keyword features from text descriptions of product information (such as product names, instructions, etc.); S420: Extracting numerical features, such as price, weight, size, and technical parameters of the product; S430: Extract classification features, including product categories (such as electronic products, textiles, and mechanical products), trading countries, and trade methods; S440. Use feature selection algorithms, such as information gain and chi-square test, to evaluate the importance of each feature to the product classification results; S450, using principal component analysis dimensionality reduction technology to project high-dimensional feature vectors into low-dimensional space while preserving the original data information as much as possible.

[0011] Preferably, in step S500, the specific steps of establishing and training the classification model are as follows: S510. Select an appropriate machine learning or deep learning model as the core model for product classification; S520: Use the pre-processed and feature-engineered commodity data to train the selected model. S530. Adjust the model's hyperparameters, such as the number and depth of decision trees (for random forests), learning rate, and batch size (for deep learning models), by evaluating the model's performance metrics (such as accuracy, recall, and F1 value) on the validation set.

[0012] Preferably, in step S600, the specific steps of classification adjustment are as follows: S610. Establish a customs commodity classification rule database, which includes the general rules stipulated by the International Customs Organization, special customs regulations of various countries, and detailed classification rules for specific commodities; S620: After the classification model outputs preliminary product classification results, the product features are matched with the rule base.

[0013] Preferably, in step S700, the specific steps of classification adjustment are as follows: S710, calculating the similarity between the commodity to be classified and the historical commodities that have been accurately classified; S720: Analyze the classification of these similar products. If the classification of most similar products is inconsistent with the model output, it is necessary to re-examine the product characteristics and the accuracy of the model, and the classification results may need to be adjusted.

[0014] Preferably, in step S800, the steps of code generation and output are as follows: S810. Generate a customs declaration code based on the final commodity classification result; S820. Organize the customs declaration code, relevant commodity information, classification basis, etc. into a standardized customs declaration document format.

[0015] Preferably, the method further includes S900, setting a manual review link, and manually intervening in the review when abnormalities occur in the classification, so as to avoid the problem of a large number of classification errors.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention utilizes a big data-based customs commodity classification method that fully utilizes vast amounts of commodity information, effectively overcoming the shortcomings of traditional classification methods. This method improves the accuracy and efficiency of customs commodity classification, ensuring the smooth conduct of international trade and the effective implementation of customs supervision. It can also effectively address complex and ever-changing commodity situations, improving the quality and efficiency of customs commodity classification and ensuring the smooth conduct of international trade.

[0017] 2. The present invention feeds back the results and process data of this commodity classification to the big data acquisition module and the training module of the classification model to increase the data source of the classification model, so that the method can provide more stable classification results based on the increasing data.

[0018] 3. The present invention sets up a manual review link, which facilitates manual intervention and review when classification anomalies occur, and facilitates timely and effective handling of classification anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the classification method of the present invention. DETAILED DESCRIPTION

[0020] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0021] The following is a further description with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 As shown in the figure, a customs commodity classification method based on big data is shown. The specific steps of the method are as follows: S100, collecting various data of historical products; the specific steps of collecting various data of historical products are as follows: S110. Obtain historical declaration data from the customs declaration system, including product name, brand, model, specifications, quantity, trading country, trade method, declaration code, and corresponding declaring company information; this data is a treasure trove of information directly generated in customs operations. For example, from a large number of historical declaration records, it is possible to obtain multiple declarations of a specific brand and model of smartphones, including the various declared parameters and corresponding classification codes.

[0022] S120. Collect commodity transaction data from e-commerce platforms. This data contains a wealth of product information and market trends. For commodities traded in large quantities on e-commerce channels, such as fashion apparel and household items, information such as product descriptions, prices, sales volume, and consumer reviews can be obtained. By analyzing this data, it is possible to understand the actual positioning and characteristics of a commodity in the market. For example, more detailed product information can be obtained from consumer reviews of the material and style of a particular garment.

[0023] S130. Integrate the manufacturer's product data, including product design documents, production processes, and raw material lists. This data reveals the essential attributes of the product from the source. For example, for an electronic product, key information such as the chip model, circuit design, and functional module composition can be obtained from the manufacturer's design documents, helping to more accurately understand the product's characteristics.

[0024] S200: Collect the commodities to be classified and their related data. The specific steps for collecting the commodity data to be classified are as follows: S210. Use data crawling technology to collect product information from public e-commerce platforms and official corporate websites. For e-commerce platforms, set appropriate crawling rules to traverse product pages based on product categories, search keywords, and other methods to extract the required data. Simultaneously, use appropriate parsing technologies, such as HTML parsing and JSON parsing, tailored to the page structure and data storage methods of different websites to extract and organize the data into a unified format.

[0025] S220. Establish data interfaces with customs internal systems and enterprise information management systems to enable real-time or periodic data collection through secure data transmission protocols. During the collection process, use data encryption technology to ensure data security and integrity, preventing data leakage and tampering.

[0026] S300: Centrally process the collected products and their related data to optimize the products and their related data. The specific steps for optimizing product data are as follows: S310: Remove duplicate data by comparing key identifiers (such as product name, model, customs declaration code, etc.) in the data records to identify and delete duplicate customs declaration records or product information; for example, for multiple duplicate customs declaration data for the same product by the same enterprise within a short period of time, only one complete record will be retained.

[0027] S320: Process missing values. For commodity data missing key information, fill in missing values ​​based on the data's distribution patterns and correlations. For example, if a product's specifications are missing but complete specifications are available in other customs declarations for the same brand and model, this missing value information can be used as a reference. Data with severe missing values ​​that cannot be filled in will be marked or temporarily excluded to avoid adverse effects on subsequent analysis.

[0028] S330. Correct erroneous data. Use data validation rules and logical checks to discover and correct errors in the data. For example, check whether the price of goods is within a reasonable market range. If there is obviously abnormal price data (such as prices that are too low or too high, which may be an entry error), adjust it based on the average price of similar goods or verify with relevant companies.

[0029] S340. Unify data formats and convert similar types of information in data from different sources into a unified format; for example, standardize the date format to "AAA-BB-CC", standardize uppercase and lowercase letters, abbreviations, etc. in product names, and convert different weight units into international standard units such as kilograms.

[0030] S350: Standardize the text data using lexical analysis and part-of-speech tagging techniques from natural language processing. For example, standardize synonyms in product descriptions into standard terms, such as "computer" and "computer" into "computer," and "mobile phone" and "mobile phone" into "mobile phone," to facilitate subsequent data analysis and matching.

[0031] S400: Extract and select key features of product data to facilitate subsequent classification of products. The specific steps for extracting product data features are as follows: S410. Extract keyword features from product information with textual descriptions (such as product names and instructions). Using a bag-of-words model or TF-IDF (term frequency-inverse document frequency) algorithm, convert the text into vectors, highlighting keywords that are important for product classification. For example, for a smartwatch, extract keywords such as "smart device," "wearable," "time display," and "health monitoring function" as features.

[0032] S420. Extract numerical features, such as the product's price, weight, dimensions, and technical parameters. Statistical features such as price range and price dispersion can be further calculated. For technical parameter features, such as chip frequency and memory capacity of electronic products, analyze their relevance to product classification and perform appropriate normalization to keep them within the same magnitude range for ease of subsequent model processing.

[0033] S430. Extract categorical features, including product category (e.g., electronics, textiles, machinery), trade country, and trade method. Perform one-hot encoding (IHEN) on these categorical features, converting them into computer-processable binary vectors. For example, if there are five product categories and a product belongs to category 3, its IHEN vector is [0, 0, 1, 0, 0].

[0034] S440. Use feature selection algorithms, such as those based on information gain and chi-square tests, to assess the importance of each feature to the product classification results. Select features that have a significant impact on classification and remove redundant or weakly relevant features to reduce data dimensionality and model complexity. For example, information gain analysis may reveal that for a certain product category, certain technical parameters contribute little to the classification and can be removed.

[0035] S450 uses principal component analysis (PCA) to reduce dimensions, projecting high-dimensional feature vectors into a low-dimensional space while preserving as much information as possible from the original data. PCA combines multiple related features into a small number of principal components that represent the main trends in the original data, improving model training and prediction efficiency.

[0036] S500: Build a classification model based on big data and train the classification model. The specific steps for building and training the classification model are as follows: S510. Select an appropriate machine learning or deep learning model as the core model for commodity classification. Consider using ensemble learning models, such as random forests and gradient boosted tree models (GBDT). These models improve classification accuracy and stability by combining multiple weak classifiers. For example, a random forest model consists of multiple decision trees, each of which votes on a commodity classification, with the final classification result determined by majority vote. Examples include convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs). CNNs are suitable for processing data with grid structures, such as product images (if available) or two-dimensional feature matrices from processed text. MLPs can process input feature vectors of any form, classifying commodities through layer-by-layer transmission and activation of multiple neurons. In model design, determine the number of nodes in the input layer based on the number and type of commodity features, and the number of nodes in the output layer based on the number of customs commodity classification categories. Design an appropriate hidden layer structure and number of neurons.

[0037] S520: Train the selected model using preprocessed and feature-engineered product data. Split the dataset into training, validation, and test sets in a ratio of 70:20:10. During training, use an appropriate loss function and optimization algorithm. For classification models, a common loss function is the cross-entropy loss function, and the optimization algorithm can be stochastic gradient descent (SGD) or its variants (such as Adam and Adagrad).

[0038] S530: Adjust model hyperparameters, such as the number and depth of decision trees (for random forests), learning rate, and batch size (for deep learning models), by evaluating model performance metrics (such as accuracy, recall, and F1 score) on the validation set. Continue optimizing the model until satisfactory performance is achieved on the validation set. Then, evaluate the final model on the test set to ensure good generalization.

[0039] S600: Use the constructed classification model to adjust the classification of the products. The specific steps of the classification adjustment are as follows: S610. Establish a customs commodity classification rule library, which includes general rules stipulated by the International Customs Organization, special customs regulations of various countries, and detailed classification rules for specific commodities. These rules are expressed in the form of logical expressions, decision trees, or production rules. For example, for commodities containing precious metals, there are rules to determine classification based on the proportion of precious metal content.

[0040] S620: After the classification model outputs preliminary product classification results, the product features are matched against the rule library. If there are rule conflicts or the model classification does not conform to the rules, the classification results are adjusted according to the rules. For example, if a product meets the conditions of multiple rules simultaneously, adjustments are made based on the priority of the rules.

[0041] S700: Verify the classified commodity data to determine whether the classification is reasonable. The specific steps for classification adjustment are as follows: S710. Calculate the similarity between the product to be classified and historical products that have been accurately classified. Various similarity calculation methods can be used, such as cosine similarity, Euclidean distance, or text similarity based on edit distance (for text features such as product names). Find a group of historical products that are highly similar to the product to be classified (e.g., the top 10 products in terms of similarity).

[0042] S720: Analyze the classification of these similar products. If the classification of most similar products is inconsistent with the model output, it is necessary to re-examine the product characteristics and the accuracy of the model, and may need to adjust the classification results. At the same time, by analyzing the differences in the classification of similar products, it is possible to identify possible ambiguous or critical information in the product characteristics, and further optimize the model and classification method.

[0043] S800: Encode the verified commodity classification data. Simultaneously, the results and process data of this commodity classification are fed back to the big data acquisition module and the classification model training module. In the big data acquisition module, historical customs declaration data and other relevant data can be updated to continuously enrich and improve data resources. In the model training module, new classification cases are used as new training data to retrain the model, enabling it to adapt to new commodity types and market changes, and continuously improving the accuracy and efficiency of commodity classification. The steps for code production and output are as follows: S810. Generate a customs declaration code based on the final product classification. The declaration code is output according to the internationally accepted customs coding system format, including chapter, heading, and subheading codes. For example, for a smartphone with a confirmed classification, the corresponding declaration code is output, such as "8517.12.10."

[0044] S820: Organize the customs declaration code, related commodity information, classification basis, and other contents into a standardized customs declaration document format. These documents can serve as the basis for customs clearance, enterprise declaration, and subsequent audits, ensuring the accuracy and completeness of commodity classification information.

[0045] Example 2 like Figure 1 As shown in the figure, a customs commodity classification method based on big data is shown. The specific steps of the method are as follows: S100, collecting various data of historical products; the specific steps of collecting various data of historical products are as follows: S110. Obtain historical declaration data from the customs declaration system, including product name, brand, model, specifications, quantity, trading country, trade method, declaration code, and corresponding declaring company information; this data is a treasure trove of information directly generated in customs operations. For example, from a large number of historical declaration records, it is possible to obtain multiple declarations of a specific brand and model of smartphones, including the various declared parameters and corresponding classification codes.

[0046] S120. Collect commodity transaction data from e-commerce platforms. This data contains a wealth of product information and market trends. For commodities traded in large quantities on e-commerce channels, such as fashion apparel and household items, information such as product descriptions, prices, sales volume, and consumer reviews can be obtained. By analyzing this data, it is possible to understand the actual positioning and characteristics of a commodity in the market. For example, more detailed product information can be obtained from consumer reviews of the material and style of a particular garment.

[0047] S130. Integrate the manufacturer's product data, including product design documents, production processes, and raw material lists. This data reveals the essential attributes of the product from the source. For example, for an electronic product, key information such as the chip model, circuit design, and functional module composition can be obtained from the manufacturer's design documents, helping to more accurately understand the product's characteristics.

[0048] S200: Collect the commodities to be classified and their related data. The specific steps for collecting the commodity data to be classified are as follows: S210. Use data crawling technology to collect product information from public e-commerce platforms and official corporate websites. For e-commerce platforms, set appropriate crawling rules to traverse product pages based on product categories, search keywords, and other methods to extract the required data. Simultaneously, use appropriate parsing technologies, such as HTML parsing and JSON parsing, tailored to the page structure and data storage methods of different websites to extract and organize the data into a unified format.

[0049] S220. Establish data interfaces with customs internal systems and enterprise information management systems to enable real-time or periodic data collection through secure data transmission protocols. During the collection process, use data encryption technology to ensure data security and integrity, preventing data leakage and tampering.

[0050] S300: Centrally process the collected products and their related data to optimize the products and their related data. The specific steps for optimizing product data are as follows: S310: Remove duplicate data by comparing key identifiers (such as product name, model, customs declaration code, etc.) in the data records to identify and delete duplicate customs declaration records or product information; for example, for multiple duplicate customs declaration data for the same product by the same enterprise within a short period of time, only one complete record will be retained.

[0051] S320: Process missing values. For commodity data missing key information, fill in missing values ​​based on the data's distribution patterns and correlations. For example, if a product's specifications are missing but complete specifications are available in other customs declarations for the same brand and model, this missing value information can be used as a reference. Data with severe missing values ​​that cannot be filled in will be marked or temporarily excluded to avoid adverse effects on subsequent analysis.

[0052] S330. Correct erroneous data. Use data validation rules and logical checks to discover and correct errors in the data. For example, check whether the price of goods is within a reasonable market range. If there is obviously abnormal price data (such as prices that are too low or too high, which may be an entry error), adjust it based on the average price of similar goods or verify with relevant companies.

[0053] S340. Unify data formats and convert similar types of information in data from different sources into a unified format; for example, standardize the date format to "AAA-BB-CC", standardize uppercase and lowercase letters, abbreviations, etc. in product names, and convert different weight units into international standard units such as kilograms.

[0054] S350: Standardize the text data using lexical analysis and part-of-speech tagging techniques from natural language processing. For example, standardize synonyms in product descriptions into standard terms, such as "computer" and "computer" into "computer," and "mobile phone" and "mobile phone" into "mobile phone," to facilitate subsequent data analysis and matching.

[0055] S400: Extract and select key features of product data to facilitate subsequent classification of products. The specific steps for extracting product data features are as follows: S410. Extract keyword features from product information with textual descriptions (such as product names and instructions). Using a bag-of-words model or TF-IDF (term frequency-inverse document frequency) algorithm, convert the text into vectors, highlighting keywords that are important for product classification. For example, for a smartwatch, extract keywords such as "smart device," "wearable," "time display," and "health monitoring function" as features.

[0056] S420. Extract numerical features, such as the product's price, weight, dimensions, and technical parameters. Statistical features such as price range and price dispersion can be further calculated. For technical parameter features, such as chip frequency and memory capacity of electronic products, analyze their relevance to product classification and perform appropriate normalization to keep them within the same magnitude range for ease of subsequent model processing.

[0057] S430. Extract categorical features, including product category (e.g., electronics, textiles, machinery), trade country, and trade method. Perform one-hot encoding (IHEN) on these categorical features, converting them into computer-processable binary vectors. For example, if there are five product categories and a product belongs to category 3, its IHEN vector is [0, 0, 1, 0, 0].

[0058] S440. Use feature selection algorithms, such as those based on information gain and chi-square tests, to assess the importance of each feature to the product classification results. Select features that have a significant impact on classification and remove redundant or weakly relevant features to reduce data dimensionality and model complexity. For example, information gain analysis may reveal that for a certain product category, certain technical parameters contribute little to the classification and can be removed.

[0059] S450 uses principal component analysis (PCA) to reduce dimensions, projecting high-dimensional feature vectors into a low-dimensional space while preserving as much information as possible from the original data. PCA combines multiple related features into a small number of principal components that represent the main trends in the original data, improving model training and prediction efficiency.

[0060] S500: Build a classification model based on big data and train the classification model. The specific steps for building and training the classification model are as follows: S510. Select an appropriate machine learning or deep learning model as the core model for commodity classification. Consider using ensemble learning models, such as random forests and gradient boosted tree models (GBDT). These models improve classification accuracy and stability by combining multiple weak classifiers. For example, a random forest model consists of multiple decision trees, each of which votes on a commodity classification, with the final classification result determined by majority vote. Examples include convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs). CNNs are suitable for processing data with grid structures, such as product images (if available) or two-dimensional feature matrices from processed text. MLPs can process input feature vectors of any form, classifying commodities through layer-by-layer transmission and activation of multiple neurons. In model design, determine the number of nodes in the input layer based on the number and type of commodity features, and the number of nodes in the output layer based on the number of customs commodity classification categories. Design an appropriate hidden layer structure and number of neurons.

[0061] S520: Train the selected model using preprocessed and feature-engineered product data. Split the dataset into training, validation, and test sets in a ratio of 70:20:10. During training, use an appropriate loss function and optimization algorithm. For classification models, a common loss function is the cross-entropy loss function, and the optimization algorithm can be stochastic gradient descent (SGD) or its variants (such as Adam and Adagrad).

[0062] S530: Adjust model hyperparameters, such as the number and depth of decision trees (for random forests), learning rate, and batch size (for deep learning models), by evaluating model performance metrics (such as accuracy, recall, and F1 score) on the validation set. Continue optimizing the model until satisfactory performance is achieved on the validation set. Then, evaluate the final model on the test set to ensure good generalization.

[0063] S600: Use the constructed classification model to adjust the classification of the products. The specific steps of the classification adjustment are as follows: S610. Establish a customs commodity classification rule library, which includes general rules stipulated by the International Customs Organization, special customs regulations of various countries, and detailed classification rules for specific commodities. These rules are expressed in the form of logical expressions, decision trees, or production rules. For example, for commodities containing precious metals, there are rules to determine classification based on the proportion of precious metal content.

[0064] S620: After the classification model outputs preliminary product classification results, the product features are matched against the rule library. If there are rule conflicts or the model classification does not conform to the rules, the classification results are adjusted according to the rules. For example, if a product meets the conditions of multiple rules simultaneously, adjustments are made based on the priority of the rules.

[0065] S700: Verify the classified commodity data to determine whether the classification is reasonable. The specific steps for classification adjustment are as follows: S710. Calculate the similarity between the product to be classified and historical products that have been accurately classified. Various similarity calculation methods can be used, such as cosine similarity, Euclidean distance, or text similarity based on edit distance (for text features such as product names). Find a group of historical products that are highly similar to the product to be classified (e.g., the top 10 products in terms of similarity).

[0066] S720: Analyze the classification of these similar products. If the classification of most similar products is inconsistent with the model output, it is necessary to re-examine the product characteristics and the accuracy of the model, and may need to adjust the classification results. At the same time, by analyzing the differences in the classification of similar products, it is possible to identify possible ambiguous or critical information in the product characteristics, and further optimize the model and classification method.

[0067] S800: Encode the verified commodity classification data. Simultaneously, the results and process data of this commodity classification are fed back to the big data acquisition module and the classification model training module. In the big data acquisition module, historical customs declaration data and other relevant data can be updated to continuously enrich and improve data resources. In the model training module, new classification cases are used as new training data to retrain the model, enabling it to adapt to new commodity types and market changes, and continuously improving the accuracy and efficiency of commodity classification. The steps for code production and output are as follows: S810. Generate a customs declaration code based on the final product classification. The declaration code is output according to the internationally accepted customs coding system format, including chapter, heading, and subheading codes. For example, for a smartphone with a confirmed classification, the corresponding declaration code is output, such as "8517.12.10."

[0068] S820: Organize the customs declaration code, related commodity information, classification basis, and other contents into a standardized customs declaration document format. These documents can serve as the basis for customs clearance, enterprise declaration, and subsequent audits, ensuring the accuracy and completeness of commodity classification information.

[0069] S900. A manual review process is implemented to manually intervene and review any classification anomalies to avoid widespread misclassification. Experts with extensive experience in customs commodity classification review the model's classification results. Based on their expertise and in-depth understanding of the commodities, these experts can adjust the model's classifications to identify potential errors or uncertainties. Furthermore, these manually reviewed cases are fed back into the model training phase as special case studies, improving the model's ability to handle similar complex situations.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A customs commodity classification method based on big data, characterized by: The specific steps of this method are as follows: S100, collecting various data of historical commodities; S200: Collect commodities to be classified and their related data; S300: Centrally process the collected products and their related data to optimize the products and their related data; S400: Extract and select key features of product data to facilitate subsequent classification of products; S500: Build a classification model based on big data and train the classification model; S600: categorize and adjust the products using the constructed classification model; S700: Verify the classified commodity data to determine whether the classification is reasonable; S800: Encode and generate the verified commodity classification data. At the same time, feed back the results and process data of this commodity classification to the big data acquisition module and the classification model training module.

2. A customs commodity classification method based on big data according to claim 1, characterized in that: In step S100, the specific steps for collecting various data of historical products are as follows: S110. Obtain historical declaration data from the customs declaration system, including commodity name, brand, model, specification, quantity, trading country, trade method, declaration code, and corresponding declaring company information; S120: Collect commodity transaction data from e-commerce platforms, which covers a wealth of commodity information and market trends; S130. Integrate the product data of the manufacturing enterprise, including product design documents, production processes, raw material lists, etc.

3. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S200, the specific steps for collecting commodity data that needs to be classified are as follows: S210. Use data crawler technology to collect product information from public e-commerce platforms and corporate official websites; S220. Establish data interfaces with the customs internal system and the enterprise's information management system to achieve real-time or periodic data collection through secure data transmission protocols.

4. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S300, the specific steps for optimizing product data are as follows: S310: Deduplication of data by comparing key identifiers (such as product name, model, customs declaration code, etc.) in the data records to identify and delete duplicate customs declaration records or product information; S320: Process missing values. For product data that lacks key information, fill in the missing values ​​based on the data's distribution patterns and correlations. S330, correcting erroneous data, discovering and correcting errors in data through data validation rules and logical checks; S340, unified data format, converting similar types of information in data from different sources into a unified format; S350. Standardize text data and apply lexical analysis and part-of-speech tagging methods in natural language processing technology.

5. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S400, the specific steps of extracting product data features are as follows: S410: Extract keyword features from text descriptions of product information (such as product names, instructions, etc.); S420: Extracting numerical features, such as price, weight, size, and technical parameters of the product; S430: Extract classification features, including product categories (such as electronic products, textiles, and mechanical products), trading countries, and trade methods; S440. Use feature selection algorithms, such as information gain and chi-square test, to evaluate the importance of each feature to the product classification results; S450, using principal component analysis dimensionality reduction technology to project high-dimensional feature vectors into low-dimensional space while preserving the original data information as much as possible.

6. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S500, the specific steps of establishing and training the classification model are as follows: S510. Select an appropriate machine learning or deep learning model as the core model for product classification; S520: Use the pre-processed and feature-engineered commodity data to train the selected model. S530. Adjust the model's hyperparameters, such as the number and depth of decision trees (for random forests), learning rate, and batch size (for deep learning models), by evaluating the model's performance metrics (such as accuracy, recall, and F1 value) on the validation set.

7. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S600, the specific steps of classification adjustment are as follows: S610. Establish a customs commodity classification rule database, which includes the general rules stipulated by the International Customs Organization, special customs regulations of various countries, and detailed classification rules for specific commodities; S620: After the classification model outputs preliminary product classification results, the product features are matched with the rule base.

8. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S700, the specific steps of classification adjustment are as follows: S710, calculating the similarity between the commodity to be classified and the historical commodities that have been accurately classified; S720: Analyze the classification of these similar products. If the classification of most similar products is inconsistent with the model output, it is necessary to re-examine the product characteristics and the accuracy of the model, and the classification results may need to be adjusted.

9. The method for customs commodity classification based on big data according to claim 1, characterized in that: In step S800, the steps of code generation and output are as follows: S810. Generate a customs declaration code based on the final commodity classification result; S820. Organize the customs declaration code, relevant commodity information, classification basis, etc. into a standardized customs declaration document format.

10. A customs commodity classification method based on big data according to any one of claims 1 to 9, characterized in that: It also includes S900 and the setting of manual review process. When there are abnormalities in classification, manual intervention and review will be carried out to avoid the problem of large-scale classification errors.

Images