Cargo type classification method, device and equipment, storage medium and product

Abstract

The invention discloses a cargo type classification method and device, equipment, a storage medium and a product, and relates to the technical field of cargo classification, and the method comprises the steps: obtaining a to-be-classified cargo export credit certificate text; carrying out feature extraction on the to-be-classified cargo export credit card text; and inputting the to-be-classified cargo export credit document text subjected to feature extraction into a pre-trained XLNet model to obtain a corresponding cargo type classification result. According to the technical scheme, the natural language text can be better processed, key information and features are extracted, manual complex text analysis is not needed, the classification accuracy is improved, the classification efficiency is improved, and the labor cost is reduced; the problems that traditional manual examination is low in efficiency and prone to making mistakes, and a traditional machine learning method is difficult to accurately capture semantic information and contextual relations, so that the classification effect is poor are solved.

Description

Technical Field

[0001] The present application relates to the technical field of cargo classification, and in particular to a cargo type classification method, device, equipment, storage medium and product. Background Art

[0002] In recent years, the classification of export letter of credit goods types mainly relies on manual review, and some traditional machine learning methods are also used. Among them:

[0003] For manual review, staff are required to analyze the terms and descriptions in the export letter of credit one by one to determine the type of goods, which consumes a lot of time and energy, is inefficient and prone to errors.

[0004] For traditional machine learning methods, some basic feature engineering techniques are usually used to process text. They often rely on artificially designed features and are difficult to adapt to complex and changeable natural language texts. They have limited ability to capture semantic information and contextual relationships, and the classification effect is poor.

[0005] In addition, traditional data preprocessing mainly relies on manual or semi-automatic methods for data cleaning, word segmentation, and feature extraction, which has low efficiency and accuracy.

[0006] Therefore, it is necessary to propose a technical solution to solve the problems of low efficiency and error-proneness of traditional manual review and poor classification effect caused by the difficulty of traditional machine learning methods to accurately capture semantic information and contextual relationships. Summary of the invention

[0007] The main purpose of this application is to provide a cargo type classification method, device, equipment, storage medium and product, aiming to solve the problems of low efficiency and error-proneness of traditional manual review and poor classification effect caused by the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships.

[0008] To achieve the above objectives, the present application proposes a cargo type classification method, the method comprising:

[0009] Obtain the text of the export letter of credit for goods to be classified;

[0010] Extracting features from the export letter of credit text to be classified;

[0011] The export letter of credit text of goods to be classified after feature extraction is input into the pre-trained XLNet model to obtain the corresponding classification results of goods types.

[0012] In one embodiment, before the step of inputting the feature-extracted export letter of credit text to be classified into a pre-trained XLNet model to obtain the corresponding goods type classification result, the step further includes:

[0013] The XLNet model is trained using a large language model.

[0014] In one embodiment, the step of using a large language model to train an XLNet model includes:

[0015] Collecting export letter of credit text data from data sources, the export letter of credit text data covering letter of credit terms, descriptions and / or related documents for different types of goods;

[0016] Preprocessing the export letter of credit text data by using a pre-trained large language model to obtain preprocessed export letter of credit text data;

[0017] The Word2Vec model is used to extract features from the preprocessed export letter of credit text data to obtain feature word vectors;

[0018] Taking the feature word vector as a data set, obtaining a corresponding training set, a validation set and a test set based on the data set;

[0019] Model training and testing are performed based on the training set, validation set and test set to obtain a trained XLNet model.

[0020] In one embodiment, the step of performing model training and testing based on the training set, validation set and test set to obtain a trained XLNet model comprises:

[0021] Instantiate XLNet model parameters, wherein the XLNet model parameters include at least hidden layer size, number of layers, and number of heads;

[0022] Convert the text data in the training set into corresponding word vectors;

[0023] Performing a model training cycle based on the word vectors, wherein in each training batch, the word vectors are input into the XLNet model to calculate the loss function, the gradient is calculated by the back propagation algorithm, and the model parameters are updated using the Adam optimizer based on the calculated loss function and gradient;

[0024] Regularly evaluate the performance of the XLNet model on the validation set;

[0025] According to the performance evaluation results of the validation set, the parameters of the XLNet model are adjusted and optimized to obtain a trained XLNet model.

[0026] In one embodiment, the step of adjusting and optimizing parameters of the XLNet model according to the performance evaluation result of the validation set includes:

[0027] Adjusting the hidden layer size, number of layers and / or number of heads of the XLNet model based on the performance evaluation results of the validation set; and / or

[0028] If the trained XLNet model is overfitting, add a regularization term to the loss function, or, if the trained XLNet model is underfitting, increase the amount of training data.

[0029] In one embodiment, the step of preprocessing the export letter of credit text data using a pre-trained large language model comprises:

[0030] De-noising the export letter of credit text data using a pre-trained large language model and preset prompt words;

[0031] Inputting the denoised export letter of credit text data and preset prompt words into the large language model for word segmentation processing;

[0032] Each word obtained by the word segmentation process is input into the large language model for part-of-speech tagging and entity recognition to obtain pre-processed export letter of credit text data.

[0033] In one embodiment, the method further comprises:

[0034] Evaluate the classification results of the cargo type using the preset performance indicators;

[0035] The XLNet model is adjusted and optimized according to the evaluation results.

[0036] The present application also proposes a cargo type classification device, the device comprising:

[0037] The acquisition module is used to obtain the export letter of credit text of goods to be classified;

[0038] A feature extraction module, used for extracting features from the export letter of credit text to be classified;

[0039] The classification module is used to input the export letter of credit text of goods to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding goods type classification result.

[0040] The present application also proposes a cargo type classification device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the cargo type classification method described above.

[0041] The present application also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the cargo type classification method as described above are implemented.

[0042] The present application also proposes a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the cargo type classification method described above are implemented.

[0043] One or more technical solutions proposed in this application have at least the following technical effects:

[0044] A method, device, equipment, storage medium and product for classifying goods types are proposed, which are as follows: obtaining the text of the export letter of credit for goods to be classified; extracting features from the text of the export letter of credit for goods to be classified; and inputting the text of the export letter of credit for goods to be classified after feature extraction into a pre-trained XLNet model to obtain the corresponding goods type classification result. Since the technical solution of this application adopts the XLNet model, the powerful language understanding ability and highly automated classification process of the XLNet model can better process natural language texts, extract key information and features, and eliminate the need for manual complex text analysis, thereby improving classification accuracy, improving classification efficiency and reducing labor costs. It solves the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results.

[0045] In addition, this application uses the large model + prompt word method to preprocess data, eliminating the tedious process of traditional data preprocessing. At the same time, the large model can automatically learn the features and semantic information in the text without human participation, further reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0048] Figure 1 A flow chart of the first embodiment of the method for classifying goods by type provided in this application;

[0049] Figure 2 A flow chart of the second embodiment of the method for classifying goods by type provided in this application;

[0050] Figure 3 This is a schematic diagram of the process of removing special symbols from a large model+prompt word in an embodiment of the present application;

[0051] Figure 4 This is a schematic diagram of the process of word segmentation using a large model + prompt words in an embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of the process of entity recognition using a large model + prompt words in an embodiment of the present application;

[0053] Figure 6 This is a schematic diagram of the specific process of training the XLNet model in the embodiment of the present application;

[0054] Figure 7 A flowchart of the third embodiment of the method for classifying goods by type provided in this application;

[0055] Figure 8 A complete flow chart of the cargo classification method provided in the embodiment of the present application;

[0056] Fig. 9 This is a schematic diagram of the module structure of the cargo type classification device according to an embodiment of the present application;

[0057] Fig.10 Schematic diagram of the device structure of the hardware operating environment involved in the cargo type classification method in the embodiment of the present application.

[0058] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0060] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0061] The main solution of the embodiment of the present application is: by obtaining the text of the export letter of credit for goods to be classified; extracting features from the text of the export letter of credit for goods to be classified; inputting the text of the export letter of credit for goods to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding classification result of the goods type. Since the technical solution of the present application adopts the XLNet model, the powerful language understanding ability and highly automated classification process of the XLNet model can better process natural language texts, extract key information and features, and do not require manual complex text analysis, thereby improving classification accuracy and efficiency and reducing labor costs, solving the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results. In addition, the present application uses the method of large model + prompt words to preprocess data, eliminating the cumbersome process of traditional data preprocessing. At the same time, the large model can automatically learn the features and semantic information in the text without manual participation, further reducing labor costs.

[0062] This application takes into account that in the prior art, in the field of export letter of credit goods type classification, it mainly relies on manual review, which is inefficient and prone to errors. Some of them also use traditional machine learning methods, but this method has limited performance when processing complex natural language texts, and it is difficult to accurately capture semantic information and contextual relationships. Specifically, manual review is time-consuming and labor-intensive, accuracy is difficult to guarantee, and costs continue to rise as business volume increases. For example, when the business volume increases, more manpower costs need to be invested, and manual review is prone to fatigue and negligence, resulting in an increase in error rate. However, the traditional machine learning method has insufficient feature extraction capabilities and poor understanding and classification effects on complex texts. Traditional machine learning methods often rely on manually designed features, are difficult to adapt to complex and changeable natural language texts, and have limited ability to capture semantic information and contextual relationships. In addition, the prior art lacks unified and efficient classification standards and methods, and different personnel may have different judgments on the types of goods in the same batch of export letters of credit, which affects the smooth progress of the trade process; traditional data preprocessing requires manual participation, and the process is cumbersome.

[0063] Therefore, the embodiment of the present application provides a solution, which uses the powerful language understanding ability and highly automated classification process of the XLNet model to better process natural language texts, extract key information and features, and eliminate the need for manual complex text analysis, thereby improving classification accuracy, improving classification efficiency, and reducing labor costs. It solves the problems of low efficiency and error-proneness of traditional manual review, as well as the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results.

[0064] In addition, this application uses the large model + prompt word method to preprocess data, eliminating the tedious process of traditional data preprocessing. At the same time, the large model can automatically learn the features and semantic information in the text without human participation, further reducing labor costs.

[0065] Technical terms involved in the embodiments of this application:

[0066] XLNet model: is a pre-trained model in the field of natural language processing (NLP), which aims to combine the advantages of autoregressive (AR) and autoencoder (AE) pre-trained language models and address some limitations of existing models. The core features of XLNet include:

[0067] Permutation Language Modeling: XLNet does not model sequences in the traditional order, but maximizes the expected log-likelihood of all possible sequence permutation combinations. This means that XLNet can consider both the order and reverse order of the text and understand the language more comprehensively.

[0068] Two-Stream Self-Attention: To solve the problem that the model in the traditional self-attention mechanism cannot perceive the target position, XLNet adopts a two-stream self-attention mechanism. This design enables XLNet to process sequential and reverse information flows through two independent self-attention sub-networks, respectively, so as to better understand and generate the complex structure of language.

[0069] Combined with Transformer-XL: XLNet also incorporates the idea of ​​Transformer-XL and enhances the model's ability to understand long-distance dependencies by introducing piecewise linear position encoding and relative position encoding.

[0070] The advantages of XLNet are:

[0071] Bidirectional contextual learning: XLNet can predict the next word given the previous context, and can also predict the previous word given the following context. This bidirectional learning mechanism enables XLNet to better understand contextual information.

[0072] Overcoming BERT’s shortcomings: XLNet is able to handle certain tasks better through its autoregressive nature, such as predicting pronouns in a sentence without being affected by previously appeared words.

[0073] Superior performance: In multiple public benchmarks, XLNet has outperformed models such as BERT, achieving the current best results in benchmarks such as GLUE, SQuAD, and RACE.

[0074] XLNet is widely used in many fields such as question answering systems, text generation, sentiment analysis, information extraction, and machine translation. Due to its strong context understanding ability, XLNet has shown excellent performance in these fields.

[0075] In summary, XLNet is an innovative pre-training model that significantly improves the performance of natural language processing tasks by combining the advantages of autoregression and autoencoding and adopting a permutation language model and a two-stream self-attention mechanism.

[0076] Based on this, the present application embodiment provides a method for classifying cargo types, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the cargo classification method of the present application.

[0077] In this embodiment, the cargo type classification method includes steps S10 to S30.

[0078] Each step is explained in detail below.

[0079] like Figure 1 As shown, the first embodiment of the present application proposes a method for classifying cargo types, the method comprising:

[0080] Step S10, obtaining the export letter of credit text of goods to be classified;

[0081] The execution subject of this embodiment may be a computing service device or equipment with data processing, network communication and program running functions, such as a device, terminal, server, etc. for realizing cargo type classification, a terminal such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, etc. The following takes a cargo type classification device as an example to illustrate the scheme of this embodiment.

[0082] The solution of this embodiment is mainly to identify the text of the export letter of credit for goods, and then to classify the types of goods in the export letter of credit.

[0083] Among them, the letter of credit plays a key role in international trade and bulk transactions. It provides a relatively safe and transparent payment mechanism for buyers and sellers. It can effectively solve the trust problem between buyers and sellers and reduce the uncertainty and risk in transactions. Especially in cross-border transactions, the two parties may face complex factors such as different legal systems, language and cultural differences, and currency exchange risks. At this time, the letter of credit, as a payment method supervised by the bank, provides reliable protection for both parties. With the bank as an intermediary, the letter of credit not only ensures that the seller can receive the payment in time, but also allows the buyer to have a stronger control over the shipment of goods and the authenticity of documents during the transaction. Therefore, the letter of credit is an extremely effective risk control tool in international trade, especially when the buyer and seller are from different countries. It can protect the interests of both parties, ensure that the transaction can proceed smoothly, and promote the development of global trade.

[0084] The technical solution of this embodiment can solve the problems that traditional manual review of the text of export letters of credit for goods is inefficient and prone to errors, and traditional machine learning methods are difficult to accurately capture semantic information and contextual relationships, resulting in poor classification results.

[0085] First, obtain the text of the export letter of credit for the goods to be classified.

[0086] Among them, the Export Letter of Credit refers to a series of services provided by the bank in the exporter's location to the exporter after receiving the letter of credit issued by the issuing bank, including letter of credit notification, order acceptance, document review, document delivery, and payment collection. In international trade, the Export Letter of Credit replaces commercial credit with bank credit, provides exporters with conditional payment commitments, thereby reducing trade risks.

[0087] The basic elements of an export letter of credit include the issuing bank, applicant, beneficiary, terms and conditions, validity period and amount.

[0088] In order to classify the text of the export letter of credit for goods, it is necessary to accurately and effectively identify the content of the export letter of credit for goods so as to better meet the corresponding business needs based on the export letter of credit for goods.

[0089] Step S20, extracting features from the export letter of credit text to be classified;

[0090] The technical solution of this application adopts the XLNet model, which uses the XLNet model's powerful language understanding capabilities and highly automated classification process to better process natural language text and extract key information and features without the need for manual complex text analysis, thereby improving classification accuracy, improving classification efficiency and reducing labor costs.

[0091] Among them, the XLNet model is a pre-trained model.

[0092] As an implementation method, the XLNet model can be trained using a large language model.

[0093] The input data of the XLNet model is feature data extracted from the text. Therefore, in this embodiment, when a new export letter of credit text needs to be classified into the type of goods, the export letter of credit text is input into the trained XLNet model for classification and recognition after feature extraction.

[0094] As an implementation method, before extracting features from the export letter of credit text to be classified, the export letter of credit text to be classified may be preprocessed to improve the classification accuracy and efficiency of the export letter of credit text to be classified.

[0095] Among them, the preprocessing methods for the classified goods export letter of credit text may include: noise removal, word segmentation, part-of-speech tagging, named entity recognition, etc. Among them, noise removal includes removing redundant spaces, removing special symbols, etc.

[0096] In this embodiment, as an implementation method, feature extraction is performed on the export letter of credit text to be classified, and the Word2Vec algorithm can be used to enrich the feature representation of the text.

[0097] Among them, Word2Vec is a machine learning algorithm for natural language processing (NLP), which is mainly used to convert words into vector representations in order to capture the semantic relationship between words in the vector space. Word2Vec contains two main training models: CBOW (Continuous Bag of Words) and Skip-gram. Among them:

[0098] The CBOW model predicts the central word through the context, which assumes that the entire context (i.e. the surrounding words) affects the prediction of the target word at the same time. In the CBOW model, the words in the context are used to predict the target word, which can be regarded as a "bag of words model".

[0099] As for the Skip-gram model, in contrast to the CBOW model, the Skip-gram model predicts context words through a central word, which focuses more on capturing the independent meaning of a single word.

[0100] During Word2Vec training, there are two optimization methods to improve speed:

[0101] Negative Sampling: This method improves training efficiency by introducing negative samples (i.e. randomly selected words that are not neighbors).

[0102] Hierarchical Softmax: This method reduces the amount of computation by constructing a Huffman tree so that the softmax probability calculation only needs to be performed along the tree structure.

[0103] The word vectors generated by Word2Vec are intuitive and interpretable, making it easier to understand complex language patterns. The training and reasoning speed of Word2Vec is relatively fast, especially with the support of libraries such as Gensim. Compared with other deep learning methods, Word2Vec has lower requirements for data size and computing resources.

[0104] Through Word2Vec, each word in the text can be mapped to a dense vector, thereby converting discrete words into continuous vector space representations, providing strong feature support for subsequent NLP tasks.

[0105] In this embodiment, the CBOW model of Word2Vec is used to implement feature extraction of the export letter of credit text to be classified. The specific extraction process may include:

[0106] Firstly, the export letter of credit texts to be classified are preprocessed as corpus.

[0107] Then, the model is trained using the preprocessed corpus and the selected Word2Vec model (CBOW) and set parameters (such as word vector dimension 200, window size 5, and learning rate 0.005).

[0108] After training is completed, each word is represented as a feature vector of a specific dimension, and the word vector is obtained through the interface provided by the model.

[0109] Step S30, inputting the export letter of credit text to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding goods type classification result.

[0110] Specifically, the feature vector of the export letter of credit text of the goods to be classified is input into the pre-trained XLNet model to obtain the corresponding goods type classification result.

[0111] Among them, the classification principle of the XLNet model is mainly based on its innovative pre-training method and architectural design. The XLNet model combines the advantages of two pre-trained language models, autoregression (AR) and autoencoding (AE), and adopts a dual-stream self-attention mechanism, which enables it to effectively utilize contextual information when processing classification tasks, better understand and generate the complex structure of language, and has excellent performance in multiple natural language processing tasks.

[0112] The technical solution of this embodiment obtains the text of the export letter of credit for goods to be classified; extracts features from the text of the export letter of credit for goods to be classified; and inputs the text of the export letter of credit for goods to be classified after feature extraction into a pre-trained XLNet model to obtain the corresponding classification result of the goods type. Since the technical solution of this application adopts the XLNet model, the powerful language understanding ability and highly automated classification process of the XLNet model can better process natural language texts, extract key information and features, and eliminate the need for manual complex text analysis, thereby improving classification accuracy, improving classification efficiency and reducing labor costs, and solving the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results.

[0113] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the first embodiment can refer to the above introduction, and will not be repeated in the following. Figure 2 In step S30, the feature-extracted export letter of credit text to be classified is input into the pre-trained XLNet model to obtain the corresponding classification result of the goods type, and the following steps are included before:

[0114] Step S23, using a large language model to train to obtain an XLNet model.

[0115] This embodiment uses the powerful language understanding ability and highly automated classification process of the XLNet model to better process natural language text and extract key information and features without the need for manual complex text analysis, thereby improving classification accuracy, improving classification efficiency and reducing labor costs.

[0116] Among them, the training process of the XLNet model includes: data collection, data preprocessing, feature extraction, model training and optimization.

[0117] Specifically, as an implementation method, step S23, the step of using a large language model to train an XLNet model may include:

[0118] Step A, collecting export letter of credit text data from a data source, wherein the export letter of credit text data covers letter of credit terms, descriptions and / or related documents of different types of goods;

[0119] For example, export letter of credit text data is collected from various possible data sources, and the collected data can cover letter of credit terms, descriptions, and related documents for different types of goods.

[0120] Step B, preprocessing the export letter of credit text data by using a pre-trained large language model to obtain preprocessed export letter of credit text data;

[0121] This embodiment combines a large language model with prompt words to implement preprocessing of export letter of credit text data.

[0122] Large Language Models, or LLMs for short, are artificial intelligence models with a large number of parameters that are trained on large amounts of data using deep learning algorithms to understand and generate natural language. These models can perform a variety of language tasks, such as text generation, translation, question answering, and text summarization.

[0123] Prompts are text input into these large models to guide the model to generate specific outputs. Prompts can be simple questions or instructions, or more complex text containing multiple instructions or contextual information.

[0124] Combining large models with prompts can create powerful interactive systems that can understand and respond to a variety of complex queries, provide personalized content, and even simulate experts in specific fields. For example, a well-designed prompt can guide a language model to generate an article on a specific topic or answer questions in a professional field. Therefore, combining large models with prompts can improve the performance of the model and the relevance of the output.

[0125] The pre-training corpus required in the large language model training process is a large-scale text data collection used in the pre-training stage of the large language model. It usually contains a wide range of text content from multiple fields and sources, and is suitable for training general basic models. These corpora can be further subdivided into web page data, language texts, book data, academic materials, codes, parallel corpora, social media, and encyclopedias.

[0126] Among them, in this embodiment, as an implementation method, the export letter of credit text data can be denoised by a pre-trained large language model and preset prompt words; then, the denoised export letter of credit text data and the preset prompt words are input into the large language model for word segmentation; each word obtained by the word segmentation is input into the large language model for part-of-speech tagging and entity recognition, so as to obtain the pre-processed export letter of credit text data.

[0127] More specifically, first, the export letter of credit text data is denoised:

[0128] The specific method of removing noise can be as follows:

[0129] Punctuation removal: First define the list of punctuation marks that need to be deleted: [".","?"","!"","、",":"], then traverse the text and delete the punctuation marks when they are encountered.

[0130] Remove redundant spaces: Traverse the text and when multiple consecutive spaces are found, only keep one space.

[0131] Remove special symbols: Input the cargo description text + prompt words (such as: Preprocess this text by removing all special symbols to make it purer and more readable and prepare it for subsequent analysis and processing. Special symbols include but are not limited to punctuation marks, mathematical symbols, currency symbols, emoticons, and various special characters. Ensure that the processed text contains only letters, numbers, and common spaces and does not contain any special symbols that may interfere with analysis.) into the large model to remove special symbols. For details, Figure 3 shown.

[0132] Then, perform word segmentation. The specific method is as follows:

[0133] The processed text and prompt words (such as: Perform accurate and efficient wordsegmentation on this text. Split the text into meaningful word units to facilitate subsequent natural language processing tasks such as textclassification, sentiment analysis, and information retrieval. Ensure that the word segmentation results can accurately reflect the semantics and structure of the text, while considering the characteristics and grammatical rules of different languages.) are input into the large model for word segmentation. Figure 4 shown.

[0134] Then, perform part-of-speech tagging. The specific method is as follows:

[0135] Input the above word segmentation results into the big model to perform part-of-speech tagging for each word.

[0136] Then, perform named entity recognition. The specific method is as follows:

[0137] Use a large model + prompt words (such as: Carefully analyze the given text and accurately extract the three key named entities of company names, product names, and country names. These named entities will serve as important basic data for subsequent feature extraction and model training, providing strong support for achieving more accurate natural language processing tasks such as information retrieval, text classification, and knowledge graph construction.) to identify specific entities (such as company names, product names, country names, etc.) in the text processed above, as follows Figure 5 shown.

[0138] Step C, using the Word2Vec model to extract features from the preprocessed export letter of credit text data to obtain feature word vectors;

[0139] As an implementation method, Word2Vec can be used to enrich the feature representation of text. The Word2Vec feature extraction method can improve the accuracy of text classification. The specific method is as follows:

[0140] First, the text data of all export goods are preprocessed as corpus.

[0141] Then, the model is trained using the preprocessed corpus and the selected Word2Vec model (CBOW) and set parameters (such as setting the word vector dimension to 200, the window size to 5, and the learning rate to 0.005).

[0142] After training, each word is represented as a vector of a specific dimension. The word vector is obtained through the interface provided by the model.

[0143] Step D, taking the feature word vector as a data set, and obtaining a corresponding training set, a validation set, and a test set based on the data set;

[0144] Use the word vectors obtained by Word2Vec above as a data set and prepare corresponding training sets, validation sets, and test sets.

[0145] Step E: Perform model training and testing based on the training set, validation set and test set to obtain a trained XLNet model.

[0146] Specifically, as an implementation method, step E, performing model training and testing based on the training set, validation set and test set to obtain a trained XLNet model may include:

[0147] Step E1, instantiating XLNet model parameters, wherein the XLNet model parameters at least include hidden layer size (hidden_size), number of layers (num_layers) and number of heads (num_heads);

[0148] In the field of deep learning and natural language processing, the above model parameters are defined as follows:

[0149] Hidden_size (hidden layer size): refers to the dimension of the hidden layer in the model, that is, the number of neurons in each layer after the Embedding layer and before the output (usually the fully connected layer or the multi-head self-attention layer in the Transformer). This value determines the capacity of the model when processing information. A larger hidden_size can provide more representation capabilities, but it will also increase the number of model parameters and computational complexity.

[0150] num_layers (number of layers): refers to the number of stacked Transformer layers or RNN layers in the model. In the Transformer model, increasing the number of layers enables the model to capture more complex patterns and longer-range dependencies.

[0151] num_heads (number of heads): specifically refers to the number of "heads" in the Multi-Head Attention mechanism in the Transformer model. Each head is responsible for processing different parts or different features of the input data. Multi-Head Self-Attention allows the model to learn information in different representation subspaces in parallel. Increasing the number of heads can improve the model's ability to capture information, but it will also increase the number of model parameters.

[0152] These parameters have a significant impact on the performance and capabilities of the model and usually need to be tuned according to the specific task and resource constraints.

[0153] Specifically, in this embodiment, as an implementation method, the XLNet model is instantiated, and the value of hidden_size, the value of num_layers, and the value of num_heads can be set to 512, 6, and 12.

[0154] Step E2, converting the text data in the training set into corresponding word vectors;

[0155] Specifically, the text data in the training set is converted into corresponding word vectors, and the input values ​​are preprocessed by padding and truncation to ensure the consistency of the length of the input data.

[0156] Step E3, performing a model training cycle based on the word vector, wherein in each training batch, the word vector is input into the XLNet model to calculate the loss function, the gradient is calculated by the back propagation algorithm, and the model parameters are updated using the Adam optimizer based on the calculated loss function and gradient;

[0157] Among them, when performing a model training cycle based on word vectors, the cross entropy loss function is first defined to measure the difference between the model prediction results and the true labels.

[0158] Cross-Entropy Loss Function, also known as Log Loss or Log Likelihood Loss, is a commonly used loss function in machine learning and statistics, especially in classification problems. It measures the difference between the probability distribution of the model prediction and the probability distribution of the true label.

[0159] The cross entropy loss function has the following characteristics:

[0160] Non-negativity: The value of the cross entropy loss function is always non-negative, and the loss is 0 if and only if the model's predicted probability distribution is exactly the same as the probability distribution of the true label.

[0161] Gradient descent: During the training process, the gradient of the loss function with respect to the model parameters is calculated, and the parameters are updated using the gradient descent algorithm to minimize the loss function.

[0162] Probabilistic interpretation: The cross entropy loss function provides a probabilistic interpretation, namely the difference between the predicted probability distribution and the true distribution.

[0163] Generalization: Using the cross entropy loss function during training helps the model learn parameters that generalize to new data.

[0164] The cross entropy loss function is a core component in neural network training. When dealing with classification tasks, the cross entropy loss function can help the model learn how to predict category probabilities more accurately.

[0165] In this embodiment, the cross entropy loss function is applied to process the multi-classification problem, wherein the mathematical formula of the cross entropy loss function is expressed as:

[0166] Among them, L(y,p) is the cross entropy, y is a real label vector of one-hot encoding, with a length of K, where y i Indicates that the sample belongs to the true label of the i-th category, p is the probability distribution vector predicted by the model, and its length is also K, p i is the probability that the model predicts that the sample belongs to the i-th category.

[0167] After defining the cross entropy loss function, you also need to select the Adam optimizer and set the learning rate. For example, set the learning rate to 0.001, and then gradually reduce the learning rate as the number of training steps increases.

[0168] Among them, the Adam optimizer uses the Adam algorithm.

[0169] The Adam algorithm is an adaptive learning rate optimization algorithm that combines the characteristics of momentum and adaptive learning rate. Its main idea is to dynamically adjust the learning rate of each parameter according to the gradient of the parameter.

[0170] The core principles of the Adam algorithm include:

[0171] Momentum: The Adam algorithm introduces a momentum term to smooth the direction of gradient updates, which helps accelerate convergence and reduce oscillation.

[0172] Adaptive learning rate: The Adam algorithm calculates an adaptive learning rate for each parameter, allowing different parameters to have different learning speeds.

[0173] Bias Correction: The Adam algorithm may be affected by bias in the early iterations, so bias correction is used to correct this problem.

[0174] In this embodiment, when performing a training cycle, in each training batch, the word vector is input into the XLNet model to calculate the loss function, the gradient is calculated by the back propagation algorithm, and the model parameters are updated using the Adam optimizer based on the calculated loss function and gradient.

[0175] Among them, the backpropagation algorithm is the core algorithm for training neural networks in deep learning. It is mainly used to calculate and update model parameters to minimize the loss function between the predicted output and the actual target. It combines the gradient descent optimization algorithm and the chain rule to efficiently calculate the gradient of each parameter in the neural network, thereby updating the model parameters to minimize the loss function.

[0176] In the deep learning framework, the neural network consists of multiple layers, each containing multiple nodes (or neurons). In the forward propagation phase, the input data is passed layer by layer in the network through a series of nonlinear transformations, and finally a predicted output is generated, including: the input data is forward propagated through each layer of the neural network, the output of each layer becomes the input of the next layer, the prediction result is generated in the last layer (output layer), and finally the loss (Loss) between the output result and the true label is calculated.

[0177] After the network makes a prediction, it is necessary to adjust the weights and biases in the network based on the difference between the predicted value and the actual label so that the model can make more accurate predictions in the future. This adjustment process is achieved through the back propagation algorithm, which includes:

[0178] Error calculation: First, calculate the gap between the predicted output of the model in the current iteration and the true target, which is usually converted into the calculation of the loss function (such as cross entropy, mean square error, etc.).

[0179] Gradient calculation: Use the chain rule to start from the output layer and calculate the gradient of each intermediate variable and parameter for the total loss function layer by layer in reverse order. In other words, calculate the derivative of the loss with respect to the weights and biases of each layer in the network.

[0180] Parameter update: Based on the calculated gradient, the network weights and biases are updated using an optimization algorithm (such as gradient descent, momentum, Adam, etc.). The direction of the update is opposite to the direction of the gradient, with the goal of reducing the value of the loss function.

[0181] In short, the core principle of back propagation is to propagate the error from the output layer back through the network layer by layer, and adjust the weights of each layer according to the error signal propagated back, so that the learning ability of the entire network for the input data is gradually improved. This process is repeated in each iterative training until the model converges or reaches the predetermined number of training rounds.

[0182] The key points of the back-propagation algorithm are:

[0183] Chain Rule: The back-propagation algorithm uses the chain rule to calculate the derivatives of complex functions, which is the basis for calculating the gradient of multi-layer networks.

[0184] Gradient caching: During backpropagation, the activation values ​​and gradients of the intermediate layers need to be saved in order to calculate the gradients of subsequent layers.

[0185] Parameter initialization: The initial values ​​of the parameters have a great impact on the training process and the final results. Different initialization methods may lead to different training dynamics.

[0186] Regularization: In order to prevent overfitting, regularization terms are usually added to the loss function, such as L1 and L2 regularization.

[0187] Learning rate scheduling: The learning rate may vary during training to speed up convergence or improve model performance.

[0188] Step E4, regularly evaluating the performance of the XLNet model on the validation set;

[0189] Regularly evaluate the performance of the model on the validation set, and determine whether to stop training early to prevent overfitting based on the results of the validation set.

[0190] It should be noted that the performance of the XLNet model can also be evaluated on the validation set from time to time.

[0191] Step E5: According to the performance evaluation result of the validation set, the parameters of the XLNet model are adjusted and optimized to obtain a trained XLNet model.

[0192] Specifically, the hidden layer size, number of layers and / or number of heads of the XLNet model can be adjusted according to the performance evaluation results of the validation set, i.e., hidden_size, num_layers, num_heads, and the learning rate can also be adjusted.

[0193] If the trained XLNet model is overfitting, add a positive L1 regularization term to the loss function, or, if the trained XLNet model is underfitting, increase the amount of training data.

[0194] The specific process of training the XLNet model in this embodiment can be referred to Figure 6 shown.

[0195] This embodiment uses the above scheme, specifically trains the XLNet model through a large model, obtains the text of the export letter of credit for goods to be classified; extracts features from the text of the export letter of credit for goods to be classified; inputs the text of the export letter of credit for goods to be classified after feature extraction into the trained XLNet model, and obtains the corresponding classification result of the goods type. Since the technical solution of this application adopts the XLNet model, the powerful language understanding ability and highly automated classification process of the XLNet model can better process natural language texts, extract key information and features, and do not require manual complex text analysis, thereby improving classification accuracy, improving classification efficiency and reducing labor costs. It solves the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results. In addition, this application uses the method of large model + prompt words to preprocess data, eliminating the cumbersome process of traditional data preprocessing. At the same time, the large model can automatically learn the features and semantic information in the text without manual participation, further reducing labor costs.

[0196] Based on the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the second embodiment can be referred to the above introduction, and will not be repeated in the following. Figure 7 In step S30, the feature-extracted export letter of credit text to be classified is input into the pre-trained XLNet model to obtain the corresponding classification result of the goods type, and the following steps are included:

[0197] Step S40, evaluating the cargo type classification result using a preset performance indicator;

[0198] Step S50, adjusting and optimizing the XLNet model according to the evaluation result.

[0199] Among them, the preset performance indicators may include: accuracy, recall rate, F1 value and other indicators.

[0200] The classification results output by the model are evaluated using indicators such as accuracy and recall, and the classification performance is measured using indicators such as accuracy, recall, and F1 value. Among them, accuracy refers to the proportion of samples correctly classified by the model to the total number of samples; recall refers to the proportion of samples that are actually in a certain category and correctly classified by the model to the total number of samples that are actually in that category; F1 value is the harmonic mean of accuracy and recall, which comprehensively considers the accuracy and comprehensiveness of the model.

[0201] After the evaluation is completed, the XLNet model can be further optimized and adjusted based on the evaluation results. If the evaluation results are not ideal, more data can be collected to retrain the model, or the parameters and structure of the model can be adjusted.

[0202] For example, if the accuracy is low, you can try increasing the number of training rounds, adjusting parameters such as the learning rate, or adding more feature extraction methods.

[0203] If the evaluation results are good but there is still room for improvement, you can try using a more advanced model architecture or optimization algorithm to improve the performance and generalization ability of the model.

[0204] In addition, the optimization results can be fed back and the optimized model can be applied again to the task of cargo type classification. This process can be repeated continuously to improve the accuracy and reliability of classification.

[0205] The overall process of training the XLNet model and applying the trained XLNet model to classify the export letter of credit goods in this embodiment can be referred to Figure 8 shown.

[0206] This embodiment proposes a method for classifying the types of goods in export letters of credit based on the XLNet model, which solves the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships. An automated cargo type classification process is proposed to solve the pain points of excessive manual intervention, difficulty in ensuring accuracy, and insufficient understanding of complex texts. A method for data preprocessing using a large model is proposed, which can automatically learn the features and semantic information in the text without manual participation and in a simple process.

[0207] In addition, the present application also proposes a cargo type classification device, such as Fig. 9 As shown, the device comprises:

[0208] The acquisition module 10 is used to acquire the export letter of credit text of goods to be classified;

[0209] A feature extraction module 20 is used to extract features from the export letter of credit text to be classified;

[0210] The classification module 30 is used to input the export letter of credit text to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding classification result of the goods type.

[0211] The cargo type classification device provided by the present application adopts the cargo type classification method in the above embodiment, which can solve the problems of low efficiency and error-proneness of traditional manual review and poor classification effect caused by the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, thereby improving classification accuracy, improving classification efficiency and reducing labor costs. Compared with the prior art, the beneficial effects of the cargo type classification device provided by the present application are the same as the beneficial effects of the cargo type classification method provided by the above embodiment, and the other technical features in the cargo type classification device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0212] In addition, to achieve the above-mentioned purpose, the present application also proposes a cargo type classification device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the cargo type classification method described above.

[0213] Reference below Fig.10 , which shows a schematic diagram of a structure of a cargo type classification device suitable for implementing an embodiment of the present application. The cargo type classification device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The cargo type classification device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0214] like Fig.10As shown, the cargo type classification device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the cargo type classification device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the cargo type classification device to communicate with other devices wirelessly or by wire to exchange data. Although the cargo type classification device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0215] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0216] The cargo type classification device provided by the present application adopts the cargo type classification method in the above embodiment, which can solve the problems of low efficiency and error-proneness of traditional manual review and poor classification effect caused by the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, thereby improving classification accuracy, improving classification efficiency and reducing labor costs. Compared with the prior art, the beneficial effects of the cargo type classification device provided by the present application are the same as the beneficial effects of the cargo type classification method provided by the above embodiment, and the other technical features in the cargo type classification device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0217] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0218] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the cargo type classification method described above are implemented.

[0219] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0220] The computer-readable storage medium may be included in the cargo type classification device; or may exist independently without being assembled into the cargo type classification device.

[0221] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the goods type classification device, the goods type classification device: obtains the export letter of credit text of goods to be classified; performs feature extraction on the export letter of credit text of goods to be classified; inputs the export letter of credit text of goods to be classified after feature extraction into a pre-trained XLNet model to obtain the corresponding goods type classification result.

[0222] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0223] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0224] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0225] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned method for classifying goods by type, and can solve the problems of low efficiency and error-proneness of traditional manual review and poor classification results caused by the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, thereby improving classification accuracy, improving classification efficiency, and reducing labor costs. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the method for classifying goods by type provided by the above-mentioned embodiment, and will not be elaborated on here.

[0226] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the cargo type classification method as described above.

[0227] The computer program product provided by this application can solve the problems of low efficiency and error-proneness of traditional manual review and poor classification effect caused by the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, thereby improving classification accuracy, improving classification efficiency and reducing labor costs. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the cargo type classification method provided by the above embodiment, and will not be elaborated here.

[0228] One or more technical solutions proposed in this application have at least the following technical effects: by obtaining the text of the export letter of credit for goods to be classified; extracting features from the text of the export letter of credit for goods to be classified; inputting the text of the export letter of credit for goods to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding classification result of the goods type. Since the technical solution of this application adopts the XLNet model, the powerful language understanding ability and highly automated classification process of the XLNet model can better process natural language texts, extract key information and features, and do not require manual complex text analysis, thereby improving classification accuracy and efficiency and reducing labor costs, solving the problems of low efficiency and error-proneness of traditional manual review and the difficulty of traditional machine learning methods in accurately capturing semantic information and contextual relationships, resulting in poor classification results. In addition, this application uses the method of large model + prompt words to preprocess data, eliminating the cumbersome process of traditional data preprocessing. At the same time, the large model can automatically learn the features and semantic information in the text without manual participation, further reducing labor costs.

[0229] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for classifying cargo types, characterized in that: The method comprises: Obtain the text of the export letter of credit for goods to be classified; Extracting features from the export letter of credit text to be classified; The export letter of credit text of goods to be classified after feature extraction is input into the pre-trained XLNet model to obtain the corresponding classification results of goods types.

2. The method according to claim 1, characterized in that The step of inputting the feature-extracted export letter of credit text to be classified into a pre-trained XLNet model to obtain the corresponding classification result of the goods type also includes: The XLNet model is trained using a large language model.

3. The method according to claim 2, characterized in that The step of using the large language model to train the XLNet model includes: Collecting export letter of credit text data from data sources, the export letter of credit text data covering letter of credit terms, descriptions and / or related documents for different types of goods; Preprocessing the export letter of credit text data by using a pre-trained large language model to obtain preprocessed export letter of credit text data; The Word2Vec model is used to extract features from the preprocessed export letter of credit text data to obtain feature word vectors; Taking the feature word vector as a data set, obtaining a corresponding training set, a validation set and a test set based on the data set; Model training and testing are performed based on the training set, validation set and test set to obtain a trained XLNet model.

4. The method according to claim 3, characterized in that The steps of performing model training and testing based on the training set, validation set and test set to obtain a trained XLNet model include: Instantiate XLNet model parameters, wherein the XLNet model parameters include at least hidden layer size, number of layers, and number of heads; Convert the text data in the training set into corresponding word vectors; Performing a model training cycle based on the word vectors, wherein in each training batch, the word vectors are input into the XLNet model to calculate the loss function, the gradient is calculated by the back propagation algorithm, and the model parameters are updated using the Adam optimizer based on the calculated loss function and gradient; Regularly evaluate the performance of the XLNet model on the validation set; According to the performance evaluation results of the validation set, the parameters of the XLNet model are adjusted and optimized to obtain a trained XLNet model.

5. The method according to claim 4, characterized in that The step of adjusting and optimizing parameters of the XLNet model according to the performance evaluation result of the validation set comprises: Adjusting the hidden layer size, number of layers and / or number of heads of the XLNet model based on the performance evaluation results of the validation set; and / or If the trained XLNet model is overfitting, add a regularization term to the loss function, or, if the trained XLNet model is underfitting, increase the amount of training data.

6. The method according to claim 3, characterized in that The step of preprocessing the export letter of credit text data by using a pre-trained large language model comprises: De-noising the export letter of credit text data using a pre-trained large language model and preset prompt words; Inputting the denoised export letter of credit text data and preset prompt words into the large language model for word segmentation processing; Each word obtained by the word segmentation process is input into the large language model for part-of-speech tagging and entity recognition to obtain pre-processed export letter of credit text data.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Evaluate the classification results of the cargo type using the preset performance indicators; The XLNet model is adjusted and optimized according to the evaluation results.

8. A cargo type classification device, characterized in that: The device comprises: The acquisition module is used to obtain the export letter of credit text of goods to be classified; A feature extraction module, used for extracting features from the export letter of credit text to be classified; The classification module is used to input the export letter of credit text of goods to be classified after feature extraction into the pre-trained XLNet model to obtain the corresponding goods type classification result.

9. A cargo type classification device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the cargo type classification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the steps of the cargo type classification method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the cargo type classification method according to any one of claims 1 to 7 are implemented.

Images