Logistics order classification method based on graph convolutional neural network and RoBERTa

Abstract

The present invention discloses a logistics order classification method based on graph convolutional neural network and RoBERTa language model, which realizes logistics order classification by extracting the fusion features of global graph, local graph and text semantics of order text.

Description

Technical Field

[0001] The present invention relates to the field of modern logistics, and specifically to the application of a graph convolutional neural network and RoBERTa-based method in logistics order classification, to construct an efficient and predictable logistics order classification method. Background Art

[0002] In recent years, the logistics industry has gradually shifted from an outsourced to an integrated model. Information systems have played a key role in the operational management of logistics companies. As the central nervous system of the logistics information system, the order subsystem has played a positive role in improving operational efficiency and customer service quality. Personalized logistics services have placed higher demands on order processing processes, and logistics companies urgently need to improve the efficiency of processing differentiated orders.

[0003] Logistics supply chain services generally include road transport, water transport, and intermodal transport. Intermodal transport services further encompass road-water transport, rail-water transport, and road-rail transport. Customers submit one-stop orders in text format through the WeChat mini-program provided by the logistics platform. The platform's customer service first breaks down the order text, then classifies the logistics order based on historical order routes and optimal routes. Finally, the order is distributed to the corresponding business system based on the order type to complete the service acceptance. To efficiently complete order classification, the platform's customer service staff not only needs to have a deep understanding of the customer's request but also be familiar with various logistics business types. However, platform customer service staff often lack systematic business training, which leads to errors in logistics order classification and unnecessary secondary distribution.

[0004] Large logistics companies handle tens of thousands of orders daily. Customer service personnel must break down and categorize each order based on information such as the origin, destination, cargo list, and service requirements. This is a massive workload and prone to errors. Manually processing massive volumes of differentiated orders is difficult to achieve the efficiency standards of modern logistics services. Therefore, research on intelligent logistics order classification is crucial for reducing logistics service cycles and improving customer satisfaction. Summary of the Invention

[0005] To overcome the inefficiency and error-proneness of manual classification, this paper proposes a logistics order classification method based on the GCN and RoBERTa models. Each sentence in the order text is parsed using an Abstract Meaning Representation (AMR) algorithm, which generates multiple AMR graphs of the input text to form local AMR graphs. A global AMR graph is constructed based on the input text keywords and the root nodes of the AMR graphs. A GCN and a stacked GCN network are used to extract features from the global and local AMR graphs. Semantic features of the order text are extracted using the RoBERTa model, and finally, the features are fused to complete the logistics order classification.

[0006] The technical solution adopted by the present invention is: a logistics order classification method based on graph convolutional neural network and RoBERTa, comprising the following steps: S1, constructing a global AMR graph based on the abstract semantic representation (AMR) results and keywords of the logistics order text, and using the graph convolutional neural network to extract features from the global AMR graph to obtain the global AMR graph representation vector of the order text S2, based on the AMR algorithm, builds a local AMR graph set of logistics order text sentences, and then uses stacked GCN to process the graph set to obtain the local AMR graph representation vector of the order text S3, use the RoBERTa model to process the logistics order text and obtain the text semantic representation vector S4, fuses three types of text representation vectors to complete logistics order classification.

[0007] As a further improvement of the present invention, in said S1, constructing a global AMR graph includes the following steps: Step 1, performing sentence processing on the order text to obtain n sentences, using the keyword extraction algorithm in the HIT NLP toolkit to extract the keywords of each sentence in the order text, and obtaining an order keyword set n keyword sets {K i}; Step 2, perform word segmentation on the n sentences of the order text, and use the AMR algorithm to abstract each sentence of the order text into n AMR graphs; Step 3, compare the AMR root nodes of the n sentences, and merge the root nodes with the same semantics; Step 4, perform keyword set {K i} is analyzed and processed, and keywords with the same semantics between different sentences are fused; Step 5, an AMR global graph is constructed based on the fusion results of n sentences.

[0008] As a further improvement of the present invention, a natural sentence can be parsed into an AMR graph G = (V, E) by an AMR parser; V represents an abstract concept node of a content word in the sentence, and E represents the relationship between a specific content word (abstracted as a directed edge with a semantic relationship label, and ignoring function words and morphological changes to reflect more abstract semantics).

[0009] As a further improvement of the present invention, the feature extraction of the global AMR graph by the graph convolutional neural network in S1 includes: Step 1, node encoding of the constructed logistics order text relationship graph. For a given logistics order text T, the node encoding first needs to be segmented to obtain the segmentation sequence T of the order text. q , and then use word embedding to embed the sequence T q Encode and obtain the encoding sequence E q :

[0010]

[0011] in, and are the word embedding vectors of the keywords in the order text T and the AMR root node, ξ m are the embedding vectors of other words;

[0012] Step 2: To encode the position information of keywords and AMR root nodes in the order text T, position encoding is performed on the order text. The encoding result is as follows:

[0013] in and are the keyword and AMR root node position embedding vectors in the order text T, p m are the embedding vectors of other words;

[0014] Step 3: The node encoding vector is obtained by taking the weighted average of the word embedding vector and the position embedding vector. The calculation formula is as follows:

[0015] e m =λ1ξ m +λ2p m (1)

[0016]

[0017]

[0018] Logistics order text sequence T q The final encoding vector of According to the nodes in the AMR graph, in the order text sequence encoding <T q,E> to find the corresponding node word encoding vector as the embedding vector of the global AMR graph node V={v1,v2,...,v i ,...}.

[0019] As a further improvement of the present invention, the GCN network is used for feature extraction, following the GCN layer-by-layer linear propagation rule:

[0020] in A is the adjacency matrix of the input graph, I is the identity matrix, X (0) =X is the initial input feature matrix; the global AMR feature representation vector of the input order text is calculated The calculation formula is as follows:

[0021]

[0022] Where W (0) is the weight matrix from the input layer to the hidden layer, W (1) is the weight matrix from the hidden layer to the output layer.

[0023] As a further improvement of the present invention, in S2, constructing a local AMR graph set of logistics order text sentences based on the AMR algorithm includes the following steps: Step 1, sentence processing of the order text to obtain n sentences; Step 2, processing the n sentences respectively using the AMR algorithm to obtain n AMR graphs; Step 3, forming a local AMR graph of the order text from the set of n AMR graphs.

[0024] As a further improvement of the present invention, in S2, stacking GCN to process the graph fusion network includes the following steps:

[0025] Step 1: Construct local AMR graph G local ={G1,G2,...,G n}, G i ={V1,E i};

[0026] Step 2, for each G i Use word embedding and position embedding to encode and get the corresponding word embedding sequence and position embedding sequence

[0027] Step 3: Use weighted average to embed the word into the sequence and position embedding sequence Fusion is performed to finally obtain the corresponding representation vector sequence E i ;

[0028] Step 4, according to AMR graph Gi The node in the input text sequence encoding pair finds the corresponding node word encoding vector as G i The embedding vector of the node V={v1,v2,...,v i ,...}, all G i The embedding vectors of the nodes constitute the embedding vector set V of the local AMR graph local ={V1,V2,...,V n};

[0029] Embedding vector set V for local AMR image local Each element in is extracted using the GCN network:

[0030]

[0031]

[0032] Among them A i is the adjacency matrix of the i-th graph in the local AMR graph;

[0033] The representation vectors of multiple sentences constitute the feature representation vector combination of the local AMR graph

[0034]

[0035] Finally, a gating mechanism is used to fuse these representation vectors:

[0036]

[0037] Where W local and B are both learnable parameters; the fusion vector A local AMR graph representation vector as order text.

[0038] As a further improvement of the present invention, in S3, RoBERTa-wwm-ext is used as the feature extraction model; for a given order text sequence X = {x1, x2, ..., x n}, where x i For each word in the order text, xi undergoes character embedding, segmentation embedding, and position embedding encoding and is fused to obtain the final fused embedding vector. The sequence of all fused embedding vectors of X is fed into the RoBERTa-wwm-ext model; RoBERTa-wwm-ext performs feature encoding on the fused embedding vector sequence, extracts the contextual semantic information of the input sequence X, and finally outputs the semantic representation vector V corresponding to each word in the order text. lm ={v1 lm ,v2 lm ,...,v slm}; Fuse the s semantic representation vectors to finally obtain the representation vector of the order text

[0039] As a further improvement of the present invention, in S4, the three types of order representation vectors Fusion to obtain V concat ,

[0040] Among them, ω1, ω2, ω3 are fusion parameters, satisfying ω1+ω2+ω3=1; V concat Input classifier for multi-label classification of completed orders: p = sigmoid(WV concat +b) (10)

[0041] Where W and b are learnable parameters, and p is the classification prediction probability of each category;

[0042] Use cross entropy for multi-label classification as the training loss function:

[0043]

[0044] where y i is the true label, y′ i is the prediction result, i is the classification category of the order.

[0045] As a further improvement of the present invention, it also includes training the model established by the logistics order classification method. First, the historical logistics order text is segmented, sentence-processed, stop words are removed, and keywords are extracted. Then, a logistics order dataset is constructed based on the processing results of the historical logistics orders; then the dataset is divided into a training set, a validation set, and a test set according to a certain ratio. The training set is used for model training, the validation set updates the model performance through continuous iteration, and the test set is used to evaluate the model performance; finally, the trained model is used to predict the logistics order type.

[0046] The present invention has the following beneficial effects: The method is based on a GCN feature extraction layer based on a global AMR graph, a stacked GCN feature extraction layer based on a local AMR graph, a semantic encoding layer based on the RoBERTa language model, and an order classification layer. By using a GCN network to extract the topic features of texts of varying lengths, and by fully utilizing large-scale unlabeled data in the pre-training phase through the RoBERTa model, general language capabilities are better mastered, surpassing the traditional model's extraction effect on text context semantics. The present invention can quickly, efficiently, and accurately implement order classification for modern logistics companies. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1Schematic diagram of the model structure constructed for the present invention.

[0048] Figure 2 This is a schematic diagram of the AMR diagram of the present invention.

[0049] Figure 3 Schematic diagram of stacked GCN feature extraction based on local AMR images of the present invention.

[0050] Figure 4 Schematic diagram of RoBERTa feature extraction of the present invention.

[0051] Figure 5 The model training process constructed for the present invention. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0053] The logistics order classification model of the present invention is mainly composed of a GCN feature extraction layer based on the global AMR graph, a stacked GCN feature extraction layer based on the local AMR graph, a semantic encoding layer based on the RoBERTa language model, and an order classification layer. Figure 1 As shown in the figure. For a given logistics order text, the global AMR algorithm is first used to construct a global AMR graph, and the GCN network is used to extract features from the nodes of the global AMR graph to obtain the global AMR graph representation vector of the text. Secondly, the local AMR algorithm is used to construct multiple AMR graphs, and then the stacked GCN network is used to extract features from multiple AMR graphs, which are then fused to obtain the local AMR graph representation vector of the text. Thirdly, the RoBERTa pre-trained language model is used to extract the contextual semantic features of the order text to obtain the order text semantic representation vector. Finally, the three types of text representation vectors are fused, and a fully connected network (FC) combined with a sigmoid function is used to complete the order classification.

[0054] A key step in extracting feature information from logistics order text is capturing both intra- and inter-sentence features. To address inter-sentence correlation, the present invention uses a global AMR algorithm to process the logistics order text to generate a global AMR graph. This graph is then encoded using a GCN network to obtain inter-sentence correlation features and thematic features of the order text. AMR is a domain-independent sentence semantic representation method that abstracts the semantics of a sentence into a single-rooted directed acyclic graph. Figure 2This example uses an AMR graph to represent the sentence "The order's shipping location is Tongling Port Logistics Company." A natural sentence can be parsed into an AMR graph G = (V, E) using an AMR parser. V represents abstract concept nodes representing the content words in the sentence, while edges represent relationships between specific content words (abstractly labeled as directed edges with semantic relationship labels, ignoring function words and morphological variations to reflect more abstract semantics). Therefore, AMR focuses on semantic relationships rather than grammatical ones. This representation is more conducive to understanding logistics orders, and its structure is closer to the "trigger-arguments" structure of orders.

[0055] Since logistics order texts vary in length, and some sentences in the order texts are irrelevant to the order subject, the present invention extracts the order text keywords and the root node of the AMR as the serial nodes of the global AMR graph, constructing a global relationship graph. This allows the association between the order subject information and the order text sentences to be extracted. The process of constructing the inter-sentence text graph is as follows:

[0056] Step 1: Segment the order text into n sentences, and use the keyword extraction algorithm in the HIT NLP toolkit to extract the keywords of each sentence in the order text to obtain the order keyword set n keyword sets {K i};

[0057] Step 2: Segment the n sentences in the order text and use the AMR algorithm to abstract each sentence in the order text into n AMR graphs;

[0058] Step 3: Compare the AMR root nodes of n sentences and merge the root nodes with the same semantics;

[0059] Step 4: For the keyword set {K i} to analyze and process, and merge keywords with the same semantics between different sentences;

[0060] Step 5 builds the AMR global graph based on the fusion results of n sentences.

[0061] Before GCN feature extraction, the constructed logistics order text relationship graph needs to be node-encoded. For a given logistics order text T, node encoding first needs to segment it and obtain the segmentation sequence T of the order text. q , and then use word embedding to embed the sequence T q Encode and obtain the encoding sequence E q :

[0062]

[0063] in, and are the word embedding vectors of the keywords in the order text T and the AMR root node, ξ m are the embedding vectors of other words;

[0064] Step 2: To encode the position information of keywords and AMR root nodes in the order text T, position encoding is performed on the order text. The encoding result is as follows:

[0065] in and are the keyword and AMR root node position embedding vectors in the order text T, p m are the embedding vectors of other words;

[0066] Step 3: The node encoding vector is obtained by taking the weighted average of the word embedding vector and the position embedding vector. The calculation formula is as follows:

[0067] e m =λ1ξ m +λ2p m (1)

[0068]

[0069]

[0070] Logistics order text sequence T q The final encoding vector of According to the nodes in the AMR graph, in the order text sequence encoding <T q ,E> to find the corresponding node word encoding vector as the embedding vector of the global AMR graph node V={v1,v2,...,v i ,...}.

[0071] 5. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 4 is characterized in that the GCN network is used for feature extraction and the GCN layer-by-layer linear propagation rule is followed:

[0072]

[0073] in A is the adjacency matrix of the input graph, I is the identity matrix, X (0) =X is the initial input feature matrix;

[0074] Calculate the global AMR feature representation vector of the input order text The calculation formula is as follows:

[0075]

[0076] Where W (0) is the weight matrix from the input layer to the hidden layer, W (1) is the weight matrix from the hidden layer to the output layer.

[0077] like Figure 3 As shown in the figure, for the relevant features within the sentence, the present invention adopts the local AMR algorithm to process the logistics order text to generate a local AMR graph, and then uses a multi-layer stacked GCN network to encode the local AMR graph, and fuses the encoding results to obtain the fusion result of the relevant features within the order text sentence.

[0078] Since each sentence in the order text has different importance for the order classification results, and some sentences are not strongly related to the order type (such as the delivery contact information), the present invention uses the AMR algorithm to process each sentence in the order text to obtain AMR graphs for multiple sentences. The specific steps are as follows:

[0079] Step 1: Segment the order text into sentences to obtain n sentences;

[0080] Step 2: Use the AMR algorithm to process n sentences and obtain n AMR graphs;

[0081] Step 3: The set of n AMR graphs is converted into a local AMR graph of the order text.

[0082] For the constructed local AMR graph G local ={G1,G2,...,G n}, G i ={V1,E i}. For each G i Use word embedding and position embedding to encode and get the corresponding word embedding sequence and position embedding sequence Use weighted average to embed words into sequences and position embedding sequence Fusion is performed to finally obtain the corresponding representation vector sequence E i According to AMR Figure G i The node in the input text sequence encoding pair finds the corresponding node word encoding vector as G i The embedding vector of the node V={v1,v2,...,v i ,...}, all G i The embedding vectors of the nodes constitute the embedding vector set V of the local AMR graph local ={V1,V2,...,V n};

[0083] Embedding vector set V for local AMR image local Each element in is extracted using the GCN network:

[0084]

[0085]

[0086] Among them A i is the adjacency matrix of the i-th graph in the local AMR graph;

[0087] The representation vectors of multiple sentences constitute the feature representation vector combination of the local AMR graph

[0088]

[0089] Finally, a gating mechanism is used to fuse these representation vectors:

[0090]

[0091] Where W local and B are both learnable parameters; the fusion vector A local AMR graph representation vector as order text.

[0092] The RoBERTa model is an improved model based on BERT. Compared with BERT, it has larger model parameters, larger batch size, and larger-scale training data, and deletes the next sentence prediction (NSP) task. This enables the RoBERTa language model to be better applied to downstream tasks than BERT, achieving better model effects. The present invention uses the RoBERTa pre-trained language model to perform feature extraction on the order text to obtain the contextual semantic representation of the order text. Since the task of the present invention is Chinese order text, RoBERTa-wwm-ext released by the Harbin Institute of Technology iFlytek Joint Laboratory is used as the feature extraction model. Figure 4 As shown, for a given order text sequence X={x1,x2,...,x n}, where x i For each word in the order text, xi undergoes character embedding, segmentation embedding, and position embedding encoding and fusion to obtain the final fused embedding vector. The sequence of all fused embedding vectors of X is fed into the RoBERTa-wwm-ext model. RoBERTa-wwm-ext performs feature encoding on the fused embedding vector sequence, extracts the contextual semantic information of the input sequence X, and finally outputs the semantic representation vector V corresponding to each word in the order text. lm ={v1 lm ,v2 lm ,...,v slm}. The s semantic representation vectors are fused to finally obtain the representation vector of the order text Three types of order representation vectors Fusion to obtain V concat ,

[0093]

[0094] Where ω1, ω2, ω3 are fusion parameters, satisfying ω1+ω2+ω3=1;

[0095] V concat Multi-label classification of completed orders in the input classifier:

[0096] p=sigmoid(WV concat +b) (10)

[0097] Where W and b are learnable parameters, and p is the classification prediction probability of each category;

[0098] Use cross entropy for multi-label classification as the training loss function:

[0099]

[0100] where y i is the true label, y′ i is the prediction result, i is the classification category of the order.

[0101] In Example 1, the network model was constructed using the CUDA 11.0-based deep learning framework pytorch 1.7.1. The experimental platform was an Ubuntu 18.04LTS system with 64G of memory and 24G of video memory.

[0102] The training process of the logistics order classification model proposed in this invention is as follows: Figure 5 As shown in the figure, we first construct a logistics order dataset by segmenting historical logistics order text, breaking it into words and sentences, removing stop words, and extracting keywords. The dataset is then divided into training, validation, and test sets based on a specific ratio. The training set is used for model training, the validation set continuously iterates to update model performance, and the test set is used to evaluate model performance. Finally, the trained model is used to predict logistics order types.

[0103] We constructed an experimental dataset from 30,000 orders selected from our company's historical logistics orders between January 1, 2017, and December 31, 2021, as shown in Table 1. The dataset contains order text data and corresponding order types. The order text represents the customer's logistics entrustment, and the order type is annotated by customer service based on the order text. This example also performed a statistical analysis of the 30,000 order texts, revealing that the average length of these logistics order texts is 276 characters.

[0104] Table 1 Dataset description

[0105]

[0106] During the overall network training process, the hyperparameters of the model in this embodiment are shown in Table 2.

[0107] Table 2 Hyperparameter settings

[0108]

[0109] Precision, recall, F1 value, accuracy, and Hamming loss are used as evaluation indicators for logistics order classification performance. In order to verify the performance of the logistics order classification method proposed in this paper, it is compared with various baseline methods.

[0110] TextCNN: Uses pre-trained word vectors to encode the input text, then uses a convolutional neural network to extract the embedding vector of the order text to obtain the features of the input text. Finally, a fully connected network combined with a sigmoid function is used to complete order classification.

[0111] HAN: Uses word-level BiGRU+Attention and sentence-level BiGRU+Attention models to extract multi-level semantic features of order text, and then uses the sigmoid function to complete order classification.

[0112] TextGCN: First, a text graph of the corpus is constructed based on word co-occurrence and document-word relationships. Then, the constructed text graph is encoded using one-hot encoding. Finally, a GCN network is used to extract the encoded text graph to complete order classification.

[0113] XLNet: We use the Chinese XLNet pre-trained language model (chinese-xlnet-base) trained by the Harbin Institute of Technology iFlytek Joint Laboratory to extract order text features. After fine-tuning, we apply it to the order classification task.

[0114] RoBERTa: We use the Chinese RoBERTa pre-trained language model (chinese-roberta-wwm-ext) trained by the Harbin Institute of Technology iFlytek Joint Laboratory to extract input text features. After fine-tuning, we apply it to the order classification task.

[0115] BERT-GCN: Builds a heterogeneous graph based on word nodes and document nodes, uses the BERT pre-trained model to initialize document nodes, and jointly trains the BERT module and GCN module to complete order classification.

[0116] The results of the baseline comparison experiment are shown in Table 3. It can be seen from the table that the logistics order classification method proposed in the present invention is superior to other baseline models in all indicators. It is worth noting that TextCNN uses the traditional CNN network to extract text features and perform classification, which is not good. This is because the CNN network can only extract local features of the text and cannot obtain text context information. HAN uses a bidirectional GRU combined with an Attention mechanism to extract contextual semantic information, but lacks the extraction of local features of the text and has poor feature extraction capabilities for long sentences. Therefore, it only achieves better results than TextCNN. For the XLNet and RoBERTa models, the feature information of the order text can be effectively extracted, so better results are achieved. TextGCN uses GCN to encode text graphs, which can effectively obtain the syntactic structure information of the text and better solve the problem of text length. However, its ability to extract text context information is poor. BERT-GCN achieves better results due to the addition of the BERT module. Since the RoBERTa model makes full use of large-scale unlabeled data in the pre-training stage, it can better master general language skills and perform better than traditional models in extracting text context semantics in most tasks. In addition, the GCN network can be used to extract the topic features of texts of varying lengths. Therefore, the order classification method based on multi-level graph structures proposed in this invention can have better performance.

[0117] Table 3 Baseline comparison results

[0118] Classification method Precision Recall F1 Accuracy HammingLoss TextCNN 0.7185 0.7761 0.7462 0.7075 0.18083 HAN 0.7384 0.7981 0.7671 0.7310 0.16600 TextGCN 0.7658 0.8202 0.7921 0.7638 0.14747 XLNet 0.7814 0.8293 0.8046 0.7785 0.13793 ROBERTa 0.7978 0.8447 0.8206 0.7979 0.12650 BERT-GCN 0.8180 0.8630 0.8399 0.8173 0.11267 The present invention 0.8466 0.8849 0.8653 0.8495 0.09437

[0119] To illustrate the effectiveness of each module of the logistics order classification model, this example conducts an ablation experiment:

[0120] ① Remove the GCN feature extraction layer based on the global AMR graph and only use the feature representation vectors of the remaining two modules, while the other parts remain unchanged.

[0121] ②Remove the stacked GCN feature extraction layer based on the local AMR image and only use the feature representation vectors of the remaining two modules, while the other parts remain unchanged.

[0122] ③Remove the semantic encoding layer based on the RoBERTa model and only use the feature representation vectors of the remaining two modules, while the other parts remain unchanged.

[0123] The ablation experiment results of each module of the logistics order classification model are shown in Table 4. It can be seen that the various evaluation indicators of the classification model proposed in the present invention are better than ①, ② and ③ (model of the embodiment of the present invention>②>①>③). ③ has the worst effect, which shows that the RoBERTa model has made a great contribution to the feature extraction of order text, and thus can achieve better results. From ②>①, it can be seen that the GCN feature extraction layer based on the global AMR map has a better effect on the feature extraction of order text than the stacked GCN feature extraction layer based on the local AMR map. It can be seen that each module of the model of the present invention can effectively improve the performance of logistics order classification.

[0124] Table 4 Ablation experiment results

[0125]

[0126]

[0127] The present invention proposes the application of graph convolutional neural networks and RoBERTa language models to physical orders in the field of modern logistics, and constructs a logistics order classification method. This method realizes logistics order classification by extracting the fusion features of the global graph, local graph and text semantics of the order text. First, a global AMR graph is constructed based on the global AMR algorithm combined with text keywords, and the structural features of the global AMR graph are extracted using GCN to obtain the global AMR graph representation vector of the order text. Secondly, the order text sentences are processed based on the local AMR algorithm to generate a local AMR graph set, and the local AMR graph set is processed using stacked-GCN to obtain the local AMR graph representation vector set, and the vector set is fused to obtain the local AMR graph representation vector of the order text. Thirdly, the contextual semantic features of the order text are extracted using the RoBERTa model to obtain the semantic representation vector of the order text. Finally, the three types of order text representation vectors are fused, and the order classification is completed using a fully connected network combined with a sigmoid function.

[0128] Those skilled in the art should know that the protection scheme of the present invention is not limited to the above-mentioned embodiments, and various arrangements, combinations and transformations can be made on the basis of the above-mentioned embodiments. Without violating the spirit of the present invention, various transformations of the present invention fall within the protection scope of the present invention.

Claims

1. A logistics order classification method based on graph convolutional neural networks and RoBERTa, including the following steps: S1, constructing a global AMR graph based on the abstract semantic representation AMR results and keywords of the logistics order text, and using a graph convolutional neural network to extract features from the global AMR graph to obtain the global AMR graph representation vector of the order text ;Constructing the global AMR graph includes the following steps: Step 1: Segment the order text to obtain n Sentences, use the keyword extraction algorithm in the Harbin Institute of Technology NLP toolkit to extract the keywords of each sentence in the order text, and get the order keyword set n Keyword set {K i }; Step 2, order text n Sentences are segmented and each sentence of the order text is abstracted into n AMR graph; Step 3, n The AMR root nodes of the sentences are compared, and the root nodes with the same semantics are fused; Step 4, n The keyword set of a sentence {K i } to analyze and process, and merge keywords with the same semantics between different sentences; Step 5, according to n The fusion results of the sentences are used to construct the AMR global graph; S2, based on the AMR algorithm, builds a local AMR graph set of logistics order text sentences, and then uses stacked GCN to process the graph set to obtain the local AMR graph representation vector of the order text The construction of a local AMR graph set for logistics order text sentences based on the AMR algorithm includes the following steps: Step 1: Sentence the order text and get n a sentence; Step 2, n The sentences are processed using the AMR algorithm to obtain n AMR graph; Step 3, n The set of AMR graphs forms a local AMR graph of the order text; S3, use the RoBERTa model to process the logistics order text and obtain the text semantic representation vector ; RoBERTa-wwm-ext is used as the feature extraction model; For a given order text sequence , where x i is each word in the order text, x i After character embedding, segmentation embedding, and position embedding encoding and fusion, the final fused embedding vector is obtained. The sequence of all fused embedding vectors of X is sent to the RoBERTa-wwm-ext model; RoBERTa-wwm-ext performs feature encoding on the fused embedding vector sequence, extracts the contextual semantic information of the input sequence X, and finally outputs the semantic representation vector corresponding to each word of the order text. ;Will s The semantic representation vectors are fused to finally obtain the representation vector of the order text. ; S4, fuses three types of text representation vectors to complete logistics order classification.

2. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 1 is characterized in that: A natural sentence can be parsed into an AMR graph G = (V, E) by an AMR parser; V represents the abstract concept node of the content words in the sentence, and E represents the relationship between a specific content word.

3. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 1 is characterized in that: The convolutional neural network in S1 extracts features from the global AMR image, including: Step 1: Node encoding is performed on the constructed logistics order text relationship graph. For a given logistics order text T, node encoding first needs to perform word segmentation to obtain the word segmentation sequence of the order text. T q , and then use word embedding to parse the sequence T q Encode and obtain the encoding sequence E q : in, and are the keyword embedding vectors in the order text T and the AMR root node, are the embedding vectors of other words; Step 2: To encode the position information of keywords and AMR root nodes in the order text T, position encoding is performed on the order text. The encoding result is as follows: in and are the keywords in the order text T and the position embedding vectors of the AMR root node, are the embedding vectors of other words; Step 3: The node encoding vector is obtained by taking the weighted average of the word embedding vector and the position embedding vector. The calculation formula is as follows: (1) (2) (3) Logistics order text sequence T q The final encoding vector of , according to the nodes in the AMR graph, in the order text sequence encoding pair < T q , E> Find the corresponding node word encoding vector as the embedding vector of the global AMR graph node .

4. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 3 is characterized in that: The GCN network is used for feature extraction, following the GCN layer-by-layer linear propagation rule: (4) in , , A is the adjacency matrix of the input graph, I is the identity matrix, is the initial input feature matrix; Calculate the global AMR feature representation vector of the input order text The calculation formula is as follows: (5) in is the weight matrix from the input layer to the hidden layer, is the weight matrix from the hidden layer to the output layer.

5. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 1, characterized in that, in S2, stacking GCN to process the graph fusion network includes the following steps: Step 1: Build a local AMR map ; Step 2, for each G i Use word embedding and position embedding to encode and get the corresponding word embedding sequence and position embedding sequence ; Step 3: Use weighted average to embed the word into the sequence and position embedding sequence Fusion is performed to finally obtain the corresponding representation vector sequence E i ; Step 4, according to AMR graph G i The node in the input text sequence encoding pair finds the corresponding node word encoding vector as G i Embedding vector of the node , all G i The embedding vectors of the nodes constitute the embedding vector set of the local AMR graph ; Embedding vector set V for local AMR image local Each element in is extracted using the GCN network: (6) (7) in A i is the first i The adjacency matrix of the graph; The representation vectors of multiple sentences constitute the feature representation vector combination of the local AMR graph ， Finally, a gating mechanism is used to fuse these representation vectors: (8) in and All are learnable parameters; fusion vector A local AMR graph representation vector as order text.

6. The logistics order classification method based on graph convolutional neural network and RoBERTa according to any one of claims 1 to 5, characterized in that: In S4, three types of order representation vectors 、 、 Fusion V concat , (9) in is the fusion parameter, satisfying ; Will V concat Multi-label classification of completed orders in the input classifier: (10) Where W and b are learnable parameters, and p is the classification prediction probability of each category; Use cross entropy for multi-label classification as the training loss function: (11) in is the true label, To predict the results, i The classification category of the order.

7. The logistics order classification method based on graph convolutional neural network and RoBERTa according to claim 6 is characterized by It also includes training the model established by the logistics order classification method. First, the historical logistics order text is segmented, sentence-processed, stop words are removed, and keywords are extracted. Then, a logistics order dataset is constructed based on the processing results of historical logistics orders. Then, the dataset is divided into training set, validation set and test set according to a certain ratio. The training set is used for model training, the validation set updates the model performance through continuous iteration, and the test set is used to evaluate the model performance. Finally, the trained model is used to predict the logistics order type.

Images