Navigation text sentiment analysis method based on sentiment enhancement and multi-task learning

Abstract

The invention discloses a shipping text sentiment analysis method based on sentiment enhancement and multi-task learning, which belongs to the field of natural language processing, and comprises the following steps: constructing a shipping text sentiment analysis model; related data in the shipping field is collected and processed to obtain a shipping text data set; performing semantic-level feature extraction on the text content through a RoBERTa model pre-training weight to obtain a coded coarse-grained feature vector; through a two-way propagation mechanism, semantic information of the preamble is embedded into a coarse-grained feature vector; the fine-grained features are transmitted into a residual neural network, and recessive feature representation is obtained; splicing the coarse and fine granularity features through an attention mechanism and endowing corresponding weights to the coarse and fine granularity features; through bottom-layer hard parameter sharing and a mechanism that each task enjoys an independent parameter, the auxiliary tasks are corrected, and the main task discriminates the emotional tendency. According to the method, the coarse and fine granularity features are fully fused while external emotion knowledge is imported for the emotion analysis model, so that the discrimination accuracy of the model is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of natural language processing, in particular to a shipping text sentiment analysis method based on sentiment enhancement and multi-task learning. BACKGROUND

[0002] The text classification task is a hot research direction in the field of natural language processing (NLP) and has wide application value. With the rapid development of the Internet, many shipping organizations or shipping enterprises regularly publish statistical analysis reports of shipping credit indicators online, providing a new way for data acquisition. Based on these online text data, shipping credit evaluation related features can be extracted, and coarse-grained and fine-grained features can be combined to realize sentiment analysis of shipping text. This method not only can more efficiently evaluate the development level of the shipping industry, but also can provide strong support for the optimization of shipping development planning, thereby enhancing the competitiveness of the industry.

[0003] However, in the current research, most methods are usually based on different granularity features and are modeled independently, and the coarse-grained feature model works independently of the fine-grained feature model, which results in the lack of interaction between coarse-grained features and fine-grained features on the one hand, and the lack of embedding of external sentiment information on the other hand. This makes it easy to fall into a local optimum and the model fitting effect is poor.

[0004] Multi-task learning is a machine learning method that aims to learn multiple tasks simultaneously. This method is widely used in many fields such as bioinformatics, computer vision, speech recognition, NLP, etc. The key to multi-task learning is how to construct shared feature representations of each sub-task. The model assigns a gating network to each task, allowing the model to selectively capture the commonalities and differences between each sub-task. Through the exploration and improvement of researchers, the mechanism of parameter soft sharing is also widely recognized. Multi-task learning produces a risk order of overfitting through shared parameters, which is smaller than the risk order of task-related parameter overfitting, and is only equal to the number of tasks. However, when different tasks are in competition, parameter optimization conflicts will occur, which will cause the performance of the task to decline. SUMMARY

[0005] Therefore, the application provides a shipping text sentiment analysis method based on sentiment enhancement and multi-task learning, which can import external sentiment knowledge into the sentiment analysis model while fully integrating coarse and fine-grained features to improve the discrimination accuracy of the model.

[0006] To this end, the application provides the following technical solutions: A shipping text sentiment analysis method based on sentiment enhancement and multi-task learning, comprising the following steps: A shipping text sentiment analysis model is constructed, which includes a data collection module, a coarse-grained extraction module, a fine-grained extraction module, a coarse-fine-grained calibration module, and a shipping credit evaluator module; Collecting relevant data in the shipping field, and integrating and labeling the data to obtain a shipping text dataset, and obtaining webpage tags for representing classification categories; Through the RoBERTa model pre-training weight, the semantic level feature of the text content is extracted, and the encoded coarse-grained feature vector is obtained; The output of the RoBERTa model is taken as the input of the BiLSTM, and the semantic information of the preceding and following texts is embedded into the coarse-grained feature vector through the bidirectional propagation mechanism; The fine-grained features are transmitted into the residual neural network, and more rich implicit feature representations are obtained through the skip connection processing of the residual neural network; The coarse-grained features and fine-grained features are spliced through the attention mechanism and are given corresponding weights; Through the mechanism of bottom layer hard parameter sharing and independent parameters for each task, the auxiliary task is corrected, and the main task makes a judgment on the sentiment tendency.

[0007] Further, the collection of relevant data in the shipping field includes obtaining part of the original data of shipping credit from the transportation-related departments and collecting a large amount of shipping evaluation data from online shipping websites; the shipping text dataset includes text and three-element sentiment labels of the text, and the sentiment dictionary contains various sentiment words.

[0008] Further, the output of the RoBERTa model is taken as the input of the BiLSTM, and the semantic information of the preceding and following texts is embedded into the coarse-grained feature vector through the bidirectional propagation mechanism; specifically including: The RoBERTa model and BiLSTM are used to represent the features in vectors; the input text is , and the word vector is obtained after Token embedding, Segment embedding, and Position embedding: (1) , , , Token embedding vector, Segment embedding vector, and Position embedding vector, respectively, is the integrated vector; Then, the is subjected to Segment embedding and Position embedding, and and are obtained, wherein ; The above task retains the key local features of word vectors. In order to combine context information, BiLSTM is used to transform word vectors. Embed emotional information in both positive and negative directions of the context; Through the forget gate in a single LSTM molecule Control the degree of retention of emotional information in the previous step memory unit: (2) in, represents the sigmiod function, W f represents the parameter matrix, h t-1 Indicates the memory status of the previous moment. x t Represents the input at this moment, b f is the bias of the function; The input gate finds the corresponding new attribute information in the coarse-grained extraction module for the attribute information discarded in the forget gate and adds it to supplement the discarded attribute information; the calculation principle is to accept the long-term memory output from the forget gate , and the short-term memory output from the learning gate , then merge the two directly and update ,get The calculation formula is as follows: (3) (4) (5) in, represents the output value of the input gate, represents the input unit of the memory unit at this moment, is the hyperbolic tangent activation function, and are their respective bias vectors, Represents the internal state of the memory unit at this moment; The output gate is calculated by the sigmiod function and The part that needs to be output is then input into the gate to obtain After being processed by the tanh function, the two are multiplied together to obtain: (6) (7) in, Represents the output value of the output gate, It indicates the memory status at this moment.

[0009] Further: the fine-grained features are input into the residual neural network, and the more rich hidden feature representations are obtained through the skip connection processing of the residual neural network; specifically including: The residual neural network is used to model the fine-grained features to obtain the embedding representation of the fine-grained features: (8) Wherein, The matrix is the weight matrix of the convolution layer in the residual network, is the input signal directly transmitted by the skip connection.

[0010] Further: the coarse-grained features and the fine-grained features are spliced and given corresponding weights through the attention mechanism; specifically including: In terms of coarse and fine-grained calibration, the attention mechanism is used to obtain the weights: (9) Wherein, The query vector is represented by q, The key vector is represented by k, The value vector is represented by v, The dimension size of ; The self-attention score value calculated for i times is spliced and linearly transformed to obtain an enhanced semantic vector with the same length as the original word vector, which is the output of the Multi-head attention layer, and the specific calculation method is: (10) (11) Wherein, The learnable weight matrix is represented by W, , , The projection matrix of the query vector, the key vector and the value vector is represented by Wq, Wk and Wv respectively, The output obtained after splicing is represented by y; Different features in the feature set have different effects on the representation of the sentence; through the right residual neural network, first, the hidden context representation of the feature is obtained, and the splicing of the word embedding vector, the position embedding vector and the part of speech embedding vector is obtained through the right BiLSTM to obtain the hidden context representation of the sentence ; Finally, the attention weight distribution of each feature corresponding to the sentence is calculated; the related calculation formula is: (12) (13) (14) where, represents the contextual awareness of the aspect term; i represents the contextual hidden state of the th word; j is the attention weight from the th word in the aspect term to the i th word in the sentence; j is the initial representation of the aspect term, tanh is a nonlinear activation function; i and are trainable parameters; The hidden representation of the attention output from the aspect to the sentence is further assigned different weights, reducing the number of learnable parameters, improving the accuracy of the shipping text sentiment analysis model, and filtering out redundant information and noise in the original text. The calculation process is: (15) (16) (17) (18) where, is the attention weight from the sentence to the word in the aspect term; is obtained by the BiLSTM hidden state; is the weight matrix; is the bias; is the output after attention.

[0011] Further, through the mechanism of bottom layer hard parameter sharing and independent parameters for each task, the auxiliary task is corrected and the main task makes a judgment on the sentiment tendency; Specifically, it includes: In the shipping credit evaluator, the expert network is divided into multiple groups, and each task has a group of expert networks responsible for mining knowledge specific to the corresponding task, while all tasks share a group of expert networks responsible for mining knowledge shared by all tasks; For task k , it combines the output of the corresponding task-specific expert network group and the output of the expert network group shared by all tasks through the corresponding task-specific gating network; Let the original input be x , the output of the expert network group specific to task k and the output of the expert network group shared by all tasks be : (19) and then get​ The process is represented as: (20) wherein, ; The outputs of each expert network are then weighted and summed based on the above weights to obtain the task k The input of the special network tower: (21) Finally, the task k The special network tower The output estimated value: (22).

[0012] Further, the coarse-grained extraction module in the shipping text sentiment analysis model is used to extract features at the semantic level of the text content through the pre-training weight of the RoBERTa model, to obtain the encoded coarse-grained feature vector; the output of RoBERTa is used as the input of BiLSTM, and the semantic information of the preceding and following texts is embedded in the coarse-grained feature vector through the bidirectional propagation mechanism thereof.

[0013] Further, the fine-grained extraction module in the shipping text sentiment analysis model is used to input the fine-grained features existing in the shipping text data set into the residual neural network, to obtain more rich implicit feature representations through the skip link processing of the residual neural network; the fine-grained features are added to the discrimination of sentiment tendency, to improve the discrimination effect of the shipping text sentiment analysis model.

[0014] Further, the coarse and fine-grained calibration module in the shipping text sentiment analysis model is used to splice the coarse-grained features and the fine-grained features, and then assign weights through the attention mechanism, to balance the weights between the coarse-grained features and the fine-grained features.

[0015] Further, the shipping credit evaluator module in the shipping text sentiment analysis model comprises a main task and an auxiliary task; through the mechanism of bottom hard parameter sharing and independent parameters for each task, the main task discriminates the sentiment tendency, and the auxiliary task corrects the discrimination result of the main task.

[0016] Advantages and positive effects of the present application: 1. The present application constructs a special data set in the shipping field, and constructs a shipping sentiment dictionary on this basis, introduces the sentiment dictionary into the shipping text sentiment analysis model through the sentiment embedding matrix, so that the shipping text sentiment analysis model can learn external sentiment knowledge more fully.

[0017] 2、The present application designs a coarse and fine granularity calibration module, which fuses and calibrates coarse granularity features and fine granularity features, and through a self-attention mechanism, the weights of coarse granularity and fine granularity are allocated, so that the extraction capability of the shipping text sentiment analysis model on context information and keyword information is greatly improved.

[0018] 3、The present application designs a shipping credit evaluator module containing a main task and an auxiliary task, and the main structure of the module is a multi-task learning framework, through a hard parameter sharing mechanism, the main task and the auxiliary task are combined with each other, so that the parameter overfitting is effectively avoided, and the discrimination accuracy of the shipping text sentiment analysis model is improved.

[0019] 4、The present application overcomes the limitations of the prior art, introduces external sentiment knowledge for the sentiment analysis model, fully fuses coarse granularity features and fine granularity features, and improves the discrimination accuracy of the shipping text sentiment analysis model. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 A flowchart of a shipping text sentiment analysis method based on sentiment enhancement and multi-task learning provided in an embodiment of the present application; Figure 2 A schematic diagram of a shipping text sentiment analysis model based on sentiment enhancement and multi-task learning provided in an embodiment of the present application; Figure 3 A detailed table of experimental results provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0024] The specific embodiments of the present application are further illustrated in the following drawings and technical solutions.

[0025] As Figure 1 shown, a shipping text sentiment analysis method based on emotion enhancement and multi-task learning, specifically includes the following contents: Step 1, a shipping text sentiment analysis model including a data collection module, a coarse-grained extraction module, a fine-grained extraction module, a coarse-fine-grained calibration module and a shipping credit evaluator module is constructed; Specifically: (1) Data collection module: collect relevant data in the shipping field, one part of the data comes from the materials provided by the transportation department, and the other part comes from the shipping online website. The obtained data is integrated and labeled to obtain a shipping text data set (Shipping Evaluation, SE). The obtained shipping evaluation text is used to represent the classification category; the constructed shipping text data set includes text and sentiment labels.

[0026] (2) Coarse-grained extraction module: the RoBERTa model pre-training weight is used to extract the semantic level features of the text content, and the encoded feature vector is obtained. Then the output of the RoBERTa model is taken as the input of the BiLSTM, and the semantic information of the front and rear texts is embedded into the feature vector through the bidirectional propagation mechanism of the BiLSTM.

[0027] (3) Fine-grained extraction module: fine-grained features exist in the shipping text data set SE, which are input into the residual neural network to obtain more rich implicit feature representation through the skip connection processing of the residual neural network. The fine-grained features are added to the discrimination of sentiment tendency to improve the discrimination effect of the model.

[0028] (4) Coarse-fine-grained calibration module: in order to balance the weight between coarse-grained features and fine-grained features, a coarse-fine-grained calibration module is introduced, which firstly splices the coarse-grained features and the fine-grained features, and then assigns weights through the attention mechanism.

[0029] (5) Shipping Credit Evaluator Module: The shipping credit evaluator module contains two tasks: the main task and the auxiliary task. Through the mechanism of sharing underlying hard parameters and each task having independent parameters, the main task makes a judgment on the sentiment tendency, and the auxiliary task makes a correction to the judgment result of the main task.

[0030] Step 2: Collect relevant data in the shipping field, integrate and annotate the data to obtain a shipping text dataset, and obtain web page labels used to represent classification categories; Specifically, we obtain some original data of shipping credit from relevant transportation departments. The Shipping Online website has collected a large amount of shipping evaluation data, totaling more than 9,000 samples. The obtained data was integrated, labeled, and preprocessed to obtain the Shipping Evaluation Dataset (SE). On this basis, a sentiment dictionary was formed for this dataset, which includes words that tend to represent various emotions.

[0031] To evaluate the annotation quality of the shipping text dataset, an experimental validation experiment was conducted. First, three groups of samples, each containing 100 examples, were randomly sampled from the dataset. Subsequently, three experts in the shipping field were trained in sentiment annotation and each was asked to independently annotate a set of examples. After obtaining the experts' manual annotation results, Cohen's Kappa was used to assess the agreement between the experts and the initial annotations.

[0032] The evaluation results show that the Cohen's Kappa values ​​for the three groups of samples are 0.86, 0.88, and 0.82, respectively, with an average of 0.853. This result indicates that the annotation quality of the dataset is high, with strong reliability and consistency.

[0033] Step 3: Use the pre-trained weights of the RoBERTa model to extract semantic-level features from the text content and obtain the encoded coarse-grained feature vector; Specifically, the RoBERTa model is generated through full-word masking training and fine-tuned on this basis. After fine-tuning, it is passed to the BiLSTM layer for contextual information embedding.

[0034] Step 4: Use the output of the RoBERTa model as the input of the BiLSTM, and embed the semantic information of the context into the coarse-grained feature vector through a bidirectional propagation mechanism; Specifically, the RoBERTa model and BiLSTM are used to vectorize the features. The input text is , after Token embedding, Segment embedding, and Position embedding, the word vector is obtained : (1) wherein, , , Token embedding vector, Segment embedding vector, Position embedding vector, respectively, is the integrated vector.

[0035] The above task retains the key local features of the word vector. In order to combine the context information, the BiLSTM is used to embed the word vector in the positive and negative directions of the context.

[0036] The forgetting gate in a single LSTM molecule controls the degree of retention of the emotional information in the previous memory unit: (2) wherein, sigmiod function, W f is the parameter matrix, h t-1 is the memory state at the previous time, x t is the input at this time, b f is the bias of the function.

[0037] The input gate finds the corresponding new attribute information in the coarse-grained extraction module for the attribute information discarded in the forgetting gate, and adds it to make up for the discarded attribute information. The calculation principle is to accept the long-term memory output from the forgetting gate, and the short-term memory output from the learning gate, and then directly combine the two to update to obtain ; the calculation formula is as follows: (3) (4) (5) wherein, is the output value of the input gate, is the input unit of the memory unit at this time, is the hyperbolic tangent activation function, and are the respective bias vectors, is the internal state of the memory unit at this time.

[0038] ​The output gate is calculated by a sigmoid function With The part of the output required in the input gate, and then input it into the Through the tanh function processing, and then multiply the two to get: (6) (7) Wherein, The output value of the output gate, Indicates the memory state at this moment.

[0039] Step 5, the fine-grained features are input into the residual neural network, and the more rich implicit feature representation is obtained through the skip connection processing of the residual neural network; Specifically, the fine-grained features are manually labeled under certain scoring rules; the residual neural network is used to model the fine-grained features to obtain the embedding representation of the fine-grained features: (8) Wherein, The matrix is the weight matrix of the convolution layer in the residual network, Is the input signal directly transmitted by the skip connection.

[0040] Step 6, the coarse-grained features and fine-grained features are spliced and given corresponding weights through the attention mechanism; Specifically, the coarse and fine-grained calibration is completed through the multi-head attention mechanism, wherein the input of the multi-head attention mechanism is the splicing of the coarse and fine-grained features, The weighted value is obtained through the attention mechanism: (10) Wherein, Indicates the query vector, Indicates the key vector, Indicates the value vector, Indicates The dimension size of

[0041] The self-attention score value calculated for i times is spliced and linearly transformed to finally obtain an enhanced semantic vector with the same length as the original word vector, which is used as the output of the Multi-head attention layer, and the specific calculation method is shown in formula (11) and formula (12): (11) (12) In the formula, denote a learnable weight matrix, , , denote projection matrices of query vector, key vector and value vector respectively, denote the output after splicing.

[0042] Different features in the feature set have different effects on the representation of the sentence. Through the residual neural network on the right, the hidden context representation of the feature is first obtained, and the splicing of the word embedding vector, the position embedding vector and the part-of-speech embedding vector is obtained through the BiLSTM on the right to obtain the hidden context representation of the sentence . Finally, the attention weight distribution of each feature corresponding to the sentence is calculated. The related calculation formula is: (13) (14) (15) wherein, denote the context awareness of the aspect term; i denote the context hidden state of the th word; j is the attention weight from the th word in the aspect term to the i th word in the sentence; j is the initial representation of the aspect term , tanh is a nonlinear activation function; i and are trainable parameters; The hidden representation of the attention output from the aspect to the sentence is further assigned different weights, the number of learnable parameters is reduced, the discrimination accuracy of the shipping text sentiment analysis model is improved, and the redundant information and noise existing in the original text are filtered out, and the calculation process is: (16) (17) (18) (19) wherein, is the attention weight of the word in the aspect term to the sentence; is obtained through the BiLSTM hidden state; is a weight matrix; is a bias; is the attention output.

[0043] ​Step 7, through the mechanism of bottom-layer hard parameter sharing and each task enjoying independent parameters, auxiliary tasks are assisted to make corrections, and the main task makes a judgment on the sentiment tendency.

[0044] Specifically, in the shipping credit evaluator, the expert network is divided into multiple groups, and each task has a group of expert networks responsible for mining knowledge specific to the task, while all tasks share a group of expert networks responsible for mining knowledge shared by all tasks. For the task k , the output of the task-specific expert network group and the output of the all-task-shared expert network group are combined through the task-specific gating network. Let the original input be x , the output of the task-specific expert network group and the output of the all-task-shared expert network group are k : (20) Then, through the Softmax function, get weights, and the process can be represented as: (21) wherein, .

[0045] Then, based on the above weights, the outputs of each expert network are weighted and summed to obtain the input of the task k specific network tower: (22) Finally, the task k specific network tower outputs the estimated value: (23) As Figure 3 ​As shown, the accuracy of the shipping text sentiment analysis model provided by the present application is better than that of the benchmark model on the four data sets, and the accuracy of the model based on the BERT framework is generally higher than that of the early model, indicating that the early model cannot effectively integrate the context information, resulting in the absence of some key context features in the model, and a large amount of noise makes the model unable to accurately extract effective information while increasing the model parameter quantity. Ultimately, it leads to poor model effect. The accuracy of most models on the ChnSentiCorp, waimai and SE three data sets is relatively high, but it is low on the restaurant data set. The reason may be that the model used is a Chinese model, and the data set is English, which leads to poor compatibility effect. Compared with the mainstream RoBERTa model, the shipping text sentiment analysis model provided by the present application improves by 2.52, 10.57, 2.61 and 4.02 percentage points on the four data sets respectively. The shipping text sentiment analysis model provided by the present application introduces external sentiment knowledge while increasing the coarse and fine-grained calibration and multi-task learning mechanism (of which the coarse and fine-grained calibration is only applicable to the SE data set containing fine-grained features), fully integrates the context feature information, and also embeds external sentiment knowledge, effectively improving the sentiment analysis performance of the model.

[0046] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A shipping text sentiment analysis method based on sentiment enhancement and multi-task learning, characterized by: The following steps are involved: Construct a shipping text sentiment analysis model that includes a data collection module, a coarse-grained extraction module, a fine-grained extraction module, a coarse-grained and fine-grained calibration module, and a shipping credit assessor module; Collect relevant data in the shipping field, integrate and annotate the data to obtain a shipping text dataset, and obtain web page labels used to represent classification categories; The pre-trained weights of the RoBERTa model are used to extract semantic features from the text content and obtain the encoded coarse-grained feature vector. The output of the RoBERTa model is used as the input of BiLSTM, and the semantic information of the context is embedded into the coarse-grained feature vector through a bidirectional propagation mechanism. The fine-grained features are passed into the residual neural network, and through the skip link processing of the residual neural network, a richer implicit feature representation is obtained; The coarse-grained features and fine-grained features are spliced ​​together through the attention mechanism and assigned corresponding weights; Through the mechanism of sharing underlying hard parameters and allowing each task to have independent parameters, the auxiliary tasks are corrected and the main task makes judgments on emotional tendencies.

2. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The collection of relevant data in the shipping field includes obtaining some original data of shipping credit from transportation-related departments and collecting a large amount of shipping evaluation data from shipping online websites; the shipping text dataset includes text and text triple sentiment tags, and the sentiment dictionary contains various sentiment words.

3. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The output of the RoBERTa model is used as the input of the BiLSTM. Through the bidirectional propagation mechanism, the semantic information of the context is embedded into the coarse-grained feature vector. Specifically, Use RoBERTa model and BiLSTM to represent features as vectors; the input text is , after Token embedding, Segment embedding, and Position embedding, the word vector is obtained : （1） in, , , They are Token embedding vector, Segment embedding vector, and Position embedding vector respectively. is the integration vector; Then Perform Segment embedding and Position embedding to obtain and ,in The above task retains the key local features of word vectors. In order to combine context information, BiLSTM is used to transform word vectors. Embed emotional information in both positive and negative directions of the context; Through the forget gate in a single LSTM molecule Control the degree of retention of emotional information in the previous step memory unit: （2） in, represents the sigmiod function, W f represents the parameter matrix, h t-1 Indicates the memory status of the previous moment. x t Represents the input at this moment, b f is the bias of the function; The input gate finds the corresponding new attribute information in the coarse-grained extraction module for the attribute information discarded in the forget gate and adds it to supplement the discarded attribute information; the calculation principle is to accept the long-term memory output from the forget gate , and the short-term memory output from the learning gate , then merge the two directly and update ,get The calculation formula is as follows: （3） （4） （5） in, represents the output value of the input gate, represents the input unit of the memory unit at this moment, is the hyperbolic tangent activation function, and are their respective bias vectors, Represents the internal state of the memory unit at this moment; The output gate is calculated by the sigmiod function and The part that needs to be output is then input into the gate to obtain After being processed by the tanh function, the two are multiplied together to obtain: （6） （7） in, Represents the output value of the output gate, It indicates the memory status at this moment.

4. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The fine-grained features are passed into the residual neural network, and through the skip link processing of the residual neural network, a richer implicit feature representation is obtained; specifically, Use residual neural network to model fine-grained features and obtain embedded representation of fine-grained features: （8） in, The matrix is ​​the weight matrix of the convolutional layer in the residual network, is the input signal directly passed by the skip connection.

5. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The coarse-grained features and fine-grained features are spliced ​​together and assigned corresponding weights through the attention mechanism; specifically, In terms of coarse-grained and fine-grained calibration, the following weights are obtained through the attention mechanism: （9） in, represents the query vector, represents the key vector, represents a value vector, express The dimension size of will pass i The self-attention score values ​​calculated twice are concatenated and linearly transformed to finally obtain an enhanced semantic vector with the same length as the original word vector, which is used as the output of the Multi-head attention layer. The specific calculation method is: （10） （11） in, represents the learnable weight matrix, , , Represent the projection matrices of query vector, key vector and value vector respectively, Represents the output after splicing; Different features in the feature set have different effects on the representation of the sentence. Through the residual neural network on the right, the hidden context representation of the feature is first obtained, and the word embedding vector, position embedding vector and part-of-speech embedding vector are concatenated and passed through the BiLSTM on the right to obtain the hidden context representation of the sentence. ; Finally, calculate the attention weight distribution of each feature corresponding to the sentence; the relevant calculation formula is: （12） （13） （14） in, Indicates the i Contextual awareness of aspect terms; Indicates the j The contextual hidden state of each word; For the first i word to the first word in the sentence j The attention weight of each word; Aspect terms i The initial representation of , tanh is a nonlinear activation function; and is a trainable parameter; Different weights are further assigned to the hidden representations of the attention output from aspects to sentences to reduce the number of learnable parameters, improve the accuracy of the shipping text sentiment analysis model, and filter out redundant information and noise in the original text. The calculation process is: （15） （16） （17） （18） in, Attention weights for words in sentences to aspect terms; is obtained through the BiLSTM hidden state; is the weight matrix; is bias; is the output after attention.

6. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: By sharing underlying hard parameters and allowing each task to have independent parameters, the auxiliary task makes corrections and the main task makes judgments on the emotional tendency. Specifically, In terms of shipping credit evaluator, the expert network is divided into multiple groups. Each task has a dedicated group of expert networks responsible for mining the knowledge specific to the corresponding task, while all tasks share a group of expert networks responsible for mining the knowledge shared by all tasks. k , which merges the output of the expert network group specific to the corresponding task and the output of the expert network group shared by all tasks through the gating network specific to the corresponding task; Let the original input be x ,Task k The output of the dedicated expert network group and the output of the expert network group shared by all tasks are : （19） Then we get it through the Softmax function The weights are expressed as: （20） in, ; Then, based on the above weights, the outputs of each expert network are weighted summed to obtain the task k Input of the VPC tower: （21） Finally, the task k Private network tower Output estimate: （22）。 7. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The coarse-grained extraction module in the shipping text sentiment analysis model is used to extract semantic-level features from the text content using the pre-trained weights of the RoBERTa model to obtain an encoded coarse-grained feature vector. The RoBERTa output is then used as the input of the BiLSTM, and the semantic information of the context is embedded in the coarse-grained feature vector through its bidirectional propagation mechanism.

8. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The fine-grained extraction module in the shipping text sentiment analysis model is used to transfer the fine-grained features existing in the shipping text dataset into the residual neural network, and obtain a richer implicit feature representation through the skip link processing of the residual neural network; the fine-grained features are added to the discrimination of emotional tendencies to improve the discrimination effect of the shipping text sentiment analysis model.

9. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The coarse-grained and fine-grained calibration module in the shipping text sentiment analysis model is used to splice coarse-grained features with fine-grained features, and then assign weights through the attention mechanism to balance the weights between coarse-grained features and fine-grained features.

10. The shipping text sentiment analysis method based on sentiment enhancement and multi-task learning according to claim 1 is characterized in that: The shipping credit evaluator module in the shipping text sentiment analysis model includes a main task and an auxiliary task. Through the mechanism of sharing underlying hard parameters and each task having independent parameters, the main task makes a judgment on the sentiment tendency, and the auxiliary task makes a correction to the judgment result of the main task.

Images