Emotion analysis model training method and device, storage medium and electronic equipment

Abstract

The invention discloses an emotion analysis model training method and device, a storage medium and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a target sample belonging to a minority emotion category in an original text set; if the quantitative proportion of the target samples is smaller than a preset proportion, context texts of the target samples are extracted to determine enhanced samples, and the enhanced samples and the original text set are jointly used as training samples; determining an initial sentiment analysis model and target category weights corresponding to different sentiment categories; inputting the training sample into the initial sentiment analysis model for aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample; and calculating a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample and target category weights corresponding to different emotion categories, so as to carry out iterative training on the initial emotion analysis model. The problem of class imbalance in the sentiment analysis model training process can be solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a sentiment analysis model training method, device, storage medium and electronic device. Background Art

[0002] The basic task of sentiment analysis is to extract and identify emotional information from text. With the rise of pre-trained language models such as BERT, sentiment analysis technology has made significant breakthroughs.

[0003] At present, in sentiment analysis tasks, samples of different sentiment categories are usually used to train sentiment analysis models. However, the number of samples of different sentiment categories in the sample training set will have a class imbalance problem. For example, the number of samples of negative and neutral sentiment categories is usually less than the number of samples of positive sentiment categories. This imbalance will cause the model to learn less about minority categories during training and tend to predict majority categories, thus affecting the overall classification performance of the sentiment analysis model and failing to guarantee sentiment classification accuracy. Summary of the invention

[0004] In view of this, the present application provides a sentiment analysis model training method, device, storage medium and electronic device, the main purpose of which is to solve the problem of category imbalance in the sentiment analysis model training process, thereby improving the sentiment classification accuracy of the model.

[0005] According to the first aspect of the present application, a sentiment analysis model training method is provided, the method comprising: Determine target samples belonging to minority sentiment categories in the original text collection, and the quantity ratio of the target samples in the original text collection; If the quantity ratio is less than a preset ratio, determining the semantic relevance between the context text corresponding to the target sample and the target sample; Based on the semantic relevance, an enhanced sample corresponding to the target sample is determined, and the enhanced sample and the original text set are used as training samples together, wherein if it is determined according to the semantic relevance that the context text is semantically related to the target sample, the enhanced sample is generated based on the context text; if it is determined according to the semantic relevance that the context text is semantically irrelevant to the target sample, the target sample is replaced with a synonym to generate the enhanced sample; Determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories, wherein the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories; Inputting the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample; Calculating a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample and the target category weights corresponding to the different emotion categories; According to the weighted cross entropy loss function value, the initial sentiment analysis model is iteratively trained to construct a preset sentiment analysis model.

[0006] According to a second aspect of the present application, a sentiment analysis model training device is provided, the device comprising: A first determination unit is used to determine target samples belonging to a minority sentiment category in an original text set, and a quantity ratio of the target samples in the original text set; The first determining unit is further configured to determine the semantic relevance between the context text corresponding to the target sample and the target sample if the quantity ratio is less than a preset ratio; The first determination unit is further used to determine an enhanced sample corresponding to the target sample based on the semantic relevance, and use the enhanced sample and the original text set as training samples, wherein if the context text is determined to be semantically relevant to the target sample according to the semantic relevance, the enhanced sample is generated based on the context text; if the context text is determined to be semantically irrelevant to the target sample according to the semantic relevance, the target sample is replaced with a synonym to generate the enhanced sample; A second determination unit is used to determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories, wherein the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories; An analysis unit, configured to input the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample; A calculation unit, used to calculate a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample, and the target category weights corresponding to the different emotion categories; The training unit is used to iteratively train the initial sentiment analysis model according to the weighted cross entropy loss function value to construct a preset sentiment analysis model.

[0007] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned sentiment analysis model training method is implemented.

[0008] According to the fourth aspect of the present application, an electronic device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned sentiment analysis model training method when executing the program.

[0009] By means of the above technical solution, the present application provides a sentiment analysis model training method, device, storage medium and electronic device. Compared with the prior art, by determining the semantic relevance between the context text and the target sample, and performing data enhancement according to the semantic relevance, the enhanced sample corresponding to the target sample is determined, which can increase the diversity and coverage of minority class samples. At the same time, the present application introduces a weighted cross entropy loss function, in which the category weights in the weighted cross entropy loss function can be dynamically adjusted according to the category distribution, that is, the category weights corresponding to the minority emotion category are made greater than the category weights corresponding to other emotion categories, thereby reducing the role of the majority category in the overall loss and improving the model's discrimination performance for minority category samples, thereby effectively reducing the impact of category imbalance during model training and improving the model's sentiment classification accuracy.

[0010] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 A flow chart of a sentiment analysis model training method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of the architecture of the sentiment analysis model provided in the embodiment of the present application is shown; Figure 3 A schematic diagram of the process of feature extraction and analysis provided by an embodiment of the present application is shown; Figure 4 A schematic diagram of the analysis results based on squid provided in an embodiment of the present application is shown; Figure 5 A schematic diagram of attention weight based on Squid provided in an embodiment of the present application is shown; Figure 6 A schematic diagram of analysis results based on appetizers provided in an embodiment of the present application is shown; Figure 7 A schematic diagram of attention weight based on appetizers provided in an embodiment of the present application is shown; Figure 8 A schematic diagram of meal-based analysis results provided in an embodiment of the present application is shown; Fig. 9A schematic diagram of meal-based attention weights provided in an embodiment of the present application is shown; Fig.10 A structural schematic diagram of a sentiment analysis model training device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0012] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0013] The number of samples of different sentiment categories in the sample training set will have the problem of category imbalance. This imbalance will cause the model to insufficiently learn the minority categories during the training process and tend to predict the majority categories, thus affecting the overall classification performance of the sentiment analysis model and failing to guarantee the accuracy of sentiment classification.

[0014] In order to solve the above problems, the embodiments of the present invention provide a sentiment analysis model training method, device, storage medium and electronic device, which are mainly applicable to aspect-based sentiment analysis, such as Figure 1 As shown, the method includes: Step 10: Determine the target samples belonging to the minority sentiment category in the original text collection, and the quantity ratio of the target samples in the original text collection.

[0015] The original text set includes multiple texts used as samples. The description languages ​​used in the multiple texts can be Chinese, English or other languages. The types of the multiple texts can specifically be commentary texts, opinion texts, news texts, descriptive texts, etc. The sentiment categories specifically include negative, positive and neutral. It should be noted that the sentiment categories of the embodiments of the present invention can be divided according to actual business needs and are not limited to the above-mentioned negative, positive and neutral sentiment categories. In addition, the target sample is the text belonging to a minority sentiment category in the original text set. For example, there are 100 texts in the original text set, of which only 10 are negative texts. These 10 negative texts are the target samples.

[0016] For the embodiment of the present invention, aspect-based sentiment analysis aims to extract specific aspects from the text, such as the attributes of a product, and determine its sentiment polarity (sentiment classification). Get an original text set consisting of N comments ,in, Indicates i Article text, Indicates the areas of concern in this text. Represents the corresponding sentiment polarity label (sentiment classification). The main purpose of the embodiment of the present invention is to learn a mapping function f , so that: f ( S i , A i ; θ ) = y i in, θ is the model parameter.

[0017] To achieve this goal, the embodiment of the present invention constructs a preset sentiment analysis model (XAI-BERT-MSO) so that it can effectively represent the input text. S i and aspects A i , and capture the semantic relationship between them. After learning the joint representation of text and aspects in a high-dimensional space, the model can accurately classify sentiment polarity.

[0018] Since sentiment polarity (sentiment classification) is usually expressed in text form, in order to enable the model to process and learn sentiment labels in the sentiment analysis task, the embodiment of the present invention converts these text labels into numerical form, that is, performs label encoding. Based on this, the embodiment of the present invention defines a mapping function to map sentiment classification labels (such as positive, negative, and neutral) into numerical values ​​respectively. =0,1,2, where Indicates i The numeric label for the item.

[0019] To facilitate model training, the embodiment of the present invention further converts the numerical label into a one-hot encoding vector y i ,in, K =3, K Represents the number of sentiment categories, the specific form is: y i = [ δ (1, y i + 1) , δ (2, y i + 1) , δ (3, y i + 1)] ⊤ in, δ ( k , y i +1) represents the Kronecker delta function, when k = y i+ 1, the value is 1, otherwise it is 0. This encoding method of the embodiment of the present invention is concise and effective, and can facilitate the loss calculation and back propagation of the model.

[0020] Furthermore, in order to solve the problem of category imbalance in the training process of the sentiment analysis model, the embodiment of the present invention needs to count the number ratio of target samples belonging to minority sentiment categories in the original text set, and when the number ratio is less than a certain value, sample enhancement is performed.

[0021] Before performing sample enhancement, the embodiment of the present invention also needs to perform data cleaning on the original text set, that is, to clean and standardize the original text data to eliminate noise and improve the performance and generalization ability of the model. For this process, the method includes: converting multiple texts in the original text set into lowercase form to obtain multiple converted texts; deleting the uniform resource locator, social media identifier and redundant spaces in the converted multiple texts to obtain multiple preprocessed texts; determining the preprocessed multiple texts as the cleaned text set. At the same time, determine the target samples belonging to the minority emotion category in the cleaned text set, and the number ratio of the target samples in the cleaned text set. Among them, the uniform resource locator can refer to a URL link.

[0022] Specifically, for each text S i Preprocessing is performed, including converting each text to lowercase, removing URL links and social media identifiers, and finally eliminating extra spaces. After the above processing, the preprocessed text is obtained , which is specifically expressed as follows:

[0023] in, Indicates the combined operation of the above preprocessing steps. After these processes, the text data of the embodiment of the present invention is standardized, laying a foundation for subsequent feature extraction and model training.

[0024] Step 20: If the quantity ratio is less than a preset ratio, determine the semantic relevance between the context text corresponding to the target sample and the target sample.

[0025] The preset ratio may be set according to actual business requirements, and the embodiment of the present invention does not specifically limit this. For example, the preset ratio is set to 20%.

[0026] In the sentiment analysis task of the embodiment of the present invention, category imbalance is a common problem. In particular, the number of samples in the negative and neutral sentiment categories is usually less than that in the positive category. This imbalance will cause the model to insufficiently learn the minority categories during training and tend to predict the majority category, thereby affecting the overall classification performance.

[0027] In order to overcome the above problems, the embodiment of the present invention introduces semantically related context sentences when the proportion of the number of target samples of minority emotion categories is less than a preset proportion, so as to increase the number and semantic information of minority category samples and improve the model's recognition ability for minority categories. In order to determine the context sentences semantically related to the target sample, it is necessary to calculate the semantic relevance between the context text and the target sample. For the calculation process of the semantic relevance, the method includes: determining the embedding vector corresponding to the context text and the embedding vector corresponding to the target sample; based on the embedding vector corresponding to the context text and the embedding vector corresponding to the target sample, calculating the similarity between the context text and the target sample; and determining the semantic relevance between the context text and the target sample based on the similarity.

[0028] Specifically, the context sentences are extracted from the original text collection for each target sample of a few sentiment categories. and , that is, the context text. When extracting the context text, it is necessary to consider the boundary conditions. i =1, extract only text ;when i = N When only text is extracted .

[0029] Furthermore, in order to ensure the semantic relevance between the context text and the target sample, an embodiment of the present invention uses the CLS vector of the BERT model to embed the sentence, and calculates the similarity between the context text and the target sample, and then determines the semantic relevance between the context text and the target sample based on the similarity.

[0030] Step 30: Based on the semantic relevance, determine the enhanced sample corresponding to the target sample, and use the enhanced sample and the original text set as training samples.

[0031] Among them, if it is determined according to the semantic relevance that the context text is semantically related to the target sample, the enhanced sample is generated based on the context text; if it is determined according to the semantic relevance that the context text is semantically unrelated to the target sample, the target sample is replaced with a synonym to generate the enhanced sample.

[0032] For the embodiment of the present invention, after calculating the similarity between the context text and the target sample, if the similarity is greater than the preset similarity, it is determined that the context text is semantically related to the target sample, and the context text and the target sample are spliced ​​to generate an enhanced sample corresponding to the target sample; if the similarity is less than or equal to the preset similarity, it is determined that the context text is semantically irrelevant to the target sample, and the target sample is replaced with a synonym, and the target sample after the synonym replacement is spliced ​​with the target sample to generate an enhanced sample corresponding to the target sample. Among them, the preset similarity can be set according to actual business needs, and the embodiment of the present invention does not specifically limit this.

[0033] Specifically, when the similarity is greater than the preset similarity, the target sample is concatenated with the context text to form an enhanced sample. , expressed as follows: =

[0034] Enhanced Samples The sentiment label of the target sample is retained , to ensure label consistency.

[0035] The embodiment of the present invention can ensure that the generated text remains semantically coherent by selecting context sentences that are semantically related to the target sample for splicing.

[0036] For example, assuming that the original target sample for" The steak was overcooked and dry. ”, its sentiment label is negative, the context text for" The ambiance of the restaurant was delightful. ”, context text for" However, the dessert was a highlight of the meal. ”, the enhanced samples generated by data augmentation are “ The ambiance of the restaurant was delightful. The steak was overcooked and dry. However, the dessert was a highlight of the meal. ” The enhanced sample contains both the negative sentiment evaluation of the target sample and the positive and neutral sentiment evaluation in the context, thereby providing the model with more comprehensive sentiment clues.

[0037] Furthermore, in order to avoid excessive increase in the number of samples of the above categories, the embodiment of the present invention sets each minority category sample to generate at most one enhanced sample. At the same time, the embodiment of the present invention only performs data enhancement on categories whose number ratio is lower than a preset ratio (such as 20%).

[0038] Traditional data enhancement methods (such as random insertion or deletion of words, etc.) may destroy the semantic integrity of the original text, or even introduce noise, affecting the learning effect of the model. Compared with traditional enhancement methods, the embodiment of the present invention can introduce semantically related context text, enhance the naturalness and coherence of the sample, avoid semantic distortion caused by random replacement or insertion, and ensure semantic integrity. At the same time, the context text of the embodiment of the present invention provides additional clues, which can help the model understand emotional expression more comprehensively, improve the model's ability to distinguish a few categories, and provide rich semantic information. In addition, the embodiment of the present invention can help reduce overfitting and improve the performance of the model on the test set by controlling the generation ratio and diversity of enhanced samples, thereby improving the generalization ability of the model. Furthermore, the enhancement method of the embodiment of the present invention is simple to operate, does not require additional data collection or complex preprocessing, and only requires a simple splicing operation on the existing data to achieve the purpose of data enhancement.

[0039] Step 40: Determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories.

[0040] The initial sentiment analysis model includes a classification layer and an input layer and an encoding layer in a bidirectional encoder pre-training model (BERT). In addition, the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories.

[0041] For the embodiment of the present invention, in addition to determining the training samples, it is also necessary to build an initial sentiment analysis model to be trained and initialize the model parameters. The overall architecture of the model is as follows: Figure 2 As shown, it includes a classification layer, an input layer and an encoding layer of a pre-trained BERT model, the classification layer includes an attention fusion mechanism and a fully connected layer, the encoding layer includes a multi-layer encoder, and each layer of the encoder adopts a multi-head attention mechanism. In the process of iterative training of the model, the embodiment of the present invention only trains the classification layer and the encoding layer, that is, fine-tunes the pre-trained BERT model.

[0042] Furthermore, in actual application scenarios, there is often a class imbalance in the training data, that is, the number of samples in certain emotion categories is significantly less than that in other categories. In order to solve this problem, the present invention introduces the category weights of different emotion categories in the cross entropy loss function, that is, the weighted cross entropy loss function is adopted to give the minority class a higher weight to reduce the dominant role of the majority class in the overall loss, thereby improving the model's discrimination performance for minority class samples. In order to fully reflect this mechanism in the optimization process, the embodiment of the present invention uses the category weight as a hyperparameter to be tuned, and together with other parameters (such as learning rate and batch size) constitutes a parameter search space.

[0043] In the process of model construction and training, the selection of hyperparameters such as learning rate, batch size, and the above-mentioned class weights is crucial to model performance and robustness. In order to efficiently find the best in the parameter search space, the embodiment of the present invention uses a grid search method to perform an exhaustive search in a predefined parameter set, and each set of candidate parameter combinations (including different learning rate, batch size, and class weight value combinations) will go through the same training and verification process. By performing performance evaluation on all candidate parameter combinations, the parameter configuration with the best average performance on the verification set can be selected.

[0044] Based on this, the method includes: constructing multiple groups of candidate parameter combinations based on the learning efficiency and batch size of model training, and the category weights corresponding to the different emotion categories; using a grid search method to perform an exhaustive search in the multiple groups of candidate parameter combinations, and in the search process, using a cross-validation method to perform a performance evaluation on each group of candidate parameter combinations to obtain a performance evaluation result of each group of candidate parameters; based on the performance evaluation result of each group of candidate parameters, screening out a target candidate parameter combination; based on the target candidate parameter combination, determining the target category weight, target batch size and target learning efficiency.

[0045] When performing performance evaluation on each set of candidate parameter combinations, the training samples are divided into multiple mutually exclusive subsets with the same number of samples, wherein the category ratio of each subset is consistent with the category ratio of the training samples; for each set of candidate parameter combinations in the multiple sets of candidate parameter combinations, the first subset in the multiple subsets is used as a first validation set, and the other subsets except the first subset are used as a first training set, and based on the first training set and the first validation set, as well as each set of candidate parameter combinations, the initial sentiment analysis model is trained and validated to obtain a model performance indicator on the first validation set; the second subset in the multiple subsets is used as a first validation set, and the first validation set is used as a first training set. Second validation set, the other subsets except the second subset are used as the second training set, and based on the second training set and the second validation set, and each group of candidate parameter combinations, the initial sentiment analysis model is trained and verified to obtain the model performance index on the second validation set; the process of replacing the training set and the validation set and the model training and verification process are repeated until the last subset is obtained as the model performance index of the validation set; based on the multiple model performance indicators corresponding to each group of candidate parameter combinations, the average performance indicator corresponding to each group of candidate parameter combinations is calculated; according to the average performance indicator, the performance evaluation result corresponding to each group of candidate parameter combinations is determined.

[0046] Specifically, in order to make full use of limited data resources and ensure the reliability of model evaluation, the embodiment of the present invention adopts a hierarchical K Fold cross validation is used to conduct a comprehensive and objective performance evaluation of each candidate parameter combination.D According to the category distribution K Mutually exclusive subsets of equal size , and ensure that the category ratio of each subset is consistent with the category ratio of the overall training samples. k , training and verification are achieved through the following process: First, Subset As a validation set ,the remaining -1 subset is combined as the training set . Then in the training set The model is trained on the current candidate parameter combination to obtain the model parameters , and then use the trained model parameters For the validation set Make predictions and get the predicted probability of each sample ,in, Indicates i The predicted probability of a sample belonging to the corresponding category is then recorded on the validation set, and the model performance indicators (such as accuracy, precision, recall, etc.) are accumulated to evaluate the stability and overall performance of the candidate parameter combination under different data partitions.

[0047] Through all K Repeat the above process to obtain the current candidate parameter combination in all K The model performance index on the folded model is then averaged to obtain the comprehensive performance of the current candidate parameter combination. K The fold cross-validation not only ensures the representative distribution of the model in terms of category proportion, but also effectively reduces the model's dependence on specific data partitioning, thereby presenting a more reliable and objective performance evaluation in the results.

[0048] The above process is executed for each group of candidate parameter combinations. After the performance indicator averages of all candidate parameter combinations are calculated, the embodiment of the present invention compares and selects the candidate parameter combinations based on these performance indicator averages, so as to find the parameter configuration with excellent performance on the validation set, including the most appropriate category weight (target category weight), learning rate, batch size and other hyperparameters. When the optimal parameter configuration is determined, the model randomly enters the final training stage.

[0049] The entire training process of the embodiment of the present invention can be divided into two key stages, namely, the model optimization stage and the model training stage. The model optimization stage is the content introduced above, that is, using the layered KThe framework of fold cross-validation is used to systematically search and evaluate multiple sets of hyperparameter configurations including category weights. All candidate parameter combinations are tried one by one through grid search, and the performance indicators of each set of parameters are recorded in each fold cross-validation. This stage ensures the rationality of hyperparameter selection and the generalization performance under the conditions of limited model data and category imbalance, and finally obtains the optimal parameter configuration suitable for training with a complete data set; the model training stage means that once the optimal hyperparameters are determined, the model will be retrained using the complete data set to maximize the use of available samples and enhance prediction capabilities. At the same time, a weighted cross entropy loss function is used to maintain robustness to category imbalance, and an early stopping strategy is finally implemented. When the verification performance no longer improves, training is stopped to prevent performance degradation due to overfitting. The specific content of the model training stage is detailed in steps 50, 60, and 70.

[0050] Step 50: Input the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample.

[0051] The initial sentiment analysis model includes a classification layer and an input layer and an encoding layer in a bidirectional encoder pre-training model.

[0052] In order to effectively represent the semantic relationship between text and aspect words, the embodiment of the present invention uses a pre-trained BERT model for feature extraction. The input of the model includes the global information of the text and the semantic information of the aspect words. Figure 3 As shown, including: Step 51: Display the marked aspect words in the training sample to obtain the marked training sample.

[0053] Among them, when displaying the mark, a special mark is used to wrap the aspect word. It should be noted that the special mark of the embodiment of the present invention can be set according to actual business needs, such as setting the special mark to ' <aspect>', the following is only ' <aspect>'Examples are given, but not' <aspect>' is limited.

[0054] In order to enhance the model's In order to address the concerns of the present invention, the embodiment of the present invention adopts a method of displaying aspect words in the text. For example, aspect words are marked with special tags. <aspect>'Package, so that the model can clearly identify the location and scope of the aspect words and enhance their attention. To ensure' <aspect>'Not split by the word segmenter of the BERT model, the embodiment of the present invention will be specially marked' <aspect>' is added to the BERT model's default vocabulary as a special non-separable token.

[0055] For aspect words composed of multiple words (such as " battery life ”), the embodiment of the present invention adopts the overall marking method, that is, adding special marks before and after the entire aspect word. For example, " battery life "Marked as <aspect> battery life < / aspect> For the i training samples , which can be expressed as .

[0056] For example, the training sample for" the steak was overcooked and dry. ", after marking the aspect words, we get for" the <aspect> steak< / aspect> was overcooked and dry. ” The way of displaying marks in the embodiment of the present invention enables the model to pay more attention to aspect words during the encoding process, thereby facilitating more accurate capture of aspect-related sentiment information.

[0057] Step 52: Input the labeled training samples to the input layer for processing to obtain an embedding vector corresponding to the labeled training samples.

[0058] For the embodiment of the present invention, the marked training sample is segmented in the input layer to obtain a training subword sequence, and a start tag and an end tag are added to the start position and the end position of the training subword sequence respectively to obtain an extended training subword sequence; based on the index in the preset vocabulary, the extended training subword sequence is converted to obtain a training input index sequence; the word embedding vector, the position embedding vector and the segment embedding vector corresponding to the training input index sequence are determined; the word embedding vector, the position embedding vector and the segment embedding vector are added to obtain the embedding vector corresponding to the marked training sample; Specifically, the BERT model’s word segmenter is first used in the input layer to segment and encode the labeled training samples. , decompose it into training subword sequences ,in, L is the sequence length. In the embodiment of the present invention, the BERT model uses the WordPiece word segmentation algorithm to split rare words into known subwords, thereby improving the word coverage. Then, the start marker [CLS] and the end marker [SEP] are added to the start and end positions of the sequence respectively to obtain the expanded training subword sequence , expressed as follows:

[0059] Then, the expanded training subword sequence is converted into an index in the preset vocabulary to generate a training input index sequence , the expression is as follows:

[0060] The input of the BERT model consists of word embeddings, position embeddings, and segment embeddings:

[0061] in, represents the embedding vector corresponding to the labeled training sample, represents the word embedding vector, represents the position embedding vector, Represents the segment embedding vector. In the single sentence input task, is equal to 0.

[0062] Step 53: input the embedding vector corresponding to the labeled training sample to the encoding layer for processing, obtain the hidden state corresponding to the labeled training sample, and determine the global sample feature vector and the aspect word sample feature vector based on the hidden state corresponding to the labeled training sample.

[0063] The encoding layer includes multiple layers of encoders, each layer of encoders adopts a multi-head attention mechanism, and a position bias term is introduced in each layer of encoders. It should be noted that the number of layers of the encoder can be set according to actual business requirements, and the embodiment of the present invention does not specifically limit this.

[0064] For the embodiment of the present invention, after obtaining the initial representation of the input layer, the encoding layer uses the pre-trained BERT model to perform deep feature extraction on the sequence data. Since the embodiment of the present invention explicitly marks the aspect words in the input text, this has a positive impact on the attention mechanism of the encoding layer.

[0065] When determining the hidden state corresponding to the marked training sample, the embedding vector corresponding to the marked training sample is input into the first layer encoder of the multi-layer encoder for encoding to obtain the output vector of the first layer encoder; the output vector of the first layer encoder is input into the second layer encoder of the multi-layer encoder for encoding to obtain a query matrix and a key matrix, and based on the query matrix, the key matrix and the position bias item, the attention weight corresponding to the second layer encoder is calculated, wherein if the position corresponding to the position bias item belongs to the aspect word position index set of the display mark, the position bias item is determined to be an initial bias parameter; if the position corresponding to the position bias item does not belong to the aspect word position index set of the display mark, the position bias item is determined to be 0; based on the attention weight corresponding to the second layer encoder, the output vector of the second layer encoder is determined; the output vector of the second layer encoder is continuously input into the third layer encoder of the multi-layer encoder for encoding until the last layer encoder completes encoding, and the output vector of the last layer encoder is determined as the hidden state corresponding to the marked training sample.

[0066] Specifically, the encoding layer of BERT consists of multiple layers of transformer encoders, each of which captures the relationship between tokens in the sequence through a multi-head attention mechanism. l The output of the -1 layer encoder is , No. l The calculation process of the layer encoder is:

[0067] In the self-attention mechanism, the calculation formula of attention weight is:

[0068] Where Q is the query matrix, Q= , K is the key matrix, K= , and is the projection matrix.

[0069] Since aspect words are displayed marked, the model can pay more attention to these positions in the attention calculation. To further enhance this effect, the embodiment of the present invention adds a position bias term to the attention score, so that the model is more inclined to pay attention to the aspect words with display marks. Specifically, the modified attention weight calculation formula is:

[0070] in, is the position bias term, when the position j When it belongs to the aspect word position index set of the display tag, ,otherwise . is a learnable bias parameter that controls the preference for showing the marker position.

[0071] By introducing this position bias term into the attention mechanism, the encoding layer of the present invention can effectively capture the relationship between the aspect words of the display mark and the context, and the generated hidden state can also better reflect the aspect-related features.

[0072] In the hidden state of the encoding layer output, the feature representation of the aspect word is extracted. Specifically, the display mark ' <aspect>' and the position index set of the aspect words wrapped in it, and then average the hidden states of these positions to obtain the feature representation of the aspect words, that is, the aspect word sample feature vector, which is specifically expressed as follows:

[0073] in, Indicates that the BERT encoder is at position j The hidden state of The position index set of aspect words that represent display tags. Represents the feature vector of aspect word samples.

[0074] In addition, the hidden state of the global [CLS] tag is extracted as the global text representation, that is, the global sample feature vector, which is specifically expressed as follows:

[0075] in, Indicates i [CLS] marker position in training samples j The hidden state of .

[0076] Step 54: input the global sample feature vector and the aspect word sample feature vector into the classification layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample.

[0077] Among them, the classification layer includes an attention fusion mechanism and a fully connected layer.

[0078] For the embodiment of the present invention, after the encoding layer extracts rich feature representations, the classification layer is responsible for mapping these features to the emotion category space. For this process, step 54 specifically includes: based on the initial weight matrix corresponding to the attention fusion mechanism, as well as the global sample feature vector and the aspect word sample feature vector, calculating the correlation between the global sample feature vector and the aspect word sample feature vector; based on the correlation, performing feature fusion on the global sample feature vector and the aspect word sample feature vector to obtain a fused feature vector; inputting the fused feature vector into the fully connected layer for emotion classification to obtain the predicted emotion category corresponding to the training sample.

[0079] Specifically, the global sample feature vector output by the encoding layer is and aspect word sample feature vector In order to make full use of the above two parts of information, the embodiment of the present invention adopts an attention-based fusion mechanism, that is, calculating the correlation between the global sample feature vector and the aspect word sample feature vector:

[0080] in, Indicates relevance, is the learnable weight matrix, is the Sigmoid activation function. Then, the generated fusion feature vector can be specifically expressed as:

[0081] in, is the fused feature vector.

[0082] The adaptive fusion method of the embodiment of the present invention enables the model to dynamically adjust the attention paid to the global text and aspect words according to the characteristics of the input data.

[0083] Next, the classification layer maps the fused feature vector to the sentiment category space through a fully connected layer:

[0084] in, is the weight matrix, is the bias vector, K is the number of sentiment categories.

[0085] Finally, the Softmax function is used to convert the linear output into a category probability distribution:

[0086] in, Represents the predicted sentiment category corresponding to the training sample.

[0087] Through this design, the classification layer of the embodiment of the present invention not only utilizes the rich features provided by the encoding layer, but also further strengthens the focus on display tags through the attention mechanism, thereby improving the accuracy of sentiment classification.

[0088] Step 60: Calculate a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample, and the target category weights corresponding to the different emotion categories.

[0089] Among them, the target category weight is the optimal parameter configuration selected based on the grid search strategy and stratified K-fold cross validation.

[0090] In order to deal with the problem of class imbalance during training, the embodiment of the present invention introduces a weighted cross entropy loss, which is defined as follows:

[0091] in, N is the total number of training samples, K is the number of sentiment categories, Indicates category k The weights are hyperparameters selected by grid search, For the i The true sentiment categories of the training samples (represented by one-hot encoding), For the model k The predicted probability value of the class, that is, the predicted sentiment category.

[0092] The embodiment of the present invention classifies the weight Search and tune so that the model can better balance the impact of various samples in loss calculation during training, thereby showing higher robustness and generalization ability when facing unbalanced data.

[0093] Step 70: Iteratively train the initial sentiment analysis model according to the weighted cross entropy loss function value to construct a preset sentiment analysis model.

[0094] For the embodiment of the present invention, after calculating the weighted cross entropy loss function value, the encoding layer and classification layer in the initial sentiment analysis model are iteratively trained according to the weighted cross entropy loss function value. When the loss function value is minimum or reaches a preset number of iterations, the final trained preset sentiment analysis model is output.

[0095] After the model is trained, a preset sentiment analysis model can be used to perform sentiment analysis prediction on the text to be predicted, and based on the interpretability component, transparent insights into how aspect words and context clues affect sentiment prediction are provided. Based on this, the method includes: obtaining the text to be predicted; inputting the text to be predicted into the preset sentiment analysis model for sentiment classification, obtaining the sentiment category corresponding to the text to be predicted, and visualizing the attention distribution during the model analysis process and the contribution of the aspect words marked in the text to be predicted to the sentiment category. Among them, the interpretability component can specifically be an XAI component.

[0096] Specifically, when performing sentiment analysis, the text to be predicted is first preprocessed to obtain the preprocessed text to be predicted, and the aspect words are marked in the preprocessed text to be predicted to generate the marked text to be predicted. Then, the marked text to be predicted is segmented using the same BERT segmenter as in the training phase to obtain a subword sequence, and special tags [CLS]HE [SEP] are added at the start and end positions of the subword sequence to obtain an extended subword sequence. Then, the extended subword sequence is converted into an index in the vocabulary to generate an input index sequence, and the input index sequence is converted into an embedding vector. Further, the embedding vector corresponding to the marked text to be predicted is input into the encoding layer for encoding to obtain a hidden state, and then the global feature vector is extracted, and the hidden state of the marked position is averaged to obtain the aspect word feature vector. Further, the global feature vector and the aspect word feature vector are fused using the attention fusion mechanism of the classification layer to generate a fused feature vector. Finally, the fused feature vector is mapped to the sentiment category space through the fully connected layer to generate a category probability distribution, and the category with the highest probability is selected as the final prediction result.

[0097] In the above prediction process, the text to be predicted goes through the same preprocessing and feature extraction steps as the training phase, ensuring that the input format is fully aligned with the model's expectations. By explicitly marking aspect words, the model can more accurately capture the same sentiment information as a specific aspect, thereby effectively improving the accuracy of the prediction. Using the model parameters optimized in the training phase, the samples to be predicted are forward propagated, the probability distribution of the sentiment category is output, and the final prediction result is selected based on the maximum probability principle, which can achieve accurate aspect sentiment analysis of the text to be predicted.

[0098] The following example illustrates the emotion category prediction process and visualization process in an embodiment of the present invention.

[0099] Example 1: The text to be predicted is " For an appetizer, their calamari is a winner. ", aspect words are " appetizer "and" calamari ", in the prediction results, for " appetizer " is neutral, for " calamari ” is positive.

[0100] In this example, the sentence contains two aspect words " appetizer ” (appetizer) and calamari ” (Squid). Models are marked by display <aspect> and< / aspect> , it is possible to perform sentiment analysis on each aspect word separately, thus avoiding confusion of sentiment polarity.

[0101] For aspect words calamari The pre - set sentiment analysis model uses display tags to focus on "squid" and related context words. The phrase in the sentence " is a winner " expresses strong positive sentiment, and the model successfully captures this sentiment and associates it with " calamari ", correctly predicting positive sentiment. Figure 4 The image in shows the analysis result, indicating that " winner " has the highest contribution to positive sentiment, which is 0.52, while the contribution of " calamari " is 0.21. Figure 5 The attention weight heatmap of shows that the model's attention is highly concentrated on " calamari " and " winner ", and the display tags further enhance this attention.

[0102] For the aspect word " appetizer ", the sentence only mentions appetizers without directly expressing any sentiment, so the sentiment polarity is classified as neutral. The model accurately identifies "appetizer" through the display tag and focuses on its context. Due to the lack of obvious sentiment words, the model finally predicts the aspect word "appetizer" as having a neutral sentiment. Figure 6 The analysis result of shows that "appetizer" has the highest neutral contribution. It can be seen from Figure 7 that the model's attention is mainly concentrated on "appetizer" and its display tag, effectively reducing interference from other sentiment words (such as " winner ").

[0103] Example 1 shows that when processing sentences containing multiple aspect words, the model uses display tags to conduct separate sentiment analysis on each word, which can avoid confusion of sentiment polarity. This method enhances the model's attention to specific aspects and improves the accuracy of sentiment classification.

[0104] Example 2: The text to be predicted is " Way too much money for such a terrible meal. ", the aspect word is " meal ", and the prediction result is negative.

[0105] This text to be predicted contains a single aspect word "meal", and the model clearly identifies its position through the display tag. Therefore, the word " terrible " obtains a relatively high attention weight, indicating strong negative sentiment. Finally, the model associates this sentiment with "meal" and predicts the sentiment of "meal" as negative. Figure 8 The analysis result of shows that " terrible " has the highest contribution to negative sentiment, which is 0.23. It can be seen from Fig. 9 that the model's attention is also highly concentrated on " terrible " and " meal ", and the display tags enhance the attention to the aspect word.

[0106] Example 2 represents a relatively simple sentiment analysis task involving a single aspect word, and the model performs well, verifying its effectiveness in processing a single aspect word.

[0107] Explicit tags play a key role in improving model performance by helping to accurately locate aspect words, thereby reducing confusion in sentences containing multiple aspects. By marking these aspects, the model is able to focus more closely on the relevant terms, thereby facilitating better sentiment analysis. In addition, explicit tags enable the model to associate the correct polarity with each aspect, minimizing the risk of misclassification.

[0108] Figure 4-Figure 9 It shows how word contributions and attention heatmaps can illuminate the model's decision-making process. For positive or negative sentiment, strong sentiment words (such as " winner "or" terrible ”) and its corresponding aspect tags have higher correlation. In contrast, for neutral sentiment, the model mainly focuses on aspect words and their surrounding contexts, avoiding interference from other sentiment words. In addition, the display tag always obtains higher attention weights, highlighting its importance in guiding aspect-based classification.

[0109] In summary, the preset sentiment analysis model provided by the embodiment of the present invention can not only capture aspect-level sentiment clues with high accuracy, but also improve transparent classification process insights, making it more explainable and practical in practical applications.

[0110] A sentiment analysis model training method provided by an embodiment of the present invention introduces a weighted cross entropy loss function. The category weights in the weighted cross entropy loss function can be dynamically adjusted according to the category distribution, that is, the category weights corresponding to the minority sentiment categories are greater than the category weights corresponding to other sentiment categories, thereby reducing the role of the majority category in the overall loss and improving the model's discrimination performance for minority category samples, thereby effectively alleviating the impact of category imbalance during model training and improving the model's sentiment classification accuracy. In addition, by displaying labeled aspect terms in the input sequence, the embodiment of the present invention enables the sentiment analysis model to better capture the complex semantic relationship between aspect words and context. This design alleviates the problems of long-distance dependency and semantic sparsity, thereby improving the accuracy of sentiment classification. Furthermore, the embodiment of the present invention combines explainable artificial intelligence with the sentiment analysis model, which can visualize attention distribution and sentiment weights, and provide clear insights into the decision-making process. This design not only increases the transparency of the model, but also provides strong support for users to understand model behavior and enhances user trust.

[0111] Further, as Figure 1 and Figure 3 The specific implementation of the method shown in the embodiment provides a sentiment analysis model training device, such as Fig.10 As shown, the device includes: a first determination unit 101, a second determination unit 102, an analysis unit 103, a calculation unit 104 and a training unit 105.

[0112] The first determination unit 101 may be used to determine target samples belonging to a minority sentiment category in an original text set, and a quantity ratio of the target samples in the original text set.

[0113] The first determination unit 101 may also be configured to determine a semantic relevance between the context text corresponding to the target sample and the target sample if the quantity ratio is less than a preset ratio.

[0114] The first determination unit 101 can also be used to determine an enhanced sample corresponding to the target sample based on the semantic relevance, and use the enhanced sample and the original text set as training samples together, wherein if it is determined according to the semantic relevance that the context text is semantically related to the target sample, the enhanced sample is generated based on the context text; if it is determined according to the semantic relevance that the context text is semantically irrelevant to the target sample, the target sample is synonymously replaced to generate the enhanced sample.

[0115] The second determination unit 102 may be used to determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories, wherein the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories.

[0116] The analysis unit 103 may be configured to input the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample.

[0117] The calculation unit 104 may be configured to calculate a weighted cross entropy loss function value based on the predicted emotion category and the true emotion category of the training sample, and the target category weights corresponding to the different emotion categories.

[0118] The training unit 105 can be used to iteratively train the initial sentiment analysis model according to the weighted cross entropy loss function value to construct a preset sentiment analysis model.

[0119] In some embodiments, the first determination unit 101 can be specifically used to determine the embedding vector corresponding to the context text and the embedding vector corresponding to the target sample; based on the embedding vector corresponding to the context text and the embedding vector corresponding to the target sample, calculate the similarity between the context text and the target sample; and determine the semantic relevance between the context text and the target sample based on the similarity.

[0120] The first determination unit 101 can also be specifically used to, if the similarity is greater than a preset similarity, concatenate the context text with the target sample to generate an enhanced sample corresponding to the target sample; if the similarity is less than or equal to the preset similarity, perform synonym replacement on the target sample, and concatenate the target sample after synonym replacement with the target sample to generate an enhanced sample corresponding to the target sample.

[0121] In some embodiments, the device further comprises: a cleaning unit.

[0122] The cleaning unit can be used to convert multiple texts in the original text set into lowercase form respectively to obtain multiple converted texts; delete the uniform resource locators, social media identifiers and redundant spaces in the converted multiple texts to obtain multiple preprocessed texts; and determine the preprocessed multiple texts as the cleaned text set.

[0123] The first determination unit 101 may also be specifically used to determine target samples belonging to minority sentiment categories in the cleaned text set, and the quantity ratio of the target samples in the cleaned text set.

[0124] In some embodiments, the initial sentiment analysis model includes a classification layer and an input layer and an encoding layer in a bidirectional encoder pre-training model, and the analysis unit 103 includes: a marking module, an input module, an encoding module and a classification module.

[0125] The marking module may be used to display and mark aspect words in the training samples to obtain marked training samples, wherein special marks are used to wrap the aspect words when displaying the marks.

[0126] The input module can be used to input the labeled training samples into the input layer for processing to obtain the embedding vectors corresponding to the labeled training samples.

[0127] The encoding module can be used to input the embedding vector corresponding to the labeled training sample into the encoding layer for processing, obtain the hidden state corresponding to the labeled training sample, and determine the global sample feature vector and the aspect word sample feature vector based on the hidden state corresponding to the labeled training sample.

[0128] The classification module can be used to input the global sample feature vector and the aspect word sample feature vector into the classification layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample.

[0129] In some embodiments, the input module can be specifically used to perform word segmentation processing on the marked training samples in the input layer to obtain a training sub-word sequence, and add a start tag and an end tag at the start position and the end position of the training sub-word sequence respectively to obtain an expanded training sub-word sequence; based on the index in a preset vocabulary, the expanded training sub-word sequence is converted to obtain a training input index sequence; the word embedding vector, the position embedding vector and the segment embedding vector corresponding to the training input index sequence are determined; the word embedding vector, the position embedding vector and the segment embedding vector are added to obtain the embedding vector corresponding to the marked training sample.

[0130] In some embodiments, the encoding layer includes a multi-layer encoder, each layer of the encoder adopts a multi-head attention mechanism, and a position bias item is introduced in each layer of the encoder. The encoding module can be specifically used to input the embedding vector corresponding to the marked training sample into the first layer encoder of the multi-layer encoder for encoding to obtain the output vector of the first layer encoder; input the output vector of the first layer encoder into the second layer encoder of the multi-layer encoder for encoding to obtain a query matrix and a key matrix, and calculate the attention weight corresponding to the second layer encoder based on the query matrix, the key matrix and the position bias item, wherein if the position corresponding to the position bias item belongs to the aspect word position index set of the display mark, the position bias item is determined to be an initial bias parameter; if the position corresponding to the position bias item does not belong to the aspect word position index set of the display mark, the position bias item is determined to be 0; based on the attention weight corresponding to the second layer encoder, the output vector of the second layer encoder is determined; the output vector of the second layer encoder is continued to be input into the third layer encoder of the multi-layer encoder for encoding until the last layer encoder completes encoding, and the output vector of the last layer encoder is determined as the hidden state corresponding to the marked training sample.

[0131] In some embodiments, the classification layer includes an attention fusion mechanism and a fully connected layer. The classification module can be specifically used to calculate the correlation between the global sample feature vector and the aspect word sample feature vector based on the initial weight matrix corresponding to the attention fusion mechanism, the global sample feature vector and the aspect word sample feature vector; based on the correlation, perform feature fusion on the global sample feature vector and the aspect word sample feature vector to obtain a fused feature vector; input the fused feature vector into the fully connected layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample.

[0132] In some embodiments, the second determining unit 102 includes a construction module, a search module, a screening module and a determination module.

[0133] The construction module can be used to construct multiple sets of candidate parameter combinations based on the learning efficiency and batch size of model training, and the category weights corresponding to the different emotion categories.

[0134] The search module can be used to perform an exhaustive search among the multiple groups of candidate parameter combinations using a grid search method, and during the search process, a performance evaluation is performed on each group of candidate parameter combinations using a cross-validation method to obtain a performance evaluation result for each group of candidate parameters.

[0135] The screening module may be used to screen out a target candidate parameter combination according to the performance evaluation result of each group of candidate parameters.

[0136] The determination module can be used to determine the target category weight, target batch size and target learning efficiency based on the target candidate parameter combination.

[0137] In some embodiments, the search module can be specifically used to divide the training sample into multiple mutually exclusive subsets with the same number of samples, wherein the category ratio of each subset is consistent with the category ratio of the training sample; for each set of candidate parameter combinations in the multiple sets of candidate parameter combinations, the first subset in the multiple subsets is used as a first validation set, and the other subsets except the first subset are used as a first training set, and based on the first training set and the first validation set, as well as each set of candidate parameter combinations, the initial sentiment analysis model is trained and validated to obtain a model performance indicator on the first validation set; the second subset in the multiple subsets is used as a validation set, and the initial sentiment analysis model is trained and validated based on the first training set and the first validation set, and the model performance indicator on the first validation set is obtained; The second validation set is used as the second validation set, and the other subsets except the second subset are used as the second training set, and based on the second training set and the second validation set, and each group of candidate parameter combinations, the initial sentiment analysis model is trained and verified to obtain the model performance index on the second validation set; the replacement process of the training set and the validation set and the training and verification process of the model are repeated until the last subset is obtained as the model performance index of the validation set; based on the multiple model performance indicators corresponding to each group of candidate parameter combinations, the average performance indicator corresponding to each group of candidate parameter combinations is calculated; according to the average performance indicator, the performance evaluation result corresponding to each group of candidate parameter combinations is determined.

[0138] In some embodiments, the preset sentiment analysis model is embedded with an interpretability component, and the analysis unit 103 can be specifically used to obtain the text to be predicted; input the text to be predicted into the preset sentiment analysis model for sentiment classification, obtain the sentiment category corresponding to the text to be predicted, and visualize the attention distribution in the model analysis process and the contribution of the aspect words marked in the text to be predicted to the sentiment category.

[0139] It should be noted that for other corresponding descriptions of the functional units involved in the sentiment analysis model training device provided in the embodiment of the present invention, please refer to Figure 1 and Figure 3 The corresponding description in will not be repeated here.

[0140] Based on the above Figure 1 and Figure 3 The method shown in the embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 and Figure 3 The sentiment analysis model training method shown.

[0141] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0142] Based on the above Figure 1 and Figure 3 The method shown, and Fig.10 In order to achieve the above-mentioned purpose, the embodiment of the present application also provides an electronic device, which can be a personal computer, a tablet computer, a server, or other network equipment, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 3 The sentiment analysis model training method shown.

[0143] Optionally, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0144] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.

[0145] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform, or by hardware.

[0147] The embodiment of the present invention introduces a weighted cross entropy loss function, in which the category weights can be dynamically adjusted according to the category distribution, that is, the category weights corresponding to the minority emotion categories are greater than the category weights corresponding to other emotion categories, thereby reducing the role of the majority category in the overall loss and improving the model's discrimination performance for minority category samples, thereby effectively alleviating the impact of category imbalance during model training and improving the model's sentiment classification accuracy. In addition, the embodiment of the present invention enables the sentiment analysis model to better capture the complex semantic relationship between aspect terms and context by displaying labeled aspect terms in the input sequence. This design alleviates the problems of long-distance dependency and semantic sparsity, thereby improving the accuracy of sentiment classification. Furthermore, the embodiment of the present invention combines explainable artificial intelligence with the sentiment analysis model, which can visualize attention distribution and sentiment weights, and provide clear insights into the decision-making process. This design not only increases the transparency of the model, but also provides strong support for users to understand model behavior and enhances user trust.

[0148] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0149] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.< / aspect> < / aspect> < / aspect> < / aspect> < / aspect> < / aspect> < / aspect>

Claims

1. A sentiment analysis model training method, characterized in that: include: Determine target samples belonging to minority sentiment categories in the original text collection, and the quantity ratio of the target samples in the original text collection; If the quantity ratio is less than a preset ratio, determining the semantic relevance between the context text corresponding to the target sample and the target sample; Based on the semantic relevance, an enhanced sample corresponding to the target sample is determined, and the enhanced sample and the original text set are used as training samples together, wherein if it is determined according to the semantic relevance that the context text is semantically related to the target sample, the enhanced sample is generated based on the context text; if it is determined according to the semantic relevance that the context text is semantically irrelevant to the target sample, the target sample is replaced with a synonym to generate the enhanced sample; Determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories, wherein the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories; Inputting the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample; Calculating a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample and the target category weights corresponding to the different emotion categories; According to the weighted cross entropy loss function value, the initial sentiment analysis model is iteratively trained to construct a preset sentiment analysis model.

2. The method according to claim 1, characterized in that The determining of the semantic relevance between the context text and the target sample includes: Determine an embedding vector corresponding to the context text and an embedding vector corresponding to the target sample; Calculate the similarity between the context text and the target sample based on the embedding vector corresponding to the context text and the embedding vector corresponding to the target sample; Determining the semantic relevance between the context text and the target sample according to the similarity; The determining, based on the semantic relevance, an enhanced sample corresponding to the target sample includes: If the similarity is greater than a preset similarity, the context text is concatenated with the target sample to generate an enhanced sample corresponding to the target sample; If the similarity is less than or equal to the preset similarity, synonym replacement is performed on the target sample, and the target sample after synonym replacement is concatenated with the target sample to generate an enhanced sample corresponding to the target sample.

3. The method according to claim 1, characterized in that Before determining the target samples belonging to the minority sentiment category in the original text set and the quantity ratio of the target samples in the original text set, the method further includes: Converting multiple texts in the original text set into lowercase form to obtain multiple converted texts; Deleting uniform resource locators, social media identifiers, and redundant spaces from the converted multiple texts to obtain multiple preprocessed texts; Determine the preprocessed multiple texts as a cleaned text set; The step of determining target samples belonging to a minority sentiment category in the original text set and the quantity ratio of the target samples in the original text set includes: Determine the target samples belonging to the minority sentiment category in the cleaned text collection and the quantity ratio of the target samples in the cleaned text collection.

4. The method according to claim 1, characterized in that: The initial sentiment analysis model includes a classification layer and an input layer and an encoding layer in a bidirectional encoder pre-training model, and the training sample is input into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample, including: Displaying and marking aspect words in the training samples to obtain marked training samples, wherein when displaying the marks, the aspect words are wrapped with special marks; Inputting the labeled training samples into the input layer for processing to obtain an embedding vector corresponding to the labeled training samples; Inputting the embedding vector corresponding to the labeled training sample into the encoding layer for processing, obtaining the hidden state corresponding to the labeled training sample, and determining the global sample feature vector and the aspect word sample feature vector based on the hidden state corresponding to the labeled training sample; The global sample feature vector and the aspect word sample feature vector are input into the classification layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample.

5. The method according to claim 4, characterized in that Inputting the labeled training samples into the input layer for processing to obtain an embedding vector corresponding to the labeled training samples includes: Performing word segmentation processing on the marked training samples in the input layer to obtain a training subword sequence, and adding a start tag and an end tag to the start position and the end position of the training subword sequence respectively to obtain an expanded training subword sequence; Based on the indexes in the preset vocabulary, converting the expanded training subword sequence to obtain a training input index sequence; Determine a word embedding vector, a position embedding vector, and a segment embedding vector corresponding to the training input index sequence; Adding the word embedding vector, the position embedding vector and the segment embedding vector to obtain the embedding vector corresponding to the labeled training sample; and / or The encoding layer includes a multi-layer encoder, each layer of the encoder adopts a multi-head attention mechanism, and a position bias term is introduced in each layer of the encoder. The embedding vector corresponding to the marked training sample is input into the encoding layer for processing to obtain the hidden state corresponding to the marked training sample, including: Inputting the embedding vector corresponding to the marked training sample into the first layer encoder of the multi-layer encoder for encoding, to obtain the output vector of the first layer encoder; Inputting the output vector of the first layer encoder into the second layer encoder of the multi-layer encoder for encoding, obtaining a query matrix and a key matrix, and calculating the attention weight corresponding to the second layer encoder based on the query matrix, the key matrix and the position bias item, wherein if the position corresponding to the position bias item belongs to the aspect word position index set of the display mark, then determining the position bias item as an initial bias parameter; if the position corresponding to the position bias item does not belong to the aspect word position index set of the display mark, then determining the position bias item to be 0; Determining an output vector of the second layer encoder based on the attention weight corresponding to the second layer encoder; Continue to input the output vector of the second layer encoder into the third layer encoder of the multi-layer encoder for encoding until the last layer encoder completes encoding, and determine the output vector of the last layer encoder as the hidden state corresponding to the marked training sample; and / or The classification layer includes an attention fusion mechanism and a fully connected layer. The global sample feature vector and the aspect word sample feature vector are input into the classification layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample, including: Based on the initial weight matrix corresponding to the attention fusion mechanism, as well as the global sample feature vector and the aspect word sample feature vector, calculating the correlation between the global sample feature vector and the aspect word sample feature vector; Based on the correlation, the global sample feature vector and the aspect word sample feature vector are subjected to feature fusion to obtain a fused feature vector; The fused feature vector is input into the fully connected layer for sentiment classification to obtain the predicted sentiment category corresponding to the training sample.

6. The method according to claim 1, characterized in that Determining the target category weights corresponding to different emotion categories includes: Based on the learning efficiency and batch size of the model training and the category weights corresponding to the different emotion categories, construct multiple sets of candidate parameter combinations; An exhaustive search is performed among the plurality of candidate parameter combinations by using a grid search method, and during the search process, a performance evaluation is performed on each candidate parameter combination by using a cross-validation method to obtain a performance evaluation result of each candidate parameter combination; Screening out a target candidate parameter combination according to the performance evaluation results of each group of candidate parameters; Determining the target category weight, target batch size and target learning efficiency based on the target candidate parameter combination; and / or The preset sentiment analysis model is embedded with an explainability component, and the method further includes: Get the text to be predicted; The text to be predicted is input into the preset sentiment analysis model for sentiment classification, the sentiment category corresponding to the text to be predicted is obtained, and the attention distribution in the model analysis process and the contribution of the aspect words marked in the text to be predicted to the sentiment category are visualized.

7. The method according to claim 6, characterized in that The grid search method is used to perform an exhaustive search among the multiple groups of candidate parameter combinations, and during the search process, a cross-validation method is used to perform a performance evaluation on each group of candidate parameter combinations to obtain a performance evaluation result of each group of candidate parameters, including: Dividing the training samples into multiple mutually exclusive subsets with the same number of samples, wherein the category ratio of each subset is consistent with the category ratio of the training samples; For each candidate parameter combination in the multiple candidate parameter combinations, use the first subset in the multiple subsets as a first validation set, and use the other subsets except the first subset as a first training set, and train and validate the initial sentiment analysis model based on the first training set and the first validation set, as well as each candidate parameter combination, to obtain a model performance indicator on the first validation set; Using a second subset of the multiple subsets as a second validation set, and using the other subsets except the second subset as second training sets, and training and validating the initial sentiment analysis model based on the second training set and the second validation set, and each set of candidate parameter combinations, to obtain a model performance indicator on the second validation set; Repeat the process of replacing the training set and the validation set and the model training and validation process until the last subset is obtained as the model performance indicator of the validation set; Based on multiple model performance indicators corresponding to each set of candidate parameter combinations, an average performance indicator corresponding to each set of candidate parameter combinations is calculated; The performance evaluation result corresponding to each group of candidate parameter combinations is determined according to the performance indicator average value.

8. A sentiment analysis model training device, characterized in that: include: A first determination unit is used to determine target samples belonging to a minority sentiment category in an original text set, and a quantity ratio of the target samples in the original text set; The first determining unit is further configured to determine the semantic relevance between the context text corresponding to the target sample and the target sample if the quantity ratio is less than a preset ratio; The first determination unit is further used to determine an enhanced sample corresponding to the target sample based on the semantic relevance, and use the enhanced sample and the original text set as training samples, wherein if the context text is determined to be semantically relevant to the target sample according to the semantic relevance, the enhanced sample is generated based on the context text; if the context text is determined to be semantically irrelevant to the target sample according to the semantic relevance, the target sample is replaced with a synonym to generate the enhanced sample; A second determination unit is used to determine the initial sentiment analysis model and target category weights corresponding to different sentiment categories, wherein the target category weights corresponding to the minority sentiment categories are greater than the target category weights corresponding to other sentiment categories; An analysis unit, configured to input the training sample into the initial sentiment analysis model to perform aspect-based sentiment analysis to obtain a predicted sentiment category of the training sample; A calculation unit, used to calculate a weighted cross entropy loss function value based on the predicted emotion category and the real emotion category of the training sample, and the target category weights corresponding to the different emotion categories; The training unit is used to iteratively train the initial sentiment analysis model according to the weighted cross entropy loss function value to construct a preset sentiment analysis model.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images