Standard detection method and system based on common sense reasoning and MMOE framework

Abstract

The invention relates to the technical field of standing field detection and language processing, in particular to a standing field detection method and system based on common sense reasoning and an MMOE framework, and the method comprises the following steps: introducing a standing field detection text set and an emotion detection data set, setting a COMET common sense reasoning model to analyze the standing field detection text set to obtain a sentence set with different relations, obtaining an emotion detection word and sentence set based on the emotion detection data set; setting an encoder to process the different relation sentence set and the emotion detection word and sentence set to obtain a feature vector e1 of the different relation sentence set and a feature vector e2 of the emotion detection word and sentence set; establishing an expert model and a gating network mechanism to respectively process the feature vector e1 and the feature vector e2 and obtain weighted vector results of different feature vectors, and decoding the weighted vector results to obtain a vertical vector; and constructing a loss function to analyze a loss value of the vertical vector, and updating parameters of the encoder, the expert network and the gating network based on the loss value to obtain a vertical detection result.

Description

Technical Field

[0001] The present invention relates to the technical field of stance detection and language processing, and in particular to a stance detection method and system based on common sense reasoning and MMOE framework. Background Art

[0002] Stance detection can identify and judge the opinions, attitudes or positions in a text towards a specific target, and then classify them as positive, negative or neutral. In the actual application of language processing and monitoring, stance detection is different from sentiment detection. Stance detection focuses more on the stance expression of a specific object in the text rather than simply the expression of emotion.

[0003] In practical applications, the results of stance detection can help the media or other organizations to clearly understand the tendency and direction of opinions. By deeply mining stance data from diverse information sources such as social media, news reports, and online comments, the public's reactions and attitudes toward specific topics and events can be fully grasped, which has important reference value for the scientific nature of decision-making, the effectiveness of management, the strategic nature of marketing, and the adjustment and optimization of programs.

[0004] Stance detection faces many challenges. Among them, the lack of common sense knowledge of the target is a major problem. Since stance detection is closely related to a specific target, stance detection methods must have a deep understanding of the target's relevant background, domain knowledge, and common sense reasoning when judging the stance of a text. The lack of relevant common sense knowledge will lead to difficulties in understanding the context and inferring implicit information.

[0005] In addition, the lack of sentiment information is also a major challenge faced by stance detection. Although sentiment analysis is closely related to stance analysis, and sentiment information is often used as an important feature to improve stance detection performance, in the actual detection process, the lack of sentiment information seriously affects the accuracy of stance judgment results. Stance detection not only provides an effective means to analyze and judge the stance of text, but also provides strong support for decision-making, management and analysis. Summary of the invention

[0006] In view of the defects of existing methods and the shortcomings in practical applications, in order to solve the understanding difficulties caused by the lack of common sense cognition of the target and the judgment bias caused by insufficient emotional information in the process of stance detection, the present invention proposes an innovative stance detection method, which can more accurately analyze and judge the stance of the text, not only effectively make up for the lack of common sense knowledge, but also maintain a high accuracy of stance detection when the emotional information is incomplete, further promoting the development of stance analysis technology, and can better understand and respond to the complex changes in public opinion. In the first aspect, the present invention provides a stance detection method based on common sense reasoning and MMOE framework, and the method comprises the following steps: introducing a stance detection text set and a sentiment detection data set, establishing a COMET common sense reasoning model, processing the stance detection text set through the COMET common sense reasoning model to obtain a set of sentences with different relationships, and obtaining a set of sentiment detection words and sentences based on the sentiment detection data set; setting a model encoder, using the model encoder to process the set of sentences with different relationships and the set of sentiment detection words and sentences, and obtaining feature vectors e1 of the set of sentences with different relationships and feature vectors e2 of the set of sentiment detection words and sentences; establishing an expert model and a gating network mechanism, using the expert model and the gating network mechanism to process the feature vectors e1 and e2 respectively and obtain weighted vector results of different feature vectors, decoding the weighted vector results to obtain a stance vector; constructing a loss function, using the loss function to analyze the loss function value of the stance vector, and updating the parameters of the encoder, the expert network and the gating network based on the loss function value to obtain a stance detection result. The present invention includes technical means such as a stance detection dataset, a model encoder, and a common sense reasoning model, which enhance the common sense reasoning ability, integrate emotional information, extract feature vectors, integrate weighted feature vectors, and optimize and update models during the stance detection task, so that the present invention has a broader application prospect and practical application value in the field of stance detection technology.

[0007] Optionally, the establishing of the COMET common sense reasoning model includes: introducing a selection relationship of text data; establishing the COMET common sense reasoning model based on the selection relationship; The COMET common sense reasoning model satisfies the following relationship: in, represents the generated relation sentence, represents the common sense reasoning model, Indicates the behavior based on St, Indicates the feelings generated based on St, The common sense reasoning model of the present invention enhances the text comprehension ability, improves the common sense reasoning ability and improves the efficiency of the stance detection, thereby improving the generalization ability of the common sense reasoning model.

[0008] Optionally, the step of processing the stance detection text set by the COMET common sense reasoning model to obtain a set of sentences with different relations includes: generating sentences with different relations based on the COMET common sense reasoning model and the stance detection text set; and concatenating the sentences with different relations with the stance detection text set to obtain a set of sentences with different relations. The different relation sentences of the present invention provide additional information and context for the original stance detection text set, revealing the common sense knowledge, causal relationships or emotional responses implicit in the text, thereby enriching the content and information content of the stance detection text, and helping to better understand the stance tendency of the text in subsequent processing and analysis.

[0009] Optionally, the setting of the model encoder includes: setting a model encoder, wherein the model encoder includes an embedding layer, an attention mechanism layer, a feedforward fully connected layer, and a normalization layer. The model encoder of the present invention mainly includes an embedding layer, an attention mechanism layer, a feedforward fully connected layer, and a normalization layer, which can make full use of the advantages of each layer to improve the overall performance and effect of the model.

[0010] Optionally, the establishment of an expert model and a gating network mechanism, processing the feature vector e1 and the feature vector e2 respectively through the expert model and the gating network mechanism and obtaining weighted vector results of different feature vectors includes: introducing an LSTM network and a gating network, establishing an expert model and a gating network mechanism based on the LSTM network and the gating network; inputting the feature vector e1 and the feature vector e2 into two expert models based on the LSTM network for training respectively, and obtaining different training results of the two expert models, the two expert models including a first expert model and a second expert model; using the gating network in the expert model and the gating network mechanism to perform weighted processing on the different training results to obtain the output allocation weights of the two expert models for different feature vectors; obtaining weighted vector results of different feature vectors based on the output allocation weights. The present invention introduces the LSTM network as part of the mechanism, which is conducive to capturing long-term dependencies in sequence data, so that the present invention can better understand the contextual information in the text, thereby improving the accuracy of the stance detection method.

[0011] Optionally, decoding the weighted vector result to obtain the stance vector includes: inputting the weighted vector result into a decoder to obtain a decoder output result; introducing a softmax function to process the decoder output result and obtain a stance vector, wherein the stance vector includes a stance detection vector and an emotion recognition vector. The decoder of the present invention converts the weighted vector result into a more specific stance vector, making the stance detection result clearer and easier to understand, thereby providing a more comprehensive text analysis result.

[0012] Optionally, the stance vector satisfies the following relationship: in, represents the kth output, represents the kth decoder, The stance vector model of the present invention introduces the MMOE framework and the decoder, so that the present invention has higher efficiency and accuracy in the stance detection process, and provides technical support for subsequent stance detection and analysis.

[0013] Optionally, constructing the loss function includes: constructing a loss function by combining the stance detection task and the emotion recognition task; the loss function satisfies the following relationship: in, represents the loss function, represents the dynamically adjusted hyperparameters, represents the loss function of the stance detection task, The loss function of the emotion recognition task is represented. The present invention combines the stance detection task and the emotion recognition task to construct a loss function, so that the function can learn the relevant features and information of the two tasks at the same time, which helps the model to better generalize to unseen data during the training process and improve the overall performance and generalization ability of the relevant model.

[0014] Optionally, the loss function is used to analyze the loss function value of the stance vector, and the parameters of the encoder, expert network, and gated network are updated based on the loss function value to obtain the stance detection result, including: using the loss function to calculate the loss function value of the stance detection vector and the emotion recognition vector; inputting the loss function value and the optimizer into the back propagation and gradient update module, and updating the parameters of the encoder, expert model, gated network, and decoder by calculating the gradient and back propagation to achieve the fusion of implicit emotional information and the improvement of emotion detection ability, thereby obtaining the stance detection result. The present invention not only focuses on the loss degree of the stance detection task, but also takes into account the loss degree of the emotion recognition task. The above-mentioned dual-task learning mechanism helps to integrate implicit emotional information, thereby improving the accuracy of the stance detection result, further understanding the text content, and improving the accuracy and scientificity of the judgment direction and stance detection results.

[0015] In the second aspect, the present invention also provides a stance detection system based on common sense reasoning and MMOE framework, which can efficiently execute a stance detection method based on common sense reasoning and MMOE framework provided by the present invention. The system includes an input device, a processor, an output device and a memory, wherein the input device, the processor, the output device and the memory are interconnected. The system provided by the present invention has a compact structure, strong applicability, and greatly improves the operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the stance detection method based on common sense reasoning and MMOE framework of the present invention; Figure 2 It is a schematic diagram of the implementation steps of the stance detection method based on common sense reasoning and MMOE framework of the present invention; Figure 3 It is a structural diagram of the stance detection system based on common sense reasoning and MMOE framework of the present invention. DETAILED DESCRIPTION

[0017] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not need to be adopted to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, software or methods are not specifically described.

[0018] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or subcombination. In addition, it should be understood by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale.

[0019] See also Figure 1 In order to promote the interactivity and integration between emotional information and stance detection, the emotional tendency and stance expression in the text can be more comprehensively understood, and the accuracy of stance detection results can be improved. At the same time, in order to optimize the model structure and training strategy, the accuracy of the detection results can be improved, and the progress and development of stance detection and natural language processing technology can be further promoted. The present invention provides a stance detection method based on common sense reasoning and MMOE framework, and the stance detection method based on common sense reasoning and MMOE framework includes the following steps: S1. Introduce stance detection text set and sentiment detection dataset, establish COMET common sense reasoning model, process stance detection text set through COMET common sense reasoning model to obtain different relationship sentence set, and obtain sentiment detection word and sentence set based on sentiment detection dataset. The specific implementation steps and related contents are as follows: Firstly, the selection relationship of text data is introduced, and the COMET common sense reasoning model is established based on the above selection relationship.

[0020] In the embodiment, a COMET common sense reasoning model is constructed according to the selection relationship of specific text data. There are multiple possible relationship types in this model. The above selection relationship determines that different relationships generate corresponding relationship sentences.

[0021] The above COMET common sense reasoning model satisfies the following relationship: in, represents the generated relation sentence, represents the common sense reasoning model, Indicates the behavior based on St, Indicates the feelings generated based on St, Data representing a stance detection text set.

[0022] Generated relation sentences refer to relation sentences generated by the COMET common sense reasoning model, which performs common sense reasoning based on the input stance detection data.

[0023] The behavior based on St refers to the behavior or event that occurs based on the input stance detection text, i.e., the data of the stance detection text set; and the feeling generated based on St refers to the feeling or reaction caused by the corresponding behavior or event.

[0024] The data of stance detection text set refers to the text dataset used for stance detection task. The above dataset contains a series of texts, and the relevant texts can express corresponding opinions or stances for a specific target. The stance detection task is to determine the stance of the text for a given target, where stance classification mainly includes support, opposition and neutrality.

[0025] The COMET common sense reasoning model is used to perform reasoning analysis on a given set of stance detection text data. The model first analyzes the content and determines which behaviors (xEffect) and feelings (xReact) are related based on the selection relationship of the text data. Then, the model generates a series of relational sentences based on the above relationship, namely, the generated relational sentences.

[0026] Then, the generated relation sentences and stance detection text set are concatenated to obtain a set of different relation sentences.

[0027] Relational sentences are generated based on the COMET common sense reasoning model and the stance detection text set. In this process, the COMET model has a rich relational library and can generate different relational sentences according to the selected relation type. In an optional embodiment, the input sentence "X has exhausted his strength", if the "XNeed" relation is selected, COMET can generate the sentence "X may want to drink something", revealing X's needs; if the "XEffect" relation is selected, it can generate "X wants to drink water", reflecting the possible consequences of X's aforementioned behavior; if the "XReact" relation is selected, it can generate "tired", expressing X's emotional state or feelings at this time.

[0028] Subsequently, the relational sentences generated by the above model are spliced ​​with the original sentences of the stance detection text set, and corresponding prompts can be added as needed to enhance the coherence and readability of the sentence set information. In an optional embodiment, the original sentence "X has exhausted his strength" can be transformed into "X has exhausted his strength, then X wants to drink water, and now X feels very tired." The above splicing result not only adds common sense information to the original sentence, but also makes the entire sentence more complete and rich in expression.

[0029] On the other hand, it is necessary to obtain a set of emotion detection words and sentences based on the emotion detection dataset. The above set mainly includes words and sentences that can express various emotional states, which can further enrich or refine the emotional expression of the text when needed.

[0030] In summary, this embodiment generates a diverse set of relational sentences through the COMET common sense reasoning model and the stance detection text set, and adds common sense and emotional information to the original sentences by splicing them with the original sentences of the stance detection text set. At the same time, a set of emotion detection words and sentences is obtained using the emotion detection data set, which is conducive to the specific implementation of the subsequent stance detection method. That is, a stance detection sentence set is obtained based on the stance detection text set and the emotion detection data set, where the stance detection sentence set mainly includes a set of different relational sentences and a set of emotion detection words and sentences.

[0031] Furthermore, in the present embodiment, the method for obtaining the stance detection statement set is only an optional condition of the present invention. In one or some other embodiments, the method for obtaining the stance detection statement set can be adjusted according to the relationship between the stance detection requirements and the reasoning model. Different application scenarios and fields have different requirements for stance detection. By adjusting the method for obtaining the stance detection statement set, it can be ensured that the stance detection method can better adapt to different requirements, thereby improving the accuracy and efficiency of detection.

[0032] S2. Set a model encoder, use the above model encoder to process different relationship sentence sets and sentiment detection word set, and obtain feature vectors e1 of different relationship sentence sets and feature vectors e2 of sentiment detection word set. The specific implementation steps and related contents are as follows: In the embodiment, a model encoder is provided, which mainly includes an embedding layer, an attention mechanism layer, a feedforward fully connected layer and a normalization layer.

[0033] The above model encoder mainly extracts feature vectors. Specifically, different sets of sentences with different relationships for stance detection and sets of words and sentences for sentiment detection are input into the model encoder for processing. After encoding the sets of sentences with different relationships and the sets of words and sentences for sentiment detection, the encoder obtains feature vectors e1 of the sets of sentences with different relationships and feature vectors e2 of the sets of words and sentences for sentiment detection.

[0034] In an optional embodiment, the conversion process of the feature vector e1 of the sentence set with different relations is described as follows: The working mechanism of the model encoder is elaborated in detail. The above model encoder is the BERT encoder. It first converts the input text into an embedding vector through the embedding layer. This layer includes three main embedding methods: word embedding, position embedding, and segment embedding. Word embedding is responsible for mapping each word to a low-dimensional vector space to capture the semantic relationship between words; position embedding is used to represent the position information of the word in the sentence to help the model understand the order relationship of the words; segment embedding is used to distinguish different fragments or sentences that may be contained in the input.

[0035] Next, the BERT encoder processes the embedded vector through the attention mechanism layer, in which the query vector Q, key vector K and numerical vector V are calculated, and the weighted result vector w is obtained through attention operation. The above process involves the application of multiple fully connected layers and softmax functions, which aims to capture the relevance and importance between different parts of the input text.

[0036] The query vector Q, key vector K and value vector V satisfy the following relationships: in, Query vector Q, represents the query vector K, represents the query vector V, represents the fully connected layer, represents the embedding vector of the input, represents the matrix containing weights and biases for the mapping process corresponding to the query vector Q, represents the matrix containing weights and biases for the mapping process corresponding to the key vector K, A matrix containing weights and biases representing the mapping process corresponding to the numeric vector V.

[0037] The above weighted result vector w satisfies the following relationship: Among them, w represents the weighted result vector, represents the attention operation processing, Query vector Q, represents the query vector K, represents the query vector V, express The transpose of a vector, express The dimensions of the vector.

[0038] Then, the BERT encoder inputs the weighted result vector w into the feedforward fully connected layer for further processing, where activation functions and linear transformations are applied to extract more abstract and useful feature information, and the above feature information needs to satisfy the following relationship: in, represents the output result vector, represents the activation function, represents the weight matrix, x represents the input vector, Represents the bias vector.

[0039] Finally, the BERT encoder applies a normalization layer (LayerNorm) to each layer for normalization. The above LayerNorm technology helps to reduce problems such as gradient disappearance and further improves the convergence speed and stability of the model.

[0040] Through the above implementation steps, the set of sentences with different relations is converted into a feature vector e1, which provides strong support for the subsequent stance detection task. At the same time, based on the above process and specific implementation steps, the sentiment detection word and sentence set is also processed in the same way, and then the feature vector e2 of the sentiment detection word and sentence set is obtained, so that the relevant information can be used in the subsequent steps to enhance the accuracy and reliability of stance detection.

[0041] At the same time, in the stance detection task, since there is a clear target, that is, the stance that needs to be detected, the target sentence and other sentences or context information that may contain relevant information need to be input into the BERT model together. The above two parts of information can be further distinguished. Position embedding is used to mark the positional relationship between the two parts of information. Based on this model, the encoder can better understand the structure and content of the input text, thereby more accurately detecting the stance information.

[0042] Furthermore, in the present embodiment, the method for obtaining the feature vector is only an optional condition of the present invention. In one or some other embodiments, the method for obtaining the feature vector may be replaced and modified according to the actual requirements of stance detection and the structural condition of the model encoder. Different stance detection tasks have different requirements on the type and dimension of the feature vector. By replacing and modifying the method for obtaining the feature vector, the accuracy and efficiency of detection can be improved, and the key information in the text can be captured more effectively.

[0043] S3. Establish an expert model and a gating network mechanism, process the feature vector e1 and the feature vector e2 respectively through the expert model and the gating network mechanism, and obtain the weighted vector results of different feature vectors, and decode the weighted vector results to obtain the stance vector. The specific implementation steps and related contents are as follows: In the embodiment, an LSTM network and a gated network are introduced, and an expert model and a gated network mechanism are established based on the LSTM network and the gated network. The specific implementation steps are as follows: First, the feature vector e1 and the feature vector e2 are respectively input into two expert models based on the LSTM network for training, and different training results of the two expert models are obtained. In the embodiment, the above two expert models mainly include a first expert model and a second expert model.

[0044] Then, the gating network in the expert model and gating network mechanism is used to perform weighted processing on different training results to obtain the output allocation weights of the two expert models for different feature vectors.

[0045] Next, weighted vector results of different feature vectors are obtained based on the above output allocation weights.

[0046] Finally, the weighted vector result is input into the decoder to obtain the decoder output result. At the same time, the softmax function is introduced to process the decoder output result to obtain the stance vector, where the stance vector mainly includes the stance detection vector and the emotion recognition vector.

[0047] In the embodiment, the above-mentioned position vector satisfies the following relationship: in, represents the kth output, represents the kth decoder, Represents the kth output vector obtained by the MMOE framework.

[0048] In an optional embodiment, a gating network is introduced to assign weights to determine the selection between two or more expert models. When two experts generate two expert vector results based on the feature vector e1, the gating network performs a weighted combination of the output vectors of the expert models according to the weights calculated by the gating network, and the weighted combination process can be expressed as the final weighted vector.

[0049] Next, the weighted vector is processed by a fully connected layer and then transformed by a softmax function to finally obtain the stance vector of the embodiment. The stance vector plays an important role in the stance detection method, which includes the result of stance detection and the information of emotion recognition, that is, the stance detection vector and the emotion recognition vector.

[0050] In order to obtain the stance vector in this embodiment, the feature vectors e1 and e2 need to be input into two LSTM network-based expert models for training respectively. In the embodiment, the two expert models include a first expert model and a second expert model, that is, subsequent expert model 1 and expert model 2, which will generate different training results according to the input feature vectors.

[0051] Then, the gating network in the gating network mechanism is used to weight the training results of the two expert models. The above process calculates the output distribution weights of different expert models according to the output of each expert model and the internal logic of the gating network.

[0052] Based on the relevant weights, the weighted vector results of different feature vectors can be obtained, and this weighted vector result will then be input into the decoder. The decoder will further process the weighted vector according to its internal logic and parameters and generate the decoder output result.

[0053] Finally, the softmax function is introduced to process the decoder output to obtain the final stance vector. The stance vector satisfies a specific relationship, that is, the kth output is the kth decoder The embodiment introduces a gated network and multiple expert models, and uses a softmax function and a decoder to achieve stance detection and emotion recognition in the stance detection method.

[0054] In the embodiment, the above-mentioned position vector satisfies the following relationship: in, represents the kth output, represents the kth decoder, Represents the kth output vector obtained by the MMOE framework.

[0055] The output k-th vector obtained by the MMOE framework needs to satisfy the following relationship: in, represents the output k-th vector obtained by the MMOE framework, Representation Gating Network The first weight of represents the output of the input vector x to the expert model 1, represents the second weight of the gating network gk, Represents the output of the input vector x input to expert model 2.

[0056] The above expert model 1 refers to the first expert model in the embodiment; the expert model 2 refers to the second expert model in the embodiment. The final output is the weighted sum of all experts. It is a feature vector produced by the expert model using the LSTM network. In the embodiment, two expert models are set, so there are only two items in the weighted sum.

[0057] The weights of the gating network need to satisfy the following relationship: in, Represents the weighted function trained according to the input x, with different weights, represents the weight matrix, represents the gating network, and x represents the input vector.

[0058] The relevant content about the kth decoder is as follows: The kth decoder is the final step of the entire decoding process. Its internal structure is carefully designed to extract and transform the required key feature vectors from the input. Specifically, the decoder consists of three main parts: an initial fully connected layer, an activation function layer, and a subsequent fully connected layer.

[0059] First, the input data x is linearly transformed through the first fully connected layer. The weight matrix of this layer is denoted as w1 and the bias matrix is ​​denoted as b1. The above steps can be expressed as , thereby realizing the preliminary processing and feature extraction of input data.

[0060] Next, the preliminarily processed data is passed to the activation function layer. In this embodiment, ReLU (Rectified Linear Unit) is selected as the activation function. The above ReLU function can enhance the nonlinear representation ability of the model while reducing the risk of gradient disappearance. The output of this layer can be expressed as , which is the feature vector after being processed by the ReLU activation function.

[0061] Finally, the data processed by the activation function layer enters a fully connected layer again for further linear transformation and feature extraction. The weight matrix of this layer is recorded as w2, and the bias matrix is ​​recorded as b2. The output of this layer is also the final output of the entire decoder, which can be specifically expressed as ,The above output feature vector mainly contains the key information extracted from the input data, which provides support for the subsequent stance detection and ,sentiment information fusion.

[0062] In this embodiment, the kth decoder realizes effective processing and feature extraction of input data through its internal structure, and provides accurate and useful feature vectors for subsequent stance detection and emotion recognition tasks.

[0063] Furthermore, in the present embodiment, the analysis method of the stance vector is only an optional condition of the present invention. In one or some other embodiments, the stance vector analysis method can be optimized and adjusted according to the application target of the stance detection method and the actual situation of the feature vector. Adjusting the analysis method of the stance vector can enhance the generalization ability of the stance detection method and enable it to better adapt to various complex detection scenarios.

[0064] S4. Construct a loss function, use the loss function to analyze the loss function value of the stance vector, and update the parameters of the encoder, expert network, and gating network based on the loss function value to obtain the stance detection result. The specific implementation steps and related contents are as follows: Firstly, a loss function is constructed by combining the stance detection task and the emotion recognition task.

[0065] In order to optimize the stance detection task and the emotion recognition task simultaneously, a composite loss function is established in the embodiment, which is implemented based on the loss function of the stance detection task and the loss function of the emotion recognition task.

[0066] The above loss function satisfies the following relationship: in, represents the loss function, represents the dynamically adjusted hyperparameters, represents the loss function of the stance detection task, Represents the loss function for the emotion recognition task.

[0067] in, is a dynamically adjusted hyperparameter that can be used to balance the contribution of the two tasks in the total loss. The value of can control the degree of focus of the model between stance detection and emotion recognition, and thus can adapt to different application scenarios and needs.

[0068] During the training process, the cross entropy loss function can be used to evaluate the difference between the probability distribution of the model output and the true label distribution. The cross entropy loss function is a commonly used loss function that can effectively measure the similarity between two probability distributions and guide the model to optimize in the right direction.

[0069] By combining the stance detection task and the emotion recognition task and designing the corresponding composite loss function, the goal of multi-task learning can be achieved and the generalization ability and adaptability of the model can be improved. At the same time, using the cross entropy loss function to evaluate the performance of the model can ensure that the model is continuously optimized during the training process and ultimately achieve better prediction results.

[0070] In summary, in this embodiment, a composite loss function is constructed, which can take into account the stance detection task and the emotion recognition task, and balance the contribution between the two tasks by adjusting the hyperparameters. During the training process, the cross entropy loss function is used to guide the model to optimize in the right direction, which helps to obtain accurate prediction results.

[0071] Among them, the loss function of the stance detection task satisfies the following relationship: in, represents the loss function of the stance detection task, Indicates the number of detection data sets, represents the i-th one-hot encoding vector in the stance detection dataset label, The stance detection dataset represents the predicted one-hot encoded vectors.

[0072] The loss function of the emotion recognition task satisfies the following relationship: in, represents the loss function of the emotion recognition task, Indicates the number of detection data sets, represents the i-th one-hot encoding vector in the emotion detection dataset label, The emotion detection dataset represents the predicted one-hot encoded vectors.

[0073] This example further describes the relationship between one-hot encoding vectors in stance detection and emotion recognition tasks. The relevant contents are as follows: In the stance detection task, each sample in the dataset has a corresponding label, which is represented as a one-hot encoding vector, also known as , which represents the i-th vector in the stance detection dataset label. At the same time, the model predicts the above sample and generates a predicted one-hot encoding vector, which is called , which represents the i-th vector predicted by the stance detection dataset.

[0074] Similarly, there is also a dataset for the emotion recognition task, in which each sample has a corresponding emotion label. The above label can also be represented as a one-hot encoded vector, called , which represents the i-th vector in the emotion detection dataset label. The model will perform emotion recognition prediction on this sample and generate a predicted one-hot encoding vector, which is similarly called , which represents the i-th vector predicted by the emotion detection dataset.

[0075] During the training process, the true label is calculated ( and ) and the predicted label ( and ), and use the cross entropy loss function to measure the relevant differences. By minimizing the loss function value, the model is guided to learn the correct mapping relationship from input features to output labels, thereby improving the accuracy of stance detection and emotion recognition.

[0076] In summary, and Represent the one-hot encoding vectors of the true label and predicted label in the stance detection task, respectively, and and They represent the one-hot encoding vectors of the true label and the predicted label in the emotion recognition task respectively. By calculating the difference between the relevant vectors and minimizing the corresponding loss function, an optimal model that can perform stance detection and emotion recognition at the same time can be trained.

[0077] Then, the loss function is used to calculate the loss function values ​​of the stance detection vector and the emotion recognition vector; and the loss function value and the optimizer are input into the back propagation and gradient update module. The parameters of the encoder, expert model, gating network, and decoder are updated by calculating the gradient and back propagation to achieve the fusion of implicit emotion information and the improvement of emotion detection ability, thereby obtaining the stance detection result.

[0078] In this embodiment, the loss function is used to calculate the loss values ​​of the stance detection vector and the emotion recognition vector. The relevant loss values ​​reflect the difference between the model prediction results and the true labels, and are key information for guiding model optimization.

[0079] Subsequently, the loss value and the optimizer are input into the back propagation and gradient update module. The above module calculates the gradient and propagates the loss information from the output layer to the input layer layer by layer, thereby guiding the update of the model parameters. In this process, the parameters of the encoder, expert model, gated network and decoder in the embodiment will be adjusted accordingly to achieve the effective fusion of implicit emotion information and the improvement of emotion detection capability.

[0080] In the embodiment, implicit sentiment information is integrated into the final stance detection result by updating the parameters of the encoder, expert network and gating network. This fusion method enables the stance detection method based on common sense reasoning and MMOE framework to more comprehensively consider the sentiment information in the detection text set when performing stance detection, thereby improving the accuracy of the detection results.

[0081] In addition, the main task of the stance detection method is stance detection, and the auxiliary task is the analysis of sentiment information. Therefore, the loss function In an optional embodiment, at the beginning of training, The hyperparameter value makes the loss values ​​of the two tasks have the same weight, which helps the sensitivity and understanding of emotional information. As the training progresses and the model's emotion detection ability gradually stabilizes, the hyperparameter value can be gradually reduced. Hyperparameter value to reduce the proportion of emotion recognition task in the total loss. Specifically, a dynamic adjustment can be adopted The method of hyperparameter value, such as as the epoch increases, The hyperparameter value is gradually reduced until the parameter value is 0. In the later stage, the stance detection method will focus on the stance detection task, and use the emotion recognition task as an auxiliary information to help improve the practical application performance of the stance detection method.

[0082] The cognitive reasoning model, information processing method, model encoder, loss function, expert model and gating network mechanism of the embodiment can realize the effective integration and utilization of implicit emotional information, improve the ability of the stance detection method, and thus further improve the overall performance and generalization ability of the stance detection method based on common sense reasoning and MMOE framework.

[0083] In order to more intuitively demonstrate the stance detection method based on common sense reasoning and MMOE framework, a schematic diagram of specific implementation steps of the stance detection method based on common sense reasoning and MMOE framework is drawn in the embodiment. For details, please refer to Figure 2 , which clearly demonstrates the process of acquiring common sense information, the design of shared encoders for stance detection tasks and emotion detection tasks, the collaboration mechanism between expert models and gating networks, and the fusion and injection of multi-task interaction and implicit emotion information.

[0084] based on Figure 2 From the information, it can be seen that the present invention performs stance detection on public text data of social networks, constructs a stance detection method including an encoder and a multi-task MMOE (Multi-gate Mixture of Experts) framework, and incorporates common sense reasoning knowledge to achieve high-precision stance detection.

[0085] In the selection and application of common sense reasoning models, the embodiment investigates and screens suitable models to generate common sense information and integrates it into the detection method. The above steps make up for the lack of background knowledge of the model when processing text, thereby improving the accuracy of the detection method.

[0086] At the same time, the emotion detection task is innovatively used as an auxiliary task, which not only enriches the training dimension of the model, but also integrates emotional features into the stance detection model, aiming to utilize the potential connection between emotion and stance, further enhancing the detection ability of the model.

[0087] This embodiment introduces a common sense reasoning module, which can accurately capture and extract common sense features missing in the text, use the advanced COMET model to generate relevant features, and provide them as key inputs to the classifier, which significantly improves the accuracy of the model's judgment on the text's stance.

[0088] Secondly, sentiment detection is used as an auxiliary task in parallel with stance detection, and the effective integration of sentiment features is achieved by jointly optimizing model parameters. Based on this, not only the sensitivity of the model to text sentiment is enhanced, but also the comprehensiveness and accuracy of stance detection are further improved. In summary, the present invention successfully constructs an efficient and accurate stance detection framework by introducing common sense reasoning modules and sentiment detection auxiliary tasks, bringing new breakthroughs and inspirations to the field of social network text analysis.

[0089] See also Figure 3 In an optional embodiment, in order to efficiently execute the stance detection method based on common sense reasoning and MMOE framework provided by the present invention, the present invention also provides a stance detection system based on common sense reasoning and MMOE framework, the system comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, wherein the memory is used to store a computer program, the computer program comprising program instructions, the processor being configured to call the program instructions, and executing the stance detection method based on common sense reasoning and MMOE framework and the specific steps of the related embodiments provided by the present invention. The stance detection system based on common sense reasoning and MMOE framework of the present invention has a complete structure and is objective and stable.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A stance detection method based on common sense reasoning and MMOE framework, characterized in that: The steps include: Introduce a stance detection text set and a sentiment detection data set, establish a COMET common sense reasoning model, process the stance detection text set through the COMET common sense reasoning model to obtain a set of sentences with different relations, and obtain a set of sentiment detection words and sentences based on the sentiment detection data set; Setting a model encoder, using the model encoder to process the different relationship sentence set and the emotion detection word and sentence set, and obtaining a feature vector e1 of the different relationship sentence set and a feature vector e2 of the emotion detection word and sentence set; Establish an expert model and a gating network mechanism, process the feature vector e1 and the feature vector e2 respectively through the expert model and the gating network mechanism to obtain weighted vector results of different feature vectors, and decode the weighted vector results to obtain a stance vector; A loss function is constructed, and the loss function is used to analyze the loss function value of the stance vector. The parameters of the encoder, the expert network, and the gating network are updated based on the loss function value to obtain a stance detection result.

2. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The establishment of the COMET common sense reasoning model includes: Introducing selection relations of text data; Establishing a COMET common sense reasoning model based on the selection relationship; The COMET common sense reasoning model satisfies the following relationship: in, represents the generated relation sentence, represents the common sense reasoning model, Indicates the behavior based on St, Indicates the feelings generated based on St, Data representing a stance detection text set.

3. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The step of processing the stance detection text set by using the COMET common sense reasoning model to obtain a set of sentences with different relations includes: Generating different relation sentences according to the COMET common sense reasoning model and the stance detection text set; The different relationship sentences and the stance detection text set are concatenated to obtain a set of different relationship sentences.

4. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The setting model encoder comprises: A model encoder is set, wherein the model encoder includes an embedding layer, an attention mechanism layer, a feed-forward fully connected layer, and a normalization layer.

5. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The establishing of the expert model and the gated network mechanism, processing the feature vector e1 and the feature vector e2 respectively through the expert model and the gated network mechanism and obtaining the weighted vector results of different feature vectors includes: Introducing an LSTM network and a gating network, and establishing an expert model and a gating network mechanism based on the LSTM network and the gating network; Inputting the feature vector e1 and the feature vector e2 into two expert models based on the LSTM network for training respectively, and obtaining different training results of the two expert models, wherein the two expert models include a first expert model and a second expert model; The different training results are weighted by using the gating network in the expert model and gating network mechanism to obtain output allocation weights of the two expert models for different feature vectors; The weights are assigned based on the output to obtain weighted vector results of different feature vectors.

6. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The decoding process of the weighted vector result to obtain the stance vector comprises: Inputting the weighted vector result into a decoder to obtain a decoder output result; A softmax function is introduced to process the decoder output result and obtain a stance vector, which includes a stance detection vector and an emotion recognition vector.

7. The stance detection method based on common sense reasoning and MMOE framework according to claim 6, characterized in that: The stance vector satisfies the following relationship: in, represents the kth output, represents the kth decoder, Represents the kth output vector obtained by the MMOE framework.

8. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The construction of the loss function includes: Construct loss function by combining stance detection task and emotion recognition task; The loss function satisfies the following relationship: in, represents the loss function, represents the dynamically adjusted hyperparameters, represents the loss function of the stance detection task, Represents the loss function for the emotion recognition task.

9. The stance detection method based on common sense reasoning and MMOE framework according to claim 1, characterized in that: The using the loss function to analyze the loss function value of the stance vector, and updating the parameters of the encoder, the expert network, and the gating network based on the loss function value to obtain the stance detection result includes: Calculating the loss function values ​​of the stance detection vector and the emotion recognition vector using the loss function; The loss function value and the optimizer are input into the back propagation and gradient update module, and the parameters of the encoder, expert model, gating network, and decoder are updated by calculating the gradient and back propagation to achieve the fusion of implicit emotional information and the improvement of emotional detection ability, thereby obtaining the stance detection result.

10. A stance detection system based on common sense reasoning and MMOE framework, characterized in that: The system includes a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the stance detection method based on common sense reasoning and MMOE framework as described in any one of claims 1-9.

Images