A stance detection method and system based on common sense reasoning and MMOE framework

Abstract

The application relates to the technical field of stand detection and language processing, in particular to a stand detection method and system based on common sense reasoning and an MMOE framework, which comprises the following steps: introducing a stand detection text set and a sentiment detection data set, setting a COMET common sense reasoning model to analyze the stand detection text set to obtain a different relationship sentence set, and obtaining a sentiment detection word and sentence set based on the sentiment detection data set; setting an encoder to process the different relationship sentence set and the sentiment detection word and sentence set to obtain a feature vector e1 of the different relationship sentence set and a feature vector e2 of the sentiment detection word and sentence set; establishing an expert model and a gating network mechanism to process the feature vector e1 and the feature vector e2 respectively and obtain a weighted vector result of different feature vectors, and decoding and processing the weighted vector result to obtain a stand vector; constructing a loss function to analyze the loss value of the stand vector, and updating parameters of the encoder, the expert network and the gating network based on the loss value to obtain a stand detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of stance detection and language processing, in particular to a stance detection method and system based on common sense reasoning and MMOE framework. BACKGROUND

[0002] Stance detection can identify and determine the views, attitudes or stances of a specific object in the text, and then classify them as positive, negative or neutral. In the practical application of language processing and monitoring, stance detection is different from sentiment detection. Stance detection focuses more on the expression of the stance of a specific object in the text, rather than simply expressing emotions.

[0003] In practical applications, the results of stance detection can help media or other institutions clearly understand the tendency of views and the direction of the wind. By deeply mining stance data in diversified information sources such as social media, news reports and online comments, the public's reaction and attitude towards a specific topic or event can be fully grasped, which has important reference value for the scientificity of decision-making, the effectiveness of management, the strategy of marketing and the adjustment and optimization of programs.

[0004] Stance detection faces many challenges. One of the biggest problems is the lack of target common sense cognition. Since stance detection is closely related to a specific target, stance detection methods must deeply understand the relevant background, domain knowledge and common sense reasoning of the target when judging the stance of the text. The lack of relevant common sense knowledge will lead to difficulties in understanding the context and inferring implicit information.

[0005] In addition, the lack of emotional information is also a big challenge for stance detection. Although sentiment analysis is closely related to stance analysis, and emotional information is often used as an important feature to improve the performance of stance detection, the lack of emotional information seriously affects the accuracy of stance judgment results in the actual detection process. Stance detection not only provides an effective means to analyze and judge the stance of the text, but also provides strong support for decision-making, management and analysis. SUMMARY

[0006] In view of the defects of the existing method and the insufficient practical application, in order to solve the understanding difficulty caused by the lack of target common sense cognition in the position detection process, and the judgment deviation caused by the lack of emotional information, the application provides an innovative position detection method, which can more accurately analyze and judge the position of the text, not only effectively makes up for the lack of common sense knowledge, but also maintains high accuracy of position detection in the case of incomplete emotional information, further promotes the development of position analysis technology, and can better understand and cope with the complex changes of public opinion. The first aspect of the application provides a position detection method based on common sense reasoning and MMOE framework, which comprises the following steps: introducing a position detection text set and an emotional detection data set, establishing a COMET common sense reasoning model, processing the position detection text set through the COMET common sense reasoning model to obtain a different relationship sentence set, and obtaining an emotional detection word set based on the emotional detection data set; setting a model encoder, processing the different relationship sentence set and the emotional detection word set by using the model encoder, and obtaining a feature vector e1 of the different relationship sentence set and a feature vector e2 of the emotional detection word set; establishing an expert model and a gating network mechanism, processing the feature vector e1 and the feature vector e2 through the expert model and the gating network mechanism respectively to obtain a weighted vector result of different feature vectors, and decoding the weighted vector result to obtain a position vector; constructing a loss function, analyzing the loss function value of the position vector by using the loss function, updating the parameters of the encoder, the expert network and the gating network based on the loss function value, and obtaining a position detection result. The application includes position detection data set, model encoder and common sense reasoning model and other technical means, which enhances the common sense reasoning ability, fuses emotional information, extracts feature vector, fuses weighted feature vector and optimizes and updates the model in the position detection task process, so that the application has a more extensive application prospect and practical application value in the field of position detection technology.

[0007] Optionally, the COMET common sense reasoning model comprises: introducing a selection relationship of text data; establishing a COMET common sense reasoning model based on the selection relationship;

[0008] The COMET common sense reasoning model satisfies the following relationship:

[0009]

[0010] Among them, denotes the generated relationship sentence, denotes the common sense reasoning model, denotes the behavior based on St, denotes the feeling based on St, Data representing a stance detection text set. The common sense reasoning model of the present application enhances the text understanding ability, improves the common sense reasoning ability, and improves the efficiency of stance detection, thereby improving the generalization ability of the common sense reasoning model.

[0011] Optionally, the processing of the stance detection text set by the COMET common sense reasoning model to obtain a different relationship sentence set comprises: generating different relationship sentences according to the COMET common sense reasoning model and the stance detection text set; and splicing the different relationship sentences and the stance detection text set to obtain a different relationship sentence set. The different relationship sentences of the present application provide additional information and background for the original stance detection text set, reveal the implicit common sense knowledge, causal relationship or emotional response in the text, thereby enriching the content and information amount of the stance detection text, and helping to better understand the stance tendency of the text in subsequent processing and analysis.

[0012] Optionally, the model encoder comprises a model encoder comprising an embedding layer, an attention mechanism layer, a feedforward fully connected layer and a normalization layer. The model encoder of the present application mainly comprises an embedding layer, an attention mechanism layer, a feedforward fully connected layer and a normalization layer, which can fully utilize the advantages of each layer to improve the overall performance and effect of the model.

[0013] Optionally, the expert model and the gating network mechanism are established, and the feature vector e1 and the feature vector e2 are processed by the expert model and the gating network mechanism to obtain a weighted vector result of different feature vectors, comprising: introducing an LSTM network and a gating network, establishing an expert model and a gating network mechanism according to the LSTM network and the gating network; inputting the feature vector e1 and the feature vector e2 into two expert models based on the LSTM network for training, and obtaining different training results of the two expert models, the two expert models comprising a first expert model and a second expert model; using the gating network in the expert model and the gating network mechanism to weight process the different training results to obtain the output distribution weight of the two expert models for different feature vectors; and obtaining a weighted vector result of different feature vectors based on the output distribution weight. The present application introduces an LSTM network as part of the mechanism, which is beneficial to capturing long-term dependencies in sequence data, so that the present application can better understand the context information in the text, thereby improving the accuracy of the stance detection method.

[0014] Optionally, the decoding processing of the weighted vector result to obtain the stance vector comprises: inputting the weighted vector result into a decoder to obtain a decoder output result; introducing a softmax function to process the decoder output result and obtain a stance vector, the stance vector comprising a stance detection vector and a sentiment recognition vector. The decoder of the present application converts the weighted vector result into a more specific stance vector, so that the stance detection result is more clear and easy to understand, thereby providing a more comprehensive text analysis result.

[0015] Optionally, the stance vector satisfies the following relationship:

[0016]

[0017] wherein, represents the kth output, represents the kth decoder, represents the kth output vector obtained by the MMOE framework. The stance vector model of the present application introduces the MMOE framework and the decoder, so that the present application has higher efficiency and accuracy in the stance detection process, thereby providing technical support for subsequent stance detection and analysis.

[0018] Optionally, the construction of the loss function comprises: constructing a loss function in combination with the stance detection task and the sentiment recognition task; the loss function satisfies the following relationship:

[0019]

[0020] wherein, represents the loss function, represents a dynamically adjusted hyperparameter, represents the loss function of the stance detection task, represents the loss function of the sentiment recognition task. The present application constructs a loss function in combination with the stance detection task and the sentiment recognition task, so that the function can learn the relevant features and information of the two tasks at the same time, which helps the model to better generalize to unseen data during the training process, thereby improving the overall performance and generalization ability of the related model.

[0021] Optionally, the loss function value of the stance vector is analyzed by using the loss function, and parameters of the encoder, the expert network and the gating network are updated based on the loss function value to obtain a stance detection result, comprising: calculating the loss function value of the stance detection vector and the emotion recognition vector by using the loss function; inputting the loss function value and the optimizer into the reverse propagation and gradient update module, updating the parameters of the encoder, the expert model, the gating network and the decoder by calculating the gradient and the reverse propagation, so as to realize the fusion of implicit emotional information and the improvement of emotion detection capability, and further obtain the stance detection result. The present application not only focuses on the loss degree of the stance detection task, but also considers the loss degree of the emotion recognition task. The above-mentioned dual-task learning mechanism helps to fuse the implicit emotional information, thereby improving the accuracy of the stance detection result, further understanding the text content, and improving the accuracy and scientificity of the judgment direction and the stance detection result.

[0022] In a second aspect, the present application further provides a stance detection system based on common sense reasoning and MMOE framework, which can efficiently execute the stance detection method based on common sense reasoning and MMOE framework provided by the present application. The system comprises an input device, a processor, an output device and a memory, wherein the input device, the processor, the output device and the memory are connected to each other. The system structure provided by the present application is compact, has strong applicability, and greatly improves the running efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 Flow chart of the stance detection method based on common sense reasoning and MMOE framework of the present application;

[0024] Figure 2 Schematic diagram of the implementation steps of the stance detection method based on common sense reasoning and MMOE framework of the present application;

[0025] Figure 3 Structure schematic diagram of the stance detection system based on common sense reasoning and MMOE framework of the present application. DETAILED DESCRIPTION

[0026] The specific embodiments of the present application will be described in detail below. It should be noted that the embodiments described herein are only used for illustration and do not limit the present application. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present application. However, it is obvious to those skilled in the art that the present application does not necessarily have to be implemented with these specific details. In other examples, in order to avoid confusion of the present application, well-known circuits, software or methods are not specifically described.

[0027] Throughout the specification, reference to “one embodiment”, “an embodiment”, “one example” or “an example” means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the application. The appearances of the phrases “in one embodiment”, “in an embodiment”, “one example” or “an example” in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments or examples. In addition, those of ordinary skill in the art will appreciate that the diagrams provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0028] See Figure 1 In order to promote the interaction and integration between sentiment information and stance detection, to more comprehensively understand the emotional tendency and stance expression in the text, to improve the accuracy of the stance detection result, and at the same time, to optimize the model structure and training strategy, to improve the precision of the detection result, and to further promote the progress and development of the field of stance detection and natural language processing technology, the application provides a stance detection method based on common sense reasoning and MMOE framework, which comprises the following steps:

[0029] S1. Introducing a stance detection text set and a sentiment detection data set, establishing a COMET common sense reasoning model, processing the stance detection text set through the COMET common sense reasoning model to obtain a different relationship sentence set, and obtaining a sentiment detection word sentence set based on the sentiment detection data set, the specific implementation steps and related content are as follows:

[0030] Firstly, the selection relationship of the text data is introduced, and the COMET common sense reasoning model is established based on the selection relationship.

[0031] In the embodiment, the COMET common sense reasoning model is constructed according to the selection relationship of the specific text data, and there are multiple possible relationship types in the model, and the selection relationship determines the generation of corresponding relationship sentences of different relationships.

[0032] The COMET common sense reasoning model satisfies the following relationships:

[0033]

[0034] Among them, represents the generated relationship sentence, represents the common sense reasoning model, represents the behavior based on St, represents the feeling based on St, represents the data of the stance detection text set.

[0035] Generated relation sentence refers to the relation sentence generated by the COMET common sense reasoning model, which performs common sense reasoning according to the input stance detection data.

[0036] The behavior based on St refers to the behavior or event that occurs based on the input stance detection text, i.e., the data of the stance detection text set; and the feeling based on St refers to the feeling or reaction triggered by the relevant behavior or event.

[0037] The data of the stance detection text set refers to the text data set used for the stance detection task. The above data set contains a series of texts, and the relevant texts can express the corresponding views or stances for a certain specific target. The stance detection task is to determine the stance of the text for a given target, and the stance classification mainly includes support, opposition and neutrality.

[0038] The COMET common sense reasoning model is used to analyze the given stance detection text set data. The model first analyzes the content and determines which behaviors (xEffect) and feelings (xReact) are relevant based on the selection relationship of the text data. Then, the model generates a series of relation sentences based on the above relationship, i.e., the generated relation sentence.

[0039] Then, the generated relation sentence and the stance detection text set are spliced to obtain different relation sentence sets.

[0040] The relation sentence generated based on the COMET common sense reasoning model and the stance detection text set. In this process, the COMET model has a rich relationship library and can generate different relation sentences according to the selected relationship type. In an optional embodiment, if the input sentence is “X has used up all his strength”, if the “XNeed” relationship is selected, the COMET can generate the sentence “X may want to drink something”, revealing X’s needs; if the “XEffect” relationship is selected, the sentence “X wants to drink water” can be generated, which reflects the possible consequences of X’s behavior; if the “XReact” relationship is selected, the sentence “tired” can be generated, which expresses X’s emotional state or feeling at that time.

[0041] Subsequently, the relation sentence generated by the above model is spliced with the original sentence of the stance detection text set, and corresponding prompt text can be added according to the needs to enhance the coherence and readability of the sentence set information. In an optional embodiment, the original sentence “X has used up all his strength” can be spliced into “X has used up all his strength, then X wants to drink water, and X now feels very tired.” The splicing result not only adds common sense information to the original sentence, but also makes the entire sentence more complete and rich in expression.

[0042] In another aspect, a set of sentiment detection words and phrases is obtained based on the sentiment detection dataset. The set mainly includes words and phrases capable of expressing various emotional states, and can further enrich or refine the emotional expression of the text when needed.

[0043] To sum up, the embodiment generates a diversified set of relationship sentences by the COMET common sense reasoning model and the stance detection text set, and increases common sense and emotional information for the original sentence by splicing the set of relationship sentences with the stance detection text set. Meanwhile, the set of sentiment detection words and phrases is obtained by using the sentiment detection dataset, which is conducive to the specific implementation of the subsequent stance detection method. That is, the set of stance detection sentences is obtained based on the stance detection text set and the sentiment detection dataset, and the set of stance detection sentences mainly includes the set of relationship sentences and the set of sentiment detection words and phrases.

[0044] Further, in the embodiment, the acquisition method of the set of stance detection sentences is only an optional condition of the application, and in other one or some embodiments, the acquisition method of the set of stance detection sentences can be adjusted according to the relationship between the stance detection requirement and the reasoning model. Different application scenarios and fields have different requirements for stance detection, and the acquisition method of the set of stance detection sentences can ensure that the stance detection method can better adapt to different requirements, thereby improving the accuracy and efficiency of detection.

[0045] S2. Set a model encoder, use the model encoder to process the set of relationship sentences and the set of sentiment detection words and phrases, and obtain the feature vector e1 of the set of relationship sentences and the feature vector e2 of the set of sentiment detection words and phrases, and the specific implementation steps and related contents are as follows:

[0046] In the embodiment, a model encoder is set, and the model encoder mainly includes an embedding layer, an attention mechanism layer, a feedforward fully connected layer, and a normalization layer.

[0047] The model encoder mainly performs feature vector extraction. Specifically, the set of relationship sentences and the set of sentiment detection words and phrases are input into the model encoder for processing, and the feature vector e1 of the set of relationship sentences and the feature vector e2 of the set of sentiment detection words and phrases are obtained after the model encoder encodes the set of relationship sentences and the set of sentiment detection words and phrases.

[0048] In an optional embodiment, the conversion process of the set of relationship sentence feature vectors e1 is described, and the related content is as follows:

[0049] The working mechanism of the model encoder is described in detail. The above-mentioned model encoder, i.e., the BERT encoder, first converts the input text into an embedding vector through an embedding layer. This layer includes three main embedding methods: word embedding, position embedding, and segment embedding. The word embedding is responsible for mapping each word to a low-dimensional vector space to capture the semantic relationship between words; the position embedding is used to represent the position information of the word in the sentence to help the model understand the sequential relationship of the word; and the segment embedding is used to distinguish different segments or sentences that may be contained in the input.

[0050] Next, the BERT encoder processes the embedding vector through an attention mechanism layer. In this layer, the query vector Q, the key vector K, and the value vector V are calculated, and the weighted result vector w is obtained through attention operation processing. The above process involves multiple fully connected layers and the application of the softmax function, aiming to capture the relevance and importance between different parts of the input text.

[0051] The above-mentioned query vector Q, key vector K, and value vector V satisfy the following relationships:

[0052]

[0053]

[0054]

[0055] wherein, the query vector Q, denotes the query vector K, denotes the query vector V, denotes a fully connected layer, denotes the input embedding vector, denotes a matrix containing weights and biases for mapping processing of the query vector Q, denotes a matrix containing weights and biases for mapping processing of the key vector K, denotes a matrix containing weights and biases for mapping processing of the value vector V.

[0056] The above-mentioned weighted result vector w satisfies the following relationship:

[0057]

[0058]

[0059] wherein, w denotes the weighted result vector, denotes attention operation processing, the query vector Q, denotes the query vector K, denotes a query vector V, denotes the transpose of the vector, denotes the dimension of the vector.

[0060] Then, the BERT encoder inputs the weighted result vector w into a feedforward fully connected layer for further processing, in which an activation function and a linear transformation method are applied to extract more abstract and useful feature information, and the above feature information needs to satisfy the following relationship:

[0061]

[0062] wherein, denotes an output result vector, denotes an activation function, denotes a weight matrix, x denotes an input vector, denotes a bias vector.

[0063] Finally, the BERT encoder applies a normalization layer (LayerNorm) at each layer to perform normalization processing, which helps to reduce the problem of gradient disappearance and further improves the convergence speed and stability of the model.

[0064] Through the above implementation steps, the different relationship sentence set is converted into a feature vector e1, which provides strong support for the subsequent stance detection task. At the same time, based on the above process and specific implementation steps, the sentiment detection word set is also processed in the same way, and the feature vector e2 of the sentiment detection word set is obtained, so as to use the relevant information to enhance the accuracy and reliability of the stance detection in the subsequent steps.

[0065] At the same time, in the stance detection task, there is a clear target, i.e. the stance to be detected, which needs to be input into the BERT model together with other sentences or context information that may contain relevant information. The position embedding is used to mark the position relationship of the two parts of information, and the model encoder can better understand the structure and content of the input text, so as to more accurately detect the stance information.

[0066] Further, in the embodiment, the feature vector obtaining method is only an optional condition of the present application, and in other one or some embodiments, the feature vector obtaining method can be replaced and modified according to the actual needs of the stance detection and the structure of the model encoder. Different stance detection tasks have different requirements for the type and dimension of the feature vector. By replacing and modifying the feature vector obtaining method, the accuracy and efficiency of the detection can be improved, and the key information in the text can be captured more effectively.

[0067] S3. Establish an expert model and a gating network mechanism, process the feature vector e1 and the feature vector e2 respectively through the expert model and the gating network mechanism, and obtain the weighted vector results of different feature vectors, and decode the weighted vector results to obtain the stand vector, the specific implementation steps and related contents are as follows:

[0068] In the embodiment, LSTM network and gating network are introduced, and an expert model and a gating network mechanism are established according to the LSTM network and the gating network, and the specific implementation steps are as follows:

[0069] First, the feature vector e1 and the feature vector e2 are respectively input into two expert models based on the LSTM network for training, and different training results of the two expert models are obtained, and the two expert models in the embodiment mainly include a first expert model and a second expert model.

[0070] Then, the gating network in the expert model and the gating network mechanism is used to weight process the different training results to obtain the output distribution weight of the two expert models for different feature vectors.

[0071] Next, the weighted vector results of different feature vectors are obtained based on the above output distribution weight.

[0072] Finally, the weighted vector results are input into the decoder to obtain the decoder output results, and a softmax function is introduced to process the decoder output results to obtain the stand vector, wherein the stand vector mainly includes a stand detection vector and an emotion recognition vector.

[0073] The stand vector in the embodiment satisfies the following relationship:

[0074]

[0075] Wherein, represents the kth output, represents the kth decoder, represents the kth output vector obtained by the MMOE framework.

[0076] In an optional embodiment, a gating network is introduced to distribute weights, so as to determine the selection between two and multiple expert models. When two expert vectors are generated based on the feature vector e1, the gating network will weight and combine the output vectors of the expert models according to the weights calculated by the gating network, and the above weight combination process can be represented as the final weighted vector.

[0077] Next, the weighted vector is processed by a fully connected layer, and then converted by a softmax function to obtain the stance vector of the embodiment, which plays an important role in the stance detection method and contains the results of stance detection and information of sentiment recognition, that is, the stance detection vector and the sentiment recognition vector.

[0078] In order to obtain the stance vector in the embodiment, the feature vectors e1 and e2 need to be input into two LSTM network-based expert models for training first. The two expert models include a first expert model and a second expert model, that is, expert model 1 and expert model 2, which will generate different training results according to the input feature vectors.

[0079] Then, the training results of the two expert models are weighted by using the gating network in the gating network mechanism. The process will calculate the output distribution weight of different expert models according to the output of each expert model and the internal logic of the gating network.

[0080] Based on the related weights, the weighted vector result of different feature vectors can be obtained. The weighted vector result will then be input into the decoder. The decoder will further process the weighted vector according to its internal logic and parameters and generate a decoder output result.

[0081] Finally, a softmax function is introduced to process the decoder output result to obtain the final stance vector. The stance vector satisfies a specific relationship, that is, the kth output is the processing result of the kth decoder. In the embodiment, the gating network and multiple expert models are introduced, and the softmax function and the decoder are used to realize the stance detection and sentiment recognition in the stance detection method.

[0082] In the embodiment, the above stance vector satisfies the following relationship:

[0083]

[0084] wherein, represents the kth output, represents the kth decoder, represents the kth output vector obtained by the MMOE framework.

[0085] The kth output vector obtained by the MMOE framework needs to satisfy the following relationship:

[0086]

[0087] wherein, denotes the kth output vector of the MMOE framework, denotes the first weight of the gating network , denotes the output of the input vector x input to the expert model 1, denotes the second weight of the gating network gk, denotes the output of the input vector x input to the expert model 2.

[0088] The above expert model 1 refers to the first expert model in the embodiment; the expert model 2 refers to the second expert model in the embodiment. The final output is the weighted sum of all experts, is the feature vector produced by the LSTM network for the expert model, and two expert models are set in the embodiment, so there are only two items in the weighted sum.

[0089] The weight of the gating network needs to satisfy the following relationship;

[0090]

[0091] wherein, denotes the weighting function trained according to the input x, with different weights, denotes the weight matrix, denotes the gating network, and x denotes the input vector.

[0092] The kth decoder is related as follows:

[0093] The kth decoder is the final link of the entire decoding process, and its internal structure is carefully designed to extract and convert the required key feature vector from the input. Specifically, the decoder consists of three main parts: an initial fully connected layer, an activation function layer, and a subsequent fully connected layer.

[0094] First, the input data x is linearly transformed by the first fully connected layer, and the weight matrix of this layer is denoted as w1 and the bias matrix is denoted as b1. The above step can be expressed as , thereby achieving the preliminary processing and feature extraction of the input data.

[0095] Next, the preliminarily processed data is passed to the activation function layer. In the embodiment, ReLU (Rectified Linear Unit) is selected as the activation function. The ReLU function can enhance the non-linear representation ability of the model and reduce the risk of gradient vanishing. The output of this layer can be represented as , that is, the feature vector after ReLU activation function processing.

[0096] Finally, the data processed by the activation function layer enters a fully connected layer again for further linear transformation and feature extraction. The weight matrix of this layer is denoted as w2, and the bias matrix is denoted as b2. The output of this layer is also the final output of the entire decoder, which can be specifically represented as The output feature vector mainly contains the key information extracted from the input data, which provides support for subsequent stance detection and emotion information fusion.

[0097] In this embodiment, the kth decoder realizes effective processing and feature extraction of the input data through its internal structure, and provides an accurate and useful feature vector for subsequent stance detection and emotion recognition tasks.

[0098] Furthermore, in this embodiment, the stance vector analysis method is only an optional condition of the present application. In other one or some embodiments, the stance vector analysis method can be optimized and adjusted according to the application target of the stance detection method and the actual situation of the feature vector. Adjusting the stance vector analysis method can enhance the generalization ability of the stance detection method, so that it can better adapt to various complex detection scenes.

[0099] S4. Construct a loss function, analyze the loss function value of the stance vector using the loss function, and update the parameters of the encoder, expert network and gate network based on the loss function value to obtain the stance detection result. The specific implementation steps and related contents are as follows:

[0100] First, a loss function is constructed in combination with the stance detection task and the emotion recognition task.

[0101] In order to optimize the stance detection task and the emotion recognition task at the same time, a composite loss function is established in the embodiment, which is realized based on the loss function of the stance detection task and the loss function of the emotion recognition task.

[0102] The above loss function satisfies the following relationship:

[0103]

[0104] wherein, represents the loss function, represents a dynamically adjusted hyperparameter, represents the loss function of the stance detection task, represents the loss function of the emotion recognition task.

[0105] wherein, is a dynamically adjusted hyperparameter that can be used to balance the contribution of the two tasks in the total loss. By adjusting The value of can control the attention degree of the model between stance detection and sentiment recognition, which can adapt to different application scenarios and requirements.

[0106] During the training process, the cross-entropy loss function can be used to evaluate the difference between the probability distribution of the model output and the real label distribution. The cross-entropy loss function is a commonly used loss function, which can effectively measure the similarity between two probability distributions and guide the model to optimize in the correct direction.

[0107] By combining the stance detection task and the sentiment recognition task and designing the corresponding composite loss function, the goal of multi-task learning can be achieved, and the generalization ability and adaptability of the model can be improved. At the same time, using the cross-entropy loss function to evaluate the performance of the model can ensure that the model is continuously optimized during the training process and ultimately achieves good prediction results.

[0108] In summary, in this embodiment, a composite loss function is constructed, which can consider the stance detection task and the sentiment recognition task, and by adjusting the hyperparameters to balance the contribution degree between the two tasks, during the training process, the cross-entropy loss function is used to guide the model to optimize in the correct direction, which helps to get accurate prediction results.

[0109] wherein the loss function of the stance detection task satisfies the following relationship:

[0110]

[0111] wherein, represents the loss function of the stance detection task, represents the number of detection data sets, represents the i-th one-hot encoding vector in the stance detection data set label, the stance detection data set represents the predicted one-hot encoding vector.

[0112] The loss function of the sentiment recognition task satisfies the following relationship:

[0113]

[0114] wherein, represents the loss function of the sentiment recognition task, represents the number of detection data sets, represents the i-th one-hot encoding vector in the sentiment detection data set label, the sentiment detection data set represents the predicted one-hot encoding vector.

[0115] This embodiment further describes the relationship between the one-hot encoding vectors in the stance detection and sentiment recognition tasks, and the relevant content is as follows:

[0116] In the process of stance detection task, each sample in the data set has a corresponding label, which is represented as a one-hot encoding vector, also known as , which represents the i-th vector in the stance detection data set label. At the same time, the model will make a prediction on the above sample, and then generate a predicted one-hot encoding vector, which is called , which represents the i-th vector predicted by the stance detection data set.

[0117] Similarly, there is also a data set for sentiment recognition task, where each sample has a corresponding sentiment label, which can also be represented as a one-hot encoding vector, called , which represents the i-th vector in the sentiment detection data set label. The model will make a sentiment recognition prediction on this sample, and generate a predicted one-hot encoding vector, which is called , which represents the i-th vector predicted by the sentiment detection data set.

[0118] During training, the difference between the true label ( and ) and the predicted label ( and ) is calculated, and the cross-entropy loss function is used to measure the difference. By minimizing the loss function value, the model learns the correct mapping relationship from input features to output labels, thereby improving the accuracy of stance detection and sentiment recognition.

[0119] In summary, and represent the one-hot encoding vectors of the true label and the predicted label in the stance detection task, respectively, while and represent the one-hot encoding vectors of the true label and the predicted label in the sentiment recognition task, respectively. By calculating the difference between the relevant vectors and minimizing the corresponding loss function, an optimal model that can simultaneously perform stance detection and sentiment recognition can be trained.

[0120] Then, the loss function value of the stance detection vector and the sentiment recognition vector is calculated using the loss function; and the loss function value and the optimizer are input into the backpropagation and gradient update module. The parameters of the encoder, expert model, gating network, and decoder are updated by calculating the gradient and backpropagation to realize the fusion of implicit sentiment information and the improvement of sentiment detection capability, and then obtain the stance detection result.

[0121] In the embodiment, the loss function is used to calculate the loss value of the stance detection vector and the sentiment recognition vector, and the relevant loss value reflects the difference between the model prediction result and the true label, which is the key information to guide the model optimization.

[0122] Subsequently, the loss value and the optimizer are input into the back propagation and gradient update module. The module guides the update of the model parameters by calculating the gradient and back propagating the loss information from the output layer to the input layer, so that the parameters of the encoder, the expert model, the gating network and the decoder in the embodiment are adjusted accordingly to realize the effective fusion of implicit sentiment information and the improvement of sentiment detection capability.

[0123] In the embodiment, the implicit sentiment information is integrated into the final stance detection result through the parameter update of the encoder, the expert network and the gating network. This integration method enables the stance detection method based on common sense reasoning and the MMOE framework to more comprehensively consider the sentiment information in the text set during stance detection, thereby improving the accuracy of the detection result.

[0124] In addition, the main task of the stance detection method is stance detection, and the auxiliary task is sentiment information analysis, so the weight of the above-mentioned related tasks is adjusted by the hyperparameter value in the loss function. In an optional embodiment, the hyperparameter value is set at the beginning of training so that the loss values of the two tasks have the same weight, which helps the sensitivity and understanding ability of the sentiment information. As the training progresses and the sentiment detection capability of the model gradually stabilizes, the hyperparameter value can be gradually reduced to reduce the proportion of the sentiment recognition task in the total loss. Specifically, a method of dynamically adjusting the hyperparameter value can be used, such as gradually reducing the hyperparameter value as the epoch increases until the parameter value is 0. In the later stage, the stance detection method will mainly focus on the stance detection task, and the sentiment recognition task will be used as auxiliary information to help improve the practical application performance of the stance detection method.

[0125] The common sense reasoning model, information processing method, model encoder, loss function and expert model and gating network mechanism of the embodiment can realize the effective fusion and utilization of implicit sentiment information, improve the capability of the stance detection method, and further improve the overall performance and generalization ability of the stance detection method based on common sense reasoning and the MMOE framework.

[0126] In order to more intuitively show the stance detection method based on common sense reasoning and the MMOE framework, the embodiment draws a specific implementation step diagram of the stance detection method based on common sense reasoning and the MMOE framework, please refer to Figure 2Wherein the acquisition process of common sense information, the design of the shared encoder of the stance detection task and the sentiment detection task, the cooperation mechanism of the expert model and the gating network, and the fusion and injection of multi-task interaction and implicit sentiment information are clearly shown.

[0127] Based on Figure 2 It can be known from the information that the present application performs stance detection on social network public text data, and constructs a stance detection method comprising an encoder and a multi-task MMOE (Multi-gate Mixture of Experts) framework, and integrates common sense reasoning knowledge to realize high-precision stance detection.

[0128] In the selection and application of the common sense reasoning model, the embodiment researches and selects a suitable model to generate common sense information and integrates it into the detection method. The above steps make up for the lack of background knowledge of the model when processing text, thereby improving the accuracy of the detection method.

[0129] At the same time, the sentiment detection task is innovatively used as an auxiliary task, which not only enriches the training dimension of the model, but also integrates sentiment features into the stance detection model, aiming to utilize the potential connection between sentiment and stance to further enhance the detection capability of the model.

[0130] The embodiment introduces a common sense reasoning module, which can accurately capture and extract missing common sense features in the text, generate relevant features using an advanced COMET model, and provide them as key inputs to the classifier, significantly improving the accuracy of the model in judging the stance of the text.

[0131] Secondly, the sentiment detection is used as an auxiliary task parallel to the stance detection, and the common optimization of model parameters is used to realize the effective integration of sentiment features, which not only enhances the sensitivity of the model to the sentiment of the text, but also further improves the comprehensiveness and accuracy of the stance detection. In summary, by introducing the common sense reasoning module and the sentiment detection auxiliary task, the present application successfully constructs an efficient and accurate stance detection framework, which brings new breakthroughs and inspirations to the field of social network text analysis.

[0132] Please refer to Figure 3In an optional embodiment, in order to efficiently implement the stance detection method based on common sense reasoning and MMOE framework provided by the present application, the present application further provides a stance detection system based on common sense reasoning and MMOE framework, which comprises a processor, an input device, an output device and a memory, and the processor, the input device, the output device and the memory are connected with each other, wherein the memory is used for storing a computer program, the computer program comprises program instructions, the processor is configured to invoke the program instructions to execute the specific steps of the stance detection method based on common sense reasoning and MMOE framework and related embodiments provided by the present application, and the stance detection system based on common sense reasoning and MMOE framework of the present application has a complete structure and is objective and stable.

[0133] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the description of the present application.

Claims

1. A position detection method based on common sense reasoning and the MMOE framework, characterized in that, Includes the following steps: A stance detection text set and a sentiment detection dataset are introduced, and a COMET commonsense reasoning model is established. The stance detection text set is processed by the COMET commonsense reasoning model to obtain a set of sentences with different relationships, and a sentiment detection word and sentence set is obtained based on the sentiment detection dataset. Set up a model encoder, and use the model encoder to process the set of sentences with different relationships and the set of words and sentences for sentiment detection, and obtain the feature vector e1 of the set of sentences with different relationships and the feature vector e2 of the set of words and sentences for sentiment detection; An expert model and a gating network mechanism are established. The feature vectors e1 and e2 are processed by the expert model and the gating network mechanism respectively to obtain weighted vector results of different feature vectors. The weighted vector results are then decoded to obtain the position vector. A loss function is constructed, and the loss function value of the position vector is analyzed using the loss function. The parameters of the encoder, expert network, and gating network are updated based on the loss function value to obtain the position detection result. The establishment of the COMET common sense reasoning model includes: Introducing selection relationships into text data; A COMET common sense reasoning model is established based on the aforementioned selection relationship; The COMET commonsense reasoning model satisfies the following relationship: ， in, Represents the generated relational sentence. This represents a common-sense reasoning model. This represents the behavior that occurs based on St. This represents the feelings generated based on St. This represents the data in the text set used for position detection.

2. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The process of processing the position detection text set using the COMET commonsense reasoning model to obtain a set of sentences with different relationships includes: Generate sentences with different relationships based on the COMET commonsense reasoning model and the position detection text set; The sentences with different relationships and the set of position detection texts are concatenated to obtain a set of sentences with different relationships.

3. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The setting model encoder includes: A model encoder is configured, which includes an embedding layer, an attention mechanism layer, a feedforward fully connected layer, and a normalization layer.

4. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The establishment of the expert model and gating network mechanism, which processes feature vector e1 and feature vector e2 respectively through the expert model and gating network mechanism to obtain weighted vector results of different feature vectors, includes: An LSTM network and a gated network are introduced, and an expert model and a gated network mechanism are established based on the LSTM network and the gated network. The feature vectors e1 and e2 are respectively input into two expert models based on LSTM network for training, and different training results of the two expert models are obtained. The two expert models include a first expert model and a second expert model. The different training results are weighted using an expert model and a gating network mechanism to obtain the weights assigned to the outputs of the two expert models for different feature vectors. The weighted vector results of different feature vectors are obtained based on the weights assigned to the output.

5. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The process of decoding the weighted vector result to obtain the position vector includes: The weighted vector result is input into the decoder to obtain the decoder output result; A softmax function is introduced to process the decoder output and obtain a stance vector, which includes a stance detection vector and a sentiment recognition vector.

6. The position detection method based on common sense reasoning and the MMOE framework according to claim 5, characterized in that, The position vector satisfies the following relationship: ， in, This represents the k-th output. This represents the k-th decoder. This represents the k-th output vector obtained by the MMOE framework.

7. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The construction loss function includes: A loss function is constructed by combining position detection and emotion recognition tasks; The loss function satisfies the following relationship: ， in, Represents the loss function. This refers to dynamically adjusted hyperparameters. This represents the loss function for the position detection task. This represents the loss function for the emotion recognition task.

8. The position detection method based on common sense reasoning and the MMOE framework according to claim 1, characterized in that, The step of analyzing the position vector using the loss function and updating the parameters of the encoder, expert network, and gating network based on the loss function value to obtain the position detection result includes: The loss function is used to calculate the loss function values ​​of the stance detection vector and the sentiment recognition vector; The loss function value and optimizer are input into the backpropagation and gradient update module. The parameters of the encoder, expert model, gating network and decoder are updated by calculating gradients and backpropagation to achieve the fusion of implicit sentiment information and the improvement of sentiment detection capability, thereby obtaining the stance detection result.

9. A position detection system based on common sense reasoning and the MMOE framework, characterized in that, The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the position detection method based on common sense reasoning and the MMOE framework as described in any one of claims 1-8.

Images