A long-sequence dialogue stance detection method combining inference chain and multi-prototype contrast

Abstract

The application provides a long sequence dialogue stance detection method combining reasoning chain and multi-prototype comparison, which is used for solving the problems of opaque reasoning process, insufficient dialogue modeling and fuzzy class boundary in social media long sequence dialogue. The method generates deep semantic reasoning representation by constructing reasoning chain and fusing with target comments; meanwhile, multi-perspective information such as global dependence, local mode, structure relationship and user consistency is fused to obtain multi-perspective semantic representation. On this basis, multiple prototype vectors are maintained for each stance category, and robust stance modeling is realized through sample and prototype similarity calculation and multi-prototype comparison loss, and the model is jointly optimized in combination with the classification loss. The application can effectively enhance the explainability of stance reasoning, improve the adaptability to the semantic diversity and class fuzzy problem in long dialogue, and is suitable for scenes such as public opinion analysis and user attitude prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and social media public opinion analysis, and particularly relates to a stance detection method suitable for a long sequence dialogue scene of social media. BACKGROUND

[0002] With the rapid development of social networks, users increasingly express their stance in social media dialogues. The core goal of the stance detection task is to identify the attitude (support, opposition, or neutrality) of the text author towards a specific target. Existing research mainly focuses on single sentence comment analysis, and pays insufficient attention to long sequence dialogues in social media, and has not yet formed an effective detection method. Although dialogue large language models (LLMs) have made significant progress in the field of natural language processing in recent years, their application in stance detection still has the problems of opaque reasoning process and insufficient interpretability, making it difficult to meet the actual needs of explicitly modeling reasoning logic. At the same time, existing methods are mostly based on single comment analysis, and fail to fully utilize the context dependency in long sequence dialogues, especially in deep dialogue scenes, the detection performance is significantly reduced. In addition, the semantic expression within different stance categories has high diversity, and a single prototype cannot cover the complex semantic distribution, which easily leads to blurred category boundaries, and further reduces the accuracy and robustness of detection. SUMMARY

[0003] To achieve the above-mentioned purpose, the present application proposes a long sequence dialogue stance detection method combining reasoning chain and multi-prototype comparison, which comprises the following steps:

[0004] S1, constructing a long sequence dialogue training data set: collecting long sequence dialogue comment data, preprocessing the long sequence dialogue comment data, and generating input data required for model training;

[0005] S2, reasoning chain construction and interaction representation generation: constructing a multi-level reasoning chain based on the target comment and its historical dialogue, and obtaining a context-fused semantic representation through a pre-trained language model and a multi-layer attention mechanism;

[0006] S3, multi-perspective dialogue semantic modeling: modeling the semantics of the historical dialogue from multiple perspectives of global dependency, local pattern, structural relationship, and user consistency, and obtaining the context information representation related to the target comment through the attention mechanism of multiple iterations;

[0007] S4, multi-prototype comparison optimization and joint training: maintaining multiple prototype vectors for each stance category, constructing a multi-prototype comparison loss function based on the similarity between samples and prototypes, and jointly optimizing with the classification loss function to realize the stance classification of data samples.

[0008] Preferably, the step S1 comprises the following sub-steps:

[0009] S11, Collect and process conversation comment data: Obtain social media platform posts and comments, and retain post topics , posts and comments; for each target comment , collect target comments before the location of the historical conversation .

[0010] S12, Obtain user information for each comment, define a "user identification" mapping function , where is used to return the user ID of each comment. Find the comments in the historical conversation that belong to the same user as the target comment through user identification , and put them into the set ; where, represents the set of all comments in the historical conversation, represents the user ID of the comment. That is, the set contains all the comments from the same user as the target comment.

[0011] S13, Each comment is sorted into form, and each sample is manually labeled with a stance label to and the stance label as a sample in the dataset.

[0012] Preferably, the step S2 includes the following sub-steps:

[0013] S21, build a prompt word template input dialogue large language model as part of the prompt for each dataset sample , which is used to ask for stance analysis key information and obtain output. The specific method is to input the target comment and its historical conversation into the dialogue large language model as the first level of prompt words to ask for the sub-topics involved in the target comment, and generate the sub-topic reasoning chain variable of the target comment; input the sub-topic reasoning chain variable and the target comment and its historical conversation into the dialogue large language model as the second level of prompt words to ask for the sentiment and rhetoric of the target comment, and generate the sentiment and rhetoric reasoning chain variable ; input the target comment , historical conversation , sub-topic reasoning chain variable and sentiment and rhetoric reasoning chain variable As the third-level prompt word input into the dialogue language model, the main viewpoint of the target comment is queried, and the main viewpoint inference chain variable of the target comment is generated. Furthermore, target comments Historical Dialogue Subtopic inference chain variables Variables in the Emotional and Rhetorical Reasoning Chain and main viewpoint reasoning chain variables As the fourth-level cue words are input into the large language model of the dialogue, the main stance analysis of the target comment is queried, and the stance analysis inference chain variables of the target comment are generated. Four types of inference chain variables will be generated sequentially: subtopic inference chain variables. Variables in the Emotional and Rhetorical Reasoning Chain Main viewpoint reasoning chain variables Position analysis inference chain variables Obtain the inference chain variables Explicit position modeling is conducted in a manner that conforms to the order of human reasoning.

[0014] S22, target comments Join the historical dialogue The end of the dialogue sequence is formed. ,by Input based on form The pre-trained language model concatenates the inference chain variables in chronological order, and further... Input based on form The pre-trained language model, for each comment or inference chain variable, obtains a token for each word, to... Using the boundary as a set, we average the token content within each boundary to obtain the comment vector set. and inference chain variable set ; target comment vector With the set of inference chain variables spliced ​​as .

[0015] S23, will As input, it is fed into the multi-layer inference attention module, and the calculation process is as follows: .in, For attention operations, For activation function, For layer normalization, The coefficient is the fusion factor; after multiple layers of interaction, the final selection is... The corresponding sequence serves as the target comment interaction representation. .

[0016] Preferably, step S3 includes the following sub-steps:

[0017] S31, using For the set of comment vectors Encode the hidden states to obtain a sequentially encoded sequence: ,in, Indicates the first The hidden state of the comment indicates that the comment is hidden; subsequently, in Global attention is calculated to capture the semantic dependencies between each comment and its historical comments. Specifically, the first... Each position relative to the front The sequence in the th order The attention weights for each sequence are: Then, through weighted aggregation, the first... Global semantic representation of each position: , among which, when When there is no historical context, then let... Stack the global semantic representations of all positions into a matrix form: .

[0018] S32, Capturing Short-Range Dependencies in Sequences: A two-layer one-dimensional convolutional network (Conv1D) with a kernel size of 2 is used to extract short-range dependencies between adjacent comments. The formula is as follows: .

[0019] S33, Constructing a comment graph based on reply relationships The relationship is defined as follows: Feature aggregation is performed using a graph neural network, where the node feature matrix is ​​a set of comment vectors. ; then, Input Graph Neural Network Perform feature aggregation, the formula is as follows ,in, Represents a non-linear activation function. It is a trainable graph neural network. The output is... Provides sentence vector representations enhanced with hierarchical structure.

[0020] S34, using user identification information to determine which comments come from the same user, arranges the comment vectors of the same user into a sequence. Its dimensions are determined by the number of comments concatenated and the vector dimension.

[0021] S35, Post Topic Input based on The pre-trained language model yields topic representations. ,by For query, let , introduce multi-hop attention mechanism MHA with multiple iterations, for the attention weight formula of sequence : , the query of multi-hop attention mechanism MHA update formula is: . Take the output of steps S21-S24 as input, and uniformly interact through the multi-hop attention mechanism MHA of step S25 to model, select the output of the deepest layer as the final output.

[0022] Preferably, the step S4 comprises the following sub-steps:

[0023] S41, for the stance category set , K-means clustering is performed on the samples under each stance, and the obtained cluster center vector of the application is taken as the initial vector of the updateable prototype vector.

[0024] S42, in the initialization stage, the cluster center is taken as the initial value of the prototype, and then continuously updated in the training process. Set the stance category set as , for each stance , maintain updateable prototype vectors to depict different expression ways under the stance. For the encoding of the target review in each sample , its true stance is recorded as . Since directly selecting the prototype can easily lead to extreme or unstable prototype updating, the application designs a soft assignment strategy, which can more smoothly reflect the multimodal distribution. Calculate the soft assignment probability of the target review in the sample to its belonging stance th prototype: . Wherein, is the cosine similarity, is the temperature coefficient, is the stance to which the current sample belongs, is the th prototype assigned under the current stance.

[0025] S43, the application accordingly defines the multi-prototype contrastive loss of the target sample : . Wherein is the temperature coefficient, For the stance category set, the loss takes the stance prototype as a special contrast sample, with the stance prototype itself as the positive sample and other stance prototypes as the negative samples, which helps the model to maintain clear stance boundaries.

[0026] S44, to maintain the stability of the prototype during training while adapting to the feature distribution of the new batch of samples. In the current batch, for samples belonging to the stance , the soft assignment probability of the sample to the first prototype is weighted and aggregated to obtain the candidate prototype representation: . Wherein, is the soft assignment probability of the sample to the first prototype of the stance , is the vector representation of the target review in a sample, is a smoothing factor to prevent the denominator from being zero. In order to keep the prototype length fixed and avoid numerical instability, the prototype is updated using the Exponential Moving Average (EMA) method: .

[0027] S45, the classification loss and the multi-prototype contrast loss of the stance are jointly optimized, and the training target is composed of the classification loss and the multi-prototype contrast loss: . Wherein is the balance coefficient, which is used to control the relative weight of the classification loss and the multi-prototype contrast loss; so as to realize the stance classification of the data sample.

[0028] Compared with the prior art, the present application has the following beneficial effects. Strong interpretability: by explicitly constructing and integrating the reasoning chain, the logical path from the dialogue context to the final stance judgment is clearly revealed, overcoming the defect of the opaque reasoning process of traditional "black box" models.

[0029] Comprehensive semantic capture: fusion of global dependency, local pattern, structural relationship and user consistency multi-perspective information, which can more accurately model the complex structure of long sequence dialogue and effectively capture the key semantics that determine the stance.

[0030] High classification robustness: introduce multi-prototype contrast learning mechanism, learn multiple prototype vectors for each stance category, which can better represent and distinguish the diversity of stance expression, significantly improve the adaptability and classification robustness of the model to the fuzzy category boundary. BRIEF DESCRIPTION OF DRAWINGS

[0031] The accompanying drawings, which are part of the specification, serve to further understand the present application, and the schematic embodiments of the present application and the description thereof serve to explain the present application but do not constitute undue limitations on the present application. Obviously, the drawings in the following description are only some embodiments, and other drawings can be obtained from these drawings by those of ordinary skill in the art without creative labor.

[0032] In the drawings:

[0033] Figure 1 is a method implementation flowchart.

[0034] Fig. 2 is a schematic diagram of the overall framework of the method.

[0035] The implementation of the present application, functional features and advantages will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0036] The technical problems solved by the embodiments of the present application, the technical solutions adopted and the technical effects achieved will be described clearly and completely below in combination with the drawings and specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other equivalent or obvious modified embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application. The embodiments of the present application can be embodied in various different ways as defined and covered in the claims.

[0037] It should be noted that in the following description, many specific details are given for the convenience of understanding. However, it is obvious that the implementation of the present application can be realized without these specific details.

[0038] It should be noted that in the absence of explicit limitations or conflicts, the various embodiments in the present application and the technical features therein can be combined with each other to form technical solutions.

[0039] To achieve the above-mentioned object, the present application proposes a long sequence dialogue stance detection method combining reasoning chain and multi-prototype comparison, which comprises the following steps:

[0040] S1, constructing a long sequence dialogue training data set: collecting long sequence dialogue comment data, preprocessing the long sequence dialogue comment data, and generating input data required for model training;

[0041] S2, reasoning chain construction and interaction representation generation: constructing a multi-level reasoning chain based on the target comment and its history dialogue, and obtaining a semantic representation fused with context through a pre-trained language model and a multi-layer attention mechanism;

[0042] S3, multi-view dialogue semantic modeling: from global dependence, local pattern, structural relationship and user consistency, the historical dialogue is semantically modeled from multiple angles, and the context information representation related to the target review is obtained through multiple iterations of attention mechanism;

[0043] S4, multi-prototype comparison optimization and joint training: multiple prototype vectors are maintained for each stance category, a multi-prototype comparison loss function is constructed based on the similarity between samples and prototypes, and a classification loss function is jointly optimized to realize the stance classification of data samples.

[0044] Preferably, the step S1 comprises the following sub-steps:

[0045] S11, collecting and processing dialogue review data: the present application obtains social media platform posts and comments, and retains post topics , post and comment; for each target review , collect the historical dialogue before the location of the target review . .

[0046] S12, obtain the user information of each comment, define a "user identification" mapping function , wherein is used to return the user ID of each comment. Through user identification, find the comments in the historical dialogue that belong to the same user as the target review , and put them into the set ; wherein, represents the set of all comments in the historical dialogue, represents the user ID of the comment. That is, the set contains all the comments from the same user as the target review. .

[0047] S13, each comment is arranged in the form of , and each sample is manually labeled with a stance label, so that and the stance label are used as a sample in the data set.

[0048] Preferably, the step S2 comprises the following sub-steps:

[0049] With each data set sample as a part of the prompt, a prompt word template is constructed to input the dialogue large language model, which is used to ask for key information of stance analysis and obtain output. The specific method is to input the target review and its historical dialogue as the first level prompt word into the dialogue large language model, ask for the sub-topic involved in the target review, and generate the sub-topic reasoning chain variable of the target review Subtopic inference chain variables and target comments and its historical dialogue As the second-level cue words input into the large language model of the dialogue, the system queries the sentiment and rhetorical devices of the target comment, generating variables for the sentiment and rhetorical inference chain. ; target comments Historical Dialogue Subtopic inference chain variables And variables in the chain of reasoning about emotion and rhetoric As the third-level prompt word input into the dialogue language model, the main viewpoint of the target comment is queried, and the main viewpoint inference chain variable of the target comment is generated. Furthermore, target comments Historical Dialogue Subtopic inference chain variables Variables in the Emotional and Rhetorical Reasoning Chain and main viewpoint reasoning chain variables As the fourth-level cue words are input into the large language model of the dialogue, the main stance analysis of the target comment is queried, and the stance analysis inference chain variables of the target comment are generated. Four types of inference chain variables will be generated sequentially: subtopic inference chain variables. Variables in the Emotional and Rhetorical Reasoning Chain Main viewpoint reasoning chain variables Position analysis inference chain variables Obtain the inference chain variables Explicit position modeling is conducted in a manner that conforms to the order of human reasoning.

[0050] S22, target comments Join the historical dialogue The end of the dialogue sequence is formed. ,by Input based on form The pre-trained language model concatenates the inference chain variables in chronological order, and further... Input based on form The pre-trained language model, for each comment or inference chain variable, obtains a token for each word, to... Using the boundary as a set, we average the token content within each boundary to obtain the comment vector set. and inference chain variable set ; target comment vector With the set of inference chain variables spliced ​​into .

[0051] S23, the As input, input to the multi-layer reasoning attention module, the calculation process is as follows: . Wherein, is the attention operation, is the activation function, is the layer normalization, is the fusion coefficient; after multi-layer interaction, the final selection Corresponding sequence as the target review interaction representation .

[0052] Preferably, the step S3 comprises the following sub-steps:

[0053] S31, using Encode the review vector set To get the sequence of encoded hidden state: , wherein, Denotes the hidden state representation of the Review; then, global attention calculation is performed on To capture the semantic dependency between each review and its historical reviews. Specifically, define the attention weight of the Position to the Sequence in the Sequence as: ; then, the global semantic representation of the Position is obtained by weighted aggregation: , wherein, when Denotes no historical context, and is set . Stack the global semantic representations of all positions into a matrix form: .

[0054] S32, capture the short-range dependency of the sequence: use a two-layer one-dimensional convolution network (Conv1D) with a convolution kernel size of 2 to extract the short-range dependency of adjacent reviews, the formula is .

[0055] S33, build a review graph based on the reply relationship , define the relationship as follows: . Use graph neural network for feature aggregation, the node feature matrix is the review vector set ; then, input Into the graph neural network For feature aggregation, the formula is , wherein, Denotes a nonlinear activation function, Is a trainable graph neural network. The output A hierarchical enhanced sentence vector representation is provided.

[0056] S34, determine which comments come from the same user using user identification information, arrange the comment vectors of the same user into a sequence , the dimension of which is determined by the number of spliced comments and the vector dimension.

[0057] S35, input the post theme into the pre-trained language model based on , obtain the theme representation , take as the query, let , introduce a multi-iteration multi-hop attention mechanism MHA, and for the attention weight formula of the sequence : , the query of the multi-hop attention mechanism MHA is updated as: . The output of steps S21-S24 is taken as input, and the multi-hop attention mechanism MHA of step S25 is used for interactive modeling, and the output of the deepest layer is selected as the final output.

[0058] Preferably, the step S4 comprises the following sub-steps:

[0059] S41, for a set of stance categories , K-means clustering is performed on samples under each stance, and the obtained cluster center vector is taken as the initial vector of prototype vectors that can be updated.

[0060] S42, in the initialization stage, the cluster center is taken as the initial value of the prototype, and then continuously updated in the training process. Set the set of stance categories as , for each stance , maintain prototype vectors that can be updated to characterize different expression ways under the stance. For the encoding of the target comment in each sample , its true stance is recorded as . Since directly selecting the prototype can easily lead to extreme or unstable updates of the prototype, the present application designs a soft assignment strategy, which can more smoothly reflect the multimodal distribution. Calculate the soft assignment probability of the target comment in the sample to its belonging stance thprototype: . Wherein, is the cosine similarity, is the temperature coefficient, the stance to which the current sample belongs, the first prototype assigned under the current stance. the first prototype assigned under the current stance.

[0061] S43, the multi-prototype contrastive loss of the target sample defined by the invention is: wherein is the temperature coefficient, is the stance category set, the loss takes the stance prototype as a special contrastive sample, the own stance prototype as the positive sample, and other stance prototypes as the negative sample, which helps the model to keep clear stance boundaries.

[0062] S44, to maintain the stability of the prototype in the training process while adapting to the feature distribution of the new batch of samples. In the current batch, for the samples belonging to the stance , the candidate prototype representation is obtained by weighted aggregation according to the soft assignment probability of the first prototype: wherein is the soft assignment probability of the sample to the first prototype of the stance , is the vector representation of the target review in a certain sample, is a smoothing factor to prevent the denominator from being zero. In order to keep the prototype length fixed and avoid numerical instability, the prototype is updated in the form of exponential moving average (EMA): .

[0063] S45, the classification loss and the multi-prototype contrastive loss of the stance are jointly optimized, and the training target is composed of the classification loss and the multi-prototype contrastive loss: wherein is the balance coefficient, used to control the relative weight of the classification loss and the multi-prototype contrastive loss; so as to realize the stance classification of the data sample.​

Claims

1. A long-sequence dialogue stance detection method combining an inference chain and multi-prototype contrast, characterized in that, The method comprises the following steps: S1, constructing a long sequence dialogue training data set: collecting long sequence dialogue comment data, preprocessing the long sequence dialogue comment data, and generating input data required for model training; S2, target comments and its historical dialogue As the first-level cue words input into the large language model of the dialogue, sub-topic inference chain variables are generated to produce the target comment. Subtopic inference chain variables and target comments and its historical dialogue As a second-level cue word input into the large language model of the dialogue, variables in the sentiment and rhetoric inference chain are generated. ; target comments Historical Dialogue Subtopic inference chain variables And variables in the chain of reasoning about emotion and rhetoric As the third-level cue words input into the large language model of the dialogue, the main opinion inference chain variables generated for the target comment are... Furthermore, the target comment, historical dialogue, sub-topic reasoning chain variables, sentiment and rhetoric reasoning chain variables, and main viewpoint reasoning chain variables are input into the dialogue language model as fourth-level cue words to generate the position analysis reasoning chain variables of the target comment. Reasoning chain construction and interaction representation generation: A multi-level reasoning chain is constructed based on the target comment and its historical dialogue, and a semantic representation fused with the context is obtained through a pre-trained language model and a multi-layer attention mechanism; The historical dialogue and the target comment are input into a... The pre-trained language model yields a set of comment vectors. Then, the inference chain variables output in the hierarchical manner Input based on The pre-trained language model yields the inference chain variable set. The target comment vector and the inference chain variable vector are input into a multi-layer attention mechanism to carry out multi-round interactive fusion, so as to obtain a semantic representation that integrates contextual inference information. S3, multi-perspective dialogue semantic modeling: modeling the semantic of the historical dialogue from the perspectives of global dependence, local pattern, structural relationship and user consistency, and obtaining the context information representation related to the target comment through the attention mechanism of multiple iterations; S4, multi-prototype comparison optimization and joint training: maintaining multiple prototype vectors for each stance category, constructing a multi-prototype comparison loss function based on the similarity between samples and prototypes, and jointly optimizing the classification loss function, thereby realizing the stance classification of data samples.

2. The method of claim 1, wherein, The step S1 comprises: S11, collecting and processing conversation review data: obtaining social media platform posts and reviews, and retaining post topics , posts and reviews, for each target review , collecting target reviews history of conversations before the location of the target review ; S12, collating user information: define a "user identification" mapping function, use the user identification to find the target review reviews belonging to the same user , and put them into a set ; S13, generate standardized data: each review is sorted into the form of and a human stance label is annotated for each sample, so that the stance label is a sample in the data set.

3. The method of claim 1, wherein, The step S3 comprises: S31, modeling based on sequence and attention mechanism to capture long-range dependence; S32, using convolutional network to extract short-range semantic pattern; S33, constructing a graph structure based on the reply relationship between comments, and performing feature aggregation through a graph neural network; S34, introducing user identification information to model the semantic consistency of users in different comments; S35, taking the post theme as the target, using the attention mechanism of multiple iterations on the output of steps S31, S32, S33 and S34, and finally fusing the semantic representations of each perspective to form a multi-perspective context representation for the target comment.

4. The method of claim 1, wherein, The step S4 comprises: S41, generating multiple initial prototype vectors for each stance category, wherein the multiple initial prototype vectors are generated by K-means clustering on the training set samples; S42, calculating the similarity distribution of sample representation and prototype of each category, and obtaining the allocation probability of the sample through a soft allocation mechanism; S43, constructing a multi-prototype comparison loss function based on the allocation probability between samples and prototypes; S44, weighting aggregation and exponential moving average update of the prototype to dynamically adapt to the data distribution; S45, finally jointly optimizing the multi-prototype comparison loss and the classification loss, thereby realizing the stance classification of data samples.

Images