Context dynamic tracking and course learning method oriented to multi-round dialogue perception

Abstract

The invention discloses a context dynamic tracking and course learning method oriented to multi-round dialogue perception. The method comprises the following steps: S1, performing utterance feature extraction in an utterance feature extraction module: extracting a semantic feature vector of each utterance in a dialogue through a first pre-training language model; the first pre-training language model is a RoBERTa model; s2, performing common sense feature extraction in a common sense feature extraction module: extracting common sense feature vectors related to emotion dynamic from the semantic feature vectors through a second pre-training knowledge generation model; the second pre-training knowledge generation model is a COMET model; s3, performing dynamic tracking modeling through a context dynamic tracker, and performing speaker information modeling through a speaker information module; and S4, course learning training: introducing a dynamic weight distribution mechanism and a weight loss function to carry out adaptive training on the dialogue sample. According to the method, dynamic contexts, multi-dimensional speaker features and common knowledge can be deeply fused, and the understanding of complex dialogue emotions is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence and natural language processing, and particularly relates to a context dynamic tracking and curriculum learning method for multi-round dialogue perception. BACKGROUND

[0002] The dialogue emotion recognition task aims to accurately determine the emotional label of each utterance in the dialogue by analyzing the dialogue context, and is one of the core technologies for constructing a high-emotional-intelligence human-machine interaction system. With the development of large language models, their powerful language understanding capabilities bring new opportunities for dialogue emotion recognition (ERC) tasks based on large models, and promote the evolution of systems from single-task robots to dialogue partners with more empathy.

[0003] Currently, ERC methods based on deep learning mainly follow three technical routes: sequence modeling, graph structure modeling, and knowledge enhancement. First, sequence modeling methods represented by DialogueCRN can capture local temporal dependencies through recurrent networks, but their one-way propagation mechanism easily leads to the decay of historical information as the number of dialogue rounds increases, making it difficult to effectively model long-distance emotional dependencies, and the modeling granularity of speaker characteristics interaction is relatively rough. Second, graph structure-based models (such as DAG-ERC) strengthen context awareness by constructing a logical relationship graph between utterance nodes, but their static graph structure cannot dynamically adapt to the real-time changes in emotional associations in the dialogue. Although some studies attempt to incorporate external psychological knowledge to construct interpersonal relationship graphs, the pre-defined graph connection rules lack adaptability to complex social dynamics interactions, and generally ignore the continuous tracking of speaker intentions, reactions, and other streaming features. Third, knowledge enhancement methods represented by COSMIC deepen semantic understanding by integrating external common sense knowledge bases, but their knowledge selection process usually adopts fixed relationship types, limiting dynamic knowledge extraction capabilities. Comparative experiments show that such methods significantly reduce the detection accuracy of emotional mutations on short dialogue datasets due to the neglect of immediate influence transmission between speakers.

[0004] In summary, existing ERC technologies generally have the following common problems and defects: (1) insufficient long-distance emotional dependency capture ability, with decay effects of historical key information in the modeling process; (2) poor adaptability between static knowledge graphs and dynamic dialogue flows, leading to insufficient real-time interaction feature extraction; (3) lack of joint modeling of multi-dimensional behavior characteristics of speakers (such as intentions, influence, and reactions), making it difficult to depict complex social interactions; (4) although some models use curriculum learning or contrastive learning to alleviate data bias, traditional curriculum learning strategies are mostly based on single-dimensional difficulty evaluation, without multi-dimensional comprehensive evaluation combined with common sense knowledge, and lack of a gradual learning mechanism for class imbalance scenarios, resulting in insufficient model robustness.

[0005] The statements herein are provided only to aid in an understanding of the present application and are not necessarily intended to constitute SUMMARY

[0006] The present application aims to provide a context dynamic tracking and curriculum learning method for multi-turn dialogue perception, which can deeply integrate dynamic context, multi-dimensional speaker features and external common sense knowledge.

[0007] To achieve the above-mentioned purpose, the present application provides a context dynamic tracking and curriculum learning method for multi-turn dialogue perception, comprising the following steps: Step S1, speech feature extraction in the speech feature extraction module: extracting the semantic feature vector of each speech in the input dialogue through a first pre-trained language model; the first pre-trained language model is a RoBERTa model; Step S2, common sense feature extraction in the common sense feature extraction module: extracting a common sense feature vector related to emotional dynamics from the semantic feature vector through a second pre-trained knowledge generation model; the second pre-trained knowledge generation model is a COMET model; Step S3, context dynamic tracking modeling by a context dynamic tracker, and speaker information modeling by a speaker information module in the context dynamic tracking modeling; Step S4, curriculum learning training: introducing a dynamic weight distribution mechanism and a weighted loss function to adaptively train the dialogue samples.

[0008] Optionally, the step S1 comprises the following steps: S1.1, for each speech, concatenating it with a special mark to form an input sequence , and inputting the input sequence into the first pre-trained language model; S1.2, extracting the activation vector corresponding to the special mark from the encoding layer of the first pre-trained language model; S1.3, performing average pooling operation on the extracted multiple activation vectors to obtain the semantic feature vector of the speech : wherein [CLS] is a special mark, is an element in the input sequence.

[0009] Optionally, the COMET model incorporates common sense relationships in the ATOMIC knowledge graph in training to deeply analyze the speech content; the common sense feature vector at least includes intent features, reaction features and influence features.

[0010] Optionally, the context dynamic tracking modeling comprises the following steps: S3.1.1, extracting spatial context information based on the semantic feature vector of the current utterance and the historical context information by using an attention mechanism; S3.1.2, fusing the semantic feature vector of the current utterance, the spatial context information and the common sense feature vector of the previous time step by using a perception gated recurrent unit to generate a context state vector of the current time step.

[0011] Optionally, the process of extracting spatial context information by using an attention mechanism in S3.1.1 is as follows: In the formula, Q represents a query, is a first learnable parameter matrix; represents an input embedding hidden state vector of the t th time step; K represents a key; represents a context state vector, is a second learnable parameter matrix; V represents a value; is a third learnable parameter matrix; In the formula, represents information reweighted according to the context; is a normalization function of multi-head attention; is a key K dimension, used for scaling dot product; In the formula, represents the final output spatial context information; represents a layer normalization processing function; represents a regularization operation.

[0012] Optionally, the process of obtaining the context state vector of the current time step in S3.1.2 is as follows: In the formula, represents a context state vector; represents a processing function of the perception gated recurrent unit, represents a context memory at the previous moment; represents a processed current utterance feature vector; represents an intention feature vector at the previous moment; represents a reaction feature vector at the previous moment; represents an influence feature vector at the previous moment; represents vector splicing.

[0013] Optionally, the speaker information modeling updates the intention feature, the reaction feature and the influence feature of the current time step t through three bidirectional gate recurrent units respectively. wherein the first bidirectional gate recurrent unit receives the intention feature vector of the previous time step, the enhanced feature of the current time and the specific intention information to update the speaker intention feature vector of the current time step; the second bidirectional gate recurrent unit receives the influence feature vector of the previous time step, the enhanced feature of the current time and the specific influence information to update the speaker influence feature vector of the current time step; the third bidirectional gate recurrent unit receives the reaction feature vector of the previous time step, the enhanced feature of the current time, the specific reaction information and the influence feature vector of the previous time step to update the speaker reaction feature vector of the current time step.

[0014] Optionally, the update process of the intention feature vector of the current time step is: wherein, represents the speaker intention feature vector of the current time step; represents the processing function of the first bidirectional gate recurrent unit; represents the intention feature vector of the previous time step; represents the enhanced feature; represents the specific intention input; the update process of the influence feature vector of the current time step is: wherein, represents the influence feature vector of the current time step; represents the processing function of the second bidirectional gate recurrent unit; represents the influence feature vector of the previous time step; represents the specific influence input; the update process of the reaction feature vector of the current time step is: wherein, represents the reaction feature vector of the current time step; represents the processing function of the third bidirectional gate recurrent unit; represents the reaction feature vector of the previous time step; represents the specific reaction input; represents the influence feature vector of the previous time step.

[0015] Optionally, the step S4 comprises the following steps: S4.1, calculating the intention cosine similarity, influence cosine similarity and reaction cosine similarity between continuous utterances of the dialogue to evaluate the emotional complexity and inertia in the dialogue; S4.2, evaluating the degree of emotional change in the dialogue through the intention cosine similarity, influence cosine similarity and reaction cosine similarity; S4.3, dynamically assigning weights to the dialogue samples in training; S4.4, performing weighted loss training on the dialogue samples after dynamic weight assignment; The calculation formula of the intention cosine similarity in step S4.1 is: In the formula, is a cosine similarity function; N represents N sentences contained in the dialogue sample; represents the intention feature vector of the i-th sentence in the dialogue sample; represents the intention feature vector of the i+1-th sentence; The calculation formula of the influence cosine similarity is: In the formula, represents the influence feature vector of the i-th sentence in the dialogue; represents the influence feature vector of the i+1-th sentence in the dialogue; The calculation formula of the reaction cosine similarity is: In the formula, represents the reaction feature vector of the i-th sentence in the dialogue; represents the reaction feature vector of the i+1-th sentence in the dialogue; The emotional evaluation formula in step S4.2 is: In the formula, is the difficulty of the sample, and the intensity of the reaction emotional change; The higher the value, the more intense the emotional change.

[0016] Optionally, the dynamic weight assignment formula in step S4.3 is as follows: In the formula, t represents the current training round, and are adjustable parameters, represents the i-th training round, t ​​​The learning weights assigned to a sample during each round of training; The calculation formula for the weighted loss training in step S4.4 is as follows: In the formula, It is the total number of training-focused dialogues. It is the first i The number of sentences in a dialogue. It is the first i In the first dialogue j The true label of a sentence is the probability distribution predicted by the model, and L is the sample result after training with weighted loss.

[0017] In summary, compared with the prior art, the present invention has the following beneficial effects: 1. The present invention provides a context dynamic tracking and course learning method for multi-turn dialogue perception. By introducing a multi-head attention mechanism, the model can directly focus on key information at any position in the historical dialogue, overcoming the problem of historical information decay caused by excessively long sequences in traditional recurrent neural networks, and significantly improving the model's ability to perceive emotional cues in long dialogues.

[0018] 2. The present invention provides a context dynamic tracking and course learning method for multi-turn dialogue perception. By designing independent recurrent neural network units to continuously track the speaker's intention, reaction and influence, and introducing feature interaction (such as response GRU considering the influence of the previous moment) in this process, the model can more meticulously depict the complex social dynamic interaction in the dialogue and deepen the understanding of emotional dynamics.

[0019] 3. This invention provides a context-based dynamic tracking and course learning method for multi-turn dialogue perception. It innovatively proposes a comprehensive difficulty assessment function based on the continuity of features across three dimensions: intent, response, and impact, combined with a course learning strategy involving dynamic weight adjustment. This progressive training method better aligns with human cognitive patterns, enabling the model to learn difficult samples more smoothly and effectively, ultimately achieving superior and more stable performance. Attached Figure Description

[0020] Figure 1 This is a flowchart of the dynamic tracking and course learning method of the present invention. Detailed Implementation

[0021] The present application will be described in further detail below with reference to the drawings. The advantages and features of the present application will become more apparent from the following description. It is to be noted that the drawings adopted a very simplified form and all use non-precise proportions, only for the purpose of facilitating, clarifying the purpose of assisting the description of the embodiments of the present application. In order to make the purpose, features and advantages of the present application more obvious and easy to understand, please refer to the drawings. It should be noted that the structure, proportion, size, etc. shown in the drawings attached to the present specification are only used to cooperate with the content disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the implementation of the present application. Therefore, any modification of structure, change of proportion relationship or adjustment of size, which does not affect the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0022] The present application provides a context dynamic tracking and curriculum learning method (ECCL) for multi-turn dialogue perception. The dynamic tracking and curriculum learning method is mainly realized through the following modules, which include: a speech feature extraction module, a common sense feature extraction module, a context dynamic tracker, and a speaker information module. The data first passes through the speech feature extraction module, and the semantic features of the speech are extracted using the RoBERTa model (robust optimization BERT pre-training method). Then, through the common sense feature extraction module, the COSMIC model (common sense inference model based on pre-training model) is used to enhance the feature representation in combination with the ATOMIC (atomic common sense) knowledge graph. Subsequently, the data enters the context dynamic tracker, which uses multi-head attention and context-aware GRU (gated recurrent unit) to capture the emotional coherence and historical dependency in the dialogue. Finally, the speaker information module further enhances the model's understanding of the speaker's dynamics by modeling the speaker's intention, reaction, and influence. The entire model adopts a curriculum learning strategy during training to address class imbalance by dynamically adjusting sample weights, optimizing model performance. These modules work together to enable ECCL to effectively handle emotional dynamics in complex dialogues, improving the accuracy and robustness of emotion recognition.

[0023] As shown in Figure 1 The dynamic tracking and curriculum learning method includes the following steps: Step S1, speech feature extraction in the speech feature extraction module: a first pre-training language model is used to extract the semantic feature vector of each speech in the input dialogue; wherein the first pre-training language model is a RoBERTa model.

[0024] The RoBERTa model is a fine-tuned version of RoBERTa Large, which inherits the architecture of the original BERT Large. The RoBERTa language model includes 24 encoding layers, 16 self-attention heads per layer, a hidden layer dimension of 1024, and a total of 355 million parameters.

[0025] Specifically, the specific process of extracting the semantic feature vector is as follows: S1.1, for each utterance, concatenate it with a special mark (such as [CLS]) to form an input sequence , and input the input sequence into the first pre-trained language model.

[0026] S1.2, extract the activation vector corresponding to the special mark from the last several layers (such as the last four layers) of the encoding layer of the first pre-trained language model.

[0027] S1.3, average pooling operation is performed on the extracted multiple activation vectors to obtain a 1024-dimensional context-independent semantic feature vector of the utterance : In the formula, [CLS] is a special mark, is an element in the input sequence.

[0028] Step S2, common sense feature extraction in the common sense feature extraction module: using a second pre-trained knowledge generation model to extract a common sense feature vector related to the emotional dynamics from the semantic feature vector ; the common sense feature vector includes at least an intention feature, a reaction feature, and an influence feature; wherein the second pre-trained knowledge generation model is a COMET model.

[0029] The intention feature represents the intention of the speaker to speak the utterance; the reaction feature represents the reaction of the speaker to the current situation or event; the influence feature represents the possible influence of the speaker's utterance on others; each feature is a 768-dimensional vector. This step provides rich semantic and common sense prior for subsequent modeling.

[0030] Specifically, the COSMIC model analyzes the dialogue in depth by using the common sense relationships in the ATOMIC knowledge graph; that is, the input sequence is regarded as the main phrase, and is combined with the semantic feature vector . Then, the combined result is input into the encoder of the COMET model for processing, and the intention feature, the reaction feature, and the influence feature are extracted from the last time step. This feature extraction method generates three different vectors for each sentence in the dialogue, and each vector has a dimension of 768.

[0031] Step S3, context dynamic tracking modeling is performed in the context dynamic tracker, and speaker information modeling is performed in the context dynamic tracking modeling by a speaker information module; Specifically, the context dynamic tracking modeling includes the following steps: S3.1.1, based on the semantic feature vector of the current utterance and the historical context information, spatial context information is extracted by using an attention mechanism (such as a multi-head attention mechanism): The process of obtaining the spatial context information is as follows: In the formula, Q represents a query, is a first learnable parameter matrix; represents an input embedding hidden state vector at the t-th time step; K represents a key; represents a context state vector, is a second learnable parameter matrix; V represents a value; represents a third learnable parameter matrix. In the formula, represents information reweighted according to the context; is a normalization function of the multi-head attention; is a key K dimension, used to scale the dot product. In the formula, represents the final output spatial context information; represents a layer normalization processing function; represents a regularization operation.

[0032] S3.1.2, a perception gated recurrent unit (C-GRU) is used to fuse the semantic feature vector of the current utterance, the spatial context information, and the common sense feature vector at the previous time step, to generate a context state vector at the current time step; The C-GRU unit is updated at each time step t of the dialog flow, which not only processes the current utterance feature vector The C-GRU unit also integrates key historical information from the previous time step, including the speaker's intention, reaction, and its impact. This design enables the C-GRU unit to receive an information-rich input vector at each time step, combining the current state of the conversation with its immediate history. As a result, the C-GRU unit can effectively track the evolution of emotions and intentions in the conversation, ensuring that the model's understanding of the conversation is continuous and deep. Additionally, the C-GRU unit's update is influenced not only by the current utterance but also significantly by the context from the previous time step, making the model more accurate and sensitive in predicting the speaker's next emotional state.

[0033] Specifically, the process of obtaining the context state vector is as follows: wherein, represents the context state vector; represents the processing function of the perception gate recurrent unit, represents the context memory at the previous time; represents the processed current utterance feature vector; represents the intention feature vector at the previous time; represents the reaction feature vector at the previous time; represents the impact feature vector at the previous time; represents vector concatenation.

[0034] The speaker information modeling updates the intention feature, reaction feature, and impact feature of the current time step t through three bidirectional gate recurrent units, respectively; Speaker intention is crucial for grasping the overall dynamics of the conversation. It reveals the underlying emotional layers and behavioral motivations in the conversation. This concept is particularly important when dealing with natural language and performing emotion recognition tasks, as it helps to reveal the implicit emotional dimensions and behavioral motivations in the conversation. From time step t-1 to time step t, this change is determined by the intention vector at the corresponding time step t and the speaker's intention at the previous time step t-1 .

[0035] Specifically, the first bidirectional gate recurrent unit receives the intention feature vector at the previous time step, the enhanced feature at the current time, and the specific intention information to update the speaker's intention feature vector at the current time step. The update process of the intention feature vector at the current time step is as follows: wherein, represents the speaker's intention feature vector at the current time step; ​a processing function of the first bidirectional gated recurrent unit; an intent feature vector of a previous time step; an enhanced feature; a specific intent input.

[0036] Speaker influence is crucial for identifying the dynamic of impact in a conversation, especially in scenarios involving group communication or multi-lateral interaction. This concept helps reveal how one person's utterance influences other participants' behavior, emotions, or decisions. At time step t , the change from to is captured by a specialized GRU unit, Ef-GRU. This unit updates the state using the common sense vector t and of the current time step , as well as the speaker influence t-1 of the previous time step . In this way, the model can continuously track the flow of influence between participants in a conversation, thus more accurately modeling and understanding the social dynamics and interactions in group decision-making processes.

[0037] Specifically, the second bidirectional gated recurrent unit is used to receive the influence feature vector of the previous time step, the enhanced feature of the current time, and the specific influence information, to update the speaker influence feature vector of the current time step, and the update process of the influence feature vector of the current time step is: wherein, represents the influence feature vector of the current time step; represents a processing function of the second bidirectional gated recurrent unit; represents the influence feature vector of the previous time step; represents a specific influence input.

[0038] Speaker reaction plays an important role in comprehensively understanding the content of a conversation, especially in situations where emotional dynamics are complex and varied. Speaker reaction not only demonstrates immediate feelings towards the current utterance, but also reflects long-term emotional reactions to the entire conversation. At time step t , the change from to is captured by a specialized GRU unit, Re-GRU, which considers the common sense vector t , of the current time step , the influence t-1 of the previous time step , and the speaker's reaction t-1 at time step to update the state. This method enables the model to continuously track the emotional reactions in the conversation, thus providing a deeper understanding of the emotional layer of the conversation and the interaction dynamics of the participants.

[0039] Specifically, the third bidirectional gated recurrent unit receives the reaction feature vector of the previous time step, the enhanced feature of the current time, the specific reaction information and the influence feature vector of the previous time step to update the speaker reaction feature vector of the current time step, and the updating process of the reaction feature vector of the current time step is as follows: In the formula, represents the reaction feature vector of the current time step; represents the processing function of the third bidirectional gated recurrent unit; represents the reaction feature vector of the previous time step; represents the specific reaction input; represents the influence feature vector of the previous time step.

[0040] Step S4, course learning training: introducing a dynamic weight distribution mechanism and a weighted loss function to adaptively train the dialogue samples.

[0041] Specifically, the step S4 comprises the following steps: S4.1, calculating the intention cosine similarity, influence cosine similarity and reaction cosine similarity between the continuous utterances of the dialogue to evaluate the emotional complexity and inertia in the dialogue.

[0042] For a dialogue containing N sentences, the intention feature vector of the i-th sentence in the dialogue is calculated , and the intention feature vector of the i+1-th sentence is calculated , and then the intention cosine similarity between the corresponding continuous utterances is calculated ; the calculation formula of the intention cosine similarity is as follows: In the formula, is a cosine similarity function.

[0043] Similarly, the calculation formula of the influence cosine similarity is as follows: In the formula, represents the influence feature vector of the i-th sentence in the dialogue; represents the influence feature vector of the i+1-th sentence in the dialogue.

[0044] The calculation formula of the reaction cosine similarity is as follows: wherein, represents the response feature vector of the i-th sentence in the dialogue; represents the response feature vector of the i+1-th sentence in the dialogue.

[0045] S4.2, evaluate the degree of emotional change in the dialogue through the intention cosine similarity, influence cosine similarity and response cosine similarity; wherein, is the difficulty of the sample, and the intensity of the response emotional change; The higher the value, the more intense the emotional change.

[0046] S4.3, dynamic weight allocation is performed on the dialogue samples in the training; Specifically, a weight function that changes with time t and classification difficulty is constructed, and the goal of the function is to assign a lower learning weight to more challenging utterances (i.e. difficult samples) in the early stage of training; as the training progresses, i.e. as the number of training rounds increases, the learning weight of these difficult samples will gradually increase.

[0047] The formula for allocating the dynamic weight is as follows: wherein, t represents the current training round, and is an adjustable parameter, represents the learning weight assigned to a certain sample in the t th training round.

[0048] S4.4, weighted loss training is performed on the dialogue samples after dynamic weight allocation; The calculation formula of the weighted loss training is as follows: wherein, is the total number of dialogues in the training set, is the number of sentences in the i th dialogue, is the true label of the i th sentence in the j th dialogue, is the probability distribution predicted by the model, and L is the sample result after weighted loss training.

[0049] The operating system used in the experiments of the present application is Ubuntu 20.04, and the programming language is Python 3.9. All experiments are carried out on a server equipped with an NVIDIA GeForce RTX 4090 graphics card, and the optimizer selected is AdamW.

[0050] To evaluate the effectiveness of the model of the present application, extensive experiments were conducted on four benchmark datasets. The IEMOCAP dataset contains 151 two-person conversation videos, totaling 7433 utterances, with an average of 11.6 words per utterance. Each utterance is labeled as one of the eight emotions: happy, sad, neutral, angry, excited, frustrated, fearful, and disgusted. MELD contains 13708 utterances from 1433 dialogues in the TV series Friends. Each utterance contains audio, visual, and textual modalities, with an average of 8.0 words per utterance. EmoryNLP is a textual dataset extracted from the script of the TV series Friends, containing 12606 utterances labeled as one of the six basic emotions in the Wilkoxs emotion wheel and a neutral label. DailyDialog is a large text dataset constructed from a website for practicing everyday English conversations, containing 13118 multi-turn dialogues, totaling 102879 utterances, with an average of 14.6 words per utterance. Each utterance is labeled as one of the six basic emotions of Ekman or other emotions.

[0051] Table 1. Comparative experimental results of ECCL on benchmark datasets From the comparative results in Table 1, on the IEMOCAP dataset, the context dynamic tracking and curriculum learning method (ECCL) for multi-turn dialogue perception provided by the present application achieved the best results, significantly outperforming other models. The average dialogue length of the IEMOCAP dataset is longer than other datasets, providing more rich context information. This verifies the ability of the dialogue dynamic tracking module (CDT) of ECCL to capture long-distance context dependencies. On the DailyDialog dataset, the Macro-F1 score of ECCL is 53.59%, which is 0.60 lower than EmotionIC. The main reason for the limited performance of ECCL is the class imbalance problem in the DailyDialog dataset, where 83% of the labels are neutral, and the dialogue length is short, which to some extent weakens the model's ability to capture speaker information. However, thanks to the curriculum learning strategy adopted by the model, which alleviates the class imbalance problem in the dataset, the model still outperforms other baseline models, demonstrating excellent dialogue understanding and broad applicability.

[0052] On the MELD dataset, the model emphasizing common sense knowledge significantly outperforms other models, highlighting the key role of common sense in understanding conversations. According to previous studies, the conversations in the MELD dataset are generally shorter, involve multiple participants, and have more complex emotional changes. Since ECCL can better observe and capture emotional coherence and fine-grained speaker information in conversations, it performs particularly well on this dataset. On the EmoryNLP dataset, the model demonstrates significant advantages by integrating common sense knowledge, enhancing the speaker information module, and the powerful memory function of CDT. These features enable the model to effectively handle complex conversation structures and emotional flow, achieving the highest score. This result not only verifies the efficiency of the model but also highlights its ability to capture deep emotional dynamics in conversations, especially in scenarios involving complex interactions and multiple speakers.

[0053] To verify the contribution of each component in the model, a series of ablation experiments were designed, as shown in Table 2 below. Specifically, the curriculum learning strategy (w / o CL), the conversation dynamic tracking module (w / o CDT), and the speaker information module (w / o SI) were removed from the complete ECCL model, respectively. When the curriculum learning strategy is not used, the performance of the model on all four datasets decreases, with a decrease of 0.70% and 0.84% on the IEMOCAP and DailyDialog datasets, respectively. This highlights the effectiveness of the curriculum learning strategy. This strategy shows particularly significant improvement on the DailyDialog dataset, demonstrating its strong ability to effectively address class imbalance issues and improve model performance.

[0054] Table 2. Ablation study results of different modules The experimental results show that when the dialogue dynamic tracking module is removed, the performance of the model on all data sets has a significant decline, especially on the IEMOCAP data set with longer dialogue length, and this downward trend is more obvious. IEMOCAP data set is known for its complex dialogue structure and rich emotional expression, and the dialogue often involves multiple rounds of interaction, and the evolution of emotional state is also more complex, which highly depends on the capture of wide sequence context information. Therefore, the importance of the CDT module on this data set is self-evident, which can effectively track the dynamic changes of the dialogue and provide key context information for the model, thereby helping the model more accurately understand the evolution process of emotion. Further, the speaker information module is also removed, and it is observed that the model has the most significant performance decline on the MELD and EmoryNLP data sets. Both of these data sets are multi-party dialogue data sets, and the dialogue not only involves multiple speakers, but also the frequent transition of emotion and intention. In this complex dialogue scenario, the speaker information module can help the model better understand the emotional interaction and intention transmission between different speakers. For example, in multi-party dialogue, the emotional expressions of different speakers may influence each other, even trigger emotional resonance or conflict. The speaker information module can help the model identify the subject of these emotional interactions, so as to more accurately grasp the evolution trend of emotion.

[0055] In summary, the present application provides a kind of context dynamic tracking and course learning method for multi-round dialogue perception, which can more accurately and robustly identify the emotional scheme in multi-round dialogue. Through the fusion of dynamic context tracking, fine-grained speaker modeling and adaptive curriculum learning, the understanding ability and recognition performance of the model for complex dialogue emotional dynamics are comprehensively improved.

[0056] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the existence of other identical elements in the process, method, article or equipment including the element.

[0057] In the description of the application, it is to be understood by the terms "center", "height", "thickness", "upper", "lower", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specified and limited.

[0058] In the description of the application, unless otherwise specified and limited, the terms "mounting", "connecting", "connecting", "fixing" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0059] In the present application, unless otherwise specified and limited, the first feature "on" or "under" the second feature can include that the first and second features are in direct contact, or that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, the first feature "on", "above" and "above" the second feature includes that the first feature is directly above and obliquely above the second feature, or only indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature "under", "below" and "below" the second feature includes that the first feature is directly below and obliquely below the second feature, or only indicates that the horizontal height of the first feature is less than that of the second feature.

[0060] Although the content of the present application has been described in detail through the above preferred embodiments, it should be recognized that the above description should not be considered as a limitation on the present application. After reading the above content, various modifications and alternatives of the present application will be obvious to those skilled in the art. Therefore, the scope of protection of the present application should be defined by the appended claims.

Claims

1. A context dynamic tracking and curriculum learning method for multi-turn dialogue perception, characterized in that, The method comprises the following steps: S1, speech feature extraction in a speech feature extraction module: extracting a semantic feature vector of each speech in the input dialogue through a first pre-trained language model; the first pre-trained language model is a RoBERTa model; S2, common sense feature extraction in a common sense feature extraction module: extracting a common sense feature vector related to emotional dynamics from the semantic feature vector through a second pre-trained knowledge generation model; the second pre-trained knowledge generation model is a COMET model; S3, context dynamic tracking modeling through a context dynamic tracker, and speaker information modeling through a speaker information module in the context dynamic tracking modeling; S4, course learning training: introducing a dynamic weight distribution mechanism and a weighted loss function to adaptively train the dialogue samples.

2. The dynamic tracking and course learning method of claim 1, wherein, The step S1 comprises the following steps: S1.1, for each utterance, concatenating it with special tokens to form an input sequence and inputting the input sequence into the first pre-trained language model; S1.2, extracting an activation vector corresponding to the special mark from an encoding layer of the first pre-trained language model; S1.3, performing an average pooling operation on the extracted plurality of activation vectors to obtain a semantic feature vector of the utterance : In the formula, [CLS] is a special token, is an element in the input sequence.

3. The dynamic tracking and course learning method of claim 2, wherein, The COMET model incorporates common sense relationships in the ATOMIC knowledge graph in training to deeply analyze the speech content; the common sense feature vector at least includes an intention feature, a reaction feature and an influence feature.

4. The dynamic tracking and course learning method of claim 3, wherein, The context dynamic tracking modeling comprises the following steps: S3.1.1, based on the semantic feature vector of the current speech and the historical context information, extracting spatial context information using an attention mechanism; S3.1.2, using a perception gated recurrent unit to fuse the semantic feature vector of the current speech, the spatial context information and the common sense feature vector of the previous time step to generate a context state vector of the current time step.

5. The method of claim 4, wherein the step of dynamically tracking and learning comprises the step of: The process of extracting spatial context information using an attention mechanism in S3.1.1 is: ​ where Q denotes a query, is a first learnable parameter matrix; denotes an input embedding of the query Q at the t-th time step, K denotes a key, denotes a context state vector, denotes a second learnable parameter matrix; V denotes a value, denotes a third learnable parameter matrix; wherein, denotes the information re-weighted according to the context; is a normalization function for multi-headed attention; is the dimension of the key K for scaling the dot product; In the formula, represents the spatial context information of the final output; represents a layer normalization processing function; represents a regularization operation.

6. The dynamic tracking and course learning method of claim 5, wherein, The process of obtaining the context state vector of the current time step in S3.1.2 is: In the formula, denotes a context state vector; denotes a processing function of the perception gating recurrent unit, denotes a context memory of the last moment; denotes a processed current utterance feature vector; denotes an intention feature vector of the last moment; denotes a reaction feature vector of the last moment; denotes an influence feature vector of the last moment; denotes vector splicing.

7. The dynamic tracking and course learning method of claim 6, wherein, The speaker information modeling updates the intention feature, the reaction feature and the influence feature of the current time step t through three bidirectional gated recurrent units respectively; wherein the first bidirectional gated recurrent unit receives the intention feature vector of the previous time step, the enhanced feature of the current time and the specific intention information to update the speaker intention feature vector of the current time step; the second bidirectional gated recurrent unit receives the influence feature vector of the previous time step, the enhanced feature of the current time and the specific influence information to update the speaker influence feature vector of the current time step; the third bidirectional gated recurrent unit receives the reaction feature vector of the previous time step, the enhanced feature of the current time, the specific reaction information and the influence feature vector of the previous time step to update the speaker reaction feature vector of the current time step.

8. The dynamic tracking and course learning method of claim 7, wherein The update process of the intention feature vector of the current time step is: In the formula, represents the speaker intention feature vector of the current time step; represents a processing function of the first bidirectional gate recurrent unit; represents the intention feature vector of the previous time step; represents an enhanced feature; representing a specific intent input; The update process of the influence feature vector of the current time step is: wherein represents an influence feature vector of a current time step; represents a processing function of a second bidirectional gated recurrent unit; represents an influence feature vector of a previous time step; represents a particular influence input; The update process of the reaction feature vector of the current time step is: wherein represents a reaction feature vector of a current time step; represents a processing function of a third bidirectional gated recurrent unit; represents a reaction feature vector of a previous time step; represents a specific reaction input; represents an influence feature vector of a previous time step.

9. The dynamic tracking and course learning method of claim 8, wherein, The step S4 comprises the following steps: S4.1, calculating the intention cosine similarity, the influence cosine similarity and the reaction cosine similarity between the continuous speeches of the dialogue to evaluate the emotional complexity and inertia in the dialogue; S4.2, evaluating the degree of emotion change in the dialogue by the intention cosine similarity, the influence cosine similarity and the reaction cosine similarity; S4.3, performing dynamic weight distribution on the dialogue samples in the training; S4.4, performing weighted loss training on the dialogue samples after dynamic weight distribution; In the step S4.1, the formula for calculating the intended cosine similarity is: In the formula, is a cosine similarity function; N represents that N sentences are contained in the dialogue sample; represents an intent feature vector of the i-th sentence in the dialogue sample; represents an intent feature vector of the i+1-th sentence. The formula for calculating the impact cosine similarity is: wherein represents the influence feature vector of the i-th sentence in the dialogue; represents the influence feature vector of the i+1-th sentence in the dialogue; The reaction cosine similarity The calculation formula is: wherein represents the reaction feature vector of the i-th sentence in the dialogue; represents the reaction feature vector of the i+1-th sentence in the dialogue; The emotion evaluation formula in the step S4.2 is: In the formula, is the degree of difficulty of the sample, the degree of change in emotional response; The higher the value, the more intense the emotional change.

10. The dynamic tracking and course learning method according to claim 9, characterized in that, The dynamic weight distribution formula in the step S4.3 is as follows: wherein t denotes the current training round, and is an adjustable parameter, denotes the learning weight assigned to a certain sample in the t round of training; The calculation formula of the weighted loss training in the step S4.4 is: In the formula, It is the total number of training-focused dialogues. It is the first i The number of sentences in a dialogue. It is the first i In the first dialogue j The true label of a sentence is the probability distribution predicted by the model, and L is the sample result after training with weighted loss.

Images