Model training method and device, and computer readable storage medium

Abstract

The application discloses a model training method and device and a computer readable storage medium. The method comprises the following steps: obtaining first sample data of a current batch; inputting the first sample data into a target model to obtain a first predicted sentiment label of each sample speech; shielding a context representation vector or an external knowledge representation vector in the first sample data to obtain second sample data; inputting the second sample data into the target model to obtain a second predicted sentiment label of each sample speech; for each sample speech, setting a corresponding first target label according to whether the first predicted sentiment label and the second predicted sentiment label of the sample speech are the same; performing supervised contrast loss calculation according to a speech representation vector of each sample speech and the first target label of each sample speech; and training the target model according to a first loss value obtained. The method can improve the performance of the model.

Description

Technical Field

[0001] This application belongs to the field of natural language processing, and in particular relates to a training method, apparatus, and computer-readable storage medium for a model. Background Technology

[0002] Existing training tools for sentiment analysis include Baidu AI Development Platform's EasyDL and Eagle AI's Natural Language Processing (NLP) platform. These platforms primarily target general sentiment analysis methods, rather than sentence-level sentiment analysis methods specific to dialogue contexts. During training, these models do not consider contextual information or the embedding of external knowledge, nor do they optimize the model's robustness to irrelevant context and external knowledge. This results in contextual and external knowledge, which offer limited help in identifying the sentiment of the current utterance, acting as "noise" and having a negative impact. Summary of the Invention

[0003] This application provides a training method, apparatus, and computer-readable storage medium for a model that can fully consider the context or embedding of external knowledge in the sample discourse, thereby improving the performance of the model.

[0004] To address the aforementioned technical problems, a first aspect of this application provides a model training method, the method comprising: acquiring first sample data of a current batch, the first sample data including at least one representation vector of a sample utterance, a context representation vector of each sample utterance, and an external knowledge representation vector of each sample utterance; inputting the first sample data into a target model to obtain a first predicted sentiment label for each sample utterance; masking the context representation vector of each sample utterance in the first sample data, or masking the external knowledge representation vector of each sample utterance in the first sample data to obtain second sample data; inputting the second sample data into the target model to obtain a second predicted sentiment label for each sample utterance; for each sample utterance, if the first predicted sentiment label and the second predicted sentiment label of the sample utterance are the same, setting the first target label corresponding to the sample utterance to a first label value, otherwise setting it to a second label value; performing supervised contrastive loss calculation based on the representation vector of each sample utterance and the first target label of each sample utterance to obtain a first loss value; and training the target model based on the first loss value.

[0005] A second aspect of this application provides a training device, which includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit, respectively. The memory stores program data, and the processor executes the program data in the memory to implement the steps in the above method.

[0006] A third aspect of this application provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the steps in the above-described method.

[0007] The beneficial effects are as follows: Unlike existing technologies, in the training process of the target model, this application first inputs the first sample data into the target model for sentiment prediction to obtain the first predicted sentiment label; then, it masks the context representation vector or the external knowledge representation vector to obtain the second sample data, and inputs the second sample data into the target model to obtain the second predicted sentiment label. The first target label is obtained by comparing the first and second predicted sentiment labels; finally, it uses supervised contrastive loss calculation to bring similar discourses closer together and distance discourses of different natures apart, reducing the negative effect of context and external knowledge, which have little help in the sentiment recognition of the current discourse, as "noise". This fully considers the embedding of context and external knowledge of the sample discourse and improves the performance of the target model. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0009] Figure 1 This is a flowchart illustrating one implementation method of the training method for the model in this application;

[0010] Figure 2 This is a schematic diagram of the interactive UI for model training in this application;

[0011] Figure 3 This is a schematic diagram of the model annotation stage in this application;

[0012] Figure 4 This is a schematic diagram of the model training phase in this application;

[0013] Figure 5 yes Figure 1 A flowchart of step S170 in an application scenario;

[0014] Figure 6 yes Figure 1 A flowchart illustrating step S170 in another application scenario;

[0015] Figure 7 yes Figure 3 A flowchart illustrating step S1707;

[0016] Figure 8 yes Figure 1 A flowchart illustrating step S170 in another application scenario;

[0017] Figure 9 yes Figure 5 A flowchart illustrating step S1710;

[0018] Figure 10 yes Figure 1 A flowchart illustrating step S170 in another application scenario;

[0019] Figure 11 yes Figure 1 A flowchart illustrating step S170 in another application scenario;

[0020] Figure 12 This is a flowchart illustrating step S130 of this application;

[0021] Figure 13 This is a schematic diagram of one embodiment of the training device of this application;

[0022] Figure 14 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that the terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0025] See Figure 1 , Figure 1This is a flowchart illustrating one embodiment of the training method for the model in this application. The method is executed by a training device, which can be any device with processing capabilities, such as a mobile phone or a computer. The method includes:

[0026] S110: Obtain the first sample data of the current batch. The first sample data includes the discourse representation vector of at least one sample discourse, the context representation vector of each sample discourse, and the external knowledge representation vector of each sample discourse.

[0027] Specifically, the sample discourse is a single sentence selected from a dialogue text. The discourse representation vector is the representation vector of the discourse text. The context representation vector is the representation vector of the discourse text before and after the occurrence of the discourse text. There are two types of external knowledge representation vectors: one is the common sense knowledge representation vector, and the other is the knowledge representation vector associated with the words contained in the discourse text.

[0028] The first sample data includes discourse representation vectors, context representation vectors, and external knowledge representation vectors, which are obtained through vectorization. In one implementation, discourse representation vectors and context representation vectors can be obtained through various word vector models and various time-series models, while the common-sense knowledge representation vectors of the external knowledge representation vectors are directly generated through the adaptive generative model COMET (generative knowledge representation vectors), and another type of knowledge representation vector, which is associated with the words contained in the discourse text, is obtained by retrieving from an external knowledge base (conceptual knowledge representation vectors).

[0029] In another embodiment, see Figure 2 In area A, users can choose between Initial Model 1, Initial Model 2, and a custom model. If they choose Initial Model 1 or Initial Model 2, they do not need to specify any variables. If they choose a custom model, they can specify context variables in area B or external knowledge variables in area C. The specified context variables or external knowledge variables are vectorized to obtain a context representation vector or external knowledge representation vector defined by the user.

[0030] S120: Input the first sample data into the target model to obtain the first predicted sentiment label for each sample utterance.

[0031] The first predicted sentiment label is the result of the target model's first sentiment prediction on the first sample data, and it is the label corresponding to the maximum probability in the sentiment state output by the target model.

[0032] Specifically, for ease of understanding, let's illustrate with an example. Assume the current batch is the first sample data containing three sentences, with sentiment categories of: gray: 0, black: 1, white: 2. The first sample data is fed into the target model, yielding the target model's output sentiment state z. This sentiment state z represents the probability that each of the three sentences will be predicted to have its corresponding sentiment category by the target model. For example, if the first sample data contains three sentences, the probability that the target model predicts the sentiment state z of the first sentence to be gray (0) is... The probability of black (1) is The probability of white (2) is Therefore, the first sentence takes the maximum probability. The corresponding gray (0) is used as the predicted sentiment label. Similarly, for each sentence in the first sample data, the maximum value of z is taken as the prediction result, and the corresponding first predicted sentiment label can be obtained.

[0033] S130: Mask the context representation vector of each sample utterance in the first sample data, or mask the external knowledge representation vector of each sample utterance in the first sample data to obtain the second sample data.

[0034] Specifically, shielding the context representation vector in the first sample data means removing the information of the context representation vector in the first sample data. Similarly, shielding the external knowledge representation vector in the first sample data means removing the information of the external knowledge representation vector in the first sample data.

[0035] In this embodiment, step S130 includes: setting the context representation vector in the first sample data to zero or setting the external knowledge representation vector in the first sample data to zero.

[0036] Alternatively, in other embodiments, step S130 includes: removing the context representation vector from the first sample data, or removing the external knowledge representation vector from the first sample data.

[0037] S140: Input the second sample data into the target model to obtain the second predicted sentiment label for each sample utterance.

[0038] Specifically, step S140 is similar to step S120, and can be found in step S120 above, which will not be repeated here.

[0039] S150: For each sample utterance, if the first predicted sentiment label and the second predicted sentiment label of the sample utterance are the same, set the first target label corresponding to the sample utterance to the first label value; otherwise, set it to the second label value.

[0040] Specifically, the results of the first and second predicted sentiment labels are compared. If the sentiment labels are the same, the first target label is set to the first label value; if the sentiment labels are different, it is set to the second label value. The first label value can be 1, indicating that the first and second predicted sentiment labels of the sample utterance are the same, representing a positive sample, i.e., context-independent or knowledge-independent utterance. The second label value can be 0, indicating that the first and second predicted sentiment labels of the sample utterance are different, representing a negative sample, i.e., context-sensitive or knowledge-sensitive utterance. For ease of understanding, an example is provided below; please refer to [link / reference]. Figure 3 Assuming the first predicted sentiment label Second predictive sentiment tag The first and second predicted sentiment labels share the same sentiment label only in the second item, representing a positive sample, for which the first label value is set. The first and third predicted sentiment labels differ, representing a negative sample, for which the second label value is set. Therefore, the first target label...

[0041] S160: Supervised contrastive loss is calculated based on the discourse representation vector of each sample discourse and the first target label of each sample discourse to obtain the first loss value.

[0042] Specifically, the calculation of the supervised contrastive loss term enables the target model to distinguish between context-sensitive utterances and context-independent utterances. During training, the target model will bring utterances of the same nature closer together and pull away utterances of different natures, thereby enhancing the target model's ability to denoise irrelevant contexts and improving the performance of the target model.

[0043] For ease of understanding, please refer to Figure 3 and Figure 4 To illustrate with examples, the encoder module (which may take different forms in different models) vectorizes the input dialogue, context, and external knowledge text, resulting in discourse representation vectors x = {x1, x2, ..., x...}. n}, Context representation vector c = {c1, c2, ..., c n External knowledge representation vector k = {k1, k2, ..., k} n}, where n represents the number of utterances, x i c i k i Let x represent the utterance representation vector, context representation vector, and knowledge representation vector of the i-th sentence, respectively. A training batch of data is extracted from the utterance representation x, context representation c, and external knowledge representation k. The figure shows a batch of data containing three sentences (the batch size is adjustable and not a fixed value). The first predicted sentiment label is... This indicates that the second predicted sentiment label output by the target model after masking the context is used, and then the first predicted sentiment label is used as the basis for the second predicted sentiment label. Second predictive sentiment label The result is used to label the discourse with its primary objective: The formula for calculating the supervised contrastive loss of the first loss value is shown below:

[0044]

[0045] Where, N b Batch data size: 3, x i This represents the vector representation of the i-th sentence in the batch data. Corresponding to the first target label, simi(.) represents calculating the cosine similarity between two vectors. The indicator function is represented by τ = 0.07, which represents the temperature parameter in supervised comparative learning.

[0046] S170: Train the target model based on the first loss value.

[0047] Specifically, training is performed based on the first loss value. During the training process, the target model will bring closer discourses of the same nature and distance discourses of different natures.

[0048] Beneficial effects: Unlike existing technologies, this application, during the training process of the target model, first inputs the first sample data into the target model for sentiment prediction to obtain the first predicted sentiment label; then, it masks the context representation vector or the external knowledge representation vector to obtain the second sample data, and inputs the second sample data into the target model to obtain the second predicted sentiment label. The first target label is obtained by comparing the first and second predicted sentiment labels; finally, it uses supervised contrastive loss calculation to bring similar discourses closer together and distance discourses of different natures apart, reducing the negative effect of context and external knowledge, which have little help in the sentiment recognition of the current discourse, as "noise". This fully considers the embedding of context and external knowledge of the sample discourse and improves the performance of the target model.

[0049] In one application scenario, the second sample data is obtained by masking the contextual representation vector of each sample utterance in the first sample data. (See [link / reference]). Figure 5 The steps for training the target model based on the first loss value include:

[0050] S1701: Mask the external knowledge representation vector of each sample utterance in the first sample data to obtain the third sample data.

[0051] Specifically, in the application scenario, the context representation vector of each utterance in the first sample data is masked to obtain the second sample data, and the external knowledge representation vector of each utterance in the first sample data is masked to obtain the third sample data.

[0052] S1702: Input the third sample data into the target model to obtain the third predicted sentiment label for each sample utterance.

[0053] Specifically, this step is similar to step S140, which can be found in step S140 above and will not be repeated here.

[0054] S1703: For each sample utterance, if the first predicted sentiment label and the third predicted sentiment label of the sample utterance are the same, set the second target label corresponding to the sample utterance to the third label value; otherwise, set it to the fourth label value.

[0055] Specifically, the results of the first and third predicted sentiment labels are compared. If the sentiment labels are the same, the second target label is set as the third label value; if the sentiment labels are different, the second target label is set as the fourth label value. The third label value can be 1, and the fourth label value can be 0 or other label values. The only requirement is to distinguish between the third and fourth label values, and there are no restrictions here.

[0056] To facilitate understanding, we will now combine... Figure 3 To explain, the first prediction is the sentiment tag. And third-party predictive sentiment tags The second and third sentiment labels of the first and third predicted sentiment labels are the same, indicating a positive sample. Therefore, the third label value is set to 1. The first sentiment label is different, indicating a negative sample. Therefore, the fourth label value is set to 0. Therefore, the second target label...

[0057] S1704: Supervised contrastive loss is calculated based on the discourse representation vector of each sample discourse and the second target label of each sample discourse to obtain the second loss value.

[0058] Specifically, the calculation of the supervised contrastive loss term enables the target model to distinguish between knowledge-sensitive discourse and knowledge-irrelevant discourse. During training, the target model will bring discourse of the same nature closer together and distance discourse of different natures apart, thereby enhancing the target model's ability to denoise irrelevant contexts and improving the performance of the target model.

[0059] To facilitate understanding, we will now illustrate this with an example. Assume that the discourse representation vector is x = {x1, x2, ..., x...} n}, Context representation vector c = {c1, c2, ..., c n External knowledge representation vector k = {k1, k2, ..., k} n}, where n represents the number of utterances, x i k i Let and represent the discourse representation and knowledge representation of the i-th sentence, respectively, and let the first predicted sentiment label be . The third predicted sentiment label output by the target model after shielding external knowledge is represented as follows: Then, based on the first predicted sentiment tag And third-party predictive sentiment tags The results are used to label the discourse with a second target: The formula for calculating the supervised contrast loss of the second loss value is as follows:

[0060]

[0061] The specific meaning of each function in the formula for calculating the supervised comparison loss has been described in detail in step S160, and will not be repeated here.

[0062] S1705: Based on the first loss value and the second loss value, obtain the total loss value.

[0063] The total loss value can be calculated by simply summing the first loss value and the second loss value, or by weighted summing the first loss value and the second loss value, or by averaging the first loss value and the second loss value. In this embodiment, the preferred method is to calculate the total loss value by weighted summing the first loss value and the second loss value.

[0064] S1706: Train the target model based on the total loss value.

[0065] Specifically, training is performed based on the total loss value. During the training process, the target model will bring closer discourses of the same nature and distance discourses of different natures.

[0066] In another application scenario, see [reference] Figure 6 Step S170, which involves training the target model based on the first loss value, specifically includes:

[0067] S1707: Calculate the loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance to obtain the third loss value.

[0068] Among them, the true sentiment label of each sample utterance, that is, the true sentiment label of the first sample data in the current batch, is certain and will not be affected by the target model.

[0069] S1708: Based on the first loss value and the third loss value, obtain the total loss value.

[0070] The total loss value can be calculated by simply summing the first loss value and the third loss value, or by weighted summing the first loss value and the third loss value, or by averaging the first loss value and the third loss value. In this embodiment, the preferred method is to calculate the total loss value by weighted summing the first loss value and the third loss value.

[0071] S1709: Train the target model based on the total loss value.

[0072] Specifically, training is performed based on the total loss value. During the training process, the target model will bring closer discourses of the same nature and distance discourses of different natures.

[0073] In this application scenario, see Figure 7 Step S1707, which calculates the loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance to obtain the third loss value, specifically includes:

[0074] S17071: Calculate the third loss value by performing cross-entropy loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance.

[0075] Specifically, the formula for calculating the cross-entropy loss of the target model is as follows:

[0076]

[0077] Where, N b N represents the size of the batch data. e Z_i represents the total number of emotion categories, class represents the category number of the true emotion, and Z_i is an emotion probability vector of size Ne.

[0078] It should be noted that, in addition to the cross-entropy loss function, the squared loss function, the log loss function (LogLoss), and the hinge loss function can also be used.

[0079] To facilitate understanding, we will now illustrate this with an example. Assume the current batch is the first sample data containing three sentences, with sentiment categories of: gray: 0, black: 1, white: 2, and the true sentiment label y = {0, 1, 2}. Then N b =3, N e =3, class={0,1,2}.

[0080] In another application scenario, see [reference] Figure 8 Step S170, which involves training the target model based on the first loss value, further includes:

[0081] S1710: Supervised contrastive loss is calculated based on the discourse representation vector of each sample discourse and the true sentiment label of each sample discourse to obtain the fourth loss value.

[0082] The formula for calculating the supervised comparison loss of the fourth loss value is as follows:

[0083]

[0084] The specific meaning of each function in the formula for calculating the supervised comparison loss has been described in detail in step S160, and will not be repeated here.

[0085] S1711: Based on the first loss value and the fourth loss value, obtain the total loss value.

[0086] The total loss value can be calculated by simply summing the first loss value and the fourth loss value, or by weighted summing the first loss value and the fourth loss value, or by averaging the first loss value and the fourth loss value.

[0087] S1712: Train the target model based on the total loss value.

[0088] Specifically, training is performed based on the total loss value. During the training process, the target model will bring closer discourses of the same nature and distance discourses of different natures.

[0089] In this application scenario, see Figure 9 Step S1710, which involves obtaining the total loss value based on the first loss value and the fourth loss value, includes:

[0090] S17101: The first loss value and the fourth loss value are weighted and summed to obtain the total loss value.

[0091] Specifically, when performing a weighted summation of the first and fourth loss values, the weights of the first and fourth loss values ​​can be set according to the actual situation. For example, if the weights of the first and second loss values ​​are set to 1 and 0.5 respectively, then the total loss value...

[0092] In another application scenario, see [reference] Figure 10 Step S170, which involves training the target model based on the first loss value, includes:

[0093] S1713: Determine the total loss function based on the first loss value and the regularization term, wherein the regularization term is determined based on the regularization coefficient and the set of trainable parameters of the target model.

[0094] Specifically, the formula for calculating the total loss is as follows:

[0095]

[0096] There is no specific numerical limit for λ1. The specific meaning of each function in the total loss calculation formula has been described in detail in step S17101, and will not be repeated here.

[0097] S1714: Train the target model based on the total loss value.

[0098] Specifically, training is performed based on the total loss value. During the training process, the target model will bring closer discourses of the same nature and distance discourses of different natures.

[0099] In another application scenario, see [reference] Figure 11 Step S170, which involves training the target model based on the first loss value, includes:

[0100] S1715: Determine the total loss value based on the first loss value, the second loss value, the third loss value, the fourth loss value, and the regularization term.

[0101]

[0102] Where β represents the coefficient of L2 regularization, θ represents the set of trainable parameters of the model, and λ1, λ2, λ3, and λ4 have no specific numerical restrictions.

[0103] Understandably, in other application scenarios, the total loss value can be calculated based on the second, third, and fourth loss values, multiple regularization terms, and the first loss value. For example, when the sample data only masks the context representation vector, the total loss value can be calculated based on the first, third, and fourth loss values ​​and the regularization term.

[0104] S1716: Train the target model based on the total loss value.

[0105] See Figure 12 Step S130, which involves masking the context representation vector of each sample utterance in the first sample data, or masking the external knowledge representation vector of each sample utterance in the first sample data, to obtain the second sample data, includes:

[0106] S1301: Set the context representation vector of each sample utterance in the first sample data to zero, or set the external knowledge representation vector of each sample utterance in the first sample data to zero, to obtain the second sample data.

[0107] Specifically, masking means setting the context representation vector or external knowledge representation vector to zero. That is, the second sample data is obtained by setting the context representation vector in the first sample data to zero, or by setting the external knowledge representation vector in the first sample data to zero.

[0108] See Figure 13 , Figure 13 This is a schematic diagram of one embodiment of the training device of this application. The training device 200 includes a processor 210, a memory 220, and a communication circuit 230. The processor 210 is coupled to the memory 220 and the communication circuit 230 respectively. The memory 220 stores program data. The processor 210 executes the program data in the memory 220 to implement the steps in any of the above embodiments. The detailed steps can be found in the above embodiments and will not be repeated here.

[0109] The training device 200 can be any device with text processing capabilities, such as a computer or a mobile phone, and there are no restrictions on its use.

[0110] See Figure 14 , Figure 14 This is a schematic diagram of one embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 300 stores a computer program 310, which can be executed by a processor to implement the steps in any of the above methods.

[0111] Specifically, the computer-readable storage medium 300 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a device that can store the computer program 310. Alternatively, it can be a server that stores the computer program 310, which can send the stored computer program 310 to other devices for execution, or it can run the stored computer program 310 itself.

[0112] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for training a model, characterized in that, The method includes: Obtain the first sample data of the current batch, the first sample data including at least one utterance representation vector of the sample utterance, the context representation vector of each sample utterance and the external knowledge representation vector of each sample utterance; Input the first sample data into the target model to obtain the first predicted sentiment label for each sample utterance; The context representation vector of each sample utterance in the first sample data is masked, or the external knowledge representation vector of each sample utterance in the first sample data is masked, to obtain the second sample data; The second sample data is input into the target model to obtain the second predicted sentiment label for each sample utterance; For each of the sample utterances, if the first predicted sentiment label and the second predicted sentiment label of the sample utterance are the same, the first target label corresponding to the sample utterance is set to the first label value; otherwise, it is set to the second label value. A first loss value is obtained by performing supervised contrastive loss calculation based on the discourse representation vector of each sample discourse and the first target label of each sample discourse. The formula for calculating the supervised contrastive loss is as follows: ， in, For the size of the batch data, This represents the vector representation of the i-th sentence in the batch of data. The first target label is represented by simi(.), which represents the cosine similarity between the two vectors. Let τ be the indicator function, and let τ represent the temperature parameter in the supervised comparative learning. The target model is trained based on the first loss value.

2. The method according to claim 1, characterized in that, The second sample data is obtained by masking the context representation vector of each sample utterance in the first sample data; The step of training the target model based on the first loss value includes: By masking the external knowledge representation vector of each sample utterance in the first sample data, a third sample data is obtained; The third sample data is input into the target model to obtain the third predicted sentiment label for each sample utterance; For each of the sample utterances, if the first predicted sentiment label of the sample utterance is the same as the third predicted sentiment label, the second target label corresponding to the sample utterance is set to the third label value; otherwise, it is set to the fourth label value. A second loss value is obtained by performing supervised contrastive loss calculation based on the discourse representation vector of each sample discourse and the second target label of each sample discourse. The formula for calculating the supervised contrastive loss is as follows: ， in, Indicates the second target label; Based on the first loss value and the second loss value, the total loss value is obtained; The target model is trained based on the total loss value.

3. The method according to claim 1, characterized in that, The step of training the target model based on the first loss value includes: A third loss value is obtained by calculating the loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance; The total loss value is obtained based on the first loss value and the third loss value; The target model is trained based on the total loss value.

4. The method according to claim 3, characterized in that, The step of calculating the loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance to obtain the third loss value includes: The third loss value is obtained by calculating cross-entropy loss based on the true sentiment label of each sample utterance and the first predicted sentiment label of each sample utterance.

5. The method according to claim 1, characterized in that, The step of training the target model based on the first loss value includes: A fourth loss value is obtained by performing supervised contrastive loss calculation based on the representation vector of each sample utterance and the true sentiment label of each sample utterance. The formula for calculating the supervised contrastive loss includes: ， Among them, y i Indicates true emotional tags; The total loss value is obtained based on the first loss value and the fourth loss value; The target model is trained based on the total loss value.

6. The method according to claim 5, characterized in that, The step of obtaining the total loss value based on the first loss value and the fourth loss value includes: The first loss value and the fourth loss value are weighted and summed to obtain the total loss value.

7. The method according to claim 1, characterized in that, The step of training the target model based on the first loss value includes: The total loss value is determined based on the first loss value and the regularization term, wherein the regularization term is determined based on the regularization coefficient and the set of trainable parameters of the target model; The target model is trained based on the total loss value.

8. The method according to claim 1, characterized in that, The step of masking the context representation vector of each sample utterance in the first sample data, or masking the external knowledge representation vector of each sample utterance in the first sample data, to obtain the second sample data includes: The second sample data is obtained by setting the context representation vector of each sample utterance in the first sample data to zero, or by setting the external knowledge representation vector of each sample utterance in the first sample data to zero.

9. A training device, characterized in that, The training device includes a processor, a memory, and a communication circuit. The processor is coupled to the memory and the communication circuit. The memory stores program data. The processor executes the program data in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images