Model training, dialogue processing method, device, equipment and storage medium

Abstract

The present disclosure relates to a model training and dialogue processing method, apparatus, device, and storage medium. This disclosure incorporates dialogue-specific information, such as turns and roles, into the training of a dialogue pre-trained language model. This allows the resulting dialogue vector representation model to effectively model the information interactions between multiple roles that are unique to dialogues. Consequently, when the dialogue vector representation model is used to generate a representation vector corresponding to target dialogue data, the accuracy of the representation vector can be improved, resulting in a superior dialogue representation.

Description

Technical Field

[0001] The present disclosure relates to the field of information technology, and in particular to a model training and dialogue processing method, apparatus, device, and storage medium. Background Art

[0002] With the continuous development of technology, the application of intelligent question-answering systems is becoming more and more widespread. During each intelligent question-answering process, a conversation is generated. In some application scenarios, each conversation needs to be represented by a vector.

[0003] However, the inventors of the present application have discovered that the prior art cannot accurately convert a conversation into a representation vector, resulting in downstream applications that need to be applied to the representation vector being unable to accurately implement their corresponding functions. Summary of the Invention

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a model training, dialogue processing method, device, equipment and storage medium to improve the accuracy of the representation vector of dialogue data so that the representation vector can achieve good dialogue representation effect.

[0005] In a first aspect, an embodiment of the present disclosure provides a model training method, comprising:

[0006] generating second dialogue data based on the first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data;

[0007] Determining first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data; determining second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information;

[0008] Calculating a first representation distance between different characters in the first dialogue data based on the first output information, and calculating a second representation distance between different characters in the second dialogue data based on the second output information;

[0009] The conversation pre-trained language model is trained by minimizing the first representation distance and maximizing the second representation distance to obtain a conversation vector representation model, which is used to generate a representation vector corresponding to the target conversation data.

[0010] In a second aspect, an embodiment of the present disclosure provides a method for processing a conversation, including:

[0011] Get target conversation data;

[0012] Determining input information for a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, wherein the dialogue vector representation model is configured to output a representation vector for a text unit included in each dialogue text in the target dialogue data based on the input information, and wherein the dialogue vector representation model is trained using the model training method described above;

[0013] A representation vector corresponding to the target conversation data is generated according to the representation vector of the text unit respectively included in each conversation text in the target conversation data.

[0014] In a third aspect, an embodiment of the present disclosure provides a model training device, comprising:

[0015] a generating module, configured to generate second dialogue data based on the first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data;

[0016] a determination module configured to determine first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data, and to determine second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information;

[0017] a calculation module, configured to calculate a first representation distance between different characters in the first dialogue data based on the first output information, and calculate a second representation distance between different characters in the second dialogue data based on the second output information;

[0018] A training module is configured to train the dialogue pre-trained language model by minimizing the first representation distance and maximizing the second representation distance to obtain a dialogue vector representation model.

[0019] In a fourth aspect, an embodiment of the present disclosure provides a conversation processing device, including:

[0020] Acquisition module, used to obtain target conversation data;

[0021] a determination module configured to determine input information of a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, the dialogue vector representation model configured to output a representation vector of a text unit included in each dialogue text in the target dialogue data based on the input information, the dialogue vector representation model being a model obtained using the model training method described above;

[0022] A generating module is used to generate a representation vector corresponding to the target dialogue data according to the representation vector of the text unit respectively included in each dialogue text in the target dialogue data.

[0023] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including:

[0024] Memory;

[0025] processor; and

[0026] computer programs;

[0027] The computer program is stored in the memory and is configured to be executed by the processor to implement the method as described in the first aspect or the second aspect.

[0028] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method described in the first aspect or the second aspect.

[0029] The model training and conversation processing methods, apparatuses, devices, and storage media provided by the embodiments of the present disclosure generate second conversation data as negative samples using first conversation data as positive samples. First input information for a conversation pre-trained language model is determined based on each conversation text in the first conversation data and the corresponding turn and role, respectively. Second input information for the conversation pre-trained language model is determined based on each conversation text in the second conversation data and the corresponding turn and role, respectively. This allows the conversation pre-trained language model to generate first output information for the first input information and second output information for the second input information. Furthermore, a first representation distance is calculated between different roles in the first conversation data based on the first output information, and a second representation distance is calculated between different roles in the second conversation data based on the second output information. Because the conversation texts between different roles in the positive samples are more semantically coherent, while the negative samples are the result of replacing the conversation text of a single role in the positive samples, the conversation texts between different roles in the negative samples may be less semantically coherent. In this case, it's generally desirable to minimize the first representation distance between different characters in positive samples, and maximize the second representation distance between different characters in negative samples. Therefore, the pre-trained conversation language model can be trained to obtain a conversation vector representation model by minimizing the first representation distance and maximizing the second representation distance. In other words, during the training of the pre-trained conversation language model, conversation-specific information such as turns and roles is incorporated, allowing the trained conversation vector representation model to effectively model the information interactions between multiple characters in a conversation. Therefore, when the conversation vector representation model is used to generate a representation vector corresponding to the target conversation data, the accuracy of the representation vector can be improved, resulting in a good conversation representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0031] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 A flow chart of the model training method provided in an embodiment of the present disclosure;

[0033] Figure 2A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0034] Figure 3 A schematic diagram of first conversation data provided by an embodiment of the present disclosure;

[0035] Figure 4 A schematic diagram of training samples in the pre-training phase provided by an embodiment of the present disclosure;

[0036] Figure 5 A schematic diagram of a pre-training phase provided in accordance with another embodiment of the present disclosure;

[0037] Figure 6 A flow chart of a model training method provided in another embodiment of the present disclosure;

[0038] Figure 7 A flow chart of a model training method provided in another embodiment of the present disclosure;

[0039] Figure 8 A flow chart of a model training method provided in another embodiment of the present disclosure;

[0040] Figure 9 A flow chart of a model training method provided in another embodiment of the present disclosure;

[0041] Figure 10 A flow chart of a conversation processing method provided by another embodiment of the present disclosure;

[0042] Figure 11 A schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0043] Figure 12 A schematic diagram of the structure of a conversation processing device provided in an embodiment of the present disclosure;

[0044] Figure 13 A schematic diagram of the structure of an electronic device embodiment provided by the present disclosure. DETAILED DESCRIPTION

[0045] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features therein can be combined with each other in the absence of conflict.

[0046] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0047] Intelligent question-answering systems are becoming increasingly popular, and each intelligent question-answering process generates a conversation. In some application scenarios, each conversation needs to be represented by a vector.

[0048] However, existing technologies cannot accurately convert a conversation into a representation vector, resulting in downstream applications that need to use this representation vector being unable to accurately implement their corresponding functions. To address this problem, the present disclosure provides a model training method, which is described below in conjunction with specific embodiments.

[0049] Figure 1 This is a flow chart of the model training method provided in the embodiment of the present disclosure. The method can be executed by a model training device, which can be implemented in software and / or hardware. The device can be configured in an electronic device, such as a server or a terminal, wherein the terminal specifically includes a mobile phone, a computer or a tablet computer. In addition, the model training method described in this embodiment can be applied to Figure 2 The application scenario shown in Figure 2 is as follows. Figure 2 As shown, this application scenario includes a terminal 21 and a server 22. Server 22 can use this model training method to train the conversational pretrain language model (CPLM), thereby obtaining a conversation vector representation model. Furthermore, server 22 can execute the conversation processing methods described in subsequent embodiments based on this trained conversation vector representation model. Alternatively, server 22 can deploy the trained conversation vector representation model on other servers or terminals 21, allowing these other servers or terminals 21 to execute the conversation processing methods described in subsequent embodiments based on this trained conversation vector representation model.

[0050] The following combination Figure 2 This paper introduces the model training method. Figure 1 As shown, the specific steps of this method are as follows:

[0051] S101. Generate second dialogue data based on first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data.

[0052] For example, the server 22 may pre-acquire multiple sample conversation data. Each sample conversation data may be a conversation between an intelligent customer service representative and a user. Furthermore, the server 22 may randomly select a sample conversation data from the multiple sample conversation data as the first conversation data, and generate the second conversation data based on the first conversation data. In this embodiment, the sample conversation data may be speech data or text data converted from speech data. This embodiment uses text-based conversation data as an example. The text-based conversation data may include the conversation text of each character, i.e., the speech content, the turn corresponding to each conversation text, and the character. A single speech by a character may be considered a turn, and the speech content of a character in a single speech may be recorded as a single conversation text. Assuming that the first conversation data includes both the conversation text of the intelligent customer service representative and the conversation text of the user, the conversation text of the user may be modified while the conversation text of the intelligent customer service representative remains fixed, thereby generating the second conversation data. Alternatively, the conversation text of the intelligent customer service representative may be modified while the conversation text of the user remains fixed, thereby generating the second conversation data. Alternatively, the conversation text of the intelligent customer service representative and the user in the first conversation data can be modified to generate the second conversation data. That is, the second conversation data differs from the first conversation data in the conversation text of at least one character, i.e., the conversation text of at least one character in the first conversation data differs from the conversation text of the at least one character in the second conversation data. It will be understood that in this embodiment, the sample conversation data, the first conversation data, or the second conversation data can be a conversation between two characters, or a conversation between more than two characters.

[0053] S102. Determine first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data. Determine second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data. The conversation pre-trained language model is used to obtain first output information based on the first input information and obtain second output information based on the second input information.

[0054] For example, the first conversation data is as follows Figure 3The conversation process shown. "XXXXXX," "YYYYY," "MMMM," ​​and "NNNNN" respectively represent conversation texts. The first conversation data includes four rounds, of which "XXXXXX" corresponds to the first round, "YYYYY" corresponds to the second round, "MMMM" corresponds to the third round, and "NNNNN" corresponds to the fourth round. The roles corresponding to "XXXXXX" and "MMMM" are intelligent customer service representatives, and the roles corresponding to "YYYYY" and "NNNNN" are users. Furthermore, based on each conversation text included in the first conversation data, and the round and role corresponding to each conversation text, first input information for a conversation pre-trained language model is generated. The conversation pre-trained language model may be a pre-trained model. Similarly, based on each conversation text in the second conversation data, and the round and role corresponding to each conversation text in the second conversation data, second input information for the conversation pre-trained language model is generated. The conversation pre-trained language model can obtain first output information based on the first input information and second output information based on the second input information. Specifically, the first input information and the second input information can be sequentially input into the dialogue pre-trained language model, so that the dialogue pre-trained language model can sequentially output the first output information and the second output information.

[0055] S103: Calculate a first representation distance between different characters in the first dialogue data based on the first output information, and calculate a second representation distance between different characters in the second dialogue data based on the second output information.

[0056] For example, the server 22 can calculate the first representation distance between different characters in the first dialogue data based on the first output information output by the dialogue pre-trained language model, and can also calculate the second representation distance between different characters in the second dialogue data based on the second output information output by the dialogue pre-trained language model.

[0057] S104: Train the conversation pre-trained language model by minimizing the first representation distance and maximizing the second representation distance to obtain a conversation vector representation model, where the conversation vector representation model is used to generate a representation vector corresponding to the target conversation data.

[0058] In this embodiment, the first conversation data can be recorded as a positive sample, and the second conversation data can be recorded as a negative sample. Each positive sample can correspond to at least one negative sample. Since the conversation text between different characters in the positive sample is more semantically coherent, while the negative sample is the result of replacing the conversation text of a single character in the positive sample, the conversation text between different characters in the negative sample may be less semantically coherent. In this case, it is generally desirable for the first representation distance between different characters in the positive sample to be as small as possible, and for the second representation distance between different characters in the negative sample to be as large as possible. Therefore, the adjustment direction for the parameters of the conversation pre-trained language model can be determined by minimizing the first representation distance and maximizing the second representation distance, and the parameters of the conversation pre-trained language model can be adjusted according to this adjustment direction. It is understood that in this embodiment, different sample conversation data can be selected as positive samples. In other words, there can be multiple positive samples, and at least one negative sample can be constructed for each positive sample. Based on a positive sample and one or more corresponding negative samples, the parameters of the conversation pre-trained language model can be adjusted once, i.e., a single iterative training of the conversation pre-trained language model can be achieved. Therefore, as the positive and negative sample pairs are continuously replaced, the parameters of the conversational pre-trained language model can be continuously adjusted, meaning that the conversational pre-trained language model can be continuously iteratively trained. When the parameters of the conversational pre-trained language model converge, or the number of iterations reaches a preset number, the conversational pre-trained language model can be recorded as a conversational vector representation model. In other words, after training the conversational pre-trained language model, a conversational vector representation model can be obtained. This conversational vector representation model is used to generate a representation vector corresponding to the target conversation data.

[0059] This disclosed embodiment generates second conversation data as a negative sample using first conversation data as a positive sample. First input information for a conversation pre-trained language model is determined based on each conversation text in the first conversation data and the corresponding turn and role. Second input information for the conversation pre-trained language model is determined based on each conversation text in the second conversation data and the corresponding turn and role. This allows the conversation pre-trained language model to generate first output information for the first input information and second output information for the second input information. Furthermore, a first representation distance is calculated between different roles in the first conversation data based on the first output information, and a second representation distance is calculated between different roles in the second conversation data based on the second output information. Because the conversation text between different roles in the positive sample is more semantically coherent, while the negative sample is the result of replacing the conversation text of a single role in the positive sample, the conversation text between different roles in the negative sample may be less semantically coherent. In this case, it's generally desirable to minimize the first representation distance between different characters in positive samples, and maximize the second representation distance between different characters in negative samples. Therefore, the pre-trained conversation language model can be trained to obtain a conversation vector representation model by minimizing the first representation distance and maximizing the second representation distance. In other words, during the training of the pre-trained conversation language model, conversation-specific information such as turns and roles is incorporated, allowing the trained conversation vector representation model to effectively model the information interactions between multiple characters in a conversation. Therefore, when the conversation vector representation model is used to generate a representation vector corresponding to the target conversation data, the accuracy of the representation vector can be improved, resulting in a good conversation representation.

[0060] In the above embodiment, the conversation pre-training language model can first go through a pre-training phase and then be trained using the model training method described in this embodiment. In some embodiments, the process of training using the model training method described in this embodiment can also be recorded as a fine-tuning process. It is understandable that the training samples used in the pre-training and fine-tuning of the conversation pre-training language model can be the same or different. Figure 4 and Figure 5 The pre-training process of the dialogue pre-training language model is introduced in detail. Figure 4The shown dialogue data can be the training samples adopted in the pre-training process. The dialogue data includes the dialogue texts "aaaaaa" and "cccc" of the intelligent customer service, as well as the dialogue texts "bbbb" and "dddd" of the user. The dialogue data includes 4 rounds. Among them, "aaaaaa" corresponds to the first round, "bbbb" corresponds to the second round, "cccc" corresponds to the third round, and "dddd" corresponds to the fourth round. It can be understood that the number of rounds included in the dialogue data is not limited to 4, and this is only for illustrative purposes here.

[0061] In this pre-training process, it can be based on, for example, Figure 4 As shown, the input information of the dialogue pre-training language model can be determined according to the text units included in each dialogue text, the position information of each text unit, and the round and role corresponding to each dialogue text respectively. The text unit can be denoted as token, and a text unit can specifically be a character, a Chinese character, a word, a sub-word, or a phrase, etc. It can be understood that a dialogue text can include one or more text units. Further, it can be determined, as shown in Figure 4 The index of each text unit included in each dialogue text in the preset table, for example, the index of a text unit in the preset table is denoted as token ID. The position information of each text unit can be the position index of each text unit, and this position index is denoted as position index. For example, Figure 4 As shown, "aaaaaa" is "Good morning", and "bbbb" is "Hello". Assuming a text unit is a Chinese character, the position indices corresponding to the 3 characters "你", "好", and "啊" respectively can be the position indices relative to the beginning of the dialogue text to which these 3 characters belong, that is, "Hello", or can be the position indices relative to Figure 4 The beginning of the first dialogue text in the entire dialogue data shown, that is, "Good morning". Among them, if the position indices corresponding to the 3 characters "你", "好", and "啊" respectively are the position indices relative to the beginning of "Hello", then this position index can be denoted as the relative position index. If the position indices corresponding to the 3 characters "你", "好", and "啊" respectively are the position indices relative to the beginning of "Good morning", then this position index can be denoted as the absolute position index. Taking the relative position index as an example, the position indices corresponding to the 3 characters "你", "好", and "啊" respectively can be 0, 1, 2. In addition, it can also be determined, as shown in Figure 4The turn index of each dialogue text shown is recorded as the turn index. The turn index can also be called the turn sequence number. For example, "aaaaaa" corresponds to the first turn, and the turn index of "aaaaaa" can be recorded as 0. Similarly, the turn index of "bbbb" can be recorded as 1, the turn index of "cccc" can be recorded as 2, and the turn index of "dddd" can be recorded as 3. It can be understood that the turn index corresponding to different tokens in the same dialogue text is the turn index of the dialogue text, that is, the turn index corresponding to different tokens in the same dialogue text is the same. In addition, different roles can correspond to different role labels, which can be recorded as role labels. For example, the role label corresponding to intelligent customer service is A, and the role label corresponding to the user is B. Therefore, the role label corresponding to "aaaaaa" and "cccc" is A, and the role label corresponding to "bbbb" and "dddd" is B, respectively. Similarly, the role labels corresponding to different tokens in the same conversation text are the role labels corresponding to the conversation text. In other words, the role labels corresponding to different tokens in the same conversation text are the same.

[0062] like Figure 5 As shown, assuming Figure 4 The "aaaaaa" shown includes 3 text units, namely "do", "you", and "have". "bbbb" includes 3 text units, namely "i", "have", and "a". "cccc" includes 2 text units, namely "that" and "is". "dddd" includes 2 text units, namely "have" and "a". In this embodiment, a special character [CLS] can be added to the beginning of each conversation. In addition, a special character [EOU] can be added between each two adjacent conversation texts. In addition, during the pre-training process, the following text units can be used: Figure 4One or more text units in each conversation text are masked. For example, the character "you" in "aaaaaa" is masked by replacing it with the special character [MASK]. Furthermore, the token ID embedding, position index embedding, turn index embedding, and role label embedding for the same text unit or special character are summed to obtain the input information for the conversation pre-trained language model (CPLM). For example, the special character [MASK] corresponding to "you" is recorded as the first special character [MASK]. Since the token IDs corresponding to "you" and the first special character [MASK] in the preset table may be different, the token ID embedding corresponding to "you" and the token ID embedding of the first special character [MASK] may be different, but the position index embedding corresponding to "you" and the position index embedding corresponding to the first special character [MASK] are the same, the turn index embedding corresponding to "you" and the turn index embedding corresponding to the first special character [MASK] are the same, and the role label embedding corresponding to "you" and the role label embedding corresponding to the first special character [MASK] are the same. Further, by adding the token ID embedding, position index embedding, turnindex embedding, and role label embedding of the first special character [MASK], the input information 51 corresponding to the first special character [MASK] can be obtained. The input information of the CPLM can be as follows: Figure 5 The set of input information corresponding to each special character and each text unit in the box 52 shown in FIG. Figure 5The output information shown may include the representation vectors corresponding to each special character and each text unit in box 52, wherein the representation vector corresponding to each special character [MASK] output by the CPLM is not the representation vector of the special character [MASK] itself, but the text unit with the greatest probability at the location of the special character [MASK] predicted by the CPLM. Furthermore, the parameters of the CPLM can be updated based on the representation vector corresponding to each special character [MASK] and the representation vector corresponding to the original actual text unit at the location of each special character [MASK], that is, the CPLM is pre-trained. When the training samples are continuously changed, the CPLM can be pre-trained continuously. Thus, a CPLM after pre-training is obtained. It is understandable that the pre-training process can be performed by server 22 or by other servers.

[0063] On the basis of the above embodiment, the second dialogue data is generated according to the first dialogue data, including: Figure 6 The following steps are shown:

[0064] S601: Acquire dialogue texts of different roles from a plurality of sample dialogue data to form a dialogue text set corresponding to each role.

[0065] For example, the server 22 may pre-acquire multiple sample conversation data. Each sample conversation data may be a conversation, which may be a conversation process between the intelligent customer service and the user. For example, the server 22 may pre-acquire 100 conversations. Furthermore, the server 22 may pre-process these 100 conversations. For example, the server 22 may divide the 100 conversations into a training set, a validation set, and a test set. The conversations in the training set can be used to fine-tune the pre-trained CPLM. The conversations in the validation set can be used to verify the effectiveness of the CPLM after each fine-tuning and select the optimal CPLM during the continuous fine-tuning process. The conversations in the test set can be used to test the effectiveness of the optimal CPLM, such as performance parameters. In addition, the server 22 may also perform routine data cleaning on the 100 conversations, such as data anonymization and filtering of abnormal sessions. Specifically, the 100 conversations may be selected from the Renren conversation log, which includes the customer service log. Therefore, the Renren conversation log can be processed into the data format required by CPLM, for example, one session per line, each adjacent session is separated by a delimiter, namely the special character [EOU], and each session is also attached with a corresponding role.

[0066] In addition, the server 22 can aggregate the conversation texts belonging to the intelligent customer service in the 100 conversations to obtain a conversation text set corresponding to the intelligent customer service, which can be recorded as the first negative sample pool. In addition, the server 22 can aggregate the conversation texts belonging to the user in the 100 conversations to obtain a conversation text set corresponding to the user, which can be recorded as the second negative sample pool.

[0067] S602: Fix the dialogue text of the first character in the first dialogue data, where the first dialogue data is any sample dialogue data among the plurality of sample dialogue data.

[0068] Assume that the server 22 randomly selects a conversation from the 100 conversations as a positive sample, which can be recorded as the first conversation data. Then, the server 22 can fix the conversation text of the first role in the positive sample, for example, the first role can be an intelligent customer service representative.

[0069] S603: Replace the dialogue text of the second character in the first dialogue data with the dialogue text in the dialogue text set corresponding to the second character to obtain the second dialogue data.

[0070] For example, after fixing the conversation text of the intelligent customer service representative in the positive sample, the server 22 can count the number of conversation texts of the second character in the positive sample. For example, if the second character can be a user and there are five user conversation texts in the positive sample, the server 22 can randomly extract five conversation texts from the second negative sample pool described above and replace the five original user conversation texts in the positive sample with the five randomly extracted conversation texts from the second negative sample pool, thereby obtaining second conversation data, which can be recorded as a negative sample. It is understood that one positive sample can correspond to one or more negative samples. In addition, after constructing the negative sample, the negative sample can also be corrected. For example, if the length of a conversation text in the negative sample is insufficient, the conversation text can be replaced again or its length can be supplemented. If the length of a conversation text in the negative sample is too long, part of the conversation text can be truncated.

[0071] For example, through the above construction process, multiple positive samples and corresponding negative samples for each positive sample can be obtained. For example, a positive sample and a negative sample can constitute a positive-negative sample pair, or a positive sample and multiple negative samples can constitute a positive-negative sample pair. After constructing the positive-negative sample pair, any positive sample and its paired negative sample can be sequentially input into the pre-trained CPLM. For example, the positive sample is recorded as the first conversation data, and the negative sample is recorded as the second conversation data.

[0072] Optionally, determining first input information of the dialogue pre-trained language model based on each dialogue text in the first dialogue data and the turns and roles respectively corresponding to each dialogue text in the first dialogue data includes: determining the first input information of the dialogue pre-trained language model based on text units included in each dialogue text in the first dialogue data, position information of each text unit in the first dialogue data, and the turns and roles respectively corresponding to each dialogue text in the first dialogue data.

[0073] For example Figure 3 As shown, the first conversation data includes "XXXXXX", "YYYYY", "MMMM", and "NNNNN", among which "XXXXXX" and "MMMM" are the conversation texts of the intelligent customer service, and "YYYYY" and "NNNNN" are the conversation texts of the user. The turn index corresponding to "XXXXXX" can be recorded as 0, the turn index corresponding to "YYYYY" can be recorded as 1, the turn index corresponding to "MMMM" can be recorded as 2, and the turn index corresponding to "NNNNN" can be recorded as 3. In addition, in this embodiment, different roles can correspond to different role labels. For example, the role label corresponding to the intelligent customer service is A, and the role label corresponding to the user is B. In addition, as Figure 3 Each of the dialogue texts shown may include one or more text units, each of which corresponds to a token ID and a position index. Further, based on the token ID, position index, turn index, and role label corresponding to each text unit in the first dialogue data, the first input information of the pre-trained CPLM can be determined. The process of determining the first input information can refer to Figure 5 The input information determination process shown here is different from Figure 5 The difference is that there is no need to Figure 3 Individual text units within each dialogue text are shown to be masked.

[0074] Optionally, determining the second input information of the dialogue pre-trained language model based on each dialogue text in the second dialogue data and the turns and roles corresponding to each dialogue text in the second dialogue data includes: determining the second input information of the dialogue pre-trained language model based on the text units included in each dialogue text in the second dialogue data, the position information of each text unit in the second dialogue data, and the turns and roles corresponding to each dialogue text in the second dialogue data.

[0075] Similarly, according to the token ID, position index, turn index, and role label corresponding to each text unit in the second conversation data, the second input information of the pre-trained CPLM can be determined. The process of determining the second input information can refer to Figure 5 The input information determination process shown here is different from Figure 5 The difference is that here, it is not necessary to mask the individual text units in each dialogue text in the second dialogue data.

[0076] For example, when first input information is input into a pre-trained CPLM, the CPLM may obtain first output information. When second input information is input into the pre-trained CPLM, the CPLM may obtain second output information.

[0077] Optionally, the first output information includes a representation vector of a text unit respectively included in each dialogue text in the first dialogue data; for example, the first output information may include: Figure 3 The representation vectors of the text units included in each conversation text in the positive sample are shown.

[0078] Specifically, calculating the first representation distance between different characters in the first dialogue data according to the first output information includes: Figure 7 The following steps are shown:

[0079] S701. Generate a first set based on the representation vectors of the text units corresponding to the first role in the first output information, and generate a second set based on the representation vectors of the text units corresponding to the second role in the first output information.

[0080] For example, in this embodiment, the first role is intelligent customer service, and the second role is the user. The first conversation data as a positive sample includes three conversation texts. It is assumed that the first conversation text and the third conversation text in the first conversation data correspond to the intelligent customer service, and the second conversation text in the first conversation data corresponds to the user. The first output information includes the representation vectors of the text units respectively included in each of the three conversation texts. Further, a first set can be generated based on the representation vectors of the text units corresponding to the intelligent customer service in the first output information, and a second set can be generated based on the representation vectors of the text units corresponding to the user in the first output information. Among them, the first set can be as follows Figure 8 shown The second set can be Figure 8 shown Assume that the first conversation text includes 3 tokens, the third conversation text includes 2 tokens, and the second conversation text includes 3 tokens, then The first, second, and third rows in the data correspond to the representation vectors of the three tokens in the first conversation text. For example, each row corresponds to the representation vector of one token. The fourth, fifth, and sixth rows in the representation vector are 0. The seventh and eighth rows in the figure correspond to the representation vectors of the two tokens in the third dialogue text. The first, second, and third rows in the representation vector are 0. The fourth, fifth, and sixth rows in the graph correspond to the representation vectors of the three tokens in the second conversation text. The seventh and eighth rows in the representation vector are 0.

[0081] S702: Calculate a first matrix and a second matrix based on the first set and the second set, where the first matrix and the second matrix are transposed matrices of each other.

[0082] Further, according to the first set and the second set The first matrix and the second matrix can be calculated, wherein the first matrix can be as follows Figure 8 shown The second matrix can be Figure 8 shown The first matrix can be As a result, the second matrix can be The result is that the first matrix and the second matrix are transposed matrices of each other.

[0083] S703. Update the first set according to the first matrix to obtain an updated first set, and update the second set according to the second matrix to obtain an updated second set. The first representation distance between different characters in the first dialogue data includes the distance between the vector generated according to the first set and the vector generated according to the updated first set, and the distance between the vector generated according to the second set and the vector generated according to the updated second set.

[0084] For example, according to the first matrix For the first set Update and obtain the updated first set, which is recorded as For example, Can be Similarly, according to the second matrix For the second set Update to obtain the updated second set, which is recorded as Can be results.

[0085] Furthermore, for the first set After pooling, the following is generated: Figure 8 The first vector 81 shown is the first set of updated After pooling, the generated Figure 8 The second vector 82 shown is for the updated second set After pooling, the generated Figure 8 The third vector 83 shown, for the second set After pooling, the generated Figure 8 The fourth vector 84 is shown. At this time, the distance between the first vector 81 and the second vector 82 can be expressed as follows Figure 8 The distance between the third vector 83 and the fourth vector 84 shown can be expressed as follows: Figure 8 d2 shown. d1 and d2 belong to the first representation distances between different characters in the positive samples.

[0086] Optionally, the second output information includes a representation vector for each text unit included in each conversation text in the second conversation data. For example, when the second input information is input into a pre-trained CPLM, the CPLM can obtain the second output information. The second output information includes a representation vector for each text unit included in each conversation text in the negative sample.

[0087] Specifically, calculating the second representation distance between different characters in the second dialogue data according to the second output information includes: Figure 9 The following steps are shown:

[0088] S901. Generate a third set based on the representation vectors of the text units corresponding to the first role in the second output information, and generate a fourth set based on the representation vectors of the text units corresponding to the second role in the second output information.

[0089] For example, in this embodiment, the first role is intelligent customer service, and the second role is the user. The second conversation data as a negative sample may also include three conversation texts. It is assumed that the first conversation text and the third conversation text in the second conversation data correspond to the intelligent customer service, respectively, and the second conversation text in the second conversation data corresponds to the user. The second output information includes the representation vectors of the text units respectively included in each of the three conversation texts. Furthermore, a third set can be generated based on the representation vectors of the text units corresponding to the intelligent customer service in the second output information, and a fourth set can be generated based on the representation vectors of the text units corresponding to the user in the second output information. The third set is similar to Figure 8 shown The fourth set is similar to Figure 8 shown

[0090] S902: Calculate a third matrix and a fourth matrix according to the third set and the fourth set, where the third matrix and the fourth matrix are transposed matrices of each other.

[0091] For example, based on the third set and the fourth set, a third matrix and a fourth matrix are calculated. The method for calculating the third matrix is ​​similar to the method for calculating the first matrix, and the method for calculating the fourth matrix is ​​similar to the method for calculating the second matrix. The specific process is not repeated here. In addition, the third matrix and the fourth matrix are transposed matrices of each other.

[0092] S903. Update the third set according to the third matrix to obtain an updated third set, and update the fourth set according to the fourth matrix to obtain an updated fourth set. The second representation distance between different characters in the second dialogue data includes the distance between the vector generated according to the third set and the vector generated according to the updated third set, and the distance between the vector generated according to the fourth set and the vector generated according to the updated fourth set.

[0093] For example, the third set is updated according to the third matrix to obtain the updated third set. The fourth set is updated according to the fourth matrix to obtain the updated fourth set. Pooling the third set before the update can generate a fifth vector, which is similar to the first vector mentioned above. Pooling the third set after the update can generate a sixth vector, which is similar to the second vector mentioned above. Pooling the fourth set after the update can generate a seventh vector, which is similar to the third vector mentioned above. Pooling the fourth set before the update can generate an eighth vector, which is similar to the fourth vector mentioned above. At this time, the distance between the fifth vector and the sixth vector can be recorded as follows Figure 8 The distance between d3, the seventh vector and the eighth vector can be expressed as Figure 8 d4 shown. d3 and d4 belong to the second representation distances between different characters in the negative samples.

[0094] Furthermore, the first representation distance between different characters in the positive sample can be minimized, and the second representation distance between different characters in the negative sample can be maximized. For example, the following can be minimized: Figure 8 d1 and d2 as shown, or the sum of d1 and d2 can be minimized. At the same time, maximize Figure 8 As shown in d3 and d4, or the sum between d3 and d4 can be maximized. Figure 8 The positive value shown in the figure represents minimization, and the negative value represents maximization. Figure 8 As shown, the first representation distance between different characters in the positive sample and the second representation distance between different characters in the negative sample can construct a loss function This loss function is used to update the parameters of the pre-trained CPLM. For example, it minimizes the first representation distance between different characters in positive samples and maximizes the second representation distance between different characters in negative samples. The parameters of the CPLM are continuously adjusted as the positive and negative sample pairs are continuously replaced. When the CPLM parameters converge, or the number of iterations reaches a preset number, the CPLM is considered a dialogue vector representation model.

[0095] This embodiment trains a pre-trained CPLM by minimizing the first representation distance between different characters in positive samples and maximizing the second representation distance between different characters in negative samples, resulting in a dialogue vector representation model. By incorporating dialogue-specific information, such as turns and roles, into this training process, the resulting dialogue vector representation model effectively models the information interactions between multiple characters that are characteristic of dialogue. Therefore, when the dialogue vector representation model is used to generate representation vectors corresponding to target dialogue data, the accuracy of the representation vectors can be improved, resulting in excellent dialogue representation.

[0096] It is understandable that the above embodiment describes how to train the pre-trained CPLM, that is, how to fine-tune the parameters of the pre-trained CPLM to obtain a dialogue vector representation model. Figure 10 Let's introduce the use of the dialogue vector representation model. For example, the dialogue vector representation model can be used to implement the following dialogue processing method, which can convert any dialogue data into a representation vector, such as a high-dimensional representation vector. Specifically, in this embodiment, the dialogue vector representation model can be deployed to Figure 2 In the terminal 21 shown in FIG. 1 , the terminal 21 can use the dialogue vector representation model to convert any dialogue data into a representation vector. Alternatively, the dialogue vector representation model can also be deployed in Figure 2 The server 22 or other servers shown in the figure can use the conversation vector representation model to convert any conversation data into a representation vector. The following is a schematic description using the server 22 as an example.

[0097] Figure 10 This is a flow chart of a conversation processing method provided by another embodiment of the present disclosure. In this embodiment, the method comprises the following specific steps:

[0098] S1001. Obtain target conversation data.

[0099] For example, the server 22 may obtain target conversation data, which may be conversation data generated or recorded by the server 22 during a conversation between the server 22 as an intelligent customer service and the user, or the target conversation data may be conversation data obtained by the server 22 from other servers, which may act as intelligent customer service to have a conversation with the user.

[0100] S1002. Determine input information of a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data. The dialogue vector representation model is configured to output representation vectors of text units respectively included in each dialogue text in the target dialogue data based on the input information. The dialogue vector representation model is trained using the model training method described above.

[0101] For example, the target conversation data can be something like Figure 3 or Figure 4 The target conversation data includes multiple conversation texts, each of which includes at least one text unit, and each text unit corresponds to a turn and a role. Based on each conversation text in the target conversation data and the turn and role corresponding to each conversation text, the input information of the conversation vector representation model can be determined. The input information of the conversation vector representation model is similar to Figure 5 The input information for the CPLM shown differs in that, for each conversation text in the target conversation data, individual text units within the conversation text do not need to be masked. Furthermore, the conversation vector representation model can output representation vectors for each text unit within the target conversation data based on the input information. The conversation vector representation model herein is trained or fine-tuned using the model training method described above.

[0102] Optionally, the input information of the dialogue vector representation model is determined based on each dialogue text in the target dialogue data, and the turns and roles respectively corresponding to each dialogue text in the target dialogue data, including: determining the input information of the dialogue vector representation model based on the text units included in each dialogue text in the target dialogue data, the position information of each text unit in the target dialogue data, and the turns and roles respectively corresponding to each dialogue text in the target dialogue data.

[0103] For example, the target conversation data includes multiple conversation texts, each conversation text includes at least one text unit, and each text unit corresponds to a token ID and a position index. In addition, each conversation text also corresponds to a turn index and a role label. Based on the tokenID, position index, turn index, and role label corresponding to each text unit in the target conversation data, the input information of the conversation vector representation model can be determined. The process of determining the input information is similar to Figure 5 The process shown is not repeated here. The difference is that, for each dialogue text in the target dialogue data, it is not necessary to mask the individual text units in the dialogue text.

[0104] S1003: Generate a representation vector corresponding to the target conversation data according to the representation vector of each text unit included in each conversation text in the target conversation data.

[0105] For example, when the dialogue vector representation model outputs the representation vector of each text unit included in each dialogue text in the target dialogue data, the representation vector of each text unit can be weighted averaged to obtain the representation vector corresponding to the target dialogue data.

[0106] This embodiment incorporates conversation-specific information, such as turns and roles, during the fine-tuning of the CLLM. This allows the trained dialogue vector representation model to effectively model the information interactions between multiple roles that are unique to conversations. Consequently, when the dialogue vector representation model is used to generate representation vectors corresponding to target conversation data, the accuracy of the representation vectors can be improved, resulting in excellent conversation representation.

[0107] Figure 11 The model training device provided by the embodiment of the present disclosure can execute the processing flow provided by the embodiment of the model training method, such as Figure 11 As shown, the model training device 110 includes:

[0108] A generating module 111 is configured to generate second dialogue data based on first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data;

[0109] Determining module 112, configured to determine first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data, and to determine second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information;

[0110] a calculation module 113, configured to calculate a first representation distance between different characters in the first dialogue data based on the first output information, and calculate a second representation distance between different characters in the second dialogue data based on the second output information;

[0111] The training module 114 is configured to train the dialogue pre-trained language model by minimizing the first representation distance and maximizing the second representation distance to obtain a dialogue vector representation model.

[0112] Optionally, when the generating module 111 generates the second conversation data based on the first conversation data, it is specifically configured to:

[0113] Obtaining dialogue texts of different characters from multiple sample dialogue data to form a dialogue text set corresponding to each character;

[0114] Fixing the dialogue text of the first character in the first dialogue data, where the first dialogue data is any sample dialogue data among the plurality of sample dialogue data;

[0115] The dialogue text of the second character in the first dialogue data is replaced with the dialogue text in the dialogue text set corresponding to the second character to obtain the second dialogue data.

[0116] Optionally, when the determination module 112 determines the first input information of the dialogue pre-trained language model based on each dialogue text in the first dialogue data and the turn and role corresponding to each dialogue text in the first dialogue data, it is specifically configured to:

[0117] Determine first input information of a dialogue pre-trained language model based on text units included in each dialogue text in the first dialogue data, position information of each text unit in the first dialogue data, and turns and roles corresponding to each dialogue text in the first dialogue data.

[0118] Optionally, when the determination module 112 determines the second input information of the dialogue pre-trained language model based on each dialogue text in the second dialogue data and the turn and role corresponding to each dialogue text in the second dialogue data, it is specifically configured to:

[0119] Determine the second input information of the dialogue pre-trained language model based on the text units included in each dialogue text in the second dialogue data, the position information of each text unit in the second dialogue data, and the turns and roles corresponding to each dialogue text in the second dialogue data.

[0120] Optionally, the first output information includes a representation vector of a text unit respectively included in each dialogue text in the first dialogue data; when the calculation module 113 calculates the first representation distance between different characters in the first dialogue data based on the first output information, it is specifically configured to:

[0121] Generate a first set based on the representation vectors of the text units corresponding to the first role in the first output information, and generate a second set based on the representation vectors of the text units corresponding to the second role in the first output information;

[0122] Calculating a first matrix and a second matrix according to the first set and the second set, where the first matrix and the second matrix are transposed matrices of each other;

[0123] The first set is updated according to the first matrix to obtain an updated first set, and the second set is updated according to the second matrix to obtain an updated second set. The first representation distance between different characters in the first dialogue data includes the distance between the vector generated according to the first set and the vector generated according to the updated first set, and the distance between the vector generated according to the second set and the vector generated according to the updated second set.

[0124] Optionally, the second output information includes a representation vector of a text unit respectively included in each dialogue text in the second dialogue data; when the calculation module 113 calculates the second representation distance between different characters in the second dialogue data based on the second output information, it is specifically configured to:

[0125] Generate a third set based on the representation vectors of the text units corresponding to the first role in the second output information, and generate a fourth set based on the representation vectors of the text units corresponding to the second role in the second output information;

[0126] Calculating a third matrix and a fourth matrix according to the third set and the fourth set, wherein the third matrix and the fourth matrix are transposed matrices of each other;

[0127] The third set is updated according to the third matrix to obtain an updated third set, and the fourth set is updated according to the fourth matrix to obtain an updated fourth set. The second representation distance between different characters in the second dialogue data includes the distance between the vector generated according to the third set and the vector generated according to the updated third set, and the distance between the vector generated according to the fourth set and the vector generated according to the updated fourth set.

[0128] Figure 11 The model training device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0129] Figure 12 The diagram of the structure of the dialogue processing device provided by the embodiment of the present disclosure. The dialogue processing device provided by the embodiment of the present disclosure can execute the processing flow provided by the dialogue processing method embodiment, such as Figure 12 As shown, the dialogue processing device 120 includes:

[0130] An acquisition module 121 is used to acquire target conversation data;

[0131] a determination module 122 configured to determine input information for a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, the dialogue vector representation model configured to output, based on the input information, representation vectors for text units included in each dialogue text in the target dialogue data, the dialogue vector representation model being a model obtained using the model training method described above;

[0132] The generating module 123 is configured to generate a representation vector corresponding to the target conversation data according to the representation vector of each text unit included in each conversation text in the target conversation data.

[0133] Optionally, when the determination module 122 determines the input information of the dialogue vector representation model based on each dialogue text in the target dialogue data and the turn and role corresponding to each dialogue text in the target dialogue data, it is specifically configured to:

[0134] The input information of the dialogue vector representation model is determined based on the text units included in each dialogue text in the target dialogue data, the position information of each text unit in the target dialogue data, and the turns and roles corresponding to each dialogue text in the target dialogue data.

[0135] Figure 12The dialogue processing device of the illustrated embodiment can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.

[0136] The above describes the internal functions and structure of the model training device or the dialogue processing device, which can be implemented as an electronic device. Figure 13 This is a schematic diagram of the structure of an electronic device embodiment provided by the present disclosure. Figure 13 As shown, the electronic device includes a memory 131 and a processor 132 .

[0137] The memory 131 is used to store programs. In addition to the aforementioned programs, the memory 131 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, images, videos, and the like.

[0138] The memory 131 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0139] The processor 132 is coupled to the memory 131 and executes the program stored in the memory 131 to:

[0140] generating second dialogue data based on the first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data;

[0141] Determining first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data; determining second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information;

[0142] Calculating a first representation distance between different characters in the first dialogue data based on the first output information, and calculating a second representation distance between different characters in the second dialogue data based on the second output information;

[0143] The conversation pre-trained language model is trained by minimizing the first representation distance and maximizing the second representation distance to obtain a conversation vector representation model, which is used to generate a representation vector corresponding to the target conversation data.

[0144] Alternatively, the processor 132 is further configured to:

[0145] Get target conversation data;

[0146] Determining input information for a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, wherein the dialogue vector representation model is configured to output a representation vector for a text unit included in each dialogue text in the target dialogue data based on the input information, and wherein the dialogue vector representation model is trained using the model training method described above;

[0147] A representation vector corresponding to the target conversation data is generated according to the representation vector of the text unit respectively included in each conversation text in the target conversation data.

[0148] Further, if Figure 13 As shown, the electronic device may further include: a communication component 133, a power component 134, an audio component 135, a display 136 and other components. Figure 13 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 13 Components shown.

[0149] The communication component 133 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 133 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 133 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0150] The power supply assembly 134 provides power to various components of the electronic device. The power supply assembly 134 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0151] The audio component 135 is configured to output and / or input audio signals. For example, the audio component 135 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 131 or transmitted via the communication component 133. In some embodiments, the audio component 135 also includes a speaker for outputting audio signals.

[0152] The display 136 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0153] In addition, an embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the model training method or dialogue processing method described in the above embodiment.

[0154] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0155] The foregoing description is intended only to provide specific embodiments of the present disclosure, intended to enable those skilled in the art to understand and implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments described herein, but rather to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A model training method, wherein: The method comprises: generating second dialogue data based on the first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data; Determining first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data; determining second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information; Calculating a first representation distance between different characters in the first dialogue data based on the first output information, and calculating a second representation distance between different characters in the second dialogue data based on the second output information; The conversation pre-trained language model is trained by minimizing the first representation distance and maximizing the second representation distance to obtain a conversation vector representation model, wherein the conversation vector representation model is used to generate a representation vector corresponding to the target conversation data; Calculating the first representation distance between different characters in the first dialogue data based on the first output information includes: generating a first set and a second set based on representation vectors of text units respectively included in each dialogue text in the first dialogue data in the first output information, wherein the characters corresponding to the first set are different from the characters corresponding to the second set; calculating a first matrix and a second matrix based on the first set and the second set, wherein the first matrix and the second matrix are transposed matrices of each other; and obtaining the first representation distance based on the first matrix and the second matrix; Calculating a second representation distance between different characters in the second dialogue data based on the second output information includes: generating a third set and a fourth set based on representation vectors of text units respectively included in each dialogue text in the second dialogue data in the second output information, wherein the characters corresponding to the third set are different from the characters corresponding to the fourth set; calculating a third matrix and a fourth matrix based on the third set and the fourth set, wherein the third matrix and the fourth matrix are transposed matrices of each other; and obtaining the second representation distance based on the third matrix and the fourth matrix.

2. The method according to claim 1, wherein Generating second conversation data according to the first conversation data includes: Obtaining dialogue texts of different characters from multiple sample dialogue data to form a dialogue text set corresponding to each character; Fixing the dialogue text of the first character in the first dialogue data, where the first dialogue data is any sample dialogue data among the plurality of sample dialogue data; The dialogue text of the second character in the first dialogue data is replaced with the dialogue text in the dialogue text set corresponding to the second character to obtain the second dialogue data.

3. The method according to claim 1, wherein Determining first input information of a dialogue pre-trained language model according to each dialogue text in the first dialogue data and the turn and role respectively corresponding to each dialogue text in the first dialogue data includes: Determine first input information of a dialogue pre-trained language model based on text units included in each dialogue text in the first dialogue data, position information of each text unit in the first dialogue data, and turns and roles corresponding to each dialogue text in the first dialogue data.

4. The method according to claim 1, wherein Determining second input information of the dialogue pre-trained language model according to each dialogue text in the second dialogue data and the turn and role respectively corresponding to each dialogue text in the second dialogue data includes: Determine the second input information of the dialogue pre-trained language model based on the text units included in each dialogue text in the second dialogue data, the position information of each text unit in the second dialogue data, and the turns and roles corresponding to each dialogue text in the second dialogue data.

5. The method according to claim 1, wherein Generating a first set and a second set according to the representation vector of each text unit respectively included in the first dialogue text in the first dialogue data in the first output information, including: Generate the first set based on the representation vector of the text unit corresponding to the first role in the first output information, and generate the second set based on the representation vector of the text unit corresponding to the second role in the first output information; The first representation distance is obtained according to the first matrix and the second matrix, including: updating the first set according to the first matrix to obtain an updated first set, and updating the second set according to the second matrix to obtain an updated second set. The first representation distance between different characters in the first dialogue data includes the distance between the vector generated according to the first set and the vector generated according to the updated first set, and the distance between the vector generated according to the second set and the vector generated according to the updated second set.

6. The method according to claim 1, wherein Generating a third set and a fourth set based on the representation vector of each text unit respectively included in the second dialogue text in the second output information, including: generating the third set based on the representation vector of the text unit corresponding to the first role in the second output information, and generating the fourth set based on the representation vector of the text unit corresponding to the second role in the second output information; The second representation distance is obtained according to the third matrix and the fourth matrix, including: updating the third set according to the third matrix to obtain an updated third set, and updating the fourth set according to the fourth matrix to obtain an updated fourth set. The second representation distance between different characters in the second dialogue data includes the distance between the vector generated according to the third set and the vector generated according to the updated third set, and the distance between the vector generated according to the fourth set and the vector generated according to the updated fourth set.

7. A method for processing a conversation, wherein: The method comprises: Get target conversation data; Determining input information for a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, the dialogue vector representation model being configured to output, based on the input information, representation vectors for text units included in each dialogue text in the target dialogue data, wherein the dialogue vector representation model is trained using the method according to any one of claims 1 to 6; A representation vector corresponding to the target conversation data is generated according to the representation vector of the text unit respectively included in each conversation text in the target conversation data.

8. The method according to claim 7, wherein: Determining input information of a dialogue vector representation model according to each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data includes: The input information of the dialogue vector representation model is determined based on the text units included in each dialogue text in the target dialogue data, the position information of each text unit in the target dialogue data, and the turns and roles corresponding to each dialogue text in the target dialogue data.

9. A model training device, wherein: include: a generating module, configured to generate second dialogue data based on the first dialogue data, wherein a dialogue text of at least one character in the first dialogue data is different from a dialogue text of the at least one character in the second dialogue data; a determination module configured to determine first input information of a conversation pre-trained language model based on each conversation text in the first conversation data and the turns and roles respectively corresponding to each conversation text in the first conversation data, and to determine second input information of the conversation pre-trained language model based on each conversation text in the second conversation data and the turns and roles respectively corresponding to each conversation text in the second conversation data, wherein the conversation pre-trained language model is configured to obtain first output information based on the first input information and obtain second output information based on the second input information; a calculation module, configured to calculate a first representation distance between different characters in the first dialogue data based on the first output information, and calculate a second representation distance between different characters in the second dialogue data based on the second output information; a training module, configured to train the dialogue pre-trained language model by minimizing the first representation distance and maximizing the second representation distance to obtain a dialogue vector representation model; The calculation module is configured to calculate a first representation distance between different characters in the first dialogue data based on the first output information by performing the following steps: generating a first set and a second set based on representation vectors of text units respectively included in each dialogue text in the first dialogue data in the first output information, wherein the characters corresponding to the first set are different from the characters corresponding to the second set; calculating a first matrix and a second matrix based on the first set and the second set, wherein the first matrix and the second matrix are transposed matrices of each other; and obtaining the first representation distance based on the first matrix and the second matrix; The calculation module is configured to calculate a second representation distance between different characters in the second dialogue data based on the second output information by performing the following steps: generating a third set and a fourth set based on representation vectors of text units respectively included in each dialogue text in the second dialogue data in the second output information, wherein the characters corresponding to the third set are different from the characters corresponding to the fourth set; calculating a third matrix and a fourth matrix based on the third set and the fourth set, wherein the third matrix and the fourth matrix are transposed matrices of each other; and obtaining the second representation distance based on the third matrix and the fourth matrix.

10. A conversation processing device, wherein: include: Acquisition module, used to obtain target conversation data; a determination module configured to determine input information of a dialogue vector representation model based on each dialogue text in the target dialogue data and the turns and roles corresponding to each dialogue text in the target dialogue data, the dialogue vector representation model configured to output, based on the input information, representation vectors of text units included in each dialogue text in the target dialogue data, the dialogue vector representation model being a model obtained using the method according to any one of claims 1 to 6; A generating module is used to generate a representation vector corresponding to the target dialogue data according to the representation vector of the text unit respectively included in each dialogue text in the target dialogue data.

11. An electronic device, wherein: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images