Text generation method, training method, device, electronic device and storage medium

Abstract

The present disclosure provides a text generation method, a training method, an apparatus, an electronic device, and a storage medium, which relate to the field of artificial intelligence technologies, and particularly to technologies such as deep learning and text processing. The specific implementation solution of the text generation method is as follows: in response to the received question text for a target scenario, the question text is processed to obtain the matching probabilities between multiple candidate words for generating a reply text and the question text; according to a predetermined word set corresponding to the target scenario and the matching probabilities, a target candidate word is determined from the multiple candidate words, where the target candidate word represents a candidate word that exists in the predetermined word set and whose matching probability is greater than a first predetermined threshold; and a reply text is generated according to the target candidate word.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, in particular to technologies such as deep learning and text processing, and specifically relates to a text generation method, a training method, an apparatus, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of the technology of large-scale pre-trained language models based on the Transformer model structure, due to the rich training corpus of large-scale pre-trained language models and the good learning effect of language models, pre-trained language models are widely used in multiple fields to adapt to different application scenarios. Summary of the Invention

[0003] The present disclosure provides a text generation method, a training method, an apparatus, an electronic device, and a storage medium.

[0004] According to one aspect of the present disclosure, there is provided a text generation method, including: in response to receiving a question text for a target scenario, processing the question text to obtain matching probabilities between multiple candidate words for generating a reply text and the question text; determining a target candidate word from the multiple candidate words according to a predetermined word set corresponding to the target scenario and the matching probabilities, where the target candidate word represents a candidate word that exists in the predetermined word set and has a matching probability greater than a first predetermined threshold; and generating a reply text according to the target candidate word.

[0005] According to another aspect of the present disclosure, there is provided a training method for a text generation model, where the text generation model includes a processing module, a selection module, and a generation module, including: inputting a sample question text into the processing module to obtain sample matching probabilities between multiple sample candidate words for generating a sample reply text and the sample question text, where the sample question text is used to describe question information of a sample target scenario; inputting the multiple sample candidate words and the sample matching probabilities into the selection module to obtain a sample target candidate word; where the sample target candidate word represents a sample candidate word that exists in a sample predetermined word set corresponding to the sample target scenario and has a sample matching probability greater than a first predetermined threshold; inputting the sample target candidate word into the generation module to obtain a sample reply text; obtaining a loss value based on a target loss function according to the sample reply text and a label text corresponding to the sample question text; and adjusting parameters of the processing module based on the loss value to obtain a trained text generation model.

[0006] According to another aspect of the present disclosure, there is provided a text generation device, including: a first processing module, a first selection module, and a first generation module. The first processing module is configured to process the problem text in response to the received problem text for a target scenario, and obtain the matching probabilities of multiple candidate words for generating a reply text with the problem text; the first selection module is configured to determine a target candidate word from the multiple candidate words according to a predetermined word set corresponding to the target scenario and the matching probabilities, where the target candidate word represents a candidate word that exists in the predetermined word set and has a matching probability greater than a first predetermined threshold; and the first generation module is configured to generate a reply text according to the target candidate word.

[0007] According to another aspect of the present disclosure, there is provided a training device for a text generation model, including: a second processing module, a second selection module, a second generation module, a loss calculation module, and a first adjustment module. The second processing module is configured to input the sample problem text into the processing module, and obtain the sample matching probabilities of multiple sample candidate words for generating a sample reply text with the sample problem text, where the sample problem text is used to describe the problem information of the sample target scenario; the second selection module is configured to input the multiple sample candidate words and the sample matching probabilities into the selection module, and obtain a sample target candidate word; wherein, the sample target candidate word represents a sample candidate word that exists in a sample predetermined word set corresponding to the sample target scenario and has a sample matching probability greater than a first predetermined threshold; the second generation module is configured to input the sample target candidate word into the generation module, and obtain a sample reply text; the loss calculation module is configured to obtain a loss value based on a target loss function according to the sample reply text and the label text corresponding to the sample problem text; and the first adjustment module is configured to adjust the parameters of the processing module based on the loss value, and obtain a trained text generation model.

[0008] According to another aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method described above.

[0009] According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described above.

[0010] According to another aspect of the present disclosure, there is provided a computer program product, including a computer program, where the computer program implements the method described above when executed by a processor.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. Description of the Drawings

[0012] The drawings are used to better understand the present solution and do not constitute a limitation to the present disclosure. Among them:

[0013] Figure 1 Schematically shows an exemplary system architecture to which the text generation method and apparatus according to embodiments of the present disclosure can be applied;

[0014] Figure 2 Schematically shows a flowchart of the text generation method according to embodiments of the present disclosure;

[0015] Figure 3 Schematically shows a schematic diagram of the training stage and application stage of the text generation large model applying the text generation method according to embodiments of the present disclosure;

[0016] Figure 4A Schematically shows a schematic diagram of generating a response text corresponding to a target scenario in a restrictive manner according to embodiments of the present disclosure;

[0017] Figure 4B Schematically shows a schematic diagram of generating a response text corresponding to a target scenario in a restrictive manner according to another embodiment of the present disclosure;

[0018] Figure 5 Schematically shows a schematic diagram of generating a response text corresponding to a target scenario in a restrictive manner according to yet another embodiment of the present disclosure;

[0019] Figure 6 Schematically shows a flowchart of the training method of the text generation model according to embodiments of the present disclosure;

[0020] Figure 7 Schematically shows a block diagram of the text generation apparatus according to embodiments of the present disclosure;

[0021] Figure 8 Schematically shows a block diagram of the training apparatus of the text generation model according to embodiments of the present disclosure; and

[0022] Figure 9 Schematically shows a block diagram of an electronic device suitable for implementing the text generation method or the training method of the text generation model according to embodiments of the present disclosure. Detailed Embodiments

[0023] Exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted below for clarity and conciseness.

[0024] Large-scale pre-trained language models based on the Transformer model structure produce relatively general and broad results. However, for specific scenarios, such as advertising, gaming, etc., the generated results may involve specific game names or professional terms in specific scenarios. Directly using the pre-trained language model to process problem texts in specific scenarios results in relatively low relevance and accuracy of the generated results to the specific scenarios.

[0025] In view of this, the present disclosure provides a text generation method. By processing the problem text, the matching probabilities of multiple candidate words for generating a response text and the problem text are obtained; according to a predetermined word set corresponding to the target scenario and the matching probabilities, target candidate words are determined from the multiple candidate words. Since the target candidate words for generating the response text are candidate words existing in the predetermined word set and having a matching probability greater than a first predetermined threshold, the relevance and accuracy of the response text to the specific scenario are improved while ensuring the response text matches the problem text.

[0026] Figure 1 An exemplary system architecture to which the text generation method and apparatus according to the embodiments of the present disclosure can be applied is schematically shown.

[0027] It should be noted that Figure 1 The illustration is only an example of a system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios. For example, in another embodiment, an exemplary system architecture to which the text generation method and apparatus can be applied may include terminal devices, but the terminal devices may implement the text generation method and apparatus provided by the embodiments of the present disclosure without interacting with a server.

[0028] As Figure 1 shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0029] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as knowledge reading applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (for example only).

[0030] Terminal devices 101, 102, and 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.

[0031] Server 105 can be a server that provides various services, such as a background management server that supports the content browsed by users using terminal devices 101, 102, and 103 (for example only). The background management server can analyze and process data such as received user requests, and feedback the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0032] It should be noted that the text generation method provided by the embodiments of the present disclosure can generally be executed by terminal devices 101, 102, or 103. Correspondingly, the text generation device provided by the embodiments of the present disclosure can also be set in terminal devices 101, 102, or 103.

[0033] Alternatively, the text generation method provided by the embodiments of the present disclosure can generally also be executed by server 105. Correspondingly, the text generation device provided by the embodiments of the present disclosure can generally be set in server 105. The text generation method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103, and / or server 105. Correspondingly, the text generation device provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from server 105 and capable of communicating with terminal devices 101, 102, 103, and / or server 105.

[0034] For example, the terminal devices 101, 102, and 103 can obtain the question text issued by the user in the target scenario, and then send the obtained question text to the server 105. The server 105 processes the question text to obtain the matching probabilities between multiple candidate words for generating the reply text and the question text. According to the predetermined word set corresponding to the target scenario and the matching probabilities, the target candidate word is determined from the multiple candidate words; and according to the target candidate word, the reply text is generated. Or a server or a server cluster capable of communicating with the terminal devices 101, 102, 103 and / or the server 105 processes the question text, and finally realizes generating the reply text corresponding to the target scenario.

[0035] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0036] In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, disclosure, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good customs.

[0037] In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the authorization or consent of the user is obtained.

[0038] Figure 2 The flowchart of the text generation method according to an embodiment of the present disclosure is schematically shown.

[0039] As Figure 2 shown, the method 200 includes operations S210 to S230.

[0040] In operation S210, in response to the received question text for the target scenario, the question text is processed to obtain the matching probabilities between multiple candidate words for generating the reply text and the question text.

[0041] In operation S220, according to the predetermined word set corresponding to the target scenario and the matching probabilities, the target candidate word is determined from the multiple candidate words.

[0042] In operation S230, according to the target candidate word, the reply text is generated.

[0043] According to an embodiment of the present disclosure, the target scenario can be various scenarios in actual applications, such as: advertisements, games, life scenarios, etc. The problem text may include words for describing the category to which the target scenario belongs. For example, the problem text may be "Please recommend a most popular game to me." Then, the target scenario can be determined as the game scenario. The problem text may not include words for describing the category to which the target scenario belongs, but directly include specific scenario description words. For example, the problem text may be "Please recommend a restaurant with private rooms that can accommodate 10 people." "Restaurant" can be classified as a life scenario category.

[0044] According to an embodiment of the present disclosure, a pre-trained language model based on the Transformer model structure can be used to process the problem text to obtain the matching probabilities between multiple candidate words for generating the response text and the problem text. For the pre-trained language model, usually a complete response text can be generated after multiple rounds of iterative processes, and multiple candidate words can be generated in each round of iterative process. The matching probability between the candidate word and the problem text can represent the matching probability between the word in the corpus of the pre-training process and the problem text.

[0045] For example: The problem text may be "Please recommend a most popular game to me." Inputting the problem text into the pre-trained language model, the matching probabilities between the obtained candidate words and the problem text may sequentially include: "M" (matching probability: 0.6), "P" (matching probability: 0.4), "U" (matching probability: 0.5).

[0046] In the related art, usually the target candidate word is determined from the candidate words based on the matching probability. Since the pre-trained language model has a high degree of generality, the response text generated based on the matching probability may be irrelevant to the target scenario. For example: In the game scenario, there is no game named "M", but the response text generated based on the matching probability is the "Mxxx" game.

[0047] Therefore, the embodiment of the present disclosure screens the candidate words based on a predetermined word set corresponding to the target scenario, so as to enable the model to generate a response text that matches the target scenario in a limited manner.

[0048] According to an embodiment of the present disclosure, the predetermined word set corresponding to the target scenario may include words with a relatively high occurrence probability in the target scenario. For example: In the game scenario, the predetermined word set may include various game names, game development institutions, etc. In the advertisement scenario, the predetermined word set may include theme keywords for describing a certain type of commodity, etc. The target candidate word represents a candidate word that exists in the predetermined word set and the matching probability is greater than the first predetermined threshold.

[0049] For example: The predetermined word set corresponding to the game scenario may include "U" and "V". According to the predetermined word set corresponding to the target scenario, it can be determined that the candidate word "U" exists in the predetermined word set. Therefore, the target candidate word can be determined as "U".

[0050] For example: The predetermined word set corresponding to the game scenario may include "U" and "M". According to the predetermined word set corresponding to the target scenario, it can be determined that both candidate words "U" and "M" exist in the predetermined word set. The predetermined threshold may be 0.5. Since only the matching probability of the candidate word "M" is greater than 0.5, the target candidate word can be determined as "M".

[0051] According to the embodiments of the present disclosure, the above selection operation is performed on the candidate words generated in each round of the text generation model to determine the target candidate word generated in each round. It should be noted that based on the characteristics of the generative model, the target candidate word generated in each round can be used as the input text for the next round together with the question text, and finally a response text matching the question text is generated in a guiding manner.

[0052] According to the embodiments of the present disclosure, by processing the question text, the matching probabilities of multiple candidate words for generating the response text and the question text are obtained; according to the predetermined word set corresponding to the target scenario and the matching probabilities, the target candidate word is determined from the multiple candidate words. Since the target candidate word for generating the response text is a candidate word that exists in the predetermined word set and the matching probability is greater than the first predetermined threshold, the relevance and accuracy of the response text to the specific scenario are improved while ensuring that the response text matches the question text.

[0053] The following refers to Figures 3 to 5 and further describes the Figure 2 shown method in combination with specific embodiments.

[0054] Figure 3 Schematically shows a schematic diagram of the training stage and application stage of the text generation large model applying the text generation method of the embodiments of the present disclosure.

[0055] As Figure 3 shown, in Embodiment 300, in the model training stage, the initial model is trained using the sample corpus 301 to obtain the general text generation large model 302. The general text generation large model 302 can be a general model applicable to various fields. Therefore, the generation results are relatively general and broad.

[0056] In the model application stage, before starting the general text generation large model 302, the predetermined word set 303 corresponding to the target scenario can be loaded. Then the question text 311 is input into the restricted text generation model 3021 loaded with the predetermined word set 303 corresponding to the target scenario.

[0057] Based on the constraints of the candidate words by the predetermined word set, a target candidate word 312 is determined from the multiple candidate words generated in each round. Finally, a response text 313 is generated according to the target candidate word 312.

[0058] Figure 4A Schematically shows a schematic diagram of generating a response text corresponding to a target scenario according to an embodiment of the present disclosure.

[0059] As Figure 4A shown, in Embodiment 400A, the question text 411 in the target scenario is input into the constrained text generation model. The constrained text generation model can be constructed based on the transformer model structure. By extracting the text features of the question text and then based on the attention mechanism, the text features are processed to obtain the matching probabilities of multiple candidate words with the question text.

[0060] According to an embodiment of the present disclosure, the predetermined word set is a trie composed of N layers of predetermined word nodes, where N is an integer greater than 1. In Embodiment 400A, N can be equal to 4. In the trie, the root node can be a preset fixed word, which can be determined according to the specific application scenario, indicating that the first word of all response texts is the same. For example, it can be "Answer".

[0061] According to an embodiment of the present disclosure, the first candidate word is obtained by the constrained text generation model performing the m-th round of processing on the question text; the second candidate word is obtained by the constrained text generation model performing the (m + 1)-th round of processing on the question text and the first target candidate word, where m is an integer greater than or equal to 1. Determine the first target predetermined word identical to the first candidate word from the multiple predetermined words in the n-th layer node of the trie, where n is an integer greater than or equal to 1 and less than or equal to N; based on the first matching probability, determine the first target candidate word from the first target predetermined words; determine the second target predetermined word identical to the second candidate word from the multiple predetermined words in the (n + 1)-th layer node associated with the first target predetermined word; and based on the second matching probability, determine the second target candidate word from the second target predetermined words.

[0062] For example: The matching probabilities 421 of the first candidate word can be: A1(0.7), A3(0.2), and M(0.8). In the trie of the target scenario, the second layer node associated with the root node may include the predetermined word A1 and the predetermined word A2, and the first target predetermined word identical to the first candidate word can be determined as the predetermined word A1. Since only one predetermined word is matched based on the trie, therefore, the predetermined word A1 is the first target candidate word 422.

[0063] According to an embodiment of the present disclosure, there may be multiple candidate words, and there may also be multiple target predetermined words selected based on the trie. For example, the matching probabilities 423 of the second candidate words may be: B1(0.7), B2(0.3), N(0.2), and B3(0.8). Among the nodes in the third layer of the trie, the nodes associated with the first target predetermined word A1 may include the predetermined word B1 and the predetermined word B2. Since the second candidate words include B1 and B2, therefore, based on the second matching probability, B1(0.7) with a high matching probability may be used as the second target candidate word 424. And based on the first target candidate word and the second target candidate word, a reply text A1B1431 is generated.

[0064] According to an embodiment of the present disclosure, when the number of target predetermined words is greater than 1, a threshold of the matching probability may also be set based on the needs of the actual application scenario, and the predetermined words with the second matching probability greater than the matching probability threshold are used as the second target candidate words. At this time, there may be multiple second target candidate words.

[0065] According to an embodiment of the present disclosure, based on the predetermined words in the trie and the association relationships between the predetermined words in the trie, the target candidate words matching the target scenario can be directionally selected from the candidate words generated in each round, thereby improving the relevance of the reply text to the target scenario and the accuracy of the reply text.

[0066] For some relatively complex application scenarios, there are more predetermined words involved and the association relationships are relatively complex. The association degrees between different predetermined words can be added in the trie, and based on the second matching probability and the association degree, the second target candidate words are determined from the second target predetermined words, so as to limit the generation result of the model.

[0067] Figure 4B Schematically shows a schematic diagram of generating a reply text corresponding to a target scenario according to another embodiment of the present disclosure.

[0068] As Figure 4B shown, in Embodiment 400B, the question text 411 in the target scenario is input into the constrained text generation model. The structure of the constrained text generation model in this embodiment is the same as that in Embodiment 400A and will not be elaborated here.

[0069] According to an embodiment of the present disclosure, the trie includes the association degrees between each predetermined word in the nth layer node and each predetermined word in the n+1th layer node. For example: the association degree between the predetermined word A1 in the second layer node and the root node may be 0.7, and the association degree between the predetermined word A2 in the second layer node and the root node may be 0.3.

[0070] According to an embodiment of the present disclosure, the selection process of the first target candidate word 422 is the same as that in Embodiment 400A, which will not be elaborated here. The matching probabilities 423 of the second candidate words may include: B1 (0.7), B2 (0.3), N (0.2), B3 (0.8). The nodes associated with the predetermined word A1 in the third-level nodes of the trie tree include the predetermined word B1 and the predetermined word B2. Since B1 and B2 are included in the second candidate words, it can be determined that the second target predetermined words are the predetermined word B1 and the predetermined word B2.

[0071] According to an embodiment of the present disclosure, the degree of association between the predetermined word A1 and the predetermined word B1 in the trie tree is 0.2, and the degree of association between the predetermined word A1 and the predetermined word B2 is 0.8.

[0072] According to an embodiment of the present disclosure, the third target predetermined word can be determined from the second target predetermined words based on the degree of association. For example: The degree of association threshold can be set to 0.5. The degree of association between the predetermined word A1 and the predetermined word B2 is 0.8, which is greater than 0.5. It can be determined that the third target predetermined word is the predetermined word B2.

[0073] It should be noted that there may also be multiple third target predetermined words determined from the second target predetermined words based on the degree of association. For multiple third target predetermined words, the second target candidate word can also be determined from the third target predetermined words based on the second matching probability. Since there is only one third target predetermined word in this Embodiment 400B, it can be determined that the second target candidate word 425 is B2.

[0074] According to an embodiment of the present disclosure, the degree of correlation between the second target predetermined word and the first target candidate word can also be obtained based on the second matching probability and the degree of association, and the second target candidate word can be determined from the second target predetermined words based on the degree of correlation.

[0075] For example: The second target predetermined words are the predetermined word B1 and the predetermined word B2. The degree of association between the predetermined word A1 and the predetermined word B1 in the trie tree is 0.2, and the degree of association between the predetermined word A1 and the predetermined word B2 is 0.8. The matching probability of B1 in the second candidate words is 0.7, and the matching probability of B2 is 0.3.

[0076] According to an embodiment of the present disclosure, the degree of correlation between the second target predetermined word and the first target candidate word can be obtained based on the product of the second matching probability and the degree of association. For example: The degree of correlation between the predetermined word B1 and the first target candidate word A1 can be 0.7×0.2 = 0.14. The degree of correlation between the predetermined word B2 and the first target candidate word A1 can be 0.3×0.8 = 0.24. Since 0.24 is greater than 0.14, it can be determined that the second target candidate word is B2.

[0077] According to an embodiment of the present disclosure, the relevance between the second target predetermined word and the first target candidate word can also determine the relevant weight according to the relevance degree first, and then obtain the relevance degree according to the second matching probability and the relevant weight.

[0078] According to an embodiment of the present disclosure, the relevant weights of multiple predetermined words can be determined according to the ratio of the relevance degrees between multiple predetermined words in the (n + 1)-th layer nodes associated with the predetermined words in the n-th layer nodes. For example, for example, in a trie, the relevance degree between the predetermined word A1 and the predetermined word B1 is 0.2, and the relevance degree between the predetermined word A1 and the predetermined word B2 is 0.8, 0.2:0.8 = 1:4. It can be determined that the relevant weight between the predetermined word A1 and the predetermined word B1 is 1, and the relevant weight between the predetermined word A1 and the predetermined word B2 is 4. Then, according to the second matching probability and the relevant weight, the relevance degree between the predetermined word B1 and the first target candidate word A1 is 0.7×1 = 0.7; the relevance degree between the predetermined word B2 and the first target candidate word A1 is 0.3×4 = 1.2. Since 0.7 is less than 1.2, it can be determined that the second target candidate word is B2.

[0079] According to an embodiment of the present disclosure, based on the method described above, based on the matching probability 426 of the third candidate word and the trie, the third target candidate word 427 can be determined from the third candidate words as C2. And finally, a response text 428 is generated.

[0080] According to an embodiment of the present disclosure, the relevance degree between different predetermined words can be determined based on the semantics between different words in the application scenario, or can be determined based on the probability that different words in the application scenario appear in the same sentence. Based on the relevance degree between the predetermined words, candidate words matching the target scenario are directionally selected, so as to be suitable for generating rare response texts in relatively complex or special scenarios and improve the accuracy of the response text.

[0081] According to an embodiment of the present disclosure, in a restrictive text generation model, each time the target candidate word selected based on the trie can be multiple. In order to improve the recommendation accuracy of the response text, the target word can be determined from the target candidate words; and a response text is generated according to the target word.

[0082] According to an embodiment of the present disclosure, for multiple target candidate words generated in the same round, the target candidate words can be sorted based on the matching probability between the target candidate words and the question text to obtain a sorting result; and based on the sorting result, the target word is determined from the target candidate words.

[0083] For example, the target candidate words obtained based on the matching probability threshold are S and T respectively, where the matching probability between S and the question text is 0.8, and the matching probability between T and the question text is 0.5, then the target word can be determined as S.

[0084] According to an embodiment of the present disclosure, while using a trie tree to improve the relevance between candidate words and the target scenario, combining the matching probability between candidate words and the question text can fully utilize the accuracy advantage of the pre-trained large model and improve the accuracy of the generated result.

[0085] According to an embodiment of the present disclosure, since the predetermined word set includes multiple predetermined words and the correlation between multiple predetermined words, multiple candidate response texts can be generated according to the target candidate word; and according to the correlation, the matching degree between multiple candidate response texts and the question text can be determined; and based on the matching degree, a response text can be determined from multiple candidate response texts.

[0086] Figure 5 Schematically shows a schematic diagram of generating a response text corresponding to the target scenario according to another embodiment of the present disclosure.

[0087] As Figure 5 shown, in Embodiment 500, the question text 511 in the target scenario is input into the constrained text generation model, and candidate words are selected based on the trie tree of the target scenario to obtain the first target candidate word A1521, the second target candidate word B2522, and the third target candidate words 523C2 and C3.

[0088] According to an embodiment of the present disclosure, the arrangement order of the target candidate word in the candidate response text can be determined according to the position of the target candidate word in the trie tree; and based on the arrangement order, multiple candidate response texts can be generated according to the target candidate word.

[0089] According to an embodiment of the present disclosure, determining the arrangement order of the target candidate word in the candidate response text according to the position of the target candidate word in the trie tree may include the following operations: in response to the i-th target candidate word and the j-th target candidate word being in the n-th layer of the trie tree, determining the n-th position of the i-th target candidate word in the first candidate response text and the n-th position of the j-th target candidate word in the second candidate response text, where n is an integer greater than or equal to 1 and less than or equal to N, and i and j are integers greater than or equal to 1 and less than or equal to I.

[0090] For example: Both the third target candidate words C2 and C3 are in the 4th layer of the trie tree. Therefore, C2 can be in the 4th position in the first candidate response text, and C3 can be in the 4th position in the second candidate response text.

[0091] According to an embodiment of the present disclosure, in response to the i-th target candidate word being in the n-th layer of the trie tree and the j-th target candidate word being in the n+1-th layer of the trie tree, determining the n-th position of the i-th target candidate word in the first candidate response text and the n+1-th position of the j-th target candidate word in the first candidate response text.

[0092] For example: The first target candidate word A1 is in the second layer of the trie, and the second target candidate word B2 is in the third layer of the trie. It can be determined that the position of the first target candidate word A1 in the first candidate response text 531 is the second character; the position of the second target candidate word B2 in the first candidate response text 531 is the third character. In this embodiment 500, the root node can be an empty character.

[0093] Thus, the first candidate response text 531 generated in this embodiment 500 is A1B2C2; the second candidate response text 532 is A1B2C3.

[0094] According to an embodiment of the present disclosure, in an actual application scenario, multiple candidate response texts can be used as the final response text for model output. It is also possible to determine the matching degree between multiple candidate response texts and the question text based on the relevance, and based on the matching degree, determine the final response text from multiple candidate response texts.

[0095] For example: In the first candidate response text 531, the relevance between the root node and A1 is 0.7, the relevance between A1 and B2 is 0.8, and the relevance between B2 and C2 is 0.4. The matching degree between the first candidate response text 531 and the question text 511 can be determined according to the average value of the relevance, that is: 0.95.

[0096] For example: In the second candidate response text 532, the relevance between the root node and A1 is 0.7, the relevance between A1 and B2 is 0.8, and the relevance between B2 and C3 is 0.6. The matching degree between the second candidate response text 532 and the question text 511 can be determined according to the average value of the relevance, that is: 1.05.

[0097] Since the matching degree 0.95 between the first candidate response text 531 and the question text 511 is less than the matching degree 1.05 between the second candidate response text 532 and the question text 511, it can be determined that the response text 533 is A1B2C3.

[0098] According to an embodiment of the present disclosure, in addition to according to the average value of the relevance, association weights can also be configured for adjacent layer nodes in the trie, and based on the association weights, the relevance is weighted to obtain the final matching degree.

[0099] According to an embodiment of the present disclosure, based on the relevance between different predetermined words in the trie, the candidate response texts are screened, which improves the accuracy of the generation result from the perspective of the semantic relevance of the overall response text.

[0100] The method provided by the embodiments of the present disclosure is implemented based on a pre-trained language model. For a language model with rich training corpora and good learning effects, it is possible to directly load the trie corresponding to the target scenario without secondary training, and then generate a response text related to the target scenario and matching the question text.

[0101] However, in actual application scenarios, the pre-trained language model may have problems such as less corpora and poor learning effects during the pre-training process. In this case, after loading the trie corresponding to the target scenario, the accuracy of the output result can be improved through secondary training.

[0102] Figure 6 The flowchart of the training method of the text generation model according to the embodiments of the present disclosure is schematically shown.

[0103] As Figure 6 shown, the training method 600 may include operations S610 to S650.

[0104] In operation S610, the sample question text is input into the processing module to obtain the sample matching probabilities between multiple sample candidate words for generating the sample response text and the sample question text.

[0105] In operation S620, the multiple sample candidate words and the sample matching probabilities are input into the selection module to obtain the sample target candidate words.

[0106] In operation S630, the sample target candidate words are input into the generation module to obtain the sample response text.

[0107] In operation S640, based on the target loss function, according to the sample response text and the label text corresponding to the sample question text, a loss value is obtained.

[0108] In operation S650, based on the loss value, the parameters of the processing module are adjusted to obtain the trained text generation model.

[0109] According to the embodiments of the present disclosure, the sample question text is used to describe the question information of the sample target scenario. The sample target candidate words represent the sample candidate words that exist in the sample predetermined word set corresponding to the sample target scenario and the sample matching probability is greater than the first predetermined threshold. The definitions of the sample question text, the sample predetermined word set, and the sample target candidate words are the same as those of the question text, the predetermined word set, and the target candidate words in the text generation method described above, and will not be elaborated here.

[0110] According to the embodiments of the present disclosure, the target loss function may adopt any loss function applicable to the training of deep learning models, such as: cross-entropy loss function, etc. The embodiments of the present disclosure do not make specific limitations in this regard.

[0111] According to an embodiment of the present disclosure, the tag text represents a standard response text adapted to the target scenario corresponding to the sample question text. Since the predetermined word set can enable the model to generate a response text related to the target scenario in a directional manner, during the training process, based on the loss value, the parameters of the pre-trained language model, that is, the parameters of the processing module, can be mainly adjusted to achieve the purpose of improving the model accuracy.

[0112] According to an embodiment of the present disclosure, since in the predetermined word set, the association relationship between different predetermined words can be pre-configured, and this association relationship also affects the accuracy of the result, during the training process, based on the loss value, the parameters of the selection module, that is, the parameters such as the association degree or association weight between different predetermined words, can also be adjusted to improve the efficiency of model training.

[0113] Figure 7 A block diagram of a text generation device according to an embodiment of the present disclosure is schematically shown.

[0114] As Figure 7 shown, the text generation device 700 may include a first processing module 710, a first selection module 720, and a first generation module 730.

[0115] The first processing module 710 is configured to process the question text in response to the received question text for the target scenario, and obtain the matching probabilities of multiple candidate words for generating the response text with the question text.

[0116] The first selection module 720 is configured to determine a target candidate word from multiple candidate words according to the predetermined word set corresponding to the target scenario and the matching probability, where the target candidate word represents a candidate word that exists in the predetermined word set and the matching probability is greater than a first predetermined threshold.

[0117] The first generation module 730 is configured to generate a response text according to the target candidate word.

[0118] According to an embodiment of the present disclosure, the predetermined word set is a trie tree composed of N layers of predetermined word nodes, N is an integer greater than 1; the multiple candidate words include a first candidate word and a second candidate word; the first candidate word is obtained by performing the m-th round of processing on the question text; the second candidate word is obtained by performing the (m + 1)-th round of processing on the question text and the first target candidate word, m is an integer greater than or equal to 1. The first selection module includes: a first selection sub-module, a first determination sub-module, a second selection sub-module, and a second determination sub-module.

[0119] The first selection sub-module is configured to determine a first target predetermined word identical to the first candidate word from multiple predetermined words in the n-th layer node of the trie tree, where n is an integer greater than or equal to 1 and less than or equal to N.

[0120] The first determination sub-module is configured to determine a first target candidate word from the first target predetermined words based on the first matching probability.

[0121] The second selection sub-module is configured to determine a second target predetermined word that is the same as the second candidate word from multiple predetermined words in the (n + 1)-th layer nodes associated with the first target predetermined words.

[0122] The second determination sub-module is configured to determine a second target candidate word from the second target predetermined words based on the second matching probability.

[0123] According to an embodiment of the present disclosure, the trie tree includes the degree of association between each predetermined word in the n-th layer nodes and each predetermined word in the (n + 1)-th layer nodes. The first selection module further includes: a third determination sub-module. The third determination sub-module is configured to determine a second target candidate word from the second target predetermined words based on the second matching probability and the degree of association.

[0124] According to an embodiment of the present disclosure, the third determination sub-module may include: a first determination unit and a second determination unit. The first determination unit is configured to determine a third target predetermined word from the second target predetermined words according to the degree of association, where the degree of association between the third target predetermined word and the first target predetermined word is greater than a second predetermined threshold. The second determination unit is configured to determine a second target candidate word from the third target predetermined words based on the second matching probability.

[0125] According to an embodiment of the present disclosure, the third determination sub-module includes: a relevance calculation unit and a third determination unit. The relevance calculation unit is configured to obtain the relevance between the second target predetermined word and the first target candidate word according to the second matching probability and the degree of association. The third determination unit is configured to determine a second target candidate word from the second target predetermined words based on the relevance.

[0126] According to an embodiment of the present disclosure, the relevance calculation unit includes: a weight calculation sub-unit and a relevance calculation sub-unit. The weight calculation sub-unit is configured to determine a relevant weight according to the degree of association. The relevance calculation sub-unit is configured to obtain the relevance according to the second matching probability and the relevant weight.

[0127] According to an embodiment of the present disclosure, the first generation module includes: a fourth determination sub-module and a first generation sub-module. The fourth determination sub-module is configured to determine a target word from the target candidate words. The first generation sub-module is configured to generate a response text according to the target word.

[0128] According to an embodiment of the present disclosure, the fourth determination sub-module may include: a sorting unit and a fourth determination unit. The sorting unit is configured to sort the target candidate words based on the matching probability between the target candidate words and the question text to obtain a sorting result. The fourth determination unit is configured to determine a target word from the target candidate words based on the sorting result.

[0129] According to an embodiment of the present disclosure, the predetermined word set includes a plurality of predetermined words and the degree of association between the plurality of predetermined words. The first generation module includes: a second generation sub-module, a matching degree calculation sub-module, and a fifth determination sub-module. The second generation sub-module is configured to generate a plurality of candidate response texts according to the target candidate words. The matching degree calculation sub-module is configured to determine the matching degree between the plurality of candidate response texts and the question text according to the degree of association. The fifth determination sub-module is configured to determine a response text from the plurality of candidate response texts based on the matching degree.

[0130] According to an embodiment of the present disclosure, the predetermined word set is a trie tree composed of N layers of predetermined word nodes, where N is an integer greater than 1; the second generation sub-module includes: a bit order determination unit and a first generation unit. The bit order determination unit is configured to determine the arrangement bit order of the target candidate word in the candidate response text according to the position of the target candidate word in the trie tree. The first generation unit is configured to generate a plurality of candidate response texts based on the arrangement bit order according to the target candidate word.

[0131] According to an embodiment of the present disclosure, the bit order determination unit includes: a first bit order determination sub-unit and a second bit order determination sub-unit. The first bit order determination sub-unit is configured to, in response to the i-th target candidate word and the j-th target candidate word being in the n-th layer of the trie tree, determine the n-th position of the i-th target candidate word in the first candidate response text and the n-th position of the j-th target candidate word in the second candidate response text, where n is an integer greater than or equal to 1 and less than or equal to N, and i and j are integers greater than or equal to 1 and less than or equal to I. The second bit order determination sub-unit is configured to, in response to the i-th target candidate word being in the n-th layer of the trie tree and the j-th target candidate word being in the n + 1-th layer of the trie tree, determine the n-th position of the i-th target candidate word in the first candidate response text and the n + 1-th position of the j-th target candidate word in the first candidate response text.

[0132] According to an embodiment of the present disclosure, the matching degree calculation sub-module includes: a first matching degree calculation unit and a second matching degree calculation unit. The first matching degree calculation unit is configured to determine the matching degree between the first candidate response text and the question text according to the degree of association between the plurality of target candidate words used to generate the first candidate response text. The second matching degree calculation unit is configured to determine the matching degree between the second candidate response text and the question text according to the degree of association between the plurality of target candidate words used to generate the second candidate response text.

[0133] According to an embodiment of the present disclosure, the first processing module includes: a feature extraction sub-module and an attention sub-module. The feature extraction sub-module is configured to extract the text features of the question text. The attention sub-module is configured to process the text features based on the attention mechanism to obtain the matching probabilities of the plurality of candidate words and the question text.

[0134] Figure 8A block diagram of a training apparatus for a text generation model according to an embodiment of the present disclosure is schematically shown.

[0135] As Figure 8 shown, the training apparatus 800 may include a second processing module 810, a second selection module 820, a second generation module 830, a loss calculation module 840, and a first adjustment module 850.

[0136] The second processing module 810 is configured to input the sample problem text into the processing module to obtain a plurality of sample candidate words for generating the sample response text and the sample matching probability between the sample candidate words and the sample problem text, where the sample problem text is used to describe the problem information of the sample target scenario.

[0137] The second selection module 820 is configured to input the plurality of sample candidate words and the sample matching probability into the selection module to obtain the sample target candidate words; wherein, the sample target candidate words represent the sample candidate words that exist in the sample predetermined word set corresponding to the sample target scenario and the sample matching probability is greater than a first predetermined threshold.

[0138] The second generation module 830 is configured to input the sample target candidate words into the generation module to obtain the sample response text.

[0139] The loss calculation module 840 is configured to obtain a loss value based on the target loss function according to the sample response text and the label text corresponding to the sample problem text.

[0140] The first adjustment module 850 is configured to adjust the parameters of the processing module based on the loss value to obtain a trained text generation model.

[0141] According to an embodiment of the present disclosure, the training apparatus may further include a second adjustment module, configured to adjust the parameters of the processing module and the parameters of the selection module based on the loss value to obtain a trained text generation module.

[0142] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0143] According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0144] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method described above.

[0145] According to an embodiment of the present disclosure, a computer program product includes a computer program which, when executed by a processor, implements the method described above.

[0146] Figure 9 FIG. shows a schematic block diagram of an exemplary electronic device 900 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0147] As Figure 9 shown, the device 900 includes a computing unit 901 which can perform various appropriate actions and processes according to a computer program stored in a read only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0148] A plurality of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0149] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 executes the various methods and processes described above, such as a text generation method or a training method of a text generation model. For example, in some embodiments, the text generation method or the training method of the text generation model can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the text generation method or the training method of the text generation model described above can be executed. Alternatively, in other embodiments, the computing unit 901 can be configured to execute the text generation method or the training method of the text generation model by any other suitable means (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0152] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0154] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0155] A computer system may include a client and a server. The client and the server are generally far away from each other and usually interact through a communication network. The client-server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0156] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0157] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A text generation method, comprising: In response to the received question text for a target scenario, using a text generation model to process the question text to obtain the matching probabilities between multiple candidate words for generating a reply text and the question text; the matching probability represents the matching probability between the words in the corpus during the pre-training process and the question text; Using the text generation model to determine a target candidate word from the multiple candidate words according to multiple predetermined words in a predetermined word set corresponding to the target scenario, the association degree between the multiple predetermined words, and the matching probability, where the target candidate word represents a candidate word that exists in the predetermined word set, the matching probability is greater than a first predetermined threshold, and the association degree is greater than a second predetermined threshold. The association degree is obtained during the process of training the text generation model using sample questions for describing the target scenario and the label text based on the loss value between the sample reply text and the label text; And Using the text generation model to generate the reply text according to the target candidate word and the association degree.

2. The method according to claim 1, wherein, The predetermined word set is a trie composed of N layers of predetermined word nodes, where N is an integer greater than 1; the multiple candidate words include a first candidate word and a second candidate word; the first candidate word is obtained by performing the m-th round of processing on the question text; The second candidate word is obtained by performing the (m + 1)-th round of processing on the question text and the first target candidate word, where m is an integer greater than or equal to 1; in the trie, the association degree between each predetermined word in the n-th layer node and each predetermined word in the (n + 1)-th layer node is included; The determining of the target candidate word from the multiple candidate words according to multiple predetermined words in a predetermined word set corresponding to the target scenario, the association degree between the multiple predetermined words, and the matching probability includes: Determining a first target predetermined word identical to the first candidate word from multiple predetermined words in the n-th layer node of the trie, where n is an integer greater than or equal to 1 and less than or equal to N; Determining the first target candidate word from the first target predetermined words based on a first matching probability; Determining a second target predetermined word identical to the second candidate word from multiple predetermined words in the (n + 1)-th layer node associated with the first target predetermined word; and Determining a second target candidate word from the second target predetermined words based on a second matching probability and the association degree.

3. The method according to claim 2, wherein, The determining of the second target candidate word from the second target predetermined words based on the second matching probability and the association degree includes: Determining a third target predetermined word from the second target predetermined words according to the association degree, where the association degree between the third target predetermined word and the first target predetermined word is greater than the second predetermined threshold; and Determining the second target candidate word from the third target predetermined words based on the second matching probability.

4. The method according to claim 2, wherein, The determining of the second target candidate word from the second target predetermined words based on the second matching probability and the association degree includes: Obtaining the relevance between the second target predetermined word and the first target candidate word according to the second matching probability and the association degree; and Determine the second target candidate word from the second target predetermined words based on the relevance.

5. The method according to claim 4, wherein The obtaining the relevance between the second target predetermined word and the first target candidate word according to the second matching probability and the relevance includes: Determine a relevant weight according to the relevance; and Obtain the relevance according to the second matching probability and the relevant weight.

6. The method according to claim 1, wherein, The generating the response text according to the target candidate word includes: Determine a target word from the target candidate words; and Generate the response text according to the target word.

7. The method according to claim 6, wherein, The determining a target word from the target candidate words includes: Rank the target candidate words based on the matching probability between the target candidate words and the question text to obtain a ranking result; and Determine the target word from the target candidate words based on the ranking result.

8. The method according to claim 1, wherein the generating the response text according to the target candidate word and the relevance includes: Generate a plurality of candidate response texts according to the target candidate word; Determine the matching degree between the plurality of candidate response texts and the question text according to the relevance; and Determine the response text from the plurality of candidate response texts based on the matching degree.

9. The method according to claim 8, wherein The predetermined word set is a trie composed of N layers of predetermined word nodes, where N is an integer greater than 1; The generating a plurality of candidate response texts according to the target candidate word includes: Determine the arrangement order of the target candidate word in the candidate response text according to the position of the target candidate word in the trie; and Generate a plurality of candidate response texts according to the target candidate word based on the arrangement order.

10. The method according to claim 9, wherein, There are I target candidate words, where I is an integer greater than 1; the determining the arrangement order of the target candidate word in the candidate response text according to the position of the target candidate word in the trie includes: In response to the i-th target candidate word and the j-th target candidate word being in the n-th layer of the trie, determine the n-th position of the i-th target candidate word in the first candidate response text and the n-th position of the j-th target candidate word in the second candidate response text, where n is an integer greater than or equal to 1 and less than or equal to N, and i, j are integers greater than or equal to 1 and less than or equal to I; and In response to the i-th target candidate word being in the n-th layer of the trie and the j-th target candidate word being in the n+1-th layer of the trie, determine the n-th position of the i-th target candidate word in the first candidate response text and the n+1-th position of the j-th target candidate word in the first candidate response text.

11. The method according to claim 8, wherein, The determining the matching degree between the plurality of candidate response texts and the question text according to the relevance includes: Determine the matching degree between the first candidate response text and the question text according to the relevance between the plurality of target candidate words used to generate the first candidate response text; and Determine the matching degree between the second candidate response text and the question text according to the relevance between the plurality of target candidate words used to generate the second candidate response text.

12. The method according to claim 1, wherein, Processing the problem text to obtain the matching probabilities between multiple candidate words for generating a response text and the problem text includes: Extracting the text features of the problem text; and Based on the attention mechanism, processing the text features to obtain the matching probabilities between multiple candidate words and the problem text.

13. A training method for a text generation model, the text generation model including a processing module, a selection module, and a generation module; including: Inputting a sample problem text into the processing module to obtain sample matching probabilities between multiple sample candidate words for generating a sample response text and the sample problem text, where the sample problem text is used to describe the problem information of a sample target scenario; the sample matching probabilities represent the matching probabilities between words in the corpus during the pre-training process and the sample problem text; Inputting the multiple sample candidate words, multiple sample predetermined words in a sample predetermined word set, the sample association degrees between the multiple sample predetermined words, and the sample matching probabilities into the selection module to obtain a sample target candidate word; where the sample target candidate word represents a sample candidate word that exists in the sample predetermined word set corresponding to the sample target scenario, the sample matching probability is greater than a first predetermined threshold, and the sample association degree is greater than a second predetermined threshold; Inputting the sample target candidate word into the generation module to obtain the sample response text; Based on a target loss function, obtaining a loss value according to the sample response text and the label text corresponding to the sample problem text; Based on the loss value, adjusting the parameters of the processing module and the parameters of the selection module to obtain the trained text generation model and the association degree between the multiple sample predetermined words.

14. A text generation device, including: A first processing module, configured to, in response to a problem text received for a target scenario, process the problem text using a text generation model to obtain the matching probabilities between multiple candidate words for generating a response text and the problem text; the matching probabilities represent the matching probabilities between words in the corpus during the pre-training process and the problem text; A first selection module, configured to use the text generation model to determine a target candidate word from the multiple candidate words according to multiple predetermined words in a predetermined word set corresponding to the target scenario, the association degrees between the multiple predetermined words, and the matching probabilities, where the target candidate word represents a candidate word that exists in the predetermined word set, the matching probability is greater than a first predetermined threshold, and the association degree is greater than a second predetermined threshold; The association degree is obtained during the process of training the text generation model using a sample problem for describing the target scenario and the label text based on the loss value between the sample response text and the label text; And A first generation module, configured to use the text generation model to generate the response text according to the target candidate word and the association degree.

15. The apparatus according to claim 14, wherein, The predetermined word set is a trie tree composed of N layers of predetermined word nodes, N is an integer greater than 1; the multiple candidate words include a first candidate word and a second candidate word; the first candidate word is obtained by processing the problem text in the m-th round; The second candidate word is obtained by performing the (m + 1)-th round of processing on the problem text and the first target candidate word, where m is an integer greater than or equal to 1; in the trie, the association degree between each predetermined word in the n-th layer node and each predetermined word in the (n + 1)-th layer node is included; The first selection module; The first selection module includes: A first selection sub-module, configured to determine a first target predetermined word identical to the first candidate word from multiple predetermined words in the n-th layer node of the trie, where n is an integer greater than or equal to 1 and less than or equal to N; A first determination sub-module, configured to determine the first target candidate word from the first target predetermined words based on a first matching probability; A second selection sub-module, configured to determine a second target predetermined word identical to the second candidate word from multiple predetermined words in the (n + 1)-th layer node associated with the first target predetermined word; and A second determination sub-module, configured to determine a second target candidate word from the second target predetermined words based on a second matching probability and the association degree.

16. The apparatus according to claim 15, wherein The second determination sub-module includes: A first determination unit, configured to determine a third target predetermined word from the second target predetermined words according to the association degree, where the association degree between the third target predetermined word and the first target predetermined word is greater than a second predetermined threshold; and A second determination unit, configured to determine the second target candidate word from the third target predetermined words based on the second matching probability.

17. The device according to claim 16, wherein, The second determination unit includes: A relevance calculation unit, configured to obtain the relevance between the second target predetermined word and the first target candidate word according to the second matching probability and the association degree; and A third determination sub-unit, configured to determine the second target candidate word from the second target predetermined words based on the relevance.

18. The apparatus according to claim 17, wherein, The relevance calculation unit includes: A weight calculation sub-unit, configured to determine a relevant weight according to the association degree; and A relevance calculation sub-unit, configured to obtain the relevance according to the second matching probability and the relevant weight.

19. The apparatus according to claim 14, wherein The first generation module includes: A fourth determination sub-module, configured to determine a target word from the target candidate words; and A first generation sub-module, configured to generate the reply text according to the target word.

20. The apparatus according to claim 19, wherein The fourth determination sub-module includes: A sorting unit, configured to sort the target candidate words based on the matching probability between the target candidate words and the problem text to obtain a sorting result; and A fourth determination unit, configured to determine the target word from the target candidate words based on the sorting result.

21. The apparatus according to claim 14, wherein the set of predetermined words includes a plurality of predetermined words and the degree of association between the plurality of predetermined words; The first generation module includes: A second generation sub-module, configured to generate multiple candidate reply texts according to the target candidate words; A matching degree calculation sub-module, configured to determine the matching degree between the multiple candidate reply texts and the problem text according to the association degree; and A fifth determination sub-module, configured to determine the reply text from the multiple candidate reply texts based on the matching degree.

22. The device according to claim 21, wherein The set of predetermined words is a trie composed of N layers of predetermined word nodes, where N is an integer greater than 1; The second generation sub-module includes: An order determination unit, configured to determine the arrangement order of the target candidate word in the candidate reply text according to the position of the target candidate word in the trie tree; and A first generation unit, configured to generate a plurality of candidate reply texts based on the arrangement order and the target candidate word.

23. The apparatus according to claim 22, wherein There are I target candidate words, where I is an integer greater than 1; The order determination unit includes: A first order determination subunit, configured to, in response to the i-th target candidate word and the j-th target candidate word being at the n-th layer in the trie tree, determine the n-th position of the i-th target candidate word in the first candidate reply text and the n-th position of the j-th target candidate word in the second candidate reply text, where n is an integer greater than or equal to 1 and less than or equal to N, and i, j are integers greater than or equal to 1 and less than or equal to I; and A second order determination subunit, configured to, in response to the i-th target candidate word being at the n-th layer in the trie tree and the j-th target candidate word being at the n+1-th layer in the trie tree, determine the n-th position of the i-th target candidate word in the first candidate reply text and the n+1-th position of the j-th target candidate word in the first candidate reply text.

24. The apparatus according to claim 21, wherein, The matching degree calculation sub-module includes: A first matching degree calculation unit, configured to determine the matching degree between the first candidate reply text and the question text according to the correlation degree between a plurality of target candidate words used to generate the first candidate reply text; and A second matching degree calculation unit, configured to determine the matching degree between the second candidate reply text and the question text according to the correlation degree between a plurality of target candidate words used to generate the second candidate reply text.

25. The apparatus according to claim 14, wherein, The first processing module includes: A feature extraction sub-module, configured to extract the text features of the question text; and An attention sub-module, configured to process the text features based on the attention mechanism to obtain the matching probabilities between a plurality of candidate words and the question text.

26. A training device for a text generation model, the text generation model including a processing module, a selection module and a generation module; including: A second processing module, configured to input a sample question text into the processing module to obtain sample matching probabilities between a plurality of sample candidate words used to generate a sample reply text and the sample question text, where the sample question text is used to describe the question information of a sample target scenario; the sample matching probability represents the matching probability between a word in the corpus during the pre-training process and the sample question text; A second selection module, configured to input the plurality of sample candidate words, a plurality of sample predetermined words in the sample predetermined word set, the sample correlation degree between the plurality of sample predetermined words and the sample matching probability into the selection module to obtain sample target candidate words; where the sample target candidate words represent sample candidate words that exist in the sample predetermined word set corresponding to the sample target scenario, the sample matching probability is greater than a first predetermined threshold, and the sample correlation degree is greater than a second predetermined threshold; A second generation module, configured to input the sample target candidate words into the generation module to obtain the sample reply text; A loss calculation module, configured to obtain a loss value based on a target loss function according to the sample response text and the label text corresponding to the sample question text; A first adjustment module, configured to adjust the parameters of the processing module and the parameters of the selection module based on the loss value, to obtain the correlation between the trained text generation model and multiple sample predetermined words.

27. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of claims 1-13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

29. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-13.

Images