A method, device, equipment and medium for generating casual replies based on dynamic prompts

Abstract

The application discloses a method and device for generating casual replies based on dynamic prompts, equipment and medium, to solve the problem that the existing prompts are the same for different queries each time, which cannot meet the requirements of the dialogue generation task for the semantic and grammatical coherence of the generated casual replies. The method comprises the following steps: determining the request and context in the training data and embedding the request and context through an embedding layer to obtain the corresponding expression result; obtaining the corresponding control layer by dynamically adjusting the neural network parameters of the prompt layer; inputting the expression result corresponding to the request into the GPT2 layer and the control layer, and inputting the expression result corresponding to the context into the GPT2 layer; determining a plurality of casual data to be replied corresponding to the expression result of the request generated by the GPT2 layer, and inputting the plurality of casual data to be replied and the context into the MMI layer to determine the target casual reply data corresponding to the training data, which meets the requirements of the context grammar and semantic coherence.

Description

Technical Field

[0001] This application relates to the field of intelligent robot technology, and in particular to a method, apparatus, device and medium for generating casual conversation responses based on dynamic prompts. Background Technology

[0002] Intelligent question answering has become an essential skill for intelligent robots. Small talk, as an important skill in intelligent question answering, often serves as a fallback function, making the user experience more user-friendly. Currently, common small talk models include follow-up search, such as the ElasticSearch framework. This requires storing a large amount of data in a database, then retrieving candidate answers based on syntactic and semantic information, finding similar questions, and returning the corresponding answers.

[0003] Another approach is through generative methods, such as the autoregressive neural network model GPT. Users can use a pre-trained GPT model and then fine-tune it for the target task. Prompt techniques can address the drawbacks of fine-tuning. By designing a prompt for each subtask, for example, in a sentiment analysis classification task to determine whether a sentence is positive or pessimistic, we can say, "The weather is nice today, and I'm in a good mood [Mask]." "I'm in a good mood [Mask]" is a prompt. At the Mask position, we predict two categories: "good" and "bad."

[0004] However, existing prompts are not refined enough for generative chatter. For different queries, the prompts are the same each time, which cannot meet the requirements of dialogue generation tasks for semantic and syntactic coherence and non-boring chatter responses. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for generating casual conversation responses based on dynamic prompts, in order to solve the technical problem that existing prompts provide the same prompt for each query, which fails to meet the requirements of dialogue generation tasks for semantic and syntactic coherence and non-boring casual conversation responses.

[0006] On the one hand, embodiments of this application provide a method for generating casual replies based on a dynamic Prompt, including:

[0007] The requests and contexts in the training data are identified, and the requests and contexts are embedded through an embedding layer to obtain the corresponding expression results;

[0008] By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained;

[0009] The expression result corresponding to the request is input to the GPT2 layer and the control layer respectively, and the expression result corresponding to the context is input to the GPT2 layer;

[0010] A number of chatter data points corresponding to the expression result of the request generated by the GPT2 layer are identified, and the number of chatter data points and the context are input into the MMI layer to determine the target chatter response data corresponding to the training data.

[0011] In one implementation of this application, after inputting the expression result corresponding to the request into the GPT2 layer and the control layer respectively, and inputting the expression result corresponding to the context into the GPT2 layer, the method further includes:

[0012] The control layer is used to pre-train the expression results corresponding to the request and the expression results corresponding to the context.

[0013] Predict the words covered in the expression results, and predict whether the expression results corresponding to the request and the expression results corresponding to the context are adjacent, so as to complete the pre-training of the expression results.

[0014] In one implementation of this application, after predicting the covered words in the expression result and predicting whether the expression result corresponding to the request and the expression result corresponding to the context are adjacent, to complete the pre-training of the expression result, the method further includes:

[0015] Based on dynamic Prompt, all downstream tasks in the expression results are converted into corresponding pre-training tasks, and based on the corresponding pre-training tasks, the words covered in the expression results are predicted.

[0016] For different training data, a dynamic prompt is generated for each training data, so as to provide prompts for the corresponding training data through the dynamic prompt.

[0017] In one implementation of this application, the step of pre-training the expression result corresponding to the request and the expression result corresponding to the context through the control layer specifically includes:

[0018] The multi-head self-attention layer in the control layer segments the expression result corresponding to the request into words, and encodes the corresponding token for the segmented expression result of the request into words.

[0019] The multidimensional vector of the token is obtained through the embedding layer, and self-attention processing is performed on the multidimensional vector of the token and the control layer to achieve pre-training of the expression result.

[0020] In one implementation of this application, after inputting the plurality of chatter data to be answered and the context into the MMI layer, the method further includes:

[0021] Generate the chat data to be replied to based on the context of the training data, and calculate the corresponding first generation probability;

[0022] Based on the preset MMI algorithm, the several pieces of chat data to be replied to are trained respectively to obtain the context corresponding to the chat data to be replied to;

[0023] Calculate the second probability of generating the corresponding context for each of the plurality of chat data to be replied to, and combine the first generation probability with the second generation probability to determine the comprehensive probability corresponding to the chat data to be replied to.

[0024] In one implementation of this application, determining the target chat response data corresponding to the training data specifically includes:

[0025] The several pieces of chat data to be answered are sorted in descending order of their comprehensive probabilities.

[0026] The chat data with the highest overall probability among the several chat data to be answered is identified as the target chat response data corresponding to the training data.

[0027] In one implementation of this application, the method further includes:

[0028] When pre-training the training data, the addiction vector corresponding to the CLS portion of the training data is determined;

[0029] The addiction vector is input into a preset classification function, and the addiction vector is classified by the preset classification function to obtain the corresponding classification result;

[0030] Based on the classification results, the type of training data corresponding to the addiction vector is determined.

[0031] On the other hand, embodiments of this application also provide an apparatus for generating casual replies based on dynamic prompts, the apparatus comprising:

[0032] The acquisition unit is used to determine the request and context in the training data, and to embed the request and context into an embedded expression through the embedding layer to obtain the corresponding expression result;

[0033] The adjustment unit is used to dynamically adjust the neural network parameters of the Prompt layer to obtain the corresponding control layer;

[0034] The input unit is used to input the expression result corresponding to the request to the GPT2 layer and the control layer respectively, and to input the expression result corresponding to the context to the GPT2 layer;

[0035] The determining unit is used to determine a number of chatter data to be answered corresponding to the expression result of the request generated by the GPT2 layer, and input the number of chatter data to be answered and the context into the MMI layer to determine the target chatter response data corresponding to the training data.

[0036] On the other hand, embodiments of this application also provide a device for generating casual conversation responses based on dynamic Prompt, the device comprising:

[0037] At least one processor;

[0038] And, a memory communicatively connected to the at least one processor;

[0039] The memory stores instructions that can be executed by the at least one processor, which enable the at least one processor to perform a method for generating casual replies based on dynamic Prompt as described above.

[0040] On the other hand, embodiments of this application also provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0041] The above is one method for generating casual replies based on dynamic prompts.

[0042] This application provides a method, apparatus, device, and medium for generating casual conversation responses based on dynamic Prompt, which includes at least the following beneficial effects:

[0043] By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained. By inputting the requests in the training data into the GPT2 layer and then into the control layer, and by inputting the context into the GPT2 layer, several pieces of chatter data to be answered can be generated based on the GPT2 layer. By inputting several pieces of chatter data to be answered and the context into the MMI layer, the target chatter response data that conforms to the context syntax and semantic coherence can be determined from several pieces of chatter data to be answered, thus avoiding the generation of repetitive and meaningless statements. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0045] Figure 1 A flowchart illustrating a method for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application;

[0046] Figure 2 A flowchart illustrating another method for generating casual conversation responses based on dynamic prompts, provided in an embodiment of this application;

[0047] Figure 3 A schematic diagram of a device for generating casual conversation responses based on dynamic prompts, provided for an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the internal structure of a device for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] The development of NLP can be divided into four stages. The first stage is based on statistical models, such as the N-Gram model. The second stage is based on neural network models. The third stage is based on the pre-training-fine-tuning model. The fourth stage is based on Prompt.

[0051] Currently, common models for casual conversation include continuous search, such as the Elasticsearch framework. This requires storing a large amount of data in a database, then using syntactic and semantic information to retrieve candidate answers, find similar questions, and return the corresponding answers. Another approach is through generative methods, such as the autoregressive neural network model GPT. Users can use a pre-trained GPT model and then fine-tune it for the target task.

[0052] However, this fine-tuning mode has the following drawbacks: the pre-training task and the downstream task are generally two different tasks. For example, in the Mask language model, during the pre-training process, we select words from the Mask and then predict the words in the Mask. If this pre-trained model is fine-tuned for a classification task, the user simply feeds the output of the hidden layer into the classifier and then fine-tunes it, which leads to the problem of inconsistent training objectives.

[0053] Existing technologies utilize Prompt techniques to address the drawbacks of fine-tuning. By designing a Prompt for each subtask, for example, in a sentiment analysis classification task to determine whether a sentence is positive or pessimistic, we can do it like this: "The weather is nice today, and I'm in a good mood [Mask]." "I'm in a good mood [Mask]" is a Prompt. We predict two categories at the Mask position: "good" and "bad."

[0054] However, existing prompts are not refined enough for generative chatter. Existing prompts are the same for every query, failing to meet the semantic and syntactic coherence and engaging nature required for dialogue generation tasks.

[0055] This application provides a method, apparatus, device, and medium for generating casual conversation responses based on dynamic Prompt. By dynamically adjusting the neural network parameters of the Prompt layer, a corresponding control layer is obtained. By inputting requests from the training data into the GPT2 layer and then into the control layer, and by inputting the context into the GPT2 layer, several pieces of casual conversation data corresponding to the requests can be generated based on the GPT2 layer. By inputting the several pieces of casual conversation data to be answered and the context into the MMI layer, target casual conversation response data that conforms to the context's syntax and semantic coherence is determined from the several pieces of casual conversation data to be answered, thus avoiding the generation of repetitive and meaningless statements. This solves the technical problem in the prior art where the Prompt prompt is the same for different queries, failing to meet the requirements of semantic and syntactic coherence and avoiding monotony in dialogue generation tasks.

[0056] Figure 1 This is a flowchart illustrating a method for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application. Figure 1 As shown in the figure, a method for generating casual replies based on dynamic prompts provided in this application embodiment may mainly include the following steps:

[0057] 101. Identify the requests and contexts in the training data, and use an embedding layer to embed the requests and contexts to obtain the corresponding representation results.

[0058] When implementing an autoregressive generative chat algorithm based on dynamic Prompt, GPT2, and MMI models, the server first needs to determine the query and context in the training data. Then, the determined query and context are input into the embedding layer. The embedding layer performs embedded representation of the query and context, thereby obtaining the expression results corresponding to the query and context respectively.

[0059] 102. By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained.

[0060] The server can obtain the corresponding control layer by dynamically adjusting the neural network parameters of the Prompt layer, that is, the parameters in the Transformer Encoder multi-head self-attention layer of the feature extractor.

[0061] 103. Input the expression results corresponding to the request into the GPT2 layer and the control layer respectively, and input the expression results corresponding to the context into the GPT2 layer.

[0062] The server inputs the expression results corresponding to the obtained request query into the GPT2 layer and the controller layer respectively. In addition, it also inputs the expression results corresponding to the context into the GPT2 layer so that the GPT2 layer can perform subsequent operations based on the expression results corresponding to the input request query and the expression results corresponding to the context.

[0063] In one embodiment of this application, before the server inputs the expression result corresponding to the request to the GPT2 layer and the control layer respectively, and inputs the expression result corresponding to the context to the GPT2 layer, the control layer pre-trains the expression result corresponding to the request query and the expression result corresponding to the context, thereby predicting the covered words in the expression result corresponding to the request query and the expression result corresponding to the context, and predicting whether the expression result corresponding to the query and the expression result corresponding to the context are adjacent, thus completing the pre-training of the expression result.

[0064] Specifically, when the server pre-trains the expression results corresponding to the request query and the expression results corresponding to the context through the control layer, it first segments the expression results corresponding to the request query into words through the multi-head self-attention layer in the control layer, and encodes the token corresponding to the segmented expression results of the request query. Then, it obtains the multi-dimensional vector of the token through the embedding layer, and performs self-attention processing on the multi-dimensional vector of the token and the control layer, thereby realizing the pre-training of the expression results.

[0065] In one embodiment of this application, after predicting the covered words in the expression result and predicting whether the expression result corresponding to the request and the expression result corresponding to the context are adjacent, the server completes the pre-training of the expression result. Then, based on a dynamic prompt, all downstream tasks in the expression result are converted into corresponding pre-training tasks. It should be noted that the downstream task in this embodiment is, for example, a sentiment analysis task. The server predicts the covered words in the expression result based on the corresponding pre-training task, and then generates dynamic prompts corresponding to different training data, thereby providing prompts for the corresponding training data through the dynamic prompts. The server transforms the sentiment analysis task into predicting masked words, for example, [CLS] the weather is nice today, [Sep] I'm in a bad mood, [Mask] [Sep], [Mask] where is the location? We can use good and bad, thus solving the target difference between the fine-tuning task and the pre-training task. A dynamic prompt, for example, "I'm in a bad mood," is a prompt. A prompt has the meaning of prompting. Each sample in the sentiment analysis task is fixed, and different prompts need to be generated for different user questions.

[0066] In one embodiment of this application, when the server pre-trains the training data, it needs to determine the addiction vector corresponding to the CLS part of the training data, then input the determined addiction vector into a preset classification function, and classify the addiction vector through the preset classification function to obtain the corresponding classification result. Thus, the type of training data corresponding to the addiction vector can be determined based on the obtained classification result.

[0067] At this point, the sentiment analysis task and the pre-training task are different, so the parameters trained by the pre-training task cannot be seamlessly integrated into the sentiment analysis task. Therefore, a large amount of sentiment analysis data is needed to fine-tune the parameters.

[0068] 104. Determine several unanswered chatter data corresponding to the expression results of the requests generated by the GPT2 layer, and input the several unanswered chatter data and context into the MMI layer to determine the target chatter response data corresponding to the training data.

[0069] The server can determine the expression result of the GPT2 layer corresponding to the query request, and generate several chat data to be answered. Then, the chat data to be answered generated by the GPT2 layer and the context input in the GPT2 layer are input into the MMI layer. The MMI layer processes the chat data to be answered and the context to determine the target chat response data corresponding to the training data, so as to realize the function of automatically generating chat responses based on dynamic prompts.

[0070] In one embodiment of this application, after inputting several pieces of chatter data to be replied to and context into the MMI layer, the server generates chatter data to be replied to corresponding to the training data based on the context of the training data, and calculates a first generation probability corresponding to the chatter data to be replied to generated from the context. Simultaneously, the server also trains on several pieces of chatter data to be replied to based on a preset MMI algorithm, thereby obtaining the context corresponding to the chatter data to be replied to. Then, it calculates a second probability for each piece of chatter data to be replied to generate the corresponding context, and combines the first generation probability with the second generation probability to determine the comprehensive probability corresponding to the chatter data to be replied to.

[0071] In one embodiment of this application, when the server determines the target chat response data corresponding to the training data, it sorts the chat response data to be answered in descending order of the comprehensive probability corresponding to the chat response data to be answered, and then determines the chat response data with the highest comprehensive probability among the chat response data to be answered from the sorted queue, and uses it as the target chat response data corresponding to the training data.

[0072] Figure 2 This is a flowchart illustrating another method for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application. For example... Figure 2 As shown, the server inputs the request query and context from the training data into the embedding layer, respectively. The embedding layer performs embedded representation of the request query and context, thereby obtaining the representation results corresponding to the request query and context. Then, the representation results corresponding to the request query are input into the GPT2 layer and the controller layer, respectively, and the representation results corresponding to the context are input into the GPT2 layer. The GPT2 layer generates several pending chatter data corresponding to the request, and then inputs these pending chatter data into the MMI layer to determine the target chatter response.

[0073] It should be noted that, Figure 2 The method shown is the same as Figure 1 The methods shown are essentially the same, therefore, Figure 2 For details not covered in the text, please refer to [the relevant source]. Figure 1 The relevant descriptions in the document will not be repeated here.

[0074] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide an apparatus for generating casual conversation responses based on dynamic prompts, the structure of which is as follows: Figure 3 As shown.

[0075] Figure 3 This is a schematic diagram of a device for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application. Figure 3 As shown, the device includes:

[0076] The acquisition unit is used to determine the request and context in the training data, and to embed the request and context into an embedded representation through the embedding layer to obtain the corresponding representation result;

[0077] The adjustment unit is used to dynamically adjust the neural network parameters of the Prompt layer to obtain the corresponding control layer;

[0078] The input unit is used to input the expression result corresponding to the request into the GPT2 layer and the control layer respectively, and to input the expression result corresponding to the context into the GPT2 layer;

[0079] The determination unit is used to determine several unanswered chatter data corresponding to the expression result of the request generated by the GPT2 layer, and inputs several unanswered chatter data and context into the MMI layer to determine the target chatter response data corresponding to the training data.

[0080] Figure 4 This is a schematic diagram of the internal structure of a device for generating casual conversation responses based on dynamic prompts, provided as an embodiment of this application. Figure 4 As shown, the device includes:

[0081] At least one processor;

[0082] And, a memory that is communicatively connected to at least one processor;

[0083] The memory stores instructions that can be executed by at least one processor, and the instructions, when executed by at least one processor, enable at least one processor to:

[0084] The requests and contexts in the training data are identified, and the requests and contexts are embedded through an embedding layer to obtain the corresponding expression results;

[0085] By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained;

[0086] The expression results corresponding to the request are input into the GPT2 layer and the control layer respectively, and the expression results corresponding to the context are input into the GPT2 layer;

[0087] The expression results of the requests generated by the GPT2 layer are identified, and several pieces of chatter data to be answered are input into the MMI layer along with the context to determine the target chatter response data corresponding to the training data.

[0088] This application embodiment also provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0089] The requests and contexts in the training data are identified, and the requests and contexts are embedded through an embedding layer to obtain the corresponding expression results;

[0090] By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained;

[0091] The expression results corresponding to the request are input into the GPT2 layer and the control layer respectively, and the expression results corresponding to the context are input into the GPT2 layer;

[0092] The expression results of the requests generated by the GPT2 layer are identified, and several pieces of chatter data to be answered are input into the MMI layer along with the context to determine the target chatter response data corresponding to the training data.

[0093] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0094] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0095] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0096] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0097] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0099] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0100] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0101] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0102] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0103] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0104] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0106] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0107] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0108] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for generating casual replies based on dynamic Prompt, characterized in that, The method includes: The requests and contexts in the training data are identified, and the requests and contexts are embedded through an embedding layer to obtain the corresponding expression results; By dynamically adjusting the neural network parameters of the Prompt layer, the corresponding control layer is obtained; The expression result corresponding to the request is input to the GPT2 layer and the control layer respectively, the expression result corresponding to the context is input to the GPT2 layer, and the output result of the control layer is input to the GPT2 layer; A number of chatter data points corresponding to the expression results of the request generated by the GPT2 layer are determined, and the number of chatter data points and the context are input into the MMI layer to determine the target chatter response data corresponding to the training data. After inputting the expression result corresponding to the request into the GPT2 layer and the control layer respectively, and inputting the expression result corresponding to the context into the GPT2 layer, the method further includes: The control layer is used to pre-train the expression results corresponding to the request and the expression results corresponding to the context. Predict the words covered in the expression results, and predict whether the expression results corresponding to the request and the expression results corresponding to the context are adjacent, so as to complete the pre-training of the expression results; After predicting the covered words in the expression results and predicting whether the expression results corresponding to the request and the expression results corresponding to the context are adjacent, in order to complete the pre-training of the expression results, the method further includes: Based on dynamic Prompt, all downstream tasks in the expression results are converted into corresponding pre-training tasks, and based on the corresponding pre-training tasks, the words covered in the expression results are predicted. For different training data, a dynamic prompt is generated for each training data, so as to provide prompts for the corresponding training data through the dynamic prompt.

2. The method for generating casual replies based on dynamic Prompt according to claim 1, characterized in that, The pre-training of the expression results corresponding to the request and the expression results corresponding to the context through the control layer specifically includes: The multi-head self-attention layer in the control layer segments the expression result corresponding to the request into words, and encodes the corresponding token for the segmented expression result of the request into words. The multidimensional vector of the token is obtained through the embedding layer, and self-attention processing is performed on the multidimensional vector of the token and the control layer to achieve pre-training of the expression result.

3. The method for generating casual replies based on dynamic Prompt according to claim 1, characterized in that, After inputting the plurality of idle chat data to be answered and the context into the MMI layer, the method further includes: Generate the chat data to be replied to based on the context of the training data, and calculate the corresponding first generation probability; Based on the preset MMI algorithm, the several pieces of chat data to be replied to are trained respectively to obtain the context corresponding to the chat data to be replied to; Calculate the second generation probability of the corresponding context for each of the plurality of chat data to be replied to, and combine the first generation probability with the second generation probability to determine the comprehensive probability corresponding to the chat data to be replied to.

4. The method for generating casual replies based on dynamic Prompt according to claim 3, characterized in that, The determination of the target casual conversation response data corresponding to the training data specifically includes: The several pieces of chat data to be answered are sorted in descending order of their comprehensive probabilities. The chat data with the highest overall probability among the several chat data to be answered is identified as the target chat response data corresponding to the training data.

5. The method for generating casual replies based on dynamic Prompt according to claim 1, characterized in that, The method further includes: During pre-training of the training data, the latent vectors corresponding to the CLS portion of the training data are determined; The latent vector is input into a preset classification function, and the latent vector is classified by the preset classification function to obtain the corresponding classification result; Based on the classification results, the type of training data corresponding to the latent vector is determined.

6. A device for generating casual replies based on dynamic prompts, characterized in that, The device includes: The acquisition unit is used to determine the request and context in the training data, and to embed the request and context into an embedded expression through the embedding layer to obtain the corresponding expression result; The adjustment unit is used to dynamically adjust the neural network parameters of the Prompt layer to obtain the corresponding control layer; The input unit is used to input the expression result corresponding to the request to the GPT2 layer and the control layer respectively, input the expression result corresponding to the context to the GPT2 layer, and input the output result of the control layer to the GPT2 layer; After inputting the expression result corresponding to the request into the GPT2 layer and the control layer respectively, and inputting the expression result corresponding to the context into the GPT2 layer, the method further includes: The control layer is used to pre-train the expression results corresponding to the request and the expression results corresponding to the context. Predict the words covered in the expression results, and predict whether the expression results corresponding to the request and the expression results corresponding to the context are adjacent, so as to complete the pre-training of the expression results; After predicting the covered words in the expression results and predicting whether the expression results corresponding to the request and the expression results corresponding to the context are adjacent, to complete the pre-training of the expression results, the method further includes: Based on dynamic Prompt, all downstream tasks in the expression results are converted into corresponding pre-training tasks, and based on the corresponding pre-training tasks, the words covered in the expression results are predicted. For different training data, a dynamic Prompt is generated for each training data, so as to provide prompts for the corresponding training data through the dynamic Prompt; The determining unit is used to determine a number of chatter data to be answered corresponding to the expression result of the request generated by the GPT2 layer, and input the number of chatter data to be answered and the context into the MMI layer to determine the target chatter response data corresponding to the training data.

7. A device for generating casual conversation responses based on dynamic prompts, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform a method for generating casual replies based on dynamic Prompt as described in any one of claims 1-5.

8. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are set as follows: A method for generating casual conversation responses based on dynamic prompts as described in any one of claims 1-5.

Images