A human-computer conversation scene construction system based on a generative language model

Abstract

The application discloses a kind of human-computer conversation scene construction systems based on generative language model, belong to human-computer interaction technical field.The application includes scene preset module, script generation module, user intent processing module, routing logic processing module, data generation and processing module and knowledge base node processing module;Above module generates the script of machine node according to user preset scene description information, judges whether there is similarity user intent according to user reply content, and carries out merging or generates new user intent, determines machine content and user content in next interaction according to preset information and language model, and forms routing chain, generates machine content based on generative language model, judges whether there is similarity intent, and forms the routing chain of machine node and user intent, triggers discrete knowledge point output node content, and realizes process memory regression, to build out general closed human-computer conversation scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction, and particularly to a human-computer dialogue scene construction system based on a generative language model. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, human-computer interaction has become an increasingly important research field. In human-computer interaction, dialogue systems are an important research direction. Dialogue systems are computer programs that can conduct human-computer dialogue. Through dialogue systems, users can interact with computers in natural language, thereby realizing various practical functions.

[0003] However, traditional dialogue systems often require a lot of human intervention and manual design, making it difficult to meet the needs of practical applications. Especially for complex dialogue scenarios, traditional dialogue systems are difficult to handle. Therefore, how to realize intelligent dialogue scene construction and automatic dialogue content generation has become a research hotspot in the field of human-computer interaction. SUMMARY

[0004] The present application relates to the technical field of human-computer interaction, and particularly to a human-computer dialogue scene construction system based on a generative language model.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] A human-computer dialogue scene construction system based on a generative language model, the system comprising a scene preset module, a script generation module, a user intent generation module, a data generation and processing module, a routing logic processing module, and a cycle processing and knowledge base management module;

[0007] The scene preset module receives and defines the overall background and target of the dialogue scene, and records it as preset information;

[0008] The script generation module obtains machine start node information and scripts from the preset information of the scene preset module, the scripts being machine content; predicts other node information in the scene through an NLP semantic model, judges whether there is similarity in the machine node according to the confidence score, and performs node script information recursion or generates a new machine node;

[0009] The user intent generation module requests a generative pre-training language model to obtain user reply content according to the preset information of the scene preset module and the machine content of the script generation module, the user reply content being user content; judges whether there is similarity in the user intent through an NLP semantic model, and performs merging or generates a new user intent;

[0010] The data generation and processing module is configured to create and define the user intention under the machine node of the script generation module;

[0011] The routing logic processing module is configured to generate the machine content and the user content of the script generation module according to the preset information of the scene preset module and the user intention generation module, request the generative pre-training language model to obtain the machine content or the user content in the next interaction, determine whether there is a similar machine node, and finally form a routing chain of the machine node and the user intention;

[0012] The cycle processing and knowledge base management module is configured to perform cycle processing according to the data generation and processing process provided by the scene preset module, the script generation module, the user intention generation module, the data generation and processing module, and the routing logic processing module, repeatedly generate and process data without limitation, and complete scene generation; and end the dialogue process by defining an end node.

[0013] The output end of the scene preset module is connected with the input end of the script generation module; the output end of the script generation module is connected with the input end of the user intention generation module; the output end of the user intention generation module is connected with the input end of the data generation and processing module; the output end of the data generation and processing module is connected with the input end of the routing logic processing module; and the output end of the routing logic processing module is connected with the input end of the cycle processing and knowledge base management module.

[0014] Further, the script generation module comprises a node information acquisition unit, a similarity node judgment unit, and a node generation and merging unit.

[0015] The node information acquisition unit is configured to obtain the machine start node information and the machine content based on the preset information of the scene preset module; input the machine start node information into the NLP semantic model of the similarity node judgment unit; predict other node information in the scene through the machine start node information; determine whether there is a similar machine node; if there is no similar node, generate a machine node; if there is a similar node, recursively input the node script information to the next sequence of the similar node script, and input the result to the node generation and merging unit; and the node script information recursion is to mark the script to the next sequence of the corresponding node.

[0016] Further, the similarity node judgment and script generation unit comprises:

[0017] The machine start node information is input into the NLP semantic model, other node information in the scene is predicted through the node information, and whether there is a similar machine node is judged according to a confidence score; wherein the confidence is calculated through a softmax function, in the NLP semantic model, a score of each category is calculated through forward propagation given an input sample, and then the softmax function is used to convert them into probabilities; the formula of the softmax function is as follows:

[0018]

[0019] Wherein, j represents data number, and the value is 1 to n, The score of the i-th category is represented, and n represents the total number of categories; the softmax function converts the score into a probability, so that the sum of all probabilities is 1;

[0020] According to the confidence returned by the sample prediction, a business threshold A is set, if the confidence is less than the business threshold A, there is no similar node, and a machine node A is generated; if the confidence is greater than or equal to the business threshold A, there is a similar node, and the script information of the node is recursively returned to the next sequence of the similar node script.

[0021] Further, the forward propagation calculation includes:

[0022] First, the input sample is represented as a vector X = [x1, x2, …, xn], wherein n is the number of input features, x1 represents the first sample feature, x2 represents the second sample feature, and so on x n n represents the n-th sample feature; n

[0023] For each layer of neural network, from the input layer to the output layer, the weighted sum is calculated, and the calculation formula is:

[0024]

[0025] Wherein, l is the number of layers of neural network, The number of neurons in the l-1 layer is represented, i represents the index of each neuron in the l-1 layer, and the range is from 1 to , The weight of connecting the i-th neuron of the l-1 layer and the j-th neuron of the l layer is represented, The output result of the i-th neuron in the l-1 layer is represented, The bias term of the j-th neuron in the l layer is represented, The weighted sum of the j-th neuron in the l layer is represented;

[0026] According to the formula: ), the output results of all neurons in each layer are calculated, wherein​ ) is the activation function, and the weighted sum The input is processed to obtain the output of the jth neuron in the lth layer ;

[0027] The final output of the neural network is the output of the last layer: ,in represents the output of the first neuron in the Lth layer, Represents the output of the second neuron in the Lth layer, and so on Represents the output of the kth neuron in the Lth layer, where L is the index of the output layer and k is the number of output categories; finally, the final output of the neural network is used as the score of each category .

[0028] Furthermore, the user intention generation module includes a user reply generation unit, a user intention similarity judgment unit, and a user intention generation and merging unit;

[0029] The user reply generation unit obtains the user reply content based on the preset information of the scene preset module and the machine content request generative pre-trained language model prediction of the script generation module; the user intention similarity judgment unit inputs the user content into the NLP semantic model to determine whether there is similar user intention; if there is no similar intention, the user intention is generated, otherwise it is merged into the similarity intention, and the judgment result is input to the user intention generation and merging unit.

[0030] Furthermore, the user intention similarity judgment unit includes:

[0031] The user's reply content is input into the NLP semantic model, and the confidence score is used to determine whether there is similar intent. The confidence score is calculated using the softmax function. In the NLP semantic model, given an input sample, the score of each category is calculated through forward propagation, and then the softmax function is used to convert them into probabilities.

[0032] Based on the confidence level returned by the sample prediction, a business threshold B is set. If the confidence level is less than the business threshold B, there is no similarity intent and user intent is generated. If the confidence level is greater than or equal to the business threshold B, it is merged into the similarity intent and the judgment result is input into the user intent generation and merging unit.

[0033] Furthermore, the routing logic processing module includes a routing chain forming unit 1 and a routing chain forming unit 2:

[0034] The routing chain forms a unit. First, the script generation module generates machine content A1 and machine node A, and the user intention generation module generates user content A and user intention A;

[0035] The creation and definition of the user intention A under the machine node A are completed by the data generation and processing module;

[0036] The preset information, the machine content A1 and the user content A information request are generated into the pre-training language model by the script generation module, and the machine content A2 is obtained;

[0037] The machine content A2 is input into the NLP semantic model, other node information in the scene is predicted, and whether there is a similar machine node is judged according to the confidence score;

[0038] In this process, if there is a similar node, the node script information is recursively assigned to the next sequence of the similar node script; otherwise, the steps of the node generation and merging unit of the script generation module are executed. In this case, there is a similar node, that is, the similarity of the machine content A2 and the machine content A1 is judged. If the machine content A2 and the machine content A1 are similar, the similar machine content A2 is recursively assigned to the node information of the similar machine node A;

[0039] The machine node A→user intention A→machine node A routing chain is formed;

[0040] The preset information, the machine content A1 information, the user content A information and the machine content A2 information request are generated into the pre-training language model by the user intention generation module to obtain the user content B. The user content B is input into the NLP semantic model to judge whether there is a similar user intention. If there is no similar intention, the user intention B is generated, otherwise the similar intention is merged;

[0041] The creation and definition of the user intention B under the machine node A are completed by the data generation and processing module;

[0042] The preset information, the machine content A1 information, the user content A information, the machine content A2 and the user content B request are generated into the pre-training language model by the script generation module to obtain the machine content B1;

[0043] The machine content B1 is input into the NLP semantic model to judge whether there is a similar machine node;

[0044] If there is no similar node, the machine node B is generated, otherwise the script sequence under the same node is merged;

[0045] The machine node A→user intention B→machine node B routing chain is formed.

[0046] Further, the loop processing and knowledge base management module includes a loop processing unit, an end node management unit and a discrete knowledge base node unit;

[0047] The circulation processing unit performs circulation processing according to the data generation and processing procedure provided by the scene preset module, the script generation module, the user intention generation module, the data generation and processing module, and the routing logic processing module, repeatedly generates a scene and performs dialogue interaction without limitation, and achieves the purpose of scene generation; the end node management unit defines any one non-start node as an end node, and in the interaction process, entering the end node means the end of the dialogue process;

[0048] The discrete knowledge base node unit defines any one non-start and end node as a discrete knowledge base node, which is placed outside the flow chain and triggered by user reply semantics to output node content; after the node content is output, the next round of dialogue defaults to the flow node left last time, completing the knowledge base node regression flow memory.

[0049] The basic process of the generative pre-training language model prediction is as follows:

[0050] The input text sequence is represented as a word embedding vector; for each time step t, and starting from the first word of the sequence; the input text sequence can be preset information and machine content, or preset information, machine content and user content;

[0051] Input word embedding vector , represents the input word embedding vector at time step t;

[0052] The vector representation of the text sequence is input into the neural network for processing, and the calculation formula is:

[0053]

[0054] where represents the hidden state at time step t, which carries the cumulative information of the previous sequence; W is the weight matrix input into the neural network, used to weight and sum the input word embedding vector and the previous hidden state; U represents the cycle weight matrix in the neural network, used to calculate the previous hidden state and the current hidden state; b represents the bias term, corresponding to the bias term in the hidden state update formula; f represents the activation function;

[0055] According to the formula , the output layer of the neural network is calculated, where V and c are the weight matrix and bias term of the output layer, g is the softmax function, which is used to convert the output into a probability distribution;

[0056] According to the probability distribution of , the word with the highest probability is selected as the next generated word, and it is added to the generated text sequence. The generated text sequence can be machine content or user content.

[0057] Compared with the prior art, the present application has the beneficial effects that: a large amount of text data can be classified and grouped, the similarities and differences between them are found out by analyzing and comparing the existing text data, and then similar text data is classified into a category to form a cluster, so that the large amount of text data can be quickly classified, and the efficiency and accuracy of the dialogue scene building are improved; the machine nodes with similar semantics are merged to reduce the repeated nodes in the dialogue scene, and due to the reduction of the number of nodes, the number of routing chains is also reduced, thereby avoiding the problem of topic quantity redundancy in open dialogue, and improving the efficiency and accuracy of the dialogue scene; the reply intentions of people with similar semantics are merged or screened, so that the number of intentions contained in the final node logical routing is less, and the problem of high complexity of dialogue scene, low efficiency and low accuracy caused by too many intentions is avoided; in the human-computer dialogue scene, according to the input of the user and the scene preset, the dialogue logic can be guided to the correct node for processing, and according to the input of the user and the scene preset, the next dialogue logic can be automatically judged, and the dialogue can be guided to the correct node for processing, in the scene, the role of the node logical routing is to ensure the correctness and accuracy of the dialogue logic, and avoid the interruption or error of the dialogue process caused by the user input error or the imperfect scene preset; in a specific scene, the user can only choose the preset topic for dialogue, and the dialogue process and logic have been preset, and the user only needs to follow the preset process to dialogue, if the user proposes a topic that does not conform to the preset logic, the dialogue direction of the user can be guided and prompted to adjust, so that the dialogue returns to the preset logic, the efficiency and accuracy of the dialogue can be improved, the user can be prevented from proposing irrelevant topics or going off topic, so that time is saved and user experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0058] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application together with the embodiments of the application, and do not constitute a limitation of the application. In the drawings:

[0059] Figure 1 is a module schematic diagram of a human-computer dialogue scene construction system based on a generative language model of the present application;

[0060] Figure 2 is a flowchart of a human-computer dialogue scene construction system based on a generative language model of the present application. DETAILED DESCRIPTION

[0061] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0062] Please refer to Figure 1 The present application provides technical solutions:

[0063] A human-computer dialogue scene construction system based on a generative language model, the system comprising a scene preset module, a script generation module, a user intention generation module, a data generation and processing module, a routing logic processing module, and a cycle processing and knowledge base management module;

[0064] The scene preset module receives and defines the overall background and target of the dialogue scene, and records it as preset information.

[0065] The script generation module obtains machine starting node information and scripts from the preset information of the scene preset module, the scripts being machine content; predicts other node information in the scene through an NLP semantic model for the machine content, judges whether there is similar machine node according to a confidence score, and performs node script information recursion or generates a new machine node.

[0066] The user intention generation module requests a generative pre-training language model to obtain user reply content from the preset information of the scene preset module and the machine content of the script generation module, the user reply content being user content; judges whether there is similar user intention for the user content through an NLP semantic model, and performs merging or generates a new user intention.

[0067] The data generation and processing module is used to create and define user intentions under machine nodes of the script generation module.

[0068] The routing logic processing module requests a generative pre-training language model to obtain machine content or user content in the next interaction according to the preset information of the scene preset module, the machine content of the script generation module, and the user content of the user intention generation module, judges whether there is similar machine node, and finally forms a routing chain of machine nodes and user intentions.

[0069] The cycle processing and knowledge base management module performs cycle processing according to the data generation and processing process provided by the scene preset module, the script generation module, the user intention generation module, the data generation and processing module, and the routing logic processing module, repeatedly generates and processes data without limitation, and completes scene generation; and then defines an end node to end the dialogue process.

[0070] The output end of the scene preset module is connected with the input end of the script generation module; the output end of the script generation module is connected with the input end of the user intention generation module; the output end of the user intention generation module is connected with the input end of the data generation and processing module; the output end of the data generation and processing module is connected with the input end of the routing logic processing module; the output end of the routing logic processing module is connected with the input end of the cycle processing and knowledge base management module.

[0071] The script generation module comprises a node information acquisition unit, a similarity node judgment unit and a node generation and merging unit.

[0072] The node information acquisition unit obtains the machine start node information and the machine content based on the preset information of the scene preset module; inputs the machine start node information into the NLP semantic model of the similarity node judgment unit, predicts other node information in the scene through the machine start node information, judges whether there is a similar machine node, generates a machine node if there is no similar node, and if there is a similar node, recursively inputs the node script information to the next sequence of the similar node script and inputs the result to the node generation and merging unit; the node script information recursion is to mark the script to the next sequence of the corresponding node.

[0073] The similarity node judgment and script generation unit comprises:

[0074] The machine start node information is input into the NLP semantic model, other node information in the scene is predicted through the node information, and whether there is a similar machine node is judged according to the confidence score; wherein the confidence is calculated by the softmax function, in the NLP semantic model, a given input sample is calculated by forward propagation to obtain the score of each category, and then the softmax function is used to convert them into probabilities; the formula of the softmax function is as follows:

[0075]

[0076] Wherein, j represents the data number, taking the value of 1 to n, The score of the i-th category, n represents the total number of categories; the softmax function converts the score into probability, so that the sum of all probabilities is 1;

[0077] According to the confidence returned by the sample prediction, a business threshold A is set, if the confidence is less than the business threshold A, there is no similar node, and a machine node A is generated; if the confidence is greater than or equal to the business threshold A, there is a similar node, and the node script information is recursively input to the next sequence of the similar node script.

[0078] The forward propagation calculation comprises:

[0079] First, the input sample is represented as a vector X=[x1,x2,...,x n ], where n is the number of input features, x1 represents the first sample feature, x2 represents the second sample feature, and so on. n Represents the nth sample feature;

[0080] For each layer of the neural network, starting from the input layer and ending at the output layer, calculate the weighted sum. The calculation formula is:

[0081]

[0082] Where l is the number of layers in the neural network, represents the number of neurons in the l-1 layer, i represents the index of each neuron in the l-1 layer, ranging from 1 to , is the weight connecting the i-th neuron in the l-1th layer and the j-th neuron in the lth layer, represents the output of the i-th neuron in the l-1th layer, represents the bias term of the jth neuron in the lth layer, represents the weighted sum of the jth neuron in the lth layer;

[0083] According to the formula: ), calculate the output results of all neurons in each layer, where ) is the activation function, and the weighted sum The input is processed to obtain the output of the jth neuron in the lth layer ;

[0084] The final output of the neural network is the output of the last layer: ,in represents the output of the first neuron in the Lth layer, Represents the output of the second neuron in the Lth layer, and so on Represents the output of the kth neuron in the Lth layer, L is the index of the output layer, k is the number of output categories, and finally the final output of the neural network is used as the score of each category .

[0085] Suppose there is a three-layer neural network with two neurons in each layer. The input sample of the input layer is [x1, x2], and the output result of the output layer is [y1, y2, y3], which represents the probability values ​​of the three categories. Use the softmax function to convert the output layer results into probability values. The specific formula is as follows:

[0086] softmax(y i ) = exp(y i) / sum(exp(y j )), j=1,2,3

[0087] Among them, y i represents the score of the i-th category, exp represents the natural exponential function, and sum represents the sum operation.

[0088] Assuming that the result of the output layer is [1.2, -0.5, 0.8], the calculation process of the softmax function is as follows:

[0089] exp(1.2) = 3.32, exp(-0.5) = 0.61, exp(0.8) = 2.23;

[0090] sum(exp(y j )) = exp(1.2) + exp(-0.5) + exp(0.8) = 6.16,

[0091] softmax(y1) = exp(1.2) / sum(exp(yj)) = 3.32 / 6.16 = 0.54,

[0092] softmax(y2) = exp(-0.5) / sum(exp(yj)) = 0.61 / 6.16 = 0.10,

[0093] softmax(y3) = exp(0.8) / sum(exp(yj)) = 2.23 / 6.16 = 0.36.

[0094] Therefore, the result of the output layer can be expressed as [0.54, 0.10, 0.36], which means that the probability of the sample belonging to the three categories is 54%, 10%, and 36% respectively.

[0095] The user intention generation module includes a user reply generation unit, a user intention similarity judgment unit, and a user intention generation and merging unit;

[0096] The user reply generation unit obtains the user reply content based on the preset information of the scenario preset module and the machine content request generative pre-trained language model prediction of the script generation module; the user intention similarity judgment unit inputs the user content into the NLP semantic model to determine whether there is a similar user intention; if there is no similar intention, the user intention is generated; otherwise, it is merged into the similar intention, and the judgment result is input to the user intention generation and merging unit;

[0097] The user intention similarity judgment unit includes:

[0098] The user reply content is input into an NLP semantic model, and whether there is a similar intention is determined according to a confidence score; the confidence score is calculated by a softmax function, in the NLP semantic model, given an input sample, the score of each category is calculated by forward propagation, and then the softmax function is used to convert them into probabilities;

[0099] According to the confidence score returned by the sample prediction, a business threshold B is set, if the confidence score is less than the business threshold B, there is no similar intention, and the user intention is generated, if the confidence score is greater than or equal to the business threshold B, it is merged into the similar intention, and the judgment result is input into the user intention generation and merging unit.

[0100] The routing logic processing module includes a routing chain forming unit and a routing chain forming unit two:

[0101] The routing chain forming unit first generates machine content A1 and machine node A by the script generation module, generates user content A and user intention A by the user intention generation module;

[0102] The creation and definition of the user intention A under the machine node A are completed by the data generation and processing module;

[0103] The preset information, machine content A1 and user content A information are requested to generate a pre-trained language model by the script generation module, and machine content A2 is obtained;

[0104] The machine content A2 is input into an NLP semantic model, other node information in the scene is predicted through node information, and whether there is a similar machine node is determined according to a confidence score;

[0105] In this process, if there is a similar node, the script information of the node is recursively assigned to the next sequence of the similar node script; otherwise, the steps of the node generation and merging unit of the script generation module are executed, and in this case, the similar node is listed, that is, the similarity of machine content A2 and machine content A1 is determined, if machine content A2 and machine content A1 are similar, the similar machine content A2 is recursively assigned to the node information of the similar machine node A;

[0106] The routing chain is formed as machine node A→user intention A→machine node A;

[0107] The routing chain forming unit two first requests the preset information, machine content A1 information, user content A information and machine content A2 information to generate a pre-trained language model by the user intention generation module, and obtains user content B; the user content B is input into an NLP semantic model to determine whether there is a similar user intention; if there is no similar intention, the user intention B is generated, otherwise the similar intention is merged;

[0108] The creation and definition of the user intention B under the machine node A are completed by the data generation and processing module;

[0109] The preset information, machine content A1 information, user content A information, machine content A2 and user content B request are generated into a generative pre-training language model by the script generation module, and machine content B1 is obtained;

[0110] The machine content B1 is input into the NLP semantic model to determine whether there is a similar machine node;

[0111] If there is no similar node, a machine node B is generated, otherwise the script sequence is merged into the same node;

[0112] The machine node A→user intention B→machine node B routing chain is formed.

[0113] The loop processing and knowledge base management module includes a loop processing unit, an end node management unit and a discrete knowledge base node unit;

[0114] The loop processing unit generates and processes the data according to the data generation and processing process provided by the scene preset module, the script generation module, the user intention generation module, the data generation and processing module and the routing logic processing module, repeatedly generates scenes and performs dialogue interaction without limitation, and achieves the purpose of scene generation; the end node management unit defines any one non-start node as an end node, and in the interaction process, entering the end node means the end of the dialogue process;

[0115] The discrete knowledge base node unit defines any one non-start and end node as a discrete knowledge base node, which is placed outside the process chain and triggered by the user reply semantics to output the node content; after the node content is output, the next round of dialogue defaults to the last flow node left, and the knowledge base node regression process memory is completed.

[0116] The basic process of the generative pre-training language model prediction is:

[0117] The input text sequence is represented as a word embedding vector; for each time step t, and from the first word of the sequence; the input text sequence can be preset information and machine content, or preset information, machine content and user content;

[0118] Input word embedding vector , represents the input word embedding vector at time step t;

[0119] The vector representation of the text sequence is input into a neural network for processing, and the calculation formula is:

[0120]

[0121] wherein denotes the hidden state at time step t, carrying the accumulation of previous sequence information; W is a weight matrix input to the neural network, used to weight-sum the input word embedding vector and the previous hidden state; U is a recurrent weight matrix in the neural network, used to recurrently compute the previous hidden state and the current hidden state; b is a bias term, corresponding to the bias term in the hidden state update formula; f is an activation function;

[0122] According to the formula , the output layer of the neural network is calculated, where V and c are the weight matrix and bias term of the output layer, g is the softmax function, used to convert the output into a probability distribution;

[0123] According to the probability distribution , the word with the highest probability is selected as the next generated word, and is added to the generated text sequence, which can be machine content or user content.

[0124] As shown in Figure 2 , the data simulation process in a standard dialogue cycle is as follows:

[0125] S1. Input the overall preset information of the scene.

[0126] S2. Based on the input preset information, obtain the machine starting node information and script A1.

[0127] S2.1. Input the machine starting node information into the NLP (Natural Language Processing) semantic model, predict other node information in the scene through the node information, and according to the confidence score, determine whether there is a similar machine node.

[0128] S2.2. If there is no similar node, generate a machine node A; if there is a similar node, recursively add the node script information to the next sequence of the similar node script, for example: node a (script 1, script 2, script 3, …), in the interaction process, the first time triggers node a, outputs script 1, the second time triggers node a, outputs script 2, and so on.

[0129] S3. Request the generative pre-trained language model (Generative Pre-trained Transformer) to generate user reply content A based on the preset information + machine content A1.

[0130] S3.1. Input the user content A into the NLP (Natural Language Processing) semantic model to determine whether there is a similar user intent.

[0131] S3.2. If there is no similarity intention, generate user intention A, otherwise merge to the similarity intention.

[0132] S4. Complete the creation and definition of user intention A under machine node A.

[0133] S5. Generate a generative pre-trained transformer with preset information + machine content A1 + user content A information request to obtain machine content A2.

[0134] S5.1. Input machine content A2 into the NLP (Natural Language Processing) semantic model to predict other node information in the scene according to the confidence score, and determine whether there is a similar machine node.

[0135] S5.2. If there is a similar node in this process, the node script information is recursively passed to the next sequence of the similar node script, otherwise execute s2.2 steps, and here it is listed that there is a similar node, i.e. machine content A2≈machine content A1=machine node A;

[0136] S5.3. Form a machine node A→user intention A→machine node A routing chain.

[0137] S6. Generate a generative pre-trained transformer with preset information + machine content A1 information + user content A information + machine content A2 information request to obtain user content B.

[0138] S6.1. Input user content B into the NLP (Natural Language Processing) semantic model to determine whether there is a similar user intention.

[0139] S6.2. If there is no similarity intention, generate user intention B, otherwise merge the similar intention, and here it is listed that there is no similar intention.

[0140] S7. Complete the creation and definition of user intention B under machine node A.

[0141] S8. Generate a generative pre-trained transformer with preset information + machine content A1 information + user content A information + machine content A2 + user content B request to obtain machine content B1.

[0142] S8.1. Input the machine content B1 into the NLP (Natural Language Processing) semantic model to determine whether there is a similar machine node.

[0143] S8.2. If there is no similar node, generate a machine node B, otherwise merge the script sequence n+1 under the same node.

[0144] S9. Form a machine node A→user intent B→machine node B routing chain.

[0145] In this embodiment, the deployment of the generative pre-training language model (Generative Pre-trained Transformer) is generated:

[0146] Use a cloud server as the deployment environment for the generative language model, which has both computing power and high scalability.

[0147] Deploy the Ubuntu operating system and install Python, TensorFlow, Flask, Gunicorn, Nginx, and Docker.

[0148] Integrate the model into the application: use the RESTful API interface to integrate the generative language model into the application. By writing a Python Flask application, an HTTP RESTful API interface is provided;

[0149] Write the start content generation interface addStartNode to obtain the StartNode (start node content) through the content (scenario description information);

[0150] Write the user content generation interface preHumanAnswer (user reply content) to obtain the preHumanAnswer (user reply content) through the content (scenario description information), StartNode (start node content);

[0151] Write the machine content generation interface preBotAnswer (machine reply content) to obtain the preBotAnswer (machine reply content) through the content (scenario description information), StartNode (start node content), and preHumanAnswer (user reply content).

[0152] Deployment of NLP (Natural Language Processing) semantic engine:

[0153] Use a cloud server as the deployment environment for the generative language model, which has both computing power and high scalability.

[0154] Deploy the Ubuntu operating system and install Python, Gensim, Flask, scikit-learn, Nginx, and Docker.

[0155] Integrate the model into the application: Use the RESTful API interface to integrate the generative language model into the application. This is achieved by writing a Python Flask application to provide an HTTP RESTful API interface.

[0156] Write a sample prediction interface Samples to predict the confidence of the sample. If the confidence < m, create a new intention A. If the confidence > m, merge the sample into the highest intention. The value of m can be adjusted according to business needs.

[0157] The confidence is calculated through the softmax function;

[0158] It should be noted that for NLP here, two scenarios need to be set. Scenario 1 is used for clustering the content of machine responses, and Scenario 2 is used for clustering the content of user responses. They are exactly the same in terms of functions and business logics.

[0159] To better describe the following business process, actual data is substituted here to simulate the generation of the process content:

[0160] 1. content = 'I need to conduct a research on personal income and need to collect 4 questions about the age, gender, occupation, and income of the surveyed objects';

[0161] 2. Through the addStartNode interface, obtain StartNode = 'Hello, I'm a research specialist. Can you spare some time to answer a few questions?';

[0162] 3. Through the sample prediction interface

[00] , use StartNode as the sample and request Scenario 1 to obtain a confidence of 0;

[0163] 4. Since the confidence < 0.6, then:

[0164] 5. Obtain botIntention = n1 through the addIntention interface;

[0165] 6. Through the user content generation interface preHumanAnswer, content = 'I need to do a survey about personal income, need to collect the respondent's age, gender, occupation, income 4 questions', StartNode = 'Hello, I am a survey officer, can you spare some time to answer a few questions?' preHumanAnswer1 = 'Yes', preHumanAnswer2 = 'I have no time'...

[0166] 7. Through the sample prediction interface Samples, preHumanAnswer1 as sample, request scene 2, get confidence = 0;

[0167] 8. Since the confidence < 0.6, then:

[0168] 9. Through the addIntention interface, humanIntention = i1 is obtained;

[0169] 10. Set the node routing relationship n1→i1;

[0170] 11. Through the machine content generation interface preBotAnswer, content = 'I need to do a survey about personal income, need to collect the respondent's age, gender, occupation, income 4 questions', StartNode = 'Hello, I am a survey officer, can you spare some time to answer a few questions?' humanIntention = 'Yes'; preHumanAnswer1 = 'Please tell me your age this year?';

[0171] 12. Perform step 7, and cycle 7→11 in this way;

[0172] It should be noted that the special processing of knowledge base nodes (discrete nodes);

[0173] The knowledge base node is a node that the user actively asks for a trigger in the dialogue process, and sets the answer content. The feature is that the node will not be triggered if the user does not actively ask;

[0174] The user can define any non-start and end machine nodes as knowledge base nodes;

[0175] When the user replies to the content at any position of the flow chain, and the Samples sample in NLP scene 2 shows a confidence > 0.6, it is not subject to node position constraints and can obtain node content through i→n relationship;

[0176] When the n node information is conveyed, the user replies to any Samples sample, and then returns to the interrupted flow node of the last round to continue the interaction;

[0177] It should be noted that the special processing of the end node, the user can define any non-start and end machine node as the end node, when entering the end node, the current session is terminated, and the dialogue record is formed for data storage.

[0178] It should be noted that in this document, the terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply these entities or operations have any such actual relationship or order. Also, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0179] Finally, it should be noted that the above only describes the preferred embodiments of the present application, and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A system for constructing a human-machine dialogue scenario based on a generative language model, characterized by comprising: The system comprises a scenario preset module, a script generation module, a user intention generation module, a data generation and processing module, a routing logic processing module, and a cycle processing and knowledge base management module; The scenario preset module receives and defines the overall background and target of the dialogue scenario, and records as preset information; The script generation module obtains machine start node information and scripts from the preset information of the scenario preset module, the scripts being machine content; predicts other node information in the scene through an NLP semantic model for the machine content, judges whether there is a similar machine node according to a confidence score, and performs node script information recursion or generates a new machine node; The user intention generation module generates user reply content according to the preset information of the scenario preset module and the machine content of the script generation module, the user reply content being user content; judges whether there is a similar user intention for the user content through an NLP semantic model, and performs merging or generates a new user intention; The data generation and processing module is used for creating and defining user intentions under machine nodes of the script generation module; The routing logic processing module requests a generative pre-training language model to obtain machine content or user content in the next interaction according to the preset information of the scenario preset module, the machine content of the script generation module, and the user content of the user intention generation module, judges whether there is a similar machine node, and finally forms a routing chain of machine nodes and user intentions; The cycle processing and knowledge base management module performs cycle processing according to the data generation and processing process provided by the scenario preset module, the script generation module, the user intention generation module, the data generation and processing module, and the routing logic processing module, repeatedly generates and processes data without limitation, and completes scenario generation; and ends the dialogue process by defining an end node; The output end of the scenario preset module is connected with the input end of the script generation module; The output end of the script generation module is connected with the input end of the user intention generation module; the output end of the user intention generation module is connected with the input end of the data generation and processing module; and the output end of the data generation and processing module is connected with the input end of the routing logic processing module; The output end of the routing logic processing module is connected with the input end of the cycle processing and knowledge base management module.

2. The system for constructing human-computer dialogue scenarios based on a generative language model according to claim 1, characterized in that: The script generation module comprises a node information acquisition unit, a similar node judgment unit, and a node generation and merging unit; The node information acquisition unit obtains machine start node information and machine content based on the preset information of the scenario preset module; inputs the machine start node information into an NLP semantic model of the similar node judgment unit, predicts other node information in the scene through the machine start node information, judges whether there is a similar machine node, generates a machine node if there is no similar node, and performs node script information recursion to the next sequence of the similar node script, and inputs the result to the node generation and merging unit; the node script information recursion is to mark the script to the next sequence of the corresponding node.

3. The system according to claim 2, wherein: The similarity node judgment and the script generation unit comprise: The machine start node information is input into the NLP semantic model, other node information in the scene is predicted through the node information, and whether there is a similar machine node is judged according to a confidence score; wherein the confidence is calculated through a softmax function, in the NLP semantic model, a given input sample is calculated through forward propagation to obtain the score of each category, and then the softmax function is used to convert them into probabilities; the formula of the softmax function is as follows: ； where j represents the data number, and takes values from 1 to n, where s i represents the score of the i-th class, and n represents the total number of classes; the softmax function converts the scores into probabilities such that the sum of all probabilities is 1; According to the confidence returned by the sample prediction, a business threshold A is set, if the confidence is less than the business threshold A, there is no similar node, and a machine node A is generated; if the confidence is greater than or equal to the business threshold A, there is a similar node, and the script information of the node is recursively returned to the next sequence of the similar node script.

4. The system according to claim 3, wherein the system is configured to: The forward propagation calculation comprises: First, the input sample is represented as a vector X=[x1,x2,...,x n ], where n is the number of input features, x1 represents the first sample feature, x2 represents the second sample feature, and so on. n Represents the nth sample feature; For each layer of the neural network, from the input layer to the output layer, the weighted sum is calculated, and the calculation formula is as follows: ； where l is the number of layers of the neural network, represents the number of neurons of the l-1th layer, i represents the index of each neuron in the l-1th layer, ranging from 1 to , is a weight connecting the i-th neuron of the l-1th layer and the j-th neuron of the lth layer, represents the output result of the i-th neuron in the l-1th layer, represents the bias term of the j-th neuron in the lth layer, represents the weighted sum of the j-th neuron in the lth layer; According to the formula: ), all neuron output results of each layer are calculated, wherein ) is an activation function, and the weighted sum of the input is processed to obtain the output of the jth neuron in the lth layer ; The final output result of the neural network is the output of the last layer: wherein represents the output of the first neuron of the Lth layer, represents the output of the second neuron of the Lth layer, and so on represents the output of the kth neuron of the Lth layer, L is the index of the output layer, and k is the number of categories of the output; and the final output result of the neural network is taken as the score of each category .

5. The system according to claim 3, wherein: The user intention generation module comprises a user reply generation unit, a user intention similarity judgment unit and a user intention generation and merging unit; The user reply generation unit generates user reply content based on the preset information of the scene preset module and the machine content request of the script generation module to obtain the user reply content through a pre-trained language model; The user intention similarity judgment unit inputs the user content into the NLP semantic model to judge whether there is a similar user intention; If there is no similar intention, a user intention is generated, otherwise it is merged into a similar intention, and the judgment result is input into the user intention generation and merging unit.

6. The human-computer dialogue scene construction system based on a generative language model according to claim 5, characterized in that: The user intention similarity judgment unit comprises: The user reply content is input into the NLP semantic model, and whether there is a similar intention is judged according to a confidence score; wherein the confidence is calculated through a softmax function, in the NLP semantic model, a given input sample is calculated through forward propagation to obtain the score of each category, and then the softmax function is used to convert them into probabilities; According to the confidence returned by the sample prediction, a business threshold B is set, if the confidence is less than the business threshold B, there is no similar intention, and a user intention is generated; if the confidence is greater than or equal to the business threshold B, it is merged into a similar intention, and the judgment result is input into the user intention generation and merging unit.

7. The system for constructing a human-computer dialogue scenario based on a generative language model according to claim 6, characterized in that: The routing logic processing module comprises a routing chain forming unit and a routing chain forming unit: The routing chain forming unit first generates machine content A1 and machine node A through the script generation module, generates user content A and user intention A through the user intention generation module; The creation and definition of user intention A under machine node A are completed through the data generation and processing module; Then, the preset information, machine content A1 and user content A information are requested to generate a pre-trained language model through the script generation module, and machine content A2 is obtained; The machine content A2 is input into the NLP semantic model, other node information in the scene is predicted through the node information, and whether there is a similar machine node is judged according to a confidence score; If there is a similarity node in the process, the node script information of the current node is recursively assigned to the next sequence of the similarity node script; otherwise, the node generation and merging unit of the script generation module is executed. In this case, there is a similarity node, i.e., judging the similarity between machine content A2 and machine content A1. If machine content A2 and machine content A1 are similar, the similar machine content A2 is recursively assigned to the node information of the similarity machine node A; A machine node A→user intent A→machine node A routing chain is formed; The routing chain forming unit first generates a preset information, machine content A1 information, user content A information, and machine content A2 information request to generate a pre-trained language model through the user intent generation module, and obtains user content B. The user content B is input into the NLP semantic model to determine whether there is a similar user intent. If there is no similar intent, a user intent B is generated, otherwise, similar intents are merged; The creation and definition of the user intent B under the machine node A are completed through the data generation and processing module; The preset information, machine content A1 information, user content A information, machine content A2, and user content B are requested to generate a pre-trained language model through the script generation module, and machine content B1 is obtained; The machine content B1 is input into the NLP semantic model to determine whether there is a similar machine node; If there is no similar node, a machine node B is generated, otherwise, it is merged into the script sequence under the same node; A machine node A→user intent B→machine node B routing chain is formed.

8. The system for constructing human-computer dialogue scenarios based on a generative language model according to claim 7, characterized in that: The cycle processing and knowledge base management module includes a cycle processing unit, an end node management unit, and a discrete knowledge base node unit; The cycle processing unit generates a cycle according to the data generation and processing process provided by the scene preset module, the script generation module, the user intent generation module, the data generation and processing module, and the routing logic processing module, repeatedly generates a scene and performs dialogue interaction without limitation, and achieves the purpose of scene generation; The end node management unit defines any one non-start node as an end node. In the interaction process, entering the end node means the end of the dialogue process; The discrete knowledge base node unit defines any one non-start and end node as a discrete knowledge base node. These nodes are placed outside the flow chain and triggered by user reply semantics to output node content. After the node content is output, the next round of dialogue defaults to the flow node left last time to complete the knowledge base node regression flow memory.

Images