A method for robot question answering and a storage medium

Abstract

The application relates to a robot question and answer method based on a Rasa self-defined NLU model and LLM combination and a storage medium, and comprises the following steps: adopting a JointBert joint model as an NLU model of a Rasa framework; labeling a specific field NLU data set according to a format requirement of the Rasa framework, and then training the NLU model of the Rasa framework; setting a reply or an action to be executed for a question of each type of intention and different entities; when receiving a user question, performing intention recognition and entity extraction through the NLU model of the Rasa framework; and performing corresponding actions according to the intention recognition. The Rasa framework self-defined NLU model simultaneously performs intention recognition and entity extraction, and sets a question sample and a template answer according to the format of the Rasa, so as to take the question sample and the template answer as a field knowledge base, thereby improving the accuracy and generalization ability of a robot answer.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, in particular to a robot question-answering method combining a self-defined NLU model and an LLM based on Rasa. BACKGROUND

[0002] In recent years, intelligent question-answering robots have become a research hotspot in many fields and have been widely applied to various scenarios such as e-commerce platforms, educational assistance, medical consultation, etc. In particular, with the advent of large language models (LLM, such as chatGPT and GLM), intelligent question-answering robots have made breakthrough progress. Through deep learning technology, the LLM learns from massive amounts of text data, generates answers through context understanding and reasoning, and has extremely high automatic question-answering capabilities. However, despite the great breakthroughs made by the LLM, it still faces some limitations and challenges. Since the LLM is trained on a general large dataset, it has general applicability but lacks knowledge in specific fields, making it unable to answer some professional questions. At the same time, for some complex or ambiguous questions, the LLM may produce hallucinations and give incorrect answers.

[0003] At the same time, the common approach to injecting domain knowledge into the LLM is to fine-tune it, using a pre-trained LLM as the initial model and further training it on small-scale data in a vertical field to improve its performance in that field. However, fine-tuning requires a large amount of computing resources and is costly. SUMMARY

[0004] In view of the above problems, the application provides a robot question-answering method combining a self-defined NLU model and an LLM based on Rasa, which solves the problem that the large language model (LLM) cannot answer some professional questions in the existing robot dialogue, and may produce hallucinations and give incorrect answers for some complex or ambiguous questions.

[0005] To achieve the above-mentioned purpose, the inventors provide a robot question-answering method combining a self-defined NLU model and an LLM based on Rasa, which comprises the following steps:

[0006] A JointBert joint model is used as the NLU model of the Rasa framework, and bert_base_chinese is used as the pre-trained model for vector representation;

[0007] A specific domain NLU dataset is labeled according to the format requirements of the Rasa framework, and then the NLU model of the Rasa framework is trained, wherein the NLU dataset includes question text, intent labels and entity labels;

[0008] The reply or action to be performed for each type of intent and different entity is set.

[0009] When receiving a user question, intent recognition and entity extraction are performed by the NLU model of the Rasa framework;

[0010] When the intent recognition is a strong intent, if the next predicted action is an utter template, the corresponding reply template is found according to the intent label and the extracted entity, and the entity is filled and replied, and if the predicted action is a custom action, the action is directly executed;

[0011] When the intent recognition is a weak intent, the first preset round of dialogue of the user is organized in the form of question and answer pair as historical chat record and delivered to the large language model for answering;

[0012] When the intent recognition is a general intent, if the next predicted action is an utter template, the corresponding reply template is found according to the intent label and the extracted entity, and the answer is generated after filling the entity, which is delivered to the large language model as reference material, and the large language model answers according to the reference material, if the predicted action is an automatic action, the user is confirmed whether the predicted intent is correct, if yes, the corresponding action is executed, if not, the large language model is called to answer in the weak intent mode.

[0013] In some embodiments, the intent recognition comprises the following steps:

[0014] The input question text sequence x=(x1,…,x T ) is input into a pre-trained model to obtain an output result H=(h1,…,h T ), x1 is a classification task mark [CLS] added, x T is an end mark [SEP] added, the middle of x1 and x T is a word, and h1 corresponds to a vectorized representation of x1;

[0015] The output result H is input into a softmax layer to obtain an intent prediction y i =softmax(W i h1+b i ), i is an intent label, W is a weight, and b is a bias;

[0016] y i greater than 0.9 is classified as a strong intent, y i between 0.5 and 0.9 is classified as a general intent, and y i less than 0.5 is classified as a weak intent.

[0017] In some embodiments, the entity extraction specifically comprises the following steps:

[0018] The question text sequence x=(x1,…,xT The vectorized representation of the sequence of characters in the input softmax layer predicts the entity label of each character s is the entity label, N represents the length of the question text, and n is the serial number of the word.

[0019] In some embodiments, the training of the Rasa framework NLU model specifically comprises the following steps:

[0020] Select JointBert model to train Rasa NLU.

[0021] In some embodiments, the following steps are further included:

[0022] If the same user has the same intent as the current question in the second preset number of conversations and has called the corresponding template reply, the template reply will not be performed for the current question, and the template answer will be sent to the large language model as reference material.

[0023] Another technical solution is also provided, which is a storage medium storing a computer program, the computer program being executed by a processor to perform the following steps:

[0024] By using JointBert joint model as the NLU model of the Rasa framework, the bert_base_chinese is used as the pre-training model for vectorized representation;

[0025] According to the format requirements of the Rasa framework, the NLU data set of a specific field is labeled, and then the NLU model of the Rasa framework is trained, the NLU data set including question text, intent label and entity label;

[0026] Set the reply or action to be performed for each type of intent and different entity question;

[0027] When receiving a user question, the intent recognition and entity extraction are performed through the NLU model of the Rasa framework;

[0028] When the intent recognition is a strong intent, if the next predicted action is a uter template type, the corresponding reply template is searched according to the intent label and the extracted entity, the entity is filled and replied, if the predicted action is a custom action, the action is directly executed;

[0029] When the intent recognition is a weak intent, the first preset number of conversations of the user is organized in the form of question and answer pair as historical chat records and transmitted to the large language model for answering;

[0030] When the intent is identified as a general intent, if the predicted next action is an utter template type, the corresponding response template is found based on the intent tag and the extracted entity. After filling in the entity to generate the answer, it is passed as reference material to the big language model, so that the big oracle model can answer based on the reference material. If the predicted action is an automatic action, the user is asked to confirm whether the user's predicted intent is correct. If confirmed, the corresponding action is executed. Otherwise, the big oracle model is called to answer according to the weak intent method.

[0031] In some embodiments, the intent recognition includes the following steps:

[0032] The input question text sequence x = (x1, ..., x...) T After pre-training the model, the output result H = (h1, ..., h) is obtained. T x1 is the added category task tag [CLS], x T Add the closing marker [SEP], x1 and x T The middle part is text, and h1 corresponds to the vectorized representation of x1;

[0033] The output result H is input into the softmax layer for classification to obtain the intent prediction y. i =softmax(W i h1+b i ), where i is the intent label, W is the weight, and b is the bias;

[0034] y i Values ​​greater than 0.9 are classified as strong intent, and y is... i A range between 0.5 and 0.9 is generally considered to be the case, with y... i Values ​​less than 0.5 are classified as weak intent.

[0035] In some embodiments, the entity extraction specifically includes the following steps:

[0036] Let the question text sequence be x = (x1, ..., x...). T The vectorized representation of the text sequence in the image is input into the softmax layer to predict the entity label for each text. s is the entity label, N represents the length of the question text, and n is the word index.

[0037] In some embodiments, training the NLU model of the Rasa framework specifically includes the following steps:

[0038] Choose the JointBert model to train RasaNLU.

[0039] In some embodiments, the following steps are also included:

[0040] If the same user has existed the same intent as the current question in the second preset number of rounds of dialogue and has called the corresponding template reply, the template reply is no longer performed for the current question, and the template answer is sent to the large language model as reference material.

[0041] Unlike the prior art, the above technical solution customizes the NLU model based on the Rasa framework to simultaneously perform intent recognition and entity extraction, sets the question sample and template answer in the format of Rasa as the domain knowledge base, and distinguishes between calling the template direct reply and calling the large language model to generate a reply according to the confidence of the question understanding, so as to improve the accuracy and generalization ability of the robot answer.

[0042] The above invention content related description is only a summary of the technical solution of the present application. In order for those skilled in the art to more clearly understand the technical solution of the present application, and then can be implemented according to the content of the description and the drawings, and in order for the above-mentioned purposes and other purposes, features and advantages of the present application can be more easily understood, the following is described in conjunction with the specific embodiments of the present application and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0043] The accompanying drawings are only used to illustrate the principles, implementation manners, applications, characteristics and effects of the specific embodiments of the present application and other related contents, and cannot be considered as limitations of the present application.

[0044] In the drawings of the specification:

[0045] Figure 1 A flowchart of the robot question and answer method based on the Rasa customized NLU model and LLM combination described in the specific embodiments;

[0046] Figure 2 A flowchart of the consciousness recognition described in the specific embodiments;

[0047] Figure 3 A flowchart of the entity extraction described in the specific embodiments;

[0048] Figure 4 Another flowchart of the robot question and answer method based on the Rasa customized NLU model and LLM combination described in the specific embodiments;

[0049] Figure 5 A structural diagram of the storage medium described in the specific embodiments.

[0050] The reference signs involved in the above drawings are explained as follows:

[0051] 510, storage medium,

[0052] 520, processor. DETAILED DESCRIPTION

[0053] To make the possible application scenarios, technical principles, specific schemes that can be implemented, purposes and effects that can be achieved of the present application clear, the following will be described in detail in combination with the specific embodiments listed and the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical schemes of the present application, and therefore only serve as examples, but cannot limit the protection scope of the present application.

[0054] In this paper, the term "embodiment" means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The term "embodiment" appearing at various places in the specification does not necessarily refer to the same embodiment, and does not particularly limit the independence or association between other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, each technical feature mentioned in each embodiment can be combined in any way to form a corresponding implementable technical scheme.

[0055] Unless otherwise defined, the meaning of the technical terms used herein is the same as that generally understood by those skilled in the art to which the present application belongs; the use of related terms in this paper is only for the purpose of describing specific embodiments, and is not intended to limit the present application.

[0056] In the description of the present application, the word "and / or" is a description of the logical relationship between the objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this paper generally represents that the associated objects before and after are a "or" logical relationship.

[0057] In the present application, such as "first" and "second", the words are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual quantity, primary and secondary or order relationship between the entities or operations.

[0058] In the present application, without more limitation, the "include", "contain", "have" or other similar expressions used in the sentence are intended to cover non-exclusive inclusion, and these expressions do not exclude the existence of other elements in the process, method or product including the described elements, so that the process, method or product including a series of elements can not only include those limited elements, but also include other elements not explicitly listed, or also include the elements inherent in such process, method or product.

[0059] As the same understanding as in the Guidelines for Review, in this application, "greater than", "less than", "exceed" and the like are understood as not including the number; "above", "below", "within" and the like are understood as including the number. In addition, in the description of the embodiments of the present application, the meaning of "multiple" is more than two (including two), and similar expressions related to "multiple" are also understood in this way, for example, "multiple groups", "multiple times" and the like, unless otherwise explicitly specified.

[0060] In the description of the embodiments of the present application, the spatially related expressions used, such as "center", "longitudinal", "transverse", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the specific embodiment or the drawing, and is only for the convenience of describing the specific embodiments of the present application or for the reader to understand, and does not indicate or imply that the indicated device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0061] Unless otherwise explicitly specified or limited, in the description of the embodiments of the present application, the terms "mount", "connect", "connect", "fix", "set", and the like should be broadly understood. For example, the "connection" can be a fixed connection, or a detachable connection, or an integral setting; it can be a mechanical connection, or an electrical connection, or a communication connection; it can be directly connected, or indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art to which the present application belongs, the specific meaning of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0062] Rasa is a rule-based and machine learning-based dialogue system framework that provides intent recognition, entity extraction, dialogue management, etc. It can build customized question and answer robots according to business scenarios and needs, but since Rasa is based on rule matching, it cannot handle problems outside the predefined range, and the degree of intelligence is not as good as large language models. At the same time, the NLU model of Rasa is pipeline type, which performs word segmentation, vectorization, intent recognition, entity extraction and other steps in turn, and needs to annotate a large amount of dialogue data for model training, and the model performs poorly when the data set is small.

[0063] Please refer to Figure 1 The embodiments of the present application provide a robot question and answer method based on a self-defined NLU model and LLM combined with Rasa, comprising the following steps:

[0064] Step S110: vectorize representation by adopting JointBert joint model as the NLU model of the Rasa framework and taking bert_base_chinese as the pre-training model; adopt the JointBert joint model to vectorize representation by taking bert_base_chinese as the pre-training model, since the intent and entity extraction are performed simultaneously, the pipeline type step-by-step component is no longer needed, therefore, the JointBert needs to inherit the Tokenizer, IntentClassifier and EntityExactor three structure bodies defined by Rasa at the same time to embed the Rasa framework adaptation.

[0065] Step S120: label the specific field NLU dataset according to the format requirement of the Rasa framework, and then train the NLU model of the Rasa framework, wherein the NLU dataset includes question text, intent label and entity label;

[0066] Step S130: set the reply or action to be performed for the question of each type of intent and different entity; then set the response and action, that is, the reply or action to be performed for the question of each type of intent and different entity, set the NLU and response to constitute the question and answer pair, that is, the knowledge base of the specific field, select the dialog strategy of Rasa to train RasaCore.

[0067] Step S130: when receiving the user's question, perform intent recognition and entity extraction through the NLU model of the Rasa framework;

[0068] When the intent recognition is a strong intent, step S151 of predicting the next action is performed, if the predicted next action is a utter template type, the corresponding reply template is searched according to the intent label and the extracted entity, the entity is filled and replied, if the predicted action is a custom action, the action is directly executed;

[0069] When the intent recognition is a weak intent, step S152 of organizing the first pre-set number of dialogues of the user according to the question and answer pair format as the historical chat record to pass to the large language model is performed, and the large language model is answered;

[0070] When the intent is identified as a general intent, step S153 is performed to predict the next action. If the predicted next action is an utter template, the corresponding reply template is found according to the intent label and the extracted entity, and the answer is generated by filling in the entity. The answer is then transmitted to the large language model as reference material, and the large language model answers according to the reference material. If the predicted action is an automatic action, the user is asked to confirm whether the predicted intent is correct. If yes, the corresponding action is performed. If no, the weak intent mode is used to call the large language model to answer.

[0071] When the intent is identified as a strong intent, if the RasaCore predicted next action is an utter template, the intent label i and the extracted entity s are transmitted to the Rasa TemplateNaturalLanguageGenerator to find the corresponding response template and fill in the entity to reply. If the predicted action is a custom action, the action is directly executed. When the intent is identified as a general intent, if the RasaCore predicted next action is an utter template, the corresponding response template is found and the answer is generated by filling in the entity. The answer is then transmitted to the LLM as reference material, and the LLM answers according to the reference material. If the predicted action is a custom action, the user is further asked to confirm whether the predicted intent is correct. If yes, the corresponding action is performed. If no, the weak intent mode is used to call the LLM to answer. When the intent is identified as a weak intent, whether the RasaCore predicted next action is an utter template or a custom action, the Tracker of Rasa is used to organize the previous three rounds of conversation between the user and the robot in the question and answer pair format as historical chat records, which are transmitted to the LLM for answering.

[0072] Based on the Rasa framework, a custom NLU model is used for intent recognition and entity extraction. The problem samples and template answers are set in the format of Rasa, which are used as domain knowledge base. According to the confidence of problem understanding, it is determined whether to call the template to reply directly or to call the large language model to generate a reply, so as to improve the accuracy and generalization ability of the robot answer.

[0073] Please refer to Figure 2 In some embodiments, the intent recognition includes the following steps:

[0074] Step S210: The input problem text sequence x=(x1,…,x T ) is input into the pre-trained model to obtain the output result H=(h1,…,h T ), x1 is the classification task mark [CLS], x Tfor the added end token [SEP], x1 and x T The middle is the text, and h1 corresponds to the vectorized representation of x1.

[0075] Step S220: input the output result H into a softmax layer to obtain an intent prediction y i = softmax(W i h1+b i ), i is an intent label, W is a weight, and b is a bias.

[0076] Step S230: divide the y i greater than 0.9 into a strong intent, divide the y i between 0.5 and 0.9 into a general intent, and divide the y i less than 0.5 into a weak intent.

[0077] When the input question text sequence is x = (x1,..., x T ), where x1 is an added classification task token [CLS], x T is an added end token [SEP], and the middle is text. The output result obtained through bert_base_chinese is H = (h1,..., h T ), h1 corresponds to the vectorized representation of x1. After inputting h1 into a softmax layer for classification, the intent prediction is y i The formula is as follows:

[0078] y i = softmax(W i h1+b i )

[0079] where i is an intent label, W is a weight, and b is a bias.

[0080] According to the confidence y i of intent recognition, divide the y i greater than 0.9 into a strong intent, which indicates that the user question is highly consistent with the data in NLU, and there is a high probability of hitting the sample type; divide the y i between 0.5 and 0.9 into a general intent, which indicates that the user question has a certain relevance to the data of the intent in NLU, but is not completely consistent; and divide the y i less than 0.5 into a weak intent, which indicates that the user question has low relevance to all intents in NLU, and there is a high probability of misclassification.

[0081] Please refer to Figure 3 In some embodiments, the entity extraction specifically includes the following steps:

[0082] Step S310; input the vectorized representation of the character sequence in the question text sequence x=(x1,…,x T ) into the softmax layer to predict the entity label of each character s is the entity label, N represents the length of the question text, and n is the serial number of the word.

[0083] For entity extraction, the vectorized representation of the remaining character sequence is also input into the softmax layer to predict the entity label of each character

[0084] s is the entity label, N represents the length of the question text, and n is the serial number of the word.

[0085] At the same time, joint intent classification and entity extraction, i.e., the learning goal is to maximize the conditional probability p(y i ,y s |x):

[0086]

[0087] In some embodiments, the training of the Rasa framework NLU model specifically includes the following steps:

[0088] Select JointBert model to train RasaNLU.

[0089] JointBert model is a fine-tuned model combined with Bert, a language representation model BERT (Bidirectional Encoder Representations from Transformers), which helps to pre-train deep bidirectional representations on large unlabelled corpora and create the latest model for various natural language processing tasks after simple fine-tuning.

[0090] Please refer to Figure 4 In some embodiments, the following steps are also included:

[0091] Step S410: If there is already an intent in the same user's previous second preset number of conversations that is the same as the current question and has called the corresponding template reply, the template reply will not be performed for the current question, and the template answer will be sent to the large language model as reference material for answering.

[0092] To avoid the factors such as bias or incompleteness of training data causing user questions to be always misclassified to a strong intent, thereby causing the robot to continuously reply to the wrong template, an error prevention mechanism is added: if there is already an intent in the same user's previous 5 conversations that is the same as the current question and has called the corresponding template reply, the template reply will not be performed, but the template answer will be passed to the LLM as reference material for answering.

[0093] In the above embodiment, a Rasa-based custom NLU model and LLM combined robot question and answer method is provided, which uses a JointBert joint model to replace the Rasa self-provided NLU model. The JointBert joint model has strong context-related sentence representation capability based on a pre-trained model, and fully utilizes the correlation between intent recognition and entity extraction. The accuracy and efficiency of intent recognition and entity extraction are higher, and the performance on small data sets is also good. Combining Rasa and LLM, on the one hand, Rasa can be used to customize the dialogue content, and the response speed is faster for more certain questions, which can effectively save the time cost and resource cost of calling LLM, and at the same time, the domain knowledge closely related to the question is transmitted to LLM for reference, which can avoid generating incorrect or low-quality answers based on massive training texts; on the other hand, LLM can be used to process various problems outside the predefined range, which has higher flexibility, and can optimize the template answer with lower matching degree to generate natural and smooth text replies.

[0094] Referring to Figure 5 , in another embodiment, a storage medium 510 stores a computer program, and the computer program is run by a processor 520 to perform the following steps:

[0095] By using the JointBert joint model as the NLU model of the Rasa framework, the bert_base_chinese is used as the pre-trained model for vectorization representation;

[0096] According to the format requirements of the Rasa framework, the NLU data set of a specific field is labeled, and then the NLU model of the Rasa framework is trained. The NLU data set includes question text, intent label and entity label;

[0097] Set the reply or action to be performed for each type of intent and different entities;

[0098] When receiving a user question, perform intent recognition and entity extraction through the NLU model of the Rasa framework;

[0099] When the intent recognition is a strong intent, if the predicted next action is a utter template type, the corresponding reply template is found according to the intent label and the extracted entity, the entity is filled and replied, and if the predicted action is a custom action, the action is directly executed;

[0100] When the intent recognition is a weak intent, the first preset number of dialogues of the user is organized in the form of question and answer pair as historical chat records and transmitted to the large language model, and the large language model answers;

[0101] When the intent is identified as a general intent, if the predicted next action is an utter template type, the corresponding response template is found based on the intent tag and the extracted entity. After filling in the entity to generate the answer, it is passed as reference material to the big language model, so that the big oracle model can answer based on the reference material. If the predicted action is an automatic action, the user is asked to confirm whether the user's predicted intent is correct. If confirmed, the corresponding action is executed. Otherwise, the big oracle model is called to answer according to the weak intent method.

[0102] A custom NLU model based on the Rasa framework is used to perform both intent recognition and entity extraction. Question samples and template answers are set according to the Rasa format and used as a domain knowledge base. The system distinguishes between calling the template to reply directly or calling the large language model to generate a reply based on the confidence level of the question understanding, so as to improve the accuracy and generalization ability of the robot's answers.

[0103] In some embodiments, the intent recognition includes the following steps:

[0104] The input question text sequence x = (x1, ..., x...) T After pre-training the model, the output result H = (h1, ..., h) is obtained. T x1 is the added category task tag [CLS], x T Add the closing marker [SEP], x1 and x T The middle part is text, and h1 corresponds to the vectorized representation of x1;

[0105] The output result H is input into the softmax layer for classification to obtain the intent prediction y. i =softmax(W i h1+b i ), where i is the intent label, W is the weight, and b is the bias;

[0106] y i Values ​​greater than 0.9 are classified as strong intent, and y is... i A range between 0.5 and 0.9 is generally considered to be the case, with y... i Values ​​less than 0.5 are classified as weak intent.

[0107] When the input question text sequence is x = (x1, ..., x...) T ), where x1 is the added category task tag [CLS], x T The added closing tag [SEP] is followed by the text. After processing with bert_base_chinese, the output is H = (h1, ..., h...). T h1 corresponds to the vectorized representation of x1. Inputting this vectorized representation into a softmax layer for classification results in the intended prediction as y. i The formula is as follows:

[0108] y i = softmax(W i h1+b i )

[0109] where i is the intent label, W is the weight, and b is the bias;

[0110] According to the confidence y i of intent recognition, the grades are divided, y i greater than 0.9 is classified as strong intent, indicating that the user's question is highly consistent with the data in NLU, and there is a high probability of hitting the sample type; y i between 0.5 and 0.9 is classified as general intent, indicating that the user's question has some relevance to the data of this intent in NLU, but it is not completely consistent; y i less than 0.5 is classified as weak intent, indicating that the user's question has low relevance to all intents in NLU, and there is a high probability of misclassification.

[0111] In some embodiments, the entity extraction specifically includes the following steps:

[0112] The vectorized representation of the character sequence in the question text sequence x = (x1, …, x T ) is input into the softmax layer to predict the entity label s of each character. s is the entity label, and N represents the length of the question text, and n is the word number.

[0113] For entity extraction, the vectorized representation of the remaining character sequence is also input into the softmax layer to predict the entity label s of each character.

[0114] where s is the entity label, N represents the length of the question text, and n is the word number.

[0115] Joint intent classification and entity extraction, the learning goal is to maximize the conditional probability p(y i , y s | x):

[0116]

[0117] In some embodiments, the training of the Rasa framework NLU model specifically includes the following steps:

[0118] Select JointBert model to train RasaNLU.

[0119] The JointBert model is a fine-tuned model combined with Bert, a language representation model BERT (Bidirectional Encoder Representations from Transformers), which helps to pre-train deep bidirectional representations on large unlabeled corpora and create state-of-the-art models for various natural language processing tasks after simple fine-tuning.

[0120] In some embodiments, the method further comprises the following steps:

[0121] If the same user has the same intent as the current question in the previous second preset number of rounds of conversation and has called the corresponding template reply, the template reply is no longer performed for the current question, and the template answer is sent to the large language model as reference material for answering.

[0122] To avoid the factors such as bias or incompleteness of training data causing the user question to be always misclassified to a strong intent, so that the robot continuously replies to the wrong template, an error prevention mechanism is added: if the same user has the same intent as the current question in the previous 5 rounds of conversation and has called the corresponding template reply, the template reply is no longer performed, but the template answer is passed to the LLM as reference material for answering.

[0123] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of the present application, they should not limit the patent protection scope of the present application. Any equivalent structure or equivalent process substitution or modification based on the essential concept of the present application, using the content described in the specification and drawings, directly or indirectly, and the technical solutions of the above embodiments applied to other related technical fields, etc., are all included in the patent protection scope of the present application.

Claims

1. A robot question-answering method based on a custom NLU model of Rasa and an LLM, characterized in that, Includes the following steps: By using the JointBert joint model as the NLU model of the Rasa framework, bert_base_chinese is used as a pre-trained model for vectorization representation; The NLU dataset for a specific domain is labeled according to the format requirements of the Rasa framework, and then the NLU model of the Rasa framework is trained. The NLU dataset includes question text, intent label and entity label. Configure the response or action to be performed for each type of intent and different entity's question; When a user query is received, intent recognition and entity extraction are performed using the NLU model of the Rasa framework; When the intent is identified as a strong intent, if the predicted next action is a uter template type, the corresponding response template is found based on the intent tag and the extracted entity, the entity is filled in and the response is made. If the predicted action is a custom action, the action is executed directly. When the intent is identified as a weak intent, the user’s first preset number of rounds of dialogue are organized in a question-and-answer pair format and passed to the large language model as a historical chat record, so that the large language model can respond. When the intent is identified as a general intent, if the predicted next action is an utter template type, the corresponding response template is found based on the intent tag and the extracted entity. After filling in the entity to generate the answer, it is passed as reference material to the big language model, so that the big oracle model can answer based on the reference material. If the predicted action is a custom action, the user is asked to confirm whether the user's predicted intent is correct. If it is correct, the corresponding action is executed. Otherwise, the big oracle model is called to answer in the weak intent mode. The intent recognition includes the following steps: input question text sequence The output result is obtained after the pre-trained model. , Add the category task tag [CLS], The added closing tag [SEP], and The middle part contains text. Corresponding to Vectorized representation of; The output H is input into a softmax layer for classification to obtain the intent prediction. , i is the intent label, W is the weight, and b is the bias; Will A value greater than 0.9 is considered a strong intent. A score between 0.5 and 0.9 indicates a general intent. Values ​​less than 0.5 are classified as weak intent.

2. The robot question answering method based on Rasa's custom NLU model and LLM combined according to claim 1, characterized in that, The entity extraction specifically includes the following steps: sequence of question text The vectorized representation of the text sequence is input into the softmax layer to predict the entity label for each text. s is the entity label, N represents the length of the question text, and n is the word index.

3. The robot question answering method based on Rasa's custom NLU model and LLM combined according to claim 1, characterized in that, The training of the Rasa framework NLU model specifically includes the following steps: Choose the JointBert model to train RasaNLU.

4. The robot question-answering method based on Rasa's custom NLU model and LLM combined according to claim 1, characterized in that, It also includes the following steps: If a user has already presented a response with the same intent to the current question in the previous second preset round of dialogue and has already used the corresponding template, then the current question will not receive a template response. Instead, the template answer will be sent to the large language model as reference material for the current user to answer the question.

5. A storage medium storing a computer program, characterized in that, The computer program, when executed by the processor, performs the following steps: By using the JointBert joint model as the NLU model of the Rasa framework, bert_base_chinese is used as a pre-trained model for vectorization representation; The NLU dataset for a specific domain is labeled according to the format requirements of the Rasa framework, and then the NLU model of the Rasa framework is trained. The NLU dataset includes question text, intent label and entity label. Configure the response or action to be performed for each type of intent and different entity's question; When a user query is received, intent recognition and entity extraction are performed using the NLU model of the Rasa framework; When the intent is identified as a strong intent, if the predicted next action is a uter template type, the corresponding response template is found based on the intent tag and the extracted entity, the entity is filled in and the response is made. If the predicted action is a custom action, the action is executed directly. When the intent is identified as a weak intent, the user’s first preset number of rounds of dialogue are organized in a question-and-answer pair format and passed to the large language model as a historical chat record, so that the large language model can respond. When the intent is identified as a general intent, if the predicted next action is an utter template type, the corresponding response template is found based on the intent tag and extracted entities. After filling in the entities to generate the answer, this template is passed as reference material to the big language model, allowing the big oracle model to answer based on the reference material. If the predicted action is an automatic action, the user is asked to confirm whether the user's predicted intent is correct. If correct, the corresponding action is executed; otherwise, the big oracle model is called to answer according to the weak intent method. The intent identification includes the following steps: input question text sequence The output result is obtained after the pre-trained model. , Add the category task tag [CLS], The added closing tag [SEP], and The middle part contains text. Corresponding to Vectorized representation of; The output H is input into a softmax layer for classification to obtain the intent prediction. , i is the intent label, W is the weight, and b is the bias; Will A value greater than 0.9 is considered a strong intent. A score between 0.5 and 0.9 indicates a general intent. Values ​​less than 0.5 are classified as weak intent.

6. The storage medium according to claim 5, characterized in that, The entity extraction specifically includes the following steps: sequence of question text The vectorized representation of the text sequence is input into the softmax layer to predict the entity label for each text. s is the entity label, N represents the length of the question text, and n is the word index.

7. The storage medium according to claim 5, characterized in that, The training of the Rasa framework NLU model specifically includes the following steps: Choose the JointBert model to train RasaNLU.

8. The storage medium according to claim 5, characterized in that, It also includes the following steps: If a user has already presented a response with the same intent to the current question in the previous second preset round of dialogue and has already used the corresponding template, then the current question will not receive a template response. Instead, the template answer will be sent to the large language model as reference material for the current user to answer the question.

Images