Dialogue content generation method, model training method, device, and storage medium

Abstract

Provided in the embodiments of the present disclosure are a dialogue content generation method, a model training method, a prompt text generation method, a device and a storage medium. The dialogue content generation method comprises: receiving an input question; determining whether the question needs to be answered in combination with external knowledge; if the question needs to be answered in combination with external knowledge, acquiring target knowledge from a preset knowledge base on the basis of the question, generating prompt text corresponding to the target knowledge, inputting into a dialogue content generation model the question, the target knowledge and the prompt text corresponding to the target knowledge, and generating dialogue content by means of the dialogue content generation model, wherein the prompt text is used for prompting the dialogue content generation model to answer the question with reference to the target knowledge; and if the question does not need to be answered in combination with external knowledge, inputting the question into the dialogue content generation model, and generating dialogue content by means of the dialogue content generation model, wherein the dialogue content generation model is obtained by means of performing fine tuning on a pre-trained language model.

Description

Method for generating conversation content, model training method, device and storage medium Technical Field

[0001] One or more embodiments of this specification relate to the field of natural language processing technology, and in particular, to a method for generating dialogue content, a method for training a dialogue content generation model, a method for generating prompt text, an electronic device, and a computer-readable storage medium. Background Art

[0002] In related technologies, the interaction between users and APPs (Applications) is mainly based on search or recommendation. For example, users enter search terms in the search box according to their own needs, and the APP pushes corresponding target objects to users based on the search terms, such as dishes, goods, services or articles, etc.; for example, the APP will use user portraits and historical behaviors as a reference, pre-determine the recommended objects and display them in the interactive interface for users to choose. The above methods can meet user needs to a certain extent, but they are weak in accepting general intents. For example, when a user wants to know "What foods can I eat to lose weight?", for search, this request is long and not specific enough, resulting in inaccurate search results; for recommendations, user portraits and historical behaviors are more considered as a reference, and it is not possible to make targeted recommendations to users' weight loss needs in real time, and the recommendation results are not accurate enough.

[0003] Summary of the Invention

[0004] In view of this, one or more embodiments of this specification provide a method for generating conversation content, a method for training a conversation content generation model, a method for generating prompt text, an electronic device, and a computer-readable storage medium.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of the present specification, a method for generating dialogue content is proposed, comprising: receiving an input question; determining whether the question needs to be answered in combination with external knowledge; if so, obtaining target knowledge from a preset knowledge base based on the question, and generating a prompt text corresponding to the target knowledge; inputting the question, the target knowledge, and the prompt text into a dialogue content generation model, and the dialogue content generation model generates dialogue content based on the question, the target knowledge, and the prompt text; the prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge; if not, inputting the question into the dialogue content generation model, and the dialogue content generation model generates dialogue content based on the question; wherein, the dialogue content generation model is obtained by fine-tuning a pre-trained language model.

[0007] In the disclosed embodiments, a pre-trained language model can be fine-tuned to generate a conversation content generation model. In actual application, after receiving a user's question, it is first determined whether the question requires external knowledge to answer. If not, the question is input into the conversation content generation model for answering. If it does, target knowledge is obtained from a pre-set knowledge base, and the question, target knowledge, and prompt text corresponding to the target knowledge are input into the conversation content generation model for answering. This embodiment takes into account that the method of answering input questions solely by the conversation content generation model cannot obtain some external knowledge in real time, and may have drawbacks such as a lack of factual knowledge and an inability to handle time-sensitive or open-ended questions. Therefore, to further improve the accuracy of responses, the disclosed embodiment injects the target knowledge into the conversation content generation model via prompt text when it is determined that the question requires external knowledge, helping the conversation content generation model acquire more external knowledge. The combination of these two conversation content generation methods facilitates accurate answers to users' general-purpose questions, improving the accuracy of conversation content and user satisfaction.

[0008] According to a second aspect of one or more embodiments of the present specification, a method for training a dialogue content generation model is proposed, comprising: obtaining a first training data set, the first training data set comprising a number of question samples and their corresponding dialogue contents; the number of question samples comprising a number of first question samples and a number of second question samples; for the second question sample, obtaining target knowledge from a preset knowledge base according to the second question sample, and generating a prompt text corresponding to the target knowledge; inputting the first question sample into a pre-trained language model, so that the pre-trained language model generates a first predicted reply according to the first question sample; and inputting the second question sample, the target knowledge corresponding to the second question sample, and the prompt text corresponding to the target knowledge into the pre-trained language model, so that the pre-trained language model generates a second predicted reply according to the second question sample, the target knowledge, and the prompt text; the prompt text is used to prompt the pre-trained language model to reply to the second question sample with reference to the target knowledge; training the pre-trained language model with the optimization objectives of minimizing the error between the first predicted reply and the dialogue content corresponding to the first question sample, and minimizing the error between the second predicted reply and the dialogue content corresponding to the second question sample, to obtain a dialogue content generation model.

[0009] In this embodiment, the dialogue content generation model implemented through the above-mentioned training method can generate dialogue content based on the input question, or can generate dialogue content based on the input question, target knowledge and prompt text. The combination of the two dialogue content generation methods is conducive to accurately answering the user's general intention questions and improving the accuracy of the dialogue content and user satisfaction.

[0010] According to a third aspect of an embodiment of the present disclosure, a prompt text generation method is provided, comprising: receiving an input question; acquiring target knowledge corresponding to the question from a preset knowledge base; generating a prompt text based on the question and the target knowledge; wherein the question, the target knowledge and the prompt text are used to be input into a dialogue content generation model to obtain dialogue content, and the prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge.

[0011] In the disclosed embodiment, the target knowledge corresponding to the question can be obtained, thereby generating a prompt text based on the question and the target knowledge. The prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge. Injecting the target knowledge into the dialogue content generation model through the prompt text is conducive to improving the accuracy of the dialogue content generation model's answers.

[0012] According to a fourth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it is used to implement the method described in the first aspect.

[0013] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of any of the above methods are implemented.

[0014] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] FIG1 is a schematic structural diagram of an interactive system provided by an exemplary embodiment.

[0016] FIG2 is a flowchart of a method for training a conversation content generation model provided by an exemplary embodiment.

[0017] FIG3 is a schematic diagram of a dish and its attributes provided by an exemplary embodiment.

[0018] FIG4 is a schematic diagram of a knowledge graph provided by an exemplary embodiment.

[0019] FIG5 is a flowchart of a method for generating conversation content provided by an exemplary embodiment.

[0020] FIG6 is a schematic diagram of displaying conversation content and a jump mark of a sales object in the conversation content in an interactive interface, provided by an exemplary embodiment.

[0021] FIG. 7 is a schematic diagram of jumping from a conversation content display page to a sales page of a sales object, provided by an exemplary embodiment.

[0022] FIG8 is a flowchart of a method for generating prompt text provided by an exemplary embodiment.

[0023] FIG9 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0025] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0026] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0027] Here is an explanation of the relevant terms:

[0028] LLM: Large Language Model (LLM) is a modeling approach based on statistics and natural language processing techniques. It utilizes high-capacity model architectures (such as the Transformer) and is trained on massive amounts of text. This pre-training allows it to acquire a wealth of prior knowledge, and model question answering is the process of guiding it to provide relevant knowledge. Common LLMs include ChatGLM, Chinese-Alpaca, Ziya-Llama, and Baichuan.

[0029] SFT: Supervised Finetuning, which obtains a smaller but higher quality dataset to further train the pre-trained language model. This dataset can appear in the original data or can be samples that the model has not seen.

[0030] Prompt: A prompt can be understood as a way to specify the focus of LLM generation. It is a piece of text or sentence that guides the LLM to generate output of a specific type, theme, or format. Using carefully designed prompts to guide a base model in the desired prior direction is currently the most accessible method, sometimes also referred to as gradient-free tuning. Prompts leverage the associations and prior information seen during training. For example, a prompt like "You are a food expert..." can be used to make the LLM output answers more inclined towards the food dimension.

[0031] In related technologies, the interaction between users and APPs (Applications) is mainly based on search or recommendation. For example, users enter search terms in the search box according to their needs, and the APP pushes the corresponding target objects to the user based on the search terms. For example, if you type "hot and sour noodles" in the search box, the APP will push stores that sell hot and sour noodles. For another example, the APP will use user profiles and historical behaviors as a reference to pre-determine the objects to be recommended and display them in the interactive interface for users to choose. For example, if a user's user profile is that they like spicy dishes, then after the user opens the APP, the APP can display Sichuan and Chongqing hot pot, spicy hot pot, or spicy fried pork and other dishes or stores in the interactive interface for users to choose.

[0032] The above method can meet user needs to a certain extent, but it is weak in taking on general intents. For example, when a user asks a question with unclear intentions such as "What foods can I eat to lose weight?", "What should I eat for breakfast?", "I don't want to eat rice, what other recommendations are there?", for search, the request is long and not specific enough, resulting in a low relevance of the search results to the question and inaccurate search results. For recommendations, user portraits and historical behaviors are more considered as references, and it is not possible to make real-time and targeted recommendations to users' immediate demands that are different from historical behaviors, and the recommendation results are not accurate enough.

[0033] Based on this, embodiments of the present disclosure provide a method for generating conversation content and a method for training a conversation content generation model. This method can fine-tune a pretrained language model (i.e., the aforementioned large language model) to obtain a conversation content generation model. In actual application, after receiving a user's question, it is first determined whether the question requires an answer based on external knowledge. If not, the question is input into the conversation content generation model for answering. If it does, target knowledge is obtained from a pre-set knowledge base, and the question, target knowledge, and prompt text corresponding to the target knowledge are input into the conversation content generation model for answering. This embodiment takes into account that the method of answering input questions solely by the conversation content generation model cannot obtain some external knowledge in real time, and may have drawbacks such as a lack of factual knowledge and an inability to handle time-sensitive or open-ended questions. Therefore, to further improve the accuracy of responses, the present disclosure, when determining that the question requires external knowledge, injects the target knowledge into the conversation content generation model through prompt text, helping the conversation content generation model acquire more external knowledge. The combination of these two conversation content generation methods facilitates accurate answers to users' general-purpose questions, improving conversation content accuracy and user satisfaction.

[0034] Please refer to Figure 1, which shows an interactive system provided by an exemplary embodiment of the present disclosure. The interactive system includes a server 100 and at least one client 200. Exemplarily, the client 200 can access the server 100 through the network to use the services provided by the server 100, including but not limited to product delivery services, product purchase services, reading services, audio and video playback services, or search services, etc.

[0035] The server 100 may be a program installed in a backend device that provides services to users. For example, as shown in FIG1 , the backend device may be a server, which may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0036] Client 200 can be a program installed on a user device that provides services to the user. Client 200 includes, but is not limited to, application programs, web pages, applets, plug-ins, or components. As shown in Figure 1 , user devices include, but are not limited to, smartphones, personal digital assistants, tablets, personal computers, laptops, virtual reality devices, augmented reality devices, and the like.

[0037] The conversation content generation method provided in the embodiment of the present disclosure can be executed by either the server 100 or the client 200, or by both of them working together (each of them executing a part of the steps), and this embodiment does not impose any restrictions on this. Taking the server 100 executing the conversation content generation method as an example, the server 100 can be deployed with a conversation content generation model, the client 200 can send the user input question to the server 100, the server 100 can execute the conversation content generation method provided in the embodiment of the present disclosure to obtain the conversation content, and then can send the conversation content to the client 200 so that the client 200 can display the conversation content. Alternatively, the conversation content generation model can be deployed in the client 200, and the client 200 can execute the conversation content generation method.

[0038] The training method for the conversation content generation model provided in the embodiment of the present disclosure may be executed by the server 100 , or may be executed by other electronic devices different from the server 100 , and this embodiment does not impose any restrictions on this.

[0039] In one example, the conversation content generation method and the conversation content generation model training method provided by the embodiments of the present disclosure can be executed by the same electronic device, or can also be executed by different electronic devices, and the present disclosure does not impose any restrictions on this.

[0040] The following is an example of the training process of the dialogue content generation model:

[0041] Referring to FIG. 2 , an embodiment of the present disclosure provides a method for training a conversation content generation model, including steps S101 to S103 .

[0042] In S101 , a first training data set is obtained, where the first training data set includes a number of question samples and their corresponding conversation contents; the number of question samples includes a number of first question samples and a number of second question samples.

[0043] Exemplarily, the first question sample is a question that does not require reference to external knowledge, and the second question sample is a question that requires reference to external knowledge. Specific questions can be set according to actual application scenarios, and this embodiment does not impose any restrictions on this.

[0044] In S102, for the second question sample, target knowledge is acquired from a preset knowledge base according to the second question sample, and a prompt text corresponding to the target knowledge is generated.

[0045] For example, a knowledge base can be pre-set, and target knowledge for answering the second question sample can be retrieved from the knowledge base. The prompt text can include role prompt text and dialogue content prompt text. The role prompt text instructs the pre-trained language model to answer from the perspective of a specified role (such as a gourmet, product reviewer, etc.); the dialogue content prompt text instructs the pre-trained language model to respond to the second question sample with reference to the target knowledge. In other words, the prompt text can control the specific form of knowledge injection, allowing the pre-trained language model to respond in the desired manner.

[0046] In S103, the first question sample is input into a pre-trained language model so that the pre-trained language model generates a first predicted answer based on the first question sample; and the second question sample, the target knowledge corresponding to the second question sample, and the prompt text corresponding to the target knowledge are input into the pre-trained language model so that the pre-trained language model generates a second predicted answer based on the second question sample, the target knowledge, and the prompt text; the prompt text is used to prompt the pre-trained language model to answer the second question sample with reference to the target knowledge; the pre-trained language model is trained with the optimization goals of minimizing the error between the first predicted answer and the dialogue content corresponding to the first question sample, and minimizing the error between the second predicted answer and the dialogue content corresponding to the second question sample, to obtain a dialogue content generation model.

[0047] In this embodiment, the pre-trained language model is fine-tuned using the following two types of training data: (1) a first question sample and its corresponding conversation content; and (2) a second question sample, the target knowledge corresponding to the second question sample, the prompt text corresponding to the target knowledge, and the conversation content corresponding to the second question sample. This not only internalizes the knowledge model but also supplements it with external knowledge, thereby improving the accuracy of responses. The introduction of the second method helps the pre-trained language model acquire more external knowledge and solves the problem of real-time knowledge injection. The combination of the two methods is conducive to accurately answering users' general intent questions, improving the accuracy of conversation content and user satisfaction.

[0048] In some embodiments, the first training sample set (several question samples and their corresponding conversation contents) is exemplarily described as follows:

[0049] The question samples included in the first training data set can be divided into two categories. That is, the first training sample set can include (1) a number of first question samples and their corresponding dialogue content; (2) a number of second question samples and their corresponding dialogue content (also known as "question-answer pairs"). The first question samples are questions that do not require reference to external knowledge, and the second question samples are questions that require reference to external knowledge.

[0050] For example, question samples in the first training sample set can be constructed based on historical questions raised by real users on the platform, and the distribution of question samples can be made the same or similar to the distribution of questions raised by real users, and the number of different types of question samples can be balanced, so that the trained dialogue content generation model can cope with different types of actual problems.

[0051] For example, a question template can be prepared, and then similar questions can be expanded on the question template to obtain more question samples. For example, if there is a question template called "what to eat for breakfast", after similar questions are expanded, sample questions such as "what to eat for lunch" and "what to eat for dinner" can be obtained.

[0052] In one possible implementation, an existing knowledge graph can be used to construct sample questions and their conversation content. A knowledge graph can be constructed based on the sales object and its attributes. Nodes in the knowledge graph include at least one entity node and at least one attribute node. Each entity node represents the sales object, and at least one attribute node connected to each entity node represents the attributes of the sales object.

[0053] For example, taking dishes as the sales object, please refer to Figure 3. A dish can contain 16 attributes, including {category, related dishes, solar terms, cities, flavors, time periods, cooking methods, scenes, effects, ingredients, seasons, regions, festivals, provinces, moods, and crowds}; taking dumplings as an example, a knowledge graph as shown in Figure 4 can be constructed based on the dish and its attributes.

[0054] For example, if the question sample is "Recommend spicy tofu to me", you can find the entity node related to "spicy tofu" in the knowledge graph, and generate the conversation content corresponding to the question sample based on the connection relationship between the entity node and other nodes.

[0055] In another possible implementation, a pre-trained language model can be used to obtain the conversation content corresponding to the question sample. For example, the question sample and the prompt text corresponding to the question sample can be input into the pre-trained language model so that the language model can output the expected conversation content.

[0056] In another possible implementation, an external search engine may be used to obtain the conversation content corresponding to the question sample, which is not limited in this embodiment.

[0057] In some embodiments, the knowledge base is exemplarily described here:

[0058] A knowledge base can store only one type of knowledge, such as a structured knowledge graph. Alternatively, to improve the accuracy and diversity of responses, the knowledge base can include at least two knowledge sub-bases of different types, for example, storing a structured knowledge graph and at least one of unstructured content such as text, images, video, and audio.

[0059] Exemplarily, the at least two knowledge sub-bases may include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base.

[0060] Referring to Figure 4 , the first knowledge sub-base may include a knowledge graph. Nodes in the knowledge graph include at least one entity node and at least one attribute node. Any entity node represents a sales object, and at least one attribute node connected to any entity node represents the attributes of the sales object. Referring to Figure 3 , taking a dish as an example, a dish may include 16 attributes: {category, dish, solar term, city, flavor, time of day, cooking method, scene, efficacy, ingredients, season, region, festival, province, mood, and crowd}. Of course, fewer or more attributes may be included, and this embodiment imposes no limitation on this.

[0061] In one possible implementation, for missing edges in the knowledge graph, pre-trained language models can be used to construct relationships between dishes and attributes, or knowledge retrieved from external search engines can be used to construct relationships between dishes and attributes. For example, for the term "crowd," where the original knowledge graph lacks a connection between a group of people and dishes, a pre-trained language model and at least one source of encyclopedic knowledge can be used to infer edges, such as the relationship between the elderly group → soft food → porridge → xx porridge, or the relationship between dieters → low-fat → low-fat ingredients and cooking method combination → xx salad, thereby constructing a connection between the group of people and dishes.

[0062] In another possible implementation, the existing connection relationships between entity nodes and attribute nodes in the knowledge graph can also be calibrated with the responses of other pre-trained language models or the search content of external search engines to improve accuracy.

[0063] The second knowledge sub-base may contain multiple popular science texts corresponding to different sales objects, such as recipes or origins of different dishes (such as the method of making spicy tofu, the origin of rice dumplings, etc.), or the parts composition, materials or origins of different commodities (such as the material of clothing, the parts contained in car models), etc., which can be obtained from relevant popular science websites.

[0064] The third knowledge sub-base may include a plurality of texts corresponding to different sales targets and their text vectors. Exemplarily, the text vectors included in the third knowledge sub-base are obtained by inputting the plurality of texts corresponding to different sales targets into a pre-trained vector generation model for conversion.

[0065] In one example, the vector generation model is trained in the following manner: a second training data set is obtained, the second training data set including a number of third question samples, wherein a portion of the third question samples carry correct dialogue content, and another portion of the third question samples carry incorrect dialogue content; the second training data set is input into the vector generation model, and the vector generation model converts each third question sample and its answer text into a text vector, and the vector generation model is trained with the optimization goal of maximizing the similarity between the text vector of the third question sample and the text vector of the correct dialogue content, and minimizing the similarity between the text vector of the third question sample and the text vector of the incorrect dialogue content. In other words, the trained vector generation model can make the text vector of the question and the text vector of the correct answer as similar as possible, and the text vector of the question and the text vector of the incorrect answer as different as possible, so that some questions with unknown intent can be answered from a vector perspective.

[0066] For example, in a dish recommendation scenario, the third question sample could be an ambiguous question such as "I don't know what to eat" or "I feel sick and don't want to eat anymore." The corresponding correct responses might be "Then I recommend you eat some XX porridge. It's nutritious and delicious...", while the corresponding incorrect responses might be "The weather is just perfect today, the temperature is 30°C...". The trained vector generation model ensures that the text vectors for the question "I feel sick and don't want to eat anymore" and the correct response "Then I recommend you eat some XX porridge. It's nutritious and delicious..." are as similar as possible, meaning the distance between the two text vectors is as short as possible. Meanwhile, the text vectors for the question "I feel sick and don't want to eat anymore" and the incorrect response "The weather is just perfect today, the temperature is 30°C..." are as different as possible, meaning the distance between the two text vectors is as far as possible.

[0067] In some embodiments, for a second question sample requiring external knowledge, target knowledge may be acquired from a preset knowledge base according to the second question sample, and a prompt text corresponding to the target knowledge may be generated.

[0068] In a possible implementation, the at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base. Target knowledge related to the second question sample can be acquired from all knowledge sub-bases included in the knowledge base.

[0069] In another possible implementation, intent recognition can be performed on the second question sample to obtain intent information; and slot extraction can be performed on the second question sample to obtain slot information; based on the intent information and the slot information, a target knowledge sub-base is determined from at least two knowledge sub-bases, and target knowledge corresponding to the second question sample is obtained from the target knowledge sub-base.

[0070] For example, the intent information may include but is not limited to "recommendation intent", "popular science intent" and "unknown intent", and the slot information may include but is not limited to the sales object and the attribute information of the sales object.

[0071] For example, intent recognition refers to determining the user's intent or needs based on the given text content. This can be done using a pre-trained intent recognition model. The training process of the intent recognition model involves the following steps:

[0072] (1) Data preparation: First, you need to prepare a labeled training dataset. The training dataset should contain multiple text samples and corresponding intent labels.

[0073] (2) Feature extraction: Convert text samples into feature representations that can be processed by machine learning algorithms. For example, use the bag-of-words model or TF-IDF method to convert text into vector representations. Alternatively, use word embeddings (such as Word2Vec and GloVe) for more advanced feature representations.

[0074] (3) Model selection and training: Select a machine learning algorithm or deep learning model suitable for the intent recognition task. Examples of machine learning algorithms include Naive Bayes, Support Vector Machine (SVM), Decision Tree, and Random Forest. Examples of deep learning models include Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and Convolutional Neural Network (CNN). Use the training data to train the selected model.

[0075] (4) Model evaluation and tuning: Use the evaluation dataset to evaluate the performance of the trained model and calculate the model's accuracy, recall, F1 value, and other indicators on the intent recognition task. For models with suboptimal performance, you can try adjusting hyperparameters, increasing the amount of data, and using regularization methods to optimize them. This will result in an intent recognition model.

[0076] After the intent recognition model is trained, the model will output the predicted intent label based on the input question.

[0077] For example, slot extraction refers to the process of identifying and extracting specific information from a given text. In natural language processing, slots typically represent key information or parameters in the text. Slot extraction can be achieved in the following ways:

[0078] (1) Rule-based approach: Use predefined rules or regular expressions to match and extract text information of specific patterns, such as phone numbers, dates, addresses, dishes, dish attributes, etc. (see Figure 3).

[0079] (2) Machine Learning Methods: Use machine learning algorithms to train models and classify text sequences into different slot types. Common methods include sequence labeling models such as conditional random fields (CRFs) or recurrent neural networks (RNNs), as well as deep learning-based models such as BERT and BiLSTM.

[0080] (3) Pre-trained model method: Use a model that has been pre-trained on a large-scale corpus to perform slot extraction through fine-tuning or transfer learning.

[0081] Exemplarily, a comprehensive model may be pre-trained through machine learning methods for use in intent recognition and slot extraction, and this embodiment does not impose any restrictions on this.

[0082] After obtaining the intent information and slot information of the second question sample, the category can be determined based on the intent information and slot information. The category indicated by the intent information and slot information can be determined in the following manner:

[0083] (1) Rule matching: Define a series of rules and determine the category based on the degree of match between intent information and slot information and the rules. The rules can be mappings between different combinations of intent information and slot information and different categories.

[0084] (2) Machine Learning Classifier: Use machine learning methods (such as Naive Bayes, Support Vector Machine, Random Forest, etc.) or deep learning models (such as Convolutional Neural Network, Recurrent Neural Network, etc.) to build an intent classifier. Convert intent information and slot information into feature vectors, and then train the classifier for classification.

[0085] (3) Hybrid approach: Combining rule matching and machine learning classifiers, rule matching is used to quickly filter out some clear intent categories, and then the remaining text is classified using machine learning classifiers. This approach can improve the efficiency and accuracy of classification.

[0086] (4) Classification based on language models: Use pre-trained language models (such as BERT, GPT, etc.) to perform text classification. Take intent information and slot information as input, and use the representation generated by the language model to perform classification judgment.

[0087] In the first possible case, if the intent information and the slot information indicate the first category, the first category represents the recommendation of a sales object having the attributes specified by the slot information, and the first knowledge sub-base is determined as the target knowledge sub-base; then, the target node corresponding to the slot information can be searched in the knowledge graph, and the target knowledge corresponding to the second question sample can be obtained based on the connection relationship between the target node and other nodes.

[0088] For example, assuming that the second question sample is "What are the spicy ones?", the intent information is "recommended dishes", and the slot information is "spicy". After determining that the first knowledge sub-base is the target knowledge sub-base, you can search for the target node corresponding to "spicy" in the knowledge graph, and further obtain dishes related to "spicy" based on the connection relationship between the target node and other nodes, such as "spicy tofu", "spicy rabbit head" and "stir-fried pork with peppers", etc. According to the connection relationship between the entity node where the dish is located and other attribute nodes, you can further obtain other attribute information as target knowledge.

[0089] In the second possible situation, if the intention information and the slot information indicate a second category, the second category represents popular science on the composition or source of the sales object, and the second knowledge sub-base is determined as the target knowledge sub-base; then, the popular science text corresponding to the sales object indicated by the slot information can be searched in the second knowledge sub-base as the target knowledge corresponding to the second question sample.

[0090] For example, assuming that the second question sample is "How to make Mapo Tofu", the intent information is "Introduce the recipe", and the slot information is "Mapo Tofu", after determining that the second knowledge sub-base is the target knowledge sub-base, you can search for the popular science text corresponding to "Mapo Tofu" as the target knowledge corresponding to the second question sample.

[0091] In the third possible scenario, if the intent information and slot information indicate that the object does not belong to either the first category or the second category, for example, if the intent information is "unknown intent" and there is no slot information, the third knowledge sub-base can be determined as the target knowledge sub-base to determine the target knowledge from a vector perspective. Furthermore, the text vector corresponding to the second question sample can be obtained based on the above-mentioned vector generation model, and then the similarity between the text vector corresponding to the second question sample and the text vectors of several texts in the third knowledge sub-base is determined, and the texts with a similarity higher than the second preset threshold among the several texts are used as the target knowledge corresponding to the second question sample. Among them, the second preset threshold can be specifically set according to the actual application scenario, and this embodiment does not impose any restrictions on this. For example, the second preset threshold is greater than 90%. This embodiment obtains target knowledge from the third knowledge sub-base, so that some questions of the user whose intentions are not clear enough can be answered, thereby improving the accuracy of the answer and user satisfaction.

[0092] This embodiment acquires external knowledge through the above three methods, which can respond to different types of user questions, improve the accuracy and diversity of external knowledge, and thus improve the accuracy and comprehensiveness of subsequent answers.

[0093] After obtaining the target knowledge corresponding to the second question sample, a prompt text corresponding to the target knowledge can be generated. Exemplarily, the prompt text may include a role prompt (role prompt text) and a reply prompt (dialogue content prompt text). The role prompt instructs the pre-trained language model to answer from the perspective of a specified role (such as a gourmet, a product reviewer, etc.); the reply prompt instructs the pre-trained language model to respond to the second question sample with reference to the target knowledge. In other words, the prompt text can control the specific form of knowledge injection so that the pre-trained language model can answer in the desired manner. For example, the prompt text may be "You are a food expert. For XX questions, please answer based on XX target knowledge."

[0094] In some embodiments, the pre-trained language model refers to a natural language processing model with a huge parameter scale and learning ability, which may include but is not limited to the GPT (Generative Pre-trained Transformer) model and its variant models, the Chinese language model and its variant models, etc.

[0095] In some embodiments, for a first question sample that does not require external knowledge, the first question sample can be directly input into a pre-trained language model, so that the pre-trained language model generates a first predicted answer based on the first question sample; and for a second question sample that requires external knowledge, after obtaining the target knowledge corresponding to the second question sample and the prompt text corresponding to the target knowledge in the above manner, the second question sample, the target knowledge corresponding to the second question sample, and the prompt text corresponding to the target knowledge can be input into the pre-trained language model, so that the pre-trained language model generates a second predicted answer based on the second question sample, the target knowledge, and the prompt text.

[0096] During the training process, an evaluation function needs to be defined to measure the difference or error between the model's prediction result and the true label. This evaluation function is usually called a loss function, which can measure the performance of the model. The loss function can be constructed based on two optimization objectives: (1) minimizing the error between the first predicted answer and the conversation content corresponding to the first question sample; (2) minimizing the error between the second predicted answer and the conversation content corresponding to the second question sample, and then minimizing the loss function through a predetermined optimization method. Common optimization methods include stochastic gradient descent (SGD), Adam, RMSprop, etc. These methods adjust the model parameters so that the loss function gradually decreases, thereby improving the model's predictive ability.

[0097] Furthermore, considering the complexity of natural language, there may be errors in the method of only calculating the loss function. Therefore, the pre-trained language model can also be trained by combining the loss function with the evaluation information. Specifically, after obtaining the first predicted reply and the second predicted reply, the evaluation information for the first predicted reply and the evaluation information for the second predicted reply can be further obtained; then, while minimizing the error between the first predicted reply and the conversation content corresponding to the first question sample, and the error between the second predicted reply and the conversation content corresponding to the second question sample as the optimization goal, the pre-trained language model is trained in combination with the evaluation information for the first predicted reply and the evaluation information for the second predicted reply, so as to further improve the accuracy of the model output results.

[0098] Exemplarily, the evaluation information for the first predicted answer and the evaluation information for the second predicted answer may be input by the user in the interactive interface.

[0099] Exemplarily, the evaluation information includes evaluation results of at least one dimension; the dimension includes at least one of the following: security dimension, response correctness dimension, response diversity dimension, relevance dimension, response format dimension, and platform feature dimension.

[0100] ① The evaluation results of the security dimension are used to indicate whether the first predicted response and the second predicted response comply with the preset regulations.

[0101] For example, assuming that the first predicted response contains sensitive words or banned words that do not comply with the preset regulations, the evaluation result in the security dimension is that it does not comply with the preset regulations.

[0102] ② The evaluation results of the response correctness dimension are used to indicate whether the contents of the first predicted response and the second predicted response are consistent with objective facts.

[0103] For example, suppose the first predicted answer is "Beans can cure cancer", which is obviously not true. Then the evaluation result of the answer correctness dimension is not consistent with the objective facts.

[0104] ③ The evaluation result of the diversity dimension of the response is used to indicate whether there is more than one sales object mentioned in the first predicted response and the second predicted response.

[0105] For example, assuming the first question is "What to eat for breakfast", and the first predicted answer includes multiple dishes such as "soy milk, fried dough sticks, steamed buns, steamed buns, spicy soup", then the evaluation result of the diversity dimension of the answer is consistent with diversity.

[0106] ④ The evaluation result of the correlation dimension is used to indicate the degree of correlation between the first question sample and the first predicted answer, and the degree of correlation between the second question sample and the second predicted answer.

[0107] For example, in a food platform, assuming the first question sample is "What to eat during the Dragon Boat Festival", the first predicted answer is "The Dragon Boat Festival is one of the traditional Chinese festivals, usually with customs such as eating rice dumplings, dragon boat racing, and hanging mugwort. Rice dumplings are one of the most representative foods of the Dragon Boat Festival. They are often made with glutinous rice, meat, egg yolks and other ingredients. They have a sweet and soft taste. Some areas also use bamboo leaves, corn leaves and other plant leaves to wrap rice dumplings to increase the fragrance and nutritional value of rice dumplings. Dragon boat racing is also one of the important activities of the Dragon Boat Festival. It is an activity that integrates sports, culture and entertainment. People will form a fleet Dragon boat races are held at the mouths of rivers, lakes, and seas to commemorate the patriotic poet Qu Yuan. Hanging mugwort is also a traditional custom of the Dragon Boat Festival. Mugwort repels mosquitoes, wards off evil spirits, and prevents disease and improves health. People hang mugwort on doorways, windows, or on their bodies to ward off evil spirits and pray for blessings. Overall, the Dragon Boat Festival's food customs are diverse, with strong local characteristics and cultural connotations. Among them, the first predicted answer's introduction to "dragon boat racing" and "hanging mugwort" is not very relevant to the first question sample, so the assessment result of the first predicted answer in the relevance dimension is 0.3 (indicating 30% relevant content).

[0108] ⑤ The evaluation result of the reply format dimension is used to indicate whether the first predicted reply and the second predicted reply meet the preset text format; the preset text format includes at least one of the total-specific-total format, the total-specific format and the specific-total format.

[0109] For example, in the above example, for the first question sample "What to eat during the Dragon Boat Festival", the first predicted answer is in the total score total format, which meets the preset text format.

[0110] ⑥ The evaluation results of the platform dimension are used to indicate whether the first predicted answer and the second predicted answer meet the scenario characteristics of the platform; for example, in a food platform, the answer tends to be given from the perspective of a gourmet; for example, in a medical platform, the answer tends to be given from the perspective of a doctor.

[0111] For example, for the second question sample, "What delicious food is nearby?", the second predicted response is, "Hello! I'm your personal gourmet. Nearby, there are XX rice noodles, XX coconut chicken, XX simple meals, XX food stalls, and XX restaurants. You can filter by region, taste, etc. to find a restaurant that suits you. Enjoy your meal!" This second predicted response is a foodie's perspective on the food platform, meeting the platform's specific needs.

[0112] For example, the pre-trained language model can be fine-tuned in the following three ways:

[0113] (1) Fine-tune the entire pre-trained language model and update the parameters of the entire pre-trained language model.

[0114] (2) Freeze most of the parameters of the pre-trained language model and only fine-tune a relatively small set of model parameters.

[0115] (3) Add new parameters to the embedding layer of the pre-trained language model and before each neural network layer.

[0116] During model training, model parameters can be gradually adjusted through iterative optimization. Each training iteration includes forward propagation, loss function calculation, backward propagation, and parameter update. The number of training iterations is typically set as a fixed number or adjusted dynamically based on the model's performance on the validation set.

[0117] For example, training and tuning can be performed through any combination of different training data, different pre-trained language models, and different fine-tuning methods, ultimately obtaining a dialogue content generation model that meets the requirements.

[0118] In some embodiments, referring to FIG. 5 , an embodiment of the present disclosure further provides a method for generating conversation content, including S201 to S204 .

[0119] In S201 , an input question is received.

[0120] In S202 , it is determined whether the question needs to be answered in combination with external knowledge.

[0121] In S203, if yes, the target knowledge is obtained from a preset knowledge base according to the question, and a prompt text corresponding to the target knowledge is generated; the question, the target knowledge and the prompt text are input into a dialogue content generation model, and the dialogue content generation model generates dialogue content according to the question, the target knowledge and the prompt text; the prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge.

[0122] In S204, if not, the question is input into the dialogue content generation model, and the dialogue content generation model generates dialogue content according to the question; wherein the dialogue content generation model is obtained by fine-tuning the pre-trained language model.

[0123] In this embodiment, considering that answering input questions solely through the conversation content generation model cannot acquire certain external knowledge in real time, there may be drawbacks such as a lack of factual knowledge and an inability to handle time-sensitive or open-ended questions. Therefore, to further improve the accuracy of responses, the present disclosure first determines whether the input question requires the integration of external knowledge. If the question is determined to require external knowledge, the target knowledge is injected into the conversation content generation model via prompt text, helping the conversation content generation model acquire more external knowledge. The combination of these two conversation content generation methods facilitates accurate answers to users' general-purpose questions, improving the accuracy and satisfaction of conversation content.

[0124] For S201, the question may be the question currently input by the user in the interactive interface. Alternatively, if the user has engaged in multiple rounds of dialogue with the electronic device, the question may be a question derived from combining the user's previous questions and the question currently input in the interactive interface during the multiple rounds of dialogue, thereby facilitating the dialogue content generation model to clarify the user's intent.

[0125] Refer to Table 1. In this example, the question sequence is a combination of past questions and the current question. Combining these two questions into the conversation content generation model allows the model to more accurately understand user intent and generate more accurate conversation content.

[0126] Table 1

[0127] Regarding S202, in one example, the dialogue content generation model is obtained by fine-tuning a pre-trained language model based on a first training dataset; the first training dataset includes a plurality of first samples and a plurality of second samples. The first samples include a first question sample and its corresponding dialogue content. The second samples include a second question sample and the target knowledge, prompt text, and dialogue content corresponding to the second question sample; the target knowledge corresponding to the second question sample is obtained from the knowledge base. The first question sample is a question that does not require reference to external knowledge, while the second question sample is a question that requires reference to external knowledge.

[0128] In one possible implementation, when determining whether an input question needs to be answered in conjunction with external knowledge, the semantic similarity between the question and several of the first question samples can be determined; and then, based on the semantic similarity between the question and several of the first question samples, it is determined whether the question needs to be answered in conjunction with external knowledge.

[0129] For example, if the semantic similarity between the question and at least one of the first question samples exceeds a first preset threshold, this indicates that the conversation content generation model has learned knowledge related to the question during the training phase and therefore determines that the question does not require external knowledge to be answered. Otherwise, this indicates that the conversation content generation model lacks knowledge related to the question and determines that the question requires external knowledge to be answered. It is understood that the first preset threshold can be set specifically based on the actual application scenario and is not limited in this embodiment. For example, the preset threshold may be 80%, 85%, or 90%.

[0130] Regarding S203 , after determining that the input question needs to be answered in combination with external knowledge, target knowledge can be acquired from a preset knowledge base according to the question.

[0131] Exemplarily, the knowledge base includes at least two knowledge sub-bases of different types. Intent recognition can be performed on the question to obtain intent information; slot extraction can also be performed on the question to obtain slot information. Based on the intent information and slot information, a target knowledge sub-base is determined from the at least two knowledge sub-bases, and target knowledge corresponding to the question is obtained from the target knowledge sub-base. The implementation of intent recognition and slot extraction can be found in the above description and will not be elaborated here.

[0132] The at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base.

[0133] (1) Please refer to Figure 4. The first knowledge sub-base includes a knowledge graph. The nodes in the knowledge graph include at least one entity node and at least one attribute node. Any entity node is used to represent a sales object, and at least one attribute node connected to any entity node is used to represent the attributes of the sales object.

[0134] (2) The second knowledge sub-base contains a plurality of popular science texts corresponding to different sales objects.

[0135] (3) The third knowledge sub-base contains a number of texts and their text vectors corresponding to different sales targets. The text vectors contained in the third knowledge sub-base are obtained by inputting the texts corresponding to different sales targets into a pre-trained vector generation model for conversion. The training process of the vector generation model can be found in the above description and will not be elaborated here.

[0136] In the first possible case, if the intention information and the slot information indicate a first category, and the first category represents a recommendation for a sales object having the attributes specified by the slot information, then the first knowledge sub-base can be determined as the target knowledge sub-base; and then the target node corresponding to the slot information can be searched in the knowledge graph, and the target knowledge corresponding to the problem can be obtained based on the connection relationship between the target node and other nodes.

[0137] In the second possible situation, if the intention information and the slot information indicate a second category, the second category represents popular science on the composition or source of the sales object, and the second knowledge sub-base is determined as the target knowledge sub-base; then, the popular science text corresponding to the sales object indicated by the slot information can be searched in the second knowledge sub-base as the target knowledge corresponding to the question.

[0138] In the third possible scenario, if the intent information and the slot information indicate that the question does not belong to either the first or second category, the third knowledge sub-base is determined as the target knowledge sub-base, the similarity between the text vector corresponding to the question and the text vectors in the third knowledge sub-base is determined, and the text corresponding to the text vector in the third knowledge sub-base whose similarity exceeds a second preset threshold is used as the target knowledge corresponding to the question. The second preset threshold can be specifically set according to the actual application scenario, and this embodiment does not impose any restrictions on this. For example, the second preset threshold is greater than 90%. This embodiment obtains target knowledge from the third knowledge sub-base, so that some user questions whose intentions are not clear can be answered, thereby improving the accuracy of the answers and user satisfaction.

[0139] This embodiment acquires external knowledge through the above three methods, which can improve the accuracy and diversity of external knowledge, thereby improving the accuracy and comprehensiveness of subsequent responses.

[0140] After obtaining the target knowledge corresponding to the question, a prompt text corresponding to the target knowledge can be generated. Exemplarily, the prompt text may include a role prompt text and a dialogue content prompt text. The role prompt text instructs the dialogue content generation model to answer from the perspective of a specified role (such as a gourmet, a product reviewer, etc.); the dialogue content prompt text instructs the dialogue content generation model to respond to the input question with reference to the target knowledge. In other words, the prompt text can control the specific form of knowledge injection so that the dialogue content generation model can respond in the desired manner. For example, the prompt text may be "You are a food expert. For XX questions, please refer to XX target knowledge to answer."

[0141] Next, the question, the target knowledge, and the prompt text corresponding to the target knowledge can be input into the conversation content generation model, which then generates conversation content based on the question, the target knowledge, and the prompt text. In this embodiment, injecting the target knowledge into the conversation content generation model through the prompt text helps the conversation content generation model acquire more external knowledge, facilitating accurate answers to users' general-intent questions and improving the accuracy and satisfaction of the conversation content.

[0142] For S204, if the input question does not need to be combined with external knowledge, the question can be directly input into the dialogue content generation model, and the dialogue content generation model generates dialogue content based on the question; wherein, the dialogue content generation model is obtained by fine-tuning the pre-trained language model.

[0143] In some embodiments, for security reasons, before determining whether the question requires an answer based on external knowledge, a quality check may be performed on the question to obtain a first quality check result. If the first quality check result satisfies a first quality check condition, step S202 is executed. Otherwise, a prompt message indicating that the question does not meet a preset requirement is displayed in the interactive interface. For example, the first quality check condition indicates that the input question does not contain preset sensitive words or preset banned words, but this is not limited to this.

[0144] In some embodiments, for security reasons, after the conversation content generation model generates the conversation content, a quality check can be performed on the conversation content generated by the conversation content generation model to obtain a second quality check result. If the second quality check result meets a second quality check condition, the conversation content corresponding to the question is displayed in the interactive interface. Exemplarily, the second quality check condition indicates that the conversation content does not contain preset sensitive words or preset banned words, and / or that the conversation content conforms to objective facts; however, this is not limited to these.

[0145] In some embodiments, the conversation content includes at least one of text, images, video, and audio corresponding to the sales target. For example, taking food as an example, refer to Figure 6. The input question is "I want to eat sweet food." The conversation content generation model generates the conversation content shown in Figure 6.

[0146] In some embodiments, to further reduce user operations, after the conversation content is generated by the conversation content generation model, a jump identifier can be set for at least one sales object in the conversation content generated by the conversation content generation model, and then the conversation content and the jump identifier of each of the at least one sales object are displayed in the interactive interface. As shown in Figure 6, in response to the user triggering an operation on any jump identifier in the interactive interface, the sales page of the sales object indicated by the jump identifier can be obtained and displayed in the interactive interface. In this embodiment, the user does not need to exit the page currently displaying the conversation content and then re-search for the sales object in the conversation content, which reduces user operation steps and further facilitates the user.

[0147] Exemplarily, the sales object includes a dish, and the sales page is used to display at least one of at least one store that sells the dish and purchase information for the dish. Exemplarily, the sales object includes a commodity, and the sales page is used to display purchase information for the commodity, such as displaying detailed product information and a purchase control.

[0148] For example, referring to FIG7 , when the user clicks on “Sliced ​​Banana”, the electronic device can respond to the click operation and display stores that include Sliced ​​Banana in the interactive interface, so that the user can select one of the stores to place an order based on their actual needs.

[0149] In some embodiments, referring to FIG8 , the present disclosure also provides a method for generating a prompt text, including:

[0150] In S301 , an input question is received.

[0151] In S302, target knowledge corresponding to the problem is acquired from a preset knowledge base.

[0152] In S303, a prompt text is generated based on the question and the target knowledge; wherein the question, the target knowledge and the prompt text are used to be input into a dialogue content generation model to obtain dialogue content, and the prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge.

[0153] In the disclosed embodiment, the target knowledge corresponding to the question can be obtained, thereby generating a prompt text based on the question and the target knowledge. The prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge. Injecting the target knowledge into the dialogue content generation model through the prompt text is conducive to improving the accuracy of the dialogue content generation model's answers.

[0154] In some embodiments, the prompt text includes role prompt text and dialogue content prompt text; the role prompt text is used to prompt the dialogue content generation model to respond from the perspective of a specified role; the dialogue content prompt text is used to prompt the dialogue content generation model to respond to the question with reference to the target knowledge.

[0155] In some embodiments, before S302 , the process further includes: determining whether the question needs to be answered in combination with external knowledge; if so, executing S302 and S303 .

[0156] In some embodiments, after S303, it also includes: inputting the question, the target knowledge and the prompt text corresponding to the target knowledge into a dialogue content generation model, and the dialogue content generation model generates dialogue content according to the question, the target knowledge and the prompt text; wherein, the dialogue content generation model is obtained by fine-tuning a pre-trained language model.

[0157] In some embodiments, if the question does not require external knowledge to answer, the question is input into the dialogue content generation model, and the dialogue content generation model generates dialogue content based on the question; wherein, the dialogue content generation model is obtained by fine-tuning the pre-trained language model.

[0158] In some embodiments, the knowledge base includes at least two knowledge sub-bases of different types. Acquiring target knowledge corresponding to the question from the preset knowledge base includes: performing intent recognition on the question to obtain intent information; and performing slot extraction on the question to obtain slot information; determining a target knowledge sub-base from the at least two knowledge sub-bases based on the intent information and the slot information, and acquiring the target knowledge corresponding to the question from the target knowledge sub-base.

[0159] In some embodiments, the at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base. The first knowledge sub-base comprises a knowledge graph, wherein the nodes in the knowledge graph comprise at least one entity node and at least one attribute node. Any entity node represents a sales target, and at least one attribute node connected to any entity node represents an attribute of the sales target. The second knowledge sub-base comprises multiple popular science texts corresponding to different sales targets. The third knowledge sub-base comprises multiple texts corresponding to different sales targets and their text vectors.

[0160] In some embodiments, the target knowledge sub-base is determined from the at least two knowledge sub-bases based on the intention information and the slot information, including: if the intention information and the slot information indicate a first category, the first category represents a recommendation of a sales object with the attributes specified by the slot information, and the first knowledge sub-base is determined as the target knowledge sub-base; if the intention information and the slot information indicate a second category, the second category represents a popularization of the composition or source of the sales object, and the second knowledge sub-base is determined as the target knowledge sub-base; if the intention information and the slot information indicate that it does not belong to either the first category or the second category, the third knowledge sub-base is determined as the target knowledge sub-base.

[0161] In some embodiments, obtaining the target knowledge corresponding to the question from the target knowledge sub-base includes: if the target knowledge sub-base is the first knowledge sub-base, searching the knowledge graph for a target node corresponding to the slot information, and obtaining the target knowledge corresponding to the question based on the connection relationship between the target node and other nodes; if the target knowledge sub-base is the second knowledge sub-base, searching the second knowledge sub-base for popular science text corresponding to the sales object indicated by the slot information as the target knowledge corresponding to the question; if the target knowledge sub-base is the third knowledge sub-base, determining the similarity between the text vector corresponding to the question and the text vector in the third knowledge sub-base, and taking the text corresponding to the text vector in the third knowledge sub-base whose similarity is higher than a second preset threshold as the target knowledge corresponding to the question.

[0162] For the specific implementation of the prompt text generation method, please refer to the relevant parts of the above embodiment and will not be repeated here.

[0163] The various technical features in the above embodiments can be combined arbitrarily as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of disclosure of this specification.

[0164] Accordingly, an embodiment of the present disclosure further provides an electronic device, comprising a processor; and a memory for storing processor-executable instructions; wherein the processor implements any one of the above methods by running the executable instructions.

[0165] For example, FIG9 is a schematic structural diagram of an electronic device provided by an exemplary embodiment. Please refer to FIG9 . At the hardware level, the device includes a processor 902, an internal bus 904, a network interface 906, a memory 908, and a non-volatile memory 910. Of course, it may also include hardware required for other services. One or more embodiments of this specification can be implemented based on software, such as the processor 902 reading the corresponding computer program from the non-volatile memory 910 into the memory 908 and then running it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0166] Accordingly, this specification also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0167] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0168] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0169] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0170] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0171] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0172] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "an," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0174] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when..." or "when..." or "in response to determining."

[0175] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of this specification shall be included in the scope of protection of one or more embodiments of this specification.

Claims

1. A method for generating dialogue content, characterized in that: include: The problem of receiving input; Determine whether the question requires external knowledge to answer; If yes, obtaining target knowledge from a preset knowledge base according to the question, and generating a prompt text corresponding to the target knowledge; Inputting the question, the target knowledge and the prompt text into a dialogue content generation model, and allowing the dialogue content generation model to generate dialogue content according to the question, the target knowledge and the prompt text; The prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge; If not, inputting the question into the dialogue content generation model, and allowing the dialogue content generation model to generate dialogue content according to the question; The dialogue content generation model is obtained by fine-tuning the pre-trained language model.

2. The method according to claim 1, characterized in that The dialogue content generation model is obtained by fine-tuning the pre-trained language model based on the first training data set; The first training data set includes a plurality of first samples and a plurality of second samples; The first sample includes a first question sample and its corresponding dialogue content; The second sample includes a second question sample and target knowledge, prompt text and dialogue content corresponding to the second question sample; wherein the target knowledge corresponding to the second question sample is obtained from the knowledge base.

3. The method according to claim 2, characterized in that The determining whether the question needs to be answered in combination with external knowledge includes: Determining semantic similarities between the question and a plurality of the first question samples respectively; According to the semantic similarities between the question and a number of the first question samples, it is determined whether the question needs to be answered in combination with external knowledge.

4. The method according to claim 3, characterized in that The step of judging whether the question needs to be answered in combination with external knowledge according to the semantic similarities between the question and a plurality of the first question samples respectively includes: If the semantic similarity between the question and at least one of the first question samples is higher than a first preset threshold, determining that the question does not need to be answered in combination with external knowledge; Otherwise, it is determined that the question requires an answer in conjunction with external knowledge.

5. The method according to claim 1, characterized in that The knowledge base includes at least two knowledge sub-bases of different types; the step of acquiring target knowledge from the preset knowledge base according to the question includes: Performing intent recognition on the question to obtain intent information; and performing slot extraction on the question to obtain slot information; According to the intention information and the slot information, a target knowledge sub-base is determined from the at least two knowledge sub-bases, and target knowledge corresponding to the question is acquired from the target knowledge sub-base.

6. The method according to claim 5, characterized in that The at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base; The first knowledge sub-base includes a knowledge graph, and the nodes in the knowledge graph include at least one entity node and at least one attribute node, any one of the entity nodes is used to represent a sales object, and at least one of the attribute nodes connected to any one of the entity nodes is used to represent an attribute of the sales object; The second knowledge sub-base includes a plurality of popular science texts corresponding to different sales objects; The third knowledge sub-base contains a number of texts corresponding to different sales objects and their text vectors.

7. The method according to claim 6, characterized in that The determining a target knowledge sub-base from the at least two knowledge sub-bases according to the intention information and the slot information includes: If the intention information and the slot information indicate a first category, the first category represents a recommendation of a sales object having an attribute specified by the slot information, and the first knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate a second category, the second category represents popularizing the composition or source of the sales object, and the second knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate that the knowledge sub-base does not belong to any of the first category and the second category, the third knowledge sub-base is determined as the target knowledge sub-base.

8. The method according to claim 6 or 7, characterized in that: The acquiring target knowledge corresponding to the problem from the target knowledge sub-base includes: If the target knowledge sub-base is the first knowledge sub-base, searching the knowledge graph for a target node corresponding to the slot information, and acquiring target knowledge corresponding to the question according to a connection relationship between the target node and other nodes; If the target knowledge sub-base is the second knowledge sub-base, searching the second knowledge sub-base for a popular science text corresponding to the sales object indicated by the slot information as the target knowledge corresponding to the question; If the target knowledge sub-base is the third knowledge sub-base, determine the similarity between the text vector corresponding to the question and the text vectors of the several texts in the third knowledge sub-base, and take the texts among the several texts whose similarity is higher than a second preset threshold as the target knowledge corresponding to the question.

9. The method according to claim 6, characterized in that The text vectors contained in the third knowledge sub-base are obtained by converting the plurality of texts corresponding to different sales objects into a pre-trained vector generation model; wherein training the vector generation model includes: Acquire a second training data set, wherein the second training data set includes a plurality of third question samples, wherein a portion of the third question samples carry correct dialogue content, and another portion of the third question samples carry incorrect dialogue content; The second training data set is input into the vector generation model, so that the vector generation model converts each of the third question samples and its answer text into text vectors respectively, and the vector generation model is trained with the optimization goals of maximizing the similarity between the text vector of the third question sample and the text vector of the correct dialogue content, and minimizing the similarity between the text vector of the third question sample and the text vector of the incorrect dialogue content.

10. The method according to claim 1, characterized in that The question is the question currently input by the user in the interactive interface; or, The question is a question obtained by combining historical questions input by the user and the question currently input in the interactive interface in multiple rounds of dialogue.

11. The method according to claim 1, characterized in that: Before determining whether the question needs to be answered in combination with external knowledge, the method further includes: Performing a quality inspection on the problem to obtain a first quality inspection result; If the first quality inspection result satisfies a first quality inspection condition, executing a step of determining whether the question needs to be answered in combination with external knowledge, wherein the first quality inspection condition indicates that the question does not contain a preset sensitive word or a preset prohibited word; Otherwise, a prompt message indicating that the problem does not meet the preset requirements is output in the interactive interface; and / or After generating the conversation content, the method further includes: Performing quality inspection on the dialogue content generated by the dialogue content generation model to obtain a second quality inspection result; If the second quality inspection result meets the second quality inspection condition, the conversation content corresponding to the question is displayed in the interactive interface, wherein the second quality inspection condition indicates that the conversation content does not contain preset sensitive words or preset banned words, and / or the conversation content conforms to objective facts.

12. The method according to claim 1, characterized in that The conversation content includes at least one of text, image, video and audio corresponding to the sales object.

13. The method according to claim 1, characterized in that Also includes: Setting a jump mark for at least one sales object in the dialogue content generated by the dialogue content generation model; Displaying the conversation content and the jump mark of each of the at least one sales object in an interactive interface; as well as In response to a triggering operation on any one of the jump identifiers, a sales page of the sales object indicated by the jump identifier is obtained, and the sales page is displayed in the interactive interface.

14. The method according to claim 13, characterized in that The sales object includes at least one of a dish and a commodity, and the sales page is used to display at least one store that includes the dish and at least one of purchase information of the dish or the commodity.

15. A method for training a dialogue content generation model, characterized in that: include: Acquire a first training data set, where the first training data set includes a number of question samples and their corresponding dialogue contents; The plurality of question samples include a plurality of first question samples and a plurality of second question samples; For the second question sample, obtaining target knowledge from a preset knowledge base according to the second question sample, and generating a prompt text corresponding to the target knowledge; Inputting the first question sample into a pre-trained language model so that the pre-trained language model generates a first predicted answer according to the first question sample; and inputting the second question sample, the target knowledge corresponding to the second question sample, and the prompt text corresponding to the target knowledge into the pre-trained language model, so that the pre-trained language model generates a second predicted answer according to the second question sample, the target knowledge, and the prompt text; the prompt text is used to prompt the pre-trained language model to refer to the target knowledge to predict the second question sample. Reply The pre-trained language model is trained with the optimization goals of minimizing the error between the first predicted response and the dialogue content corresponding to the first question sample, and minimizing the error between the second predicted response and the dialogue content corresponding to the second question sample to obtain a dialogue content generation model.

16. The method according to claim 15, characterized in that The training of the pre-trained language model further includes: The pre-trained language model is trained based on the evaluation information for the first predicted response and the evaluation information for the second predicted response.

17. The method according to claim 16, characterized in that The evaluation information includes an evaluation result of at least one dimension; The dimension includes at least one of the following: a security dimension, a response correctness dimension, a response diversity dimension, a relevance dimension, and a response format dimension; The evaluation result of the security dimension is used to indicate whether the first predicted answer and the second predicted answer meet the preset regulations; The evaluation result of the answer correctness dimension is used to indicate whether the contents of the first predicted answer and the second predicted answer are consistent with objective facts; The evaluation result of the diversity dimension of the answer is used to indicate whether there is more than one sales object mentioned in the first predicted answer and the second predicted answer; The evaluation result of the correlation dimension is used to indicate the correlation degree between the first question sample and the first predicted answer, and the correlation degree between the second question sample and the second predicted answer; The evaluation result of the reply format dimension is used to indicate whether the first predicted reply and the second predicted reply meet the preset text format; the preset text format includes at least one of the total-score-total format, the total-score format and the score-total format.

18. The method according to claim 15, characterized in that The knowledge base includes at least two knowledge sub-bases of different types; the step of acquiring target knowledge from a preset knowledge base according to the second question sample includes: performing intent recognition on the second question sample to obtain intent information; and performing slot extraction on the second question sample to obtain slot information; According to the intention information and the slot information, a target knowledge sub-base is determined from the at least two knowledge sub-bases, and target knowledge corresponding to the second question sample is acquired from the target knowledge sub-base.

19. The method according to claim 18, characterized in that The at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base; The first knowledge sub-base includes a knowledge graph, and the nodes in the knowledge graph include at least one entity node and at least one attribute node, any one of the entity nodes is used to represent a sales object, and at least one attribute node connected to any one of the entity nodes is used to represent an attribute of the sales object; The second knowledge sub-base includes a number of popular science texts corresponding to different sales objects; The third knowledge sub-base contains a number of texts corresponding to different sales objects and their text vectors.

20. The method according to claim 19, characterized in that The determining a target knowledge sub-base from the at least two knowledge sub-bases according to the intention information and the slot information includes: If the intention information and the slot information indicate a first category, the first category represents a recommendation of a sales object having an attribute specified by the slot information, and the first knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate a second category, the second category represents popularizing the composition or source of the sales object, and the second knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate that the knowledge sub-base does not belong to any of the first category and the second category, the third knowledge sub-base is determined as the target knowledge sub-base.

21. The method according to claim 19 or 20, characterized in that The acquiring target knowledge corresponding to the second question sample from the target knowledge sub-base includes: If the target knowledge sub-base is the first knowledge sub-base, searching the knowledge graph for a target node corresponding to the slot information, and acquiring target knowledge corresponding to the second question sample according to a connection relationship between the target node and other nodes; If the target knowledge sub-base is the second knowledge sub-base, searching the second knowledge sub-base for a popular science text corresponding to the sales object indicated by the slot information as the target knowledge corresponding to the second question sample; If the target knowledge subbase is the third knowledge subbase, determine the similarities between the text vectors corresponding to the second question sample and the text vectors of the several texts in the third knowledge subbase, and take the texts among the several texts whose similarity is higher than a second preset threshold as the target knowledge corresponding to the second question sample.

22. A method for generating a prompt text, characterized in that: include: The problem of receiving input; Acquire target knowledge corresponding to the problem from a preset knowledge base; Generate a prompt text based on the question and the target knowledge; wherein the question, the target knowledge and the prompt text are used to be input into a dialogue content generation model to obtain dialogue content, and the prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge.

23. The method according to claim 22, characterized in that The prompt text includes a role prompt text and a dialogue content prompt text; The role prompt text is used to prompt the dialogue content generation model to respond from the perspective of the specified role; The dialogue content prompt text is used to prompt the dialogue content generation model to answer the question with reference to the target knowledge.

24. The method according to claim 22, characterized in that The knowledge base includes at least two knowledge sub-bases of different types; The step of acquiring target knowledge corresponding to the problem from a preset knowledge base includes: Performing intent recognition on the question to obtain intent information; and performing slot extraction on the question to obtain slot information; According to the intention information and the slot information, a target knowledge sub-base is determined from the at least two knowledge sub-bases, and target knowledge corresponding to the question is acquired from the target knowledge sub-base.

25. The method according to claim 24, characterized in that The at least two knowledge sub-bases include at least two of a first knowledge sub-base, a second knowledge sub-base, and a third knowledge sub-base; The first knowledge sub-base includes a knowledge graph, and the nodes in the knowledge graph include at least one entity node and at least one attribute node, any one of the entity nodes is used to represent a sales object, and at least one of the attribute nodes connected to any one of the entity nodes is used to represent an attribute of the sales object; The second knowledge sub-base includes a plurality of popular science texts corresponding to different sales objects; The third knowledge sub-base contains a number of texts corresponding to different sales objects and their text vectors.

26. The method according to claim 25, characterized in that The determining a target knowledge sub-base from the at least two knowledge sub-bases according to the intention information and the slot information includes: If the intention information and the slot information indicate a first category, the first category represents a recommendation of a sales object having an attribute specified by the slot information, and the first knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate a second category, the second category represents popularizing the composition or source of the sales object, and the second knowledge sub-base is determined as a target knowledge sub-base; If the intention information and the slot information indicate that the knowledge sub-base does not belong to any of the first category and the second category, the third knowledge sub-base is determined as the target knowledge sub-base.

27. The method according to claim 25 or 26, characterized in that The acquiring target knowledge corresponding to the problem from the target knowledge sub-base includes: If the target knowledge sub-base is the first knowledge sub-base, searching the knowledge graph for a target node corresponding to the slot information, and acquiring target knowledge corresponding to the question according to a connection relationship between the target node and other nodes; If the target knowledge sub-base is the second knowledge sub-base, searching the second knowledge sub-base for a popular science text corresponding to the sales object indicated by the slot information as the target knowledge corresponding to the question; If the target knowledge sub-base is the third knowledge sub-base, determine the similarity between the text vector corresponding to the question and the text vectors of the several texts in the third knowledge sub-base, and take the texts among the several texts whose similarity is higher than a second preset threshold as the target knowledge corresponding to the question.

28. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 27 by running the executable instructions.

29. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the steps of the method according to any one of claims 1 to 27 are implemented.

Images