Content generation, model construction, and data processing

Abstract

Provided are a content generation method and apparatus, a model construction method and apparatus, and a data processing method and apparatus, a device, and a medium. The method comprises: in response to a dialogue initiated by a user, obtaining dialogue data (202); when this round of dialogue is the first round of dialogue, the dialogue data being a current question of the user in this round of dialogue; when this round of dialogue is not the first round of dialogue, the dialogue data comprising a current question of the user and previous historical questions of the user; inputting the dialogue data to an independent question transformation model, so that the independent question transformation model transforms the dialogue data into an independent question that represents a user intention of the user in this round of dialogue (204), wherein training samples used by the model comprise a plurality of rounds of dialogue samples and an independent question sample, and the independent question sample indicates a user intention of a latest round of dialogue sample among the plurality of rounds of dialogue samples; and inputting the transformed independent question to a multi-round dialogue system, so that the multi-round dialogue system generates reply content for this round of dialogue on the basis of the independent question (206). The reply accuracy of the multi-round dialogue system can be improved.

Description

Content generation, model building and data processing Technical Field

[0001] The present disclosure relates to the field of natural language processing technology, and in particular to content generation, model building, data processing methods, devices, equipment and media. Background Art

[0002] With the rapid development of large-scale language models, current dialogue systems possess superior text understanding and conversational capabilities, enabling multi-turn conversations between humans and machines. The key to multi-turn conversations lies in understanding and resolving contextually relevant questions. Multi-turn dialogue systems that support multi-turn conversations have been applied in various fields, including intelligent customer service, intelligent assistants, and chatbots. However, the accuracy of responses generated by existing multi-turn dialogue systems remains to be improved.

[0003] Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides content generation, model construction, data processing methods, devices, equipment and media.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for generating conversation content is provided, comprising:

[0006] In response to a conversation initiated by a user, conversation data is obtained; where, if the current conversation round is the first conversation round, the conversation data includes the user's current question in the current conversation round; if the current conversation round is not the first conversation round, the conversation data includes the user's current question in the current conversation round and the user's historical questions in each historical conversation round of at least one previous conversation round;

[0007] Inputting the conversation data into an independent question conversion model, so that the independent question conversion model converts the conversation data into independent questions that represent the user's intention in the current conversation round; wherein the independent question conversion model is obtained by training a preset language model using a training data set including multiple training samples, each training sample in the training data set includes multiple conversation rounds and an independent question sample, and the independent question sample indicates the user intention corresponding to the most recent conversation round sample in the multiple conversation rounds;

[0008] The transformed independent question is input into the multi-round dialogue system, so that the multi-round dialogue system generates a response content for the current round of dialogue according to the independent question.

[0009] According to a second aspect of an embodiment of the present disclosure, a method for constructing an independent question conversion model is provided. The independent question conversion model is used to generate independent questions based on conversation data to represent the user intention in the latest round of conversation in the conversation data. The method includes:

[0010] Obtaining a training data set comprising a plurality of training samples, each training sample in the training data set comprising a plurality of dialogue round samples and an independent question sample, wherein the independent question sample indicates a user intention corresponding to a latest dialogue round sample in the plurality of dialogue round samples;

[0011] Inputting multiple rounds of conversation samples contained in each training sample in the training data set into a preset language model, so that the language model predicts an independent question representing the user intention of the latest round of conversation sample in the multiple rounds of conversation samples based on the input multiple rounds of conversation samples, and training the language model with minimizing the error between the predicted independent question and the corresponding independent question sample as an optimization goal; and

[0012] Based on the trained language model, the independent question conversion model is obtained. According to a third aspect of an embodiment of the present disclosure, a method for processing dialogue data is provided, comprising:

[0013] In response to a conversation initiated by a user, conversation data is obtained; where, if the current conversation round is the first conversation round, the conversation data includes the user's current question in the current conversation round; if the current conversation round is not the first conversation round, the conversation data includes the user's current question in the current conversation round and the user's historical questions in each historical conversation round of at least one previous conversation round;

[0014] Inputting the conversation data into an independent question conversion model, so that the independent question conversion model converts the conversation data into independent questions representing the user intention in the current round of conversation;

[0015] The independent question conversion model is obtained by training a preset language model using a training data set including multiple training samples, each training sample in the training data set contains multiple rounds of dialogue samples and independent question samples, and the independent question sample indicates the user intention corresponding to the latest round of dialogue samples in the multiple rounds of dialogue samples.

[0016] According to a fourth aspect of an embodiment of the present disclosure, there is provided a device for generating conversation content, including:

[0017] An acquisition module is configured to: acquire conversation data in response to a conversation initiated by a user; wherein, if the current conversation round is the first conversation round, the conversation data includes the user's current question in the current conversation round; if the current conversation round is not the first conversation round, the conversation data includes the user's current question in the current conversation round and the user's historical questions in each historical conversation round of at least one previous conversation round;

[0018] an input module, configured to input the conversation data into an independent question conversion model, so that the independent question conversion model converts the conversation data into independent questions that represent the user's intention in the current conversation round; wherein the independent question conversion model is obtained by training a preset language model using a training data set comprising a plurality of training samples, each training sample in the training data set comprising multiple conversation rounds and an independent question sample, wherein the independent question sample indicates the user's intention corresponding to the most recent conversation round sample in the multiple conversation rounds;

[0019] The answer module is used to input the converted independent question into the multi-round dialogue system, so that the multi-round dialogue system generates the answer content for the current round of dialogue according to the independent question.

[0020] According to a fifth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method embodiment described in the first aspect, the second aspect, or the third aspect are implemented.

[0021] According to a sixth aspect of an embodiment of the present disclosure, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method embodiments described in the first, second, or third aspects are implemented.

[0022] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects.

[0023] In the disclosed embodiment, an independent question conversion model is designed. The independent question conversion model is obtained by training a pre-trained language model using a training dataset. The training samples in the training dataset may include multi-round dialogue samples and independent question samples. The independent question samples may indicate the user intent of the latest dialogue sample in the multi-round dialogue samples. Therefore, the independent question conversion model can convert one or more rounds of user dialogue into independent questions that represent the user intent in the current dialogue. When responding to the user's current dialogue, the independent question converted by the independent question conversion model is input into the downstream multi-round dialogue system. This provides the multi-round dialogue system with a clear indication of the user's intent, allowing the multi-round dialogue system to generate responses that meet the user's intent, thereby improving the accuracy of the multi-round dialogue system's responses.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the disclosure.

[0026] FIG1 is a flowchart of a method for constructing an independent question conversion model according to an exemplary embodiment of the present disclosure.

[0027] FIG2A is a schematic diagram showing a conversation scenario according to an exemplary embodiment of the present disclosure.

[0028] FIG2B is a flowchart showing a method for generating conversation content according to an exemplary embodiment of the present disclosure.

[0029] FIG2C is a flowchart showing a method for processing conversation data according to an exemplary embodiment of the present disclosure.

[0030] FIG3 is a hardware structure diagram of a computer device where a device for generating conversation content is located according to an exemplary embodiment of the present disclosure.

[0031] FIG4 is a block diagram showing a device for generating conversation content according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0033] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0034] It should be understood that although the present disclosure may employ the terms first, second, third, etc. to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. Depending on the context, the term "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0035] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] First, the relevant terms are explained.

[0037] Text generation refers to the use of computer programs to automatically generate text that conforms to specific rules, structure, or language style. This could be a paragraph, an article, an email, or something else. Text generation is typically based on artificial intelligence and natural language processing technologies. By understanding and analyzing the text's language structure, grammatical rules, and lexical semantics, it generates text content that meets expectations. Text generation has a wide range of applications, such as automatically generating news articles, scientific literature, novels, and advertising copy.

[0038] LLM: A Large Language Model (LLM) is a natural language processing model based on deep learning that can be trained using large datasets to generate text that resembles natural language. This model, typically composed of a large number of neural networks, can automatically learn and understand the patterns and structures in language. LLMs can be applied to fields such as machine translation, question-answering systems, and chatbots, generating high-quality text and thus improving the efficiency and accuracy of natural language processing. The most famous LLM is the GPT series (GPT-1, GPT-2, and GPT-3) developed by OpenAI, which has achieved outstanding performance in natural language generation and understanding.

[0039] Multi-turn conversation: refers to an interactive conversation between humans and computers or humans and can include multiple rounds or turns. In a multi-turn conversation, each round usually includes a speaker (such as a user or computer) and a receiver, who can communicate through voice or text until a certain goal is reached or the conversation ends. Multi-turn conversation can be applied to many fields, such as intelligent customer service, intelligent assistants, chatbots, etc. The key to multi-turn conversation is to understand and solve context-related problems, including understanding the conversation history, extracting key information, generating relevant responses, etc., which requires the use of natural language processing and artificial intelligence technologies.

[0040] Text retrieval refers to the automated search and retrieval of text information relevant to user queries or keywords within large text collections. Text retrieval systems typically consist of two main components: query processing and text matching. Query processing involves parsing and processing user queries, typically using natural language processing techniques to convert them into a searchable text representation. Text matching involves comparing the query with text in the text collection, finding the most relevant text, and returning it to the user. Text retrieval systems can be applied in many fields, such as internet search, email filtering, and information management.

[0041] Intent recognition: Intent is a classification abstraction of user instructions. When a user says something to the dialogue system, the dialogue system needs to determine whether the user actually wants to ask about the weather, set an alarm, or something else. Intent recognition is a typical classification task, the purpose of which is to identify the object that the user's search query is looking for. In addition to clear nouns, many queries are ambiguous and may satisfy multiple categories, so the intent module mainly outputs the probability distribution of a category. In addition, text matching can be used to prepare a batch of standard statements for each intent. When the actual user inputs, the user's words are compared with each entry in the statement library for similarity. If the similarity reaches a certain threshold, it means that the current user's words belong to the intent.

[0042] Prompt: A prompt can be understood as a way to specify the focus of LLM generation. It is a piece of text or sentence that guides the LLM to produce output of a specific type, theme, or format. Using carefully designed prompts to guide a base model in the desired prior direction is currently the most accessible approach, sometimes also referred to as gradient-free tuning. Prompts leverage the associations and prior information seen during training. For example, a prompt like "You are a food expert..." can be used to make the LLM output answers more oriented towards the food dimension.

[0043] In practical applications, many conversational tasks require multiple rounds of interaction with users to obtain more detailed and accurate information, allowing the dialogue system to provide more precise answers or solutions. However, current multi-turn dialogue systems face several challenges in specific domains, such as difficulty determining user intent, information redundancy, and interaction complexity. These issues hinder their efficiency and accuracy.

[0044] In an embodiment of the present disclosure, an independent question conversion model is designed. The independent question conversion model is obtained by training a pre-trained language model using a training data set. The training samples in the training data set may include multi-round dialogue samples and independent question samples. The independent question samples may indicate the user intention of the latest dialogue sample in the multi-round dialogue samples. Therefore, the independent question conversion model can convert one or more rounds of dialogue of a user into independent questions that represent the user intention of the user in this round of dialogue. When answering the user's current round of dialogue, the independent question converted by the independent question conversion model is input into the downstream multi-round dialogue system, giving the multi-round dialogue system a clear indication of the user's intention, so that the multi-round dialogue system can generate a reply content that meets the user's intention, thereby improving the accuracy of the reply content of the multi-round dialogue system. The following is a detailed description of the embodiment of the present disclosure.

[0045] The dialogue content generation scheme of the disclosed embodiment can be applied to any interactive system, which includes a server and at least one client. For example, the client can access the server through the network to use the services provided by the server, including but not limited to product delivery services, product purchase services, reading services, audio and video playback services, or search services, etc.

[0046] The server can be a program installed in the background device to provide services to users. The background device can be a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0047] A client is a program installed on a user device that provides services to the user. Clients include but are not limited to application programs (APPs), web pages, mini-programs, plug-ins, or components. User devices include but are not limited to smartphones, personal digital assistants, tablets, personal computers, laptops, virtual reality devices, augmented reality devices, etc.

[0048] The method for generating the conversation content provided by the embodiment of the present disclosure can be executed by either the server or the client, or by the cooperation of the two (each of which performs a part of the steps), and this embodiment does not impose any restrictions on this. Taking the execution on the server as an example, the server can be deployed with an independent question conversion model and a multi-round dialogue system. The client can send the question input by the user to the server. The server can execute the method for generating the conversation content provided by the embodiment of the present disclosure to obtain the reply content, and then send the reply content to the client so that the client can display the reply content. Alternatively, an independent question conversion model and a multi-round dialogue system can be deployed in the client, and the reply content generation method can be executed by the client.

[0049] The training method for the reply content generation model provided in the embodiment of the present disclosure may be executed by the server, or may be executed by other computer devices different from the server, and this embodiment does not impose any restrictions on this.

[0050] In another exemplary embodiment, the method for generating conversation content and the method for constructing an independent question conversion model provided by the embodiment of the present disclosure may be executed by the same computer device, or may be executed by different computer devices, and this embodiment does not impose any restrictions on this.

[0051] The solution of this embodiment can be applied to a variety of conversation scenarios. For example, a multi-turn conversation system can be an online customer service system, an intelligent assistant system, an intelligent question-and-answer system, and so on. Furthermore, a multi-turn conversation system can be a multi-domain conversation system that can handle conversations in a variety of different fields. For example, a multi-domain conversation system can handle user inquiries and exchanges on topics as diverse as weather, movies, and restaurant reservations. A multi-turn conversation system can also be applied to vertical domain conversation systems. Vertical domain conversation systems are conversation systems focused on specific fields or industries, used for tasks and conversation scenarios in specific fields. Such conversation systems can provide more in-depth and precise information and feedback in specific fields because they have a better understanding of the relevant knowledge and content in that field.

[0052] First, an embodiment of a method for constructing an independent question conversion model is described. As shown in FIG1 , FIG1 is a flowchart of a method for constructing an independent question conversion model according to an exemplary embodiment of the present disclosure, which is applicable to a server and / or a client, and includes the following steps 102 to 104 .

[0053] Step 102: Obtain a training data set.

[0054] The training samples in the training dataset include multi-round conversation samples and independent question samples, where the independent question samples indicate the user's intention in the latest round of conversation samples. As an example, the training dataset can be stored in various appropriate formats, such as tables, on the server and / or client.

[0055] Step 104: Input the training samples in the training data set into a preset pre-trained language model, so that the pre-trained language model can predict independent questions that represent the user's intention in the current round of dialogue samples based on the input multi-round dialogue samples, and train the pre-trained language model with the optimization goal of minimizing the error between the predicted independent questions and the independent question samples to obtain an independent question conversion model.

[0056] Exemplarily, a training dataset can be pre-set. The method for obtaining the training dataset is not limited in this embodiment. For example, dialogue (Question Answering, QA) is an enduring task, and large-scale, high-quality datasets are one of the key factors for training and evaluating models. In dialogue QA tasks, massive dialogue datasets are indispensable because they can be used to train and evaluate natural language processing models, thereby improving the accuracy and performance of the models. Massive dialogue datasets may include different types of dialogue data, such as human-computer dialogue, human-to-human dialogue, social media dialogue, etc. These datasets can use real data from people in different fields and backgrounds to ensure the authenticity and diversity of the data. These datasets are usually annotated with useful information, such as dialogue history, intent, answers, emotions, etc. Therefore, open source dialogue datasets can be obtained, and the training dataset of this embodiment can be obtained based on processing one or more open source dialogue datasets.

[0057] The publicly available datasets mentioned above are all mixed together across various fields. If this embodiment is applied to a question-and-answer scenario in a specific vertical field, there may not be a dataset related to the desired vertical field. Therefore, it is possible to filter out a conversation dataset that matches the vertical field from one or more open-source conversation datasets, thereby training an independent question conversion model that performs well in that vertical field.

[0058] As an example, the conversation scenario of this embodiment is applied to the catering / food field of local life services. The conversation data set that conforms to this field can be screened out based on the keywords in the catering field. For example, a vocabulary library of this vertical field can be pre-set, which contains the proprietary keywords of this vertical field. The conversation data that is not related to the keywords in the vocabulary is removed from the existing conversation data set, thereby obtaining a conversation data set that conforms to this vertical field. The current amount of data may be in the tens of millions. Using text matching and regular expressions to filter data may be more complex. Large-scale text data cleaning, searching, or AC (Aho-Corasick, AC automaton) dictionary tree and other solutions can also be used.

[0059] Taking into account the fact that the aforementioned publicly available datasets are applied to different fields than the dialogue scenarios of this embodiment, for example, some of the publicly available datasets are generated based on character dialogues in film and television works (such as movies or TV series), and some datasets are generated based on chat records of users in some forums. The multi-round dialogue data in these datasets may not match the actual dialogue scenarios between users and the multi-round dialogue system in this embodiment. For example, the scheme of this embodiment is applied to a recommendation scenario, in which the user's dialogue requirement is to hope that the dialogue system can recommend objects in a certain field. In the actual dialogue scenario, the user inputs a descriptive text related to the recommended object so that the dialogue system can reply to the relevant recommended object based on the input descriptive text. However, the dialogue scenarios in the publicly available dialogue datasets do not match the above-mentioned recommendation scenarios. If the independent question conversion model trained using these dialogue datasets is used, the prediction effect of the model may not be accurate enough. In order to obtain dialogue data that matches the dialogue scenario of this embodiment and to ensure that the trained model has good prediction results, in some examples, the multi-round dialogue samples can be obtained in the following way:

[0060] Obtain a historical search term sequence, which includes one or more search terms that the user searched for within a historical time period. The search terms are used to search for target objects.

[0061] A first prompt text is generated based on each search word in the historical search word sequence, and the first prompt text is input into a first pre-trained language model, which generates multiple rounds of dialogue corresponding to the historical search word sequence based on the first prompt text; wherein the first prompt text is used to instruct the first pre-trained language model to conduct multiple rounds of dialogue as a user and as a recommendation expert, and the dialogue topics are related to each search word in the historical search word sequence.

[0062] As an example, existing clients interact with users through a search function. The search function can be specifically a search box (of course, there may be other display forms), allowing users to enter search terms to find the required target objects, such as dishes, goods, services or articles, etc. The search box of the client is the interface for users to interact with the platform. When the user enters a search term in the search box, the client will match and search the search term with the relevant information in the object library based on the search engine or a customized search algorithm. The matching results will be displayed to the user in a list format, which may include the name, description, price and other information of the relevant recommended objects.

[0063] The client can store a user's historical search history through logs or other means. This historical search history stores one or more search term sequences that the user has searched for over a period of time. Historical search history is used in related technologies for personalized recommendations. For example, based on a user's search history and behavior data, the client can make personalized recommendations based on the user's interests and preferences.

[0064] However, this embodiment constructs a historical search term sequence based on historical search records to automatically generate multiple rounds of dialogue samples that conform to actual dialogue scenarios. This embodiment limits the historical search term sequence to include search terms entered by the user within a historical set time period. This is because when the user uses the client to search for the desired target object, the user will enter a search term. According to the search results of the client, if it does not meet the user's needs, the user may enter other search terms again until the user finds the desired target object. In this way, the search terms entered by the user multiple times in a relatively short period of time represent the user's search behavior from the beginning to the end of the search. The changes in these search terms also represent the changes and evolution of the user's search intentions. For example, the user may enter a relatively broad search term at the beginning. As the search results are screened and adjusted, the search scope may be gradually narrowed or the search keywords may be changed until a satisfactory target object is found.

[0065] The improvement involves using a multi-turn dialogue system to provide recommended objects to users. The interaction process between the user and the multi-turn dialogue system is actually similar to the search behavior described above. The user may initially ask the multi-turn dialogue system a few questions. As the multi-turn dialogue system responds, the user may gradually adjust the question until the multi-turn dialogue system returns a satisfactory target object. Therefore, this embodiment can construct multi-turn dialogue samples that match the actual dialogue process based on historical search term sequences.

[0066] The pre-trained language model refers to a natural language processing model with a large parameter size and learning capability, and may include but is not limited to the Generative Pre-trained Transformer (GPT) model and its variants, the Chinese language model and its variants, etc. The first pre-trained language model of this embodiment can be flexibly selected according to actual needs, for example, ChatGPT or ChatGLM, etc.

[0067] In some examples, the first prompt text may include one or more of the following information.

[0068] ① Instruct the first pre-trained language model to use role information to conduct conversations as the user and as the recommendation expert respectively.

[0069] For example, the following text can be used: "You are a user, having a conversation with a food recommendation expert." This can be adjusted flexibly as needed in actual applications and is not limited in this embodiment. This can indicate the role the model needs to play and activate the model's professional capabilities in the field of conversation.

[0070] ② Instruct the first pre-trained language model to generate multiple rounds of dialogue, the user's questions in each round of dialogue are related to one or more search terms in the historical search term sequence, and the recommended expert's answers in each round of dialogue are related to the recommended objects corresponding to one or more search terms in the historical search term sequence. Task information.

[0071] Taking the search terms including "fried noodles, vermicelli and vermicelli hot pot, duck blood vermicelli soup, cart noodles" in the historical search word sequence as an example, the text can be as follows: "Given the topics of fried noodles, vermicelli and vermicelli hot pot, duck blood vermicelli soup, cart noodles, multiple rounds of dialogue are generated based on the above content. You need to provide the filtering conditions for food and your opinions on the food recommendation experts' replies. The food recommendation experts will give recommendations based on the filtering conditions you provided." In actual applications, it can be flexibly adjusted as needed, and this embodiment does not limit this. In this way, a clear task definition can be provided to the model so that it can perform the task correctly.

[0072] ③Format information indicating the format of the multi-round dialogue output by the first pre-trained language model.

[0073] In actual applications, configuration can be performed as needed, for example, it can include "each round of dialogue should not exceed 50 words", etc. This embodiment does not limit this.

[0074] Optionally, other additional conditional information may be set as needed, such as "do not include any sensitive information in the conversation", or based on the consideration of recommending specific products, it may be required not to recommend stores such as "do not recommend restaurants and similar content", etc. This embodiment does not limit this.

[0075] As an example, the first prompt text may be as follows:

[0076] The first prompt text is input into the first pre-trained language model, and the first pre-trained language model generates multiple rounds of dialogue. As an example, the multiple rounds of dialogue generated by the first pre-trained language model based on the first prompt text in the table above can be as follows:

[0077] User: Do you have any recommendations for authentic fried noodles?

[0078] Expert: There are many types of fried noodles. What flavor do you need?

[0079] User: I want to try Guangdong fried noodles. Do you have any recommendations?

[0080] Expert: The more famous fried noodles in Guangdong are those in Guangzhou and Shenzhen. You can consider these two.

[0081] User: I like eating vermicelli. Do you have any recommendations for delicious vermicelli casserole?

[0082] Expert: Vermicelli stew is quite famous in Hunan. You can try pepper vermicelli stew or fish head vermicelli stew.

[0083] User: I heard that duck blood vermicelli soup is very good. Do you have any recommendations?

[0084] Expert: Duck blood vermicelli soup is a specialty food in Nanjing, Jiangsu. The soup is fresh and delicious. You can try it.

[0085] User: I want to eat a bowl of authentic cart noodles. How do I choose?

[0086] Expert: It is best to try cart noodles at street stalls in Hong Kong or Guangdong to ensure the authentic taste.

[0087] In practical applications, if the correlation between search terms in a historical search term sequence is poor, the second pre-trained language model may not be able to generate coherent multi-round conversations. This is because the language model generates responses based on statistics and contextual understanding. If there is no obvious connection or context between historical search terms, the model may have difficulty accurately capturing user intent and generating coherent responses. For example, the search terms "hot pot" and "coffee" in a historical search term sequence represent two different food and beverage categories and are unlikely to be mentioned simultaneously in the same context or topic. Therefore, in the multi-round conversation generated by the language model, the QA dialogue corresponding to the two queries is completely disconnected.

[0088] For sequences with strong correlation, such as the search term sequence containing "hen" and "old hen", the generated dialogue context has strong continuity.

[0089] In the actual user usage scenario, when the dialogue system asks questions, there are two common dialogue modes:

[0090] 1. Single topic in-depth study; that is, strong correlation with the query;

[0091] 2. Multi-topic jump; that is, weak query correlation.

[0092] The proportion of these two types of data in the training data may affect the model's effectiveness. For example, the second scenario is a multi-topic task, while the task of the independent question conversion model based on this embodiment primarily focuses on the user intent in the latest round of multi-round conversations. That is, the independent question conversion model primarily retains the topic of the latest round of conversations, making multi-topic tasks simpler for the independent question conversion model. The first conversation scenario, on the other hand, is slightly more challenging for the independent question conversion model. It can utilize single-topic, multi-round conversations generated from historical search term sequences that are strongly correlated with the query. Therefore, the proportion of training data for strongly correlated queries should be higher, for example, set to 80%. That is, in the training dataset, the number of in-depth conversation data samples on a single topic is relatively large, while the number of conversation data samples that jump from multiple topics is relatively small. The number of in-depth conversation data samples on a single topic is greater than the number of conversation data samples that jump from multiple topics. Optionally, in-depth conversation data samples on a single topic can be generated from sequences containing multiple search terms belonging to the same topic; conversation data samples that jump from multiple topics can be generated from sequences containing multiple search terms belonging to different topics.

[0093] In some examples, the historical search term sequence includes multiple search terms arranged in chronological order of search time. The historical search term sequence refers to a sequence in which the overall correlation between the multiple search terms satisfies a set correlation condition.

[0094] The overall correlation between each search term in multiple search terms is determined based on the average of the search term correlations between each pair of adjacent search terms in the multiple search terms, and the search term correlation between each pair of adjacent search terms is determined based on the weighted value of the text similarity between the pair of search terms and the weighted value of the domain similarity between the pair of search terms, wherein the weight of the domain similarity is higher than the weight of the text similarity.

[0095] The domain of a search term refers to a specific topic, field, or industry related to the search term. For example, considering a single-topic in-depth scenario in a conversation, this embodiment can set a higher weight for domain similarity than for text similarity. This increases the probability that each search term in a historical search term sequence belongs to the same domain, and can subsequently generate single-topic in-depth conversation data based on this.

[0096] In some examples, the text similarity between a pair of adjacent search terms is the difference between the edit distance between the pair of adjacent search terms and the target string length, where the target string length refers to the string length of the search term with the longer string length in the pair of adjacent search terms.

[0097] Among them, the edit distance refers to the minimum number of operations required to convert one string into another string, including inserting, deleting, and replacing characters. In some examples, when each operation occurs, the resulting distance can be set to 1.

[0098] Taking the strings "milk tea" and "coconut coffee" as an example, the edit distance between them is 4. The following is a possible sequence of edit operations to convert "milk tea" into "coconut coffee":

[0099] 1. Replace the character '奶' -> '生'

[0100] 2. Replace the character '茶' -> '椰'

[0101] 3. Insert the character '咖'

[0102] 4. Insert the character '啡'

[0103] Therefore, 4 operations are required to convert "milk tea" into "coconut coffee", so their edit distance is 4.

[0104] In some examples, the target object includes products, and the domain similarity between a pair of search terms can refer to the category similarity of the products corresponding to the pair of search terms respectively.

[0105] As an example, the overall relevance of the historical search term sequence can be expressed by the following formula:

[0106] relation_score is the overall relevance, which is obtained by dividing the sum of the search term relevance between each pair of adjacent search terms by the number of pairs of adjacent search terms in the historical search term sequence.

[0107] relevance(q i ,q i+1 ) is the search term relevance between a pair of adjacent search terms (search term q i and search term q i+1 ).

[0108] query_seq refers to the number of search terms in the historical search term sequence, and query_seq minus 1 is the number of pairs of adjacent search terms in the sequence.

[0109] The calculation method of relevance(q i ,q i+1 ) can be: r * (text_sim(q i ,q i+1 )) + (1 - r) * domain_sim(q i ,q i+1 )

[0110] Wherein, r is a weight, which can be configured according to actual needs; wherein, in this embodiment, the weight of domain similarity can be higher than the weight of text similarity, and can be flexibly configured according to needs in actual applications. wherein, the range of r can be set between 0 and 1, and the value of r can be flexibly set in actual applications, and different sequences can be obtained by setting different values ​​of r. for example, r can be set to a value such as 0.1, so that the weight of text similarity is 0.1 and the weight of domain similarity is 0.9, so that a sequence with low text similarity but high domain similarity can be obtained. In such a sequence, each query is highly concentrated in the same field but involves different topics, belonging to a multi-topic sequence in the same field, and these sequences can be used to generate multi-topic conversation data in the same field. r can also be a value such as 0.5, that is, the difference between text similarity and domain similarity is small, and the queries in the sequence obtained in this way belong to the same field and the same topic, that is, a single-topic sequence in the same field, and these sequences can be used to generate conversation data in the same field and a single topic.

[0111] text_sim(q i ,q i+1 ) refers to the search term q i and search term q i+1 The text similarity between , as an example, can be calculated as follows: m=max(len(q i ),len(q i+1 ))

[0112] Among them, len(q i ) refers to the search term q i The length of the string is m, which is the target string length (the length of the longer search term in a pair of adjacent search terms). Where distance is less than or equal to m.

[0113] Among them, distance(q i ,q i+1 ) refers to the search term q i and search term q i+1 The edit distance between .

[0114] Among them, domain_sim(q i ,q i+1 ) refers to the search term q i and search term q i+1 The domain similarity between the two search terms is determined by the category similarity of the target object corresponding to the search term. If the categories are the same, the value is 1; if the categories are different, the value is 0.

[0115] In practical applications, multiple historical search term sequences can be obtained from the client's search behavior records. The overall relevance (relation_score) of each historical search term sequence can be calculated. Relevance conditions can be set as needed, for example, by sorting in descending order based on relation_score. The required n historical search term sequences can be extracted according to set rules to generate multi-round conversation data. For example, a set number n can be used to select the first n historical search term sequences. Optionally, an overall relevance threshold can be set to select historical search term sequences with a relevance greater than the set threshold, and so on. This embodiment does not limit this.

[0116] As mentioned above, one of the application scenarios of this embodiment is the local life service field. In this field, the client functions provided by the business party to users may include product services in the catering field, product services in the pharmaceutical field, and product services in the retail field, etc.; after research, it was found that in fields such as pharmaceuticals or retail, product names usually have a high degree of similarity. This is because these fields have certain standards or specifications for product names, just like the same medicine, although it has different manufacturers, the drug name is usually the same. However, the catering field is completely different. The names of catering products sold in catering stores are customized by the merchants, so the names of catering products usually have a low degree of similarity. Since the client provides a variety of product services for different commodities, the historical search term sequences obtained from the client's historical search records may contain search terms from different fields. In order to apply the independent question conversion model to recommend target objects in the catering field, in actual applications, before using the aforementioned historical search term sequences to generate conversation data, the historical search term sequences obtained from the client's historical search records can be filtered through various methods such as manual labor. In order to reduce labor costs, based on the above analysis, this embodiment provides a method for filtering based on text similarity between search terms, from which historical search term sequences containing search terms in the catering field can be filtered. In other examples, text similarity can also be used to filter out search terms in the catering field, and search term sequences in fields such as medicine or retail with high similarity in product names can be excluded first, thereby reducing labor costs. For example, for each historical search term sequence obtained from the client's historical search records, the text similarity between each pair of adjacent search terms in each historical search term sequence can be obtained, and the overall text similarity of the sequence can be calculated based on the text similarity between each pair of adjacent search terms. The overall text similarity can be calculated by dividing the sum of the text similarities text_sim between each pair of adjacent search terms by the logarithm of the adjacent search terms in the sequence; historical search term sequences with an overall text similarity greater than a set threshold are most likely search term sequences in fields such as medicine or retail, and these sequences can be deleted.

[0117] Optionally, in actual applications, the historical search word sequence obtained through the above embodiment may also be manually reviewed to obtain a high-quality historical search word sequence.

[0118] Furthermore, the historical search word sequence and the first pre-trained language model can be used to automatically generate multi-round dialogue data that matches the actual dialogue scenario.

[0119] In real-world applications, conversation datasets often contain only conversation data but lack annotated conversation summaries. A common approach is to manually annotate the conversation data. However, manual annotation has several drawbacks, making the annotated data unsuitable for model training. First, individual expertise varies, leading to varying degrees of annotated data quality. Second, manual annotation can be stereotyped, resulting in similar summary annotation structures within the same batch of data, lacking diversity.

[0120] Based on this, the independent question samples in this embodiment can be obtained in the following manner.

[0121] The multi-round dialogue samples and the second prompt text are input into the second pre-trained language model, and the second pre-trained model generates independent question samples corresponding to the multi-round dialogue samples based on the multi-round dialogue samples and the second prompt text; wherein the second prompt text is used to instruct the second pre-trained language model to identify the user intention of the latest round of dialogue samples from the multi-round dialogue samples, and generate an independent question based on the identified user intention of the latest round of dialogue samples.

[0122] The multi-round conversation sample may include conversation data from at least two rounds of conversation. The second pre-trained language model of this embodiment can be flexibly selected according to actual needs, for example, it can be ChatGPT or ChatGLM, etc. The second pre-trained language model can be the same as or different from the first pre-trained language model described above, and this embodiment does not limit this.

[0123] As an example: the second prompt text may include one or more of the following information.

[0124] ① Role information indicating the identity of the second pre-trained language model as an intent recognition expert.

[0125] For example, the following text can be "You are an expert in identifying conversation intent" or "You are an expert in identifying user intent," etc. In actual applications, this can be flexibly adjusted as needed, and this embodiment does not limit this. This can indicate the role the model needs to play and activate the model's professional ability in intent recognition.

[0126] ② Instruct the second pre-trained language model to identify the user intent of the latest round of dialogue samples in multiple rounds of dialogue samples, and generate task information for independent questions based on the identified user intent of the latest round of dialogue samples.

[0127] For example, the following text could be used: "Your task is to rewrite the question I asked, summarizing it in one sentence based on the historical questions I asked and the current question. The discussion should focus on my current question. If the content discussed in the historical questions is irrelevant to the current question, it should be discarded." In actual applications, this can be flexibly adjusted as needed, and this embodiment does not limit this. In this way, a clear task definition can be provided to the model, enabling it to perform the task correctly.

[0128] ③ One or more examples for reference by the second pre-trained language model, the examples including: a historical question sequence containing one or more historical questions of a user, a current question example, and an independent question example.

[0129] As an example, the input to the second pre-trained language model may specifically be the text shown in the second column of the following table, with the corresponding description shown in the first column:

[0130] Based on this, through the above embodiment, the second pre-trained language model can be used to automatically generate labels for multi-round dialogue data (i.e., independent questions corresponding to the multi-round dialogue data). Optionally, the labels automatically generated by the second pre-trained language model can be reviewed by other means, such as manual review, to obtain a high-quality training dataset.

[0131] After preparing the training dataset, the pre-trained language model can be trained to obtain an independent question conversion model. The pre-trained language model can be flexibly selected as needed and is not limited in this embodiment. For example, it can include a ChatGLM model or a Chinese language model, such as the Chinese-alpaca-plus-13b model.

[0132] For example, the pre-trained language model can be fine-tuned in any of the following three ways:

[0133] (1) Fine-tune the entire pre-trained language model and update the parameters of the entire pre-trained language model.

[0134] (2) Freeze most of the parameters of the pre-trained language model and only fine-tune a relatively small set of model parameters.

[0135] (3) Add new parameters to the embedding layer of the pre-trained language model and before each neural network layer.

[0136] For example, a pre-trained language model can predict independent questions that represent the user's intention in the current round of dialogue samples based on the input multi-round dialogue samples. The pre-trained language model is trained with the optimization goal of minimizing the error between the predicted independent questions and the independent question samples to obtain an independent question conversion model.

[0137] Among them, the loss function of the pre-trained language model can be adjusted as needed. As an example, the existing loss function of the pre-trained language model can be used, and the error between the predicted independent question and the independent question sample can be the cross entropy loss between the predicted independent question and the independent question sample.

[0138] During model training, model parameters are gradually adjusted through iterative optimization. Each training iteration includes forward propagation, loss function calculation, backward propagation, and parameter update. The number of training iterations is typically set as a fixed number or adjusted dynamically based on the model's performance on the validation set.

[0139] Through the above embodiment, an independent question conversion model can be trained. The independent question conversion model can be used to generate independent questions that represent the user's intention in the current round of conversation based on the input conversation data. After testing, the independent question conversion model can accurately generate independent questions that represent the user's intention in the current round of conversation from the input conversation data, and also well preserves the model's existing language capabilities. Please see the following test results for examples:

[0140] The first three rows in the above table show that the independent question conversion model can accurately output independent questions that represent the user's intention in this round of conversation based on the input conversation data (including the user's current question and the user's historical questions).

[0141] The last two rows in the table contain other conversations unrelated to dish recommendations, and the independent question conversion model still retains the existing language capabilities.

[0142] Based on this, the independent question conversion model can be used to improve the output accuracy of the multi-turn dialogue system. Figure 2A is a schematic diagram of a dialogue scenario according to an exemplary embodiment of the present disclosure. Figure 2B is a flowchart of a method for generating dialogue content according to an exemplary embodiment of the present disclosure. The method may include the following steps 202 to 206.

[0143] Step 202: Acquire conversation data in response to the conversation initiated by the user.

[0144] Among them, when this round of dialogue is the first round of dialogue, the dialogue data is the user's current question in this round of dialogue; when this round of dialogue is not the first round of dialogue, the dialogue data includes the user's current question in this round of dialogue and the user's historical questions in each round of historical dialogue in at least one round of historical dialogue before this round of dialogue.

[0145] Step 204 : Input the dialogue data into an independent question conversion model, so that the independent question conversion model converts the dialogue data into independent questions that represent the user's intention in the current dialogue round.

[0146] Among them, the independent question conversion model is obtained by training a preset language model using a training data set including multiple training samples. Each training sample in the training data set contains multiple rounds of dialogue samples and independent question samples. The independent question sample indicates the user intention corresponding to the latest round of dialogue samples in the multiple rounds of dialogue samples.

[0147] In step 206 , the converted independent question is input into the multi-turn dialogue system, so that the multi-turn dialogue system generates a response content for the current dialogue round based on the independent question.

[0148] The above embodiments can refer to the aforementioned embodiments and will not be described in detail here.

[0149] FIG2C is a flowchart showing a method for processing conversation data according to an exemplary embodiment of the present disclosure. The method may include steps 212 to 214 .

[0150] Step 212: Acquire conversation data in response to the conversation initiated by the user.

[0151] Among them, when this round of dialogue is the first round of dialogue, the dialogue data includes the user's current question in this round of dialogue; when this round of dialogue is not the first round of dialogue, the dialogue data includes the user's current question in this round of dialogue and the user's historical questions in each round of historical dialogue in at least one round of historical dialogue before the current round of dialogue.

[0152] Step 214 : Input the dialogue data into an independent question conversion model, so that the independent question conversion model converts the dialogue data into independent questions that represent the user's intention in this round of dialogue.

[0153] The independent question conversion model is obtained by training a preset language model using a training dataset comprising multiple training samples. Each training sample in the training dataset contains multiple rounds of dialogue samples and an independent question sample, wherein the independent question sample indicates the user intent corresponding to the most recent round of dialogue samples in the multiple rounds of dialogue samples. The independent question conversion model converts the dialogue data into an independent question, which is then input into the multi-turn dialogue system, which then generates a response to the current round of dialogue based on the independent question.

[0154] The above embodiments can refer to the aforementioned embodiments and will not be described in detail here.

[0155] As can be seen from the above embodiments, this embodiment provides an independent question conversion model that can compress multiple rounds of dialogue into an independent question. The independent question retains key information and can indicate the user's intention in this round of dialogue. Therefore, the converted independent question can help simplify downstream tasks.

[0156] As an example, the method of this embodiment can be applied to a food recommendation scenario in the catering industry. The client includes a user-triggered dialogue portal. In response to the dialogue portal being triggered, the client can interact with the user through the independent question conversion model and multi-turn dialogue system of this embodiment. The dialogue portal can be, for example, the search box mentioned above.

[0157] This embodiment uses an independent question conversion model based on a large language model, combined with a multi-round dialogue system based on natural language generation (Text Generation) technology. The independent question conversion model is used to automatically compress multi-round dialogues into a single-round dialogue, based on which the multi-round dialogue system can generate accurate answers. The independent question conversion model of this embodiment can be applied to many tasks, such as online customer service, intelligent assistants, intelligent question and answer, etc., which can simplify downstream tasks and improve user experience and operational efficiency. Compared with traditional multi-round dialogue systems, this embodiment has higher efficiency and better user experience, while maintaining high accuracy and reliability.

[0158] Corresponding to the embodiments of the aforementioned method for generating conversation content, the present disclosure also provides embodiments of an apparatus for generating conversation content and a computer device used therein.

[0159] The embodiment of the device for generating the conversation content of the present disclosure can be applied on a computer device, such as a server or a terminal device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, as shown in Figure 3, it is a hardware structure diagram of the computer device where the device for generating the conversation content of the present disclosure is located. In addition to the processor 310, memory 330, network interface 320, and non-volatile memory 340 shown in Figure 3, the computer device 300 where the device for generating the conversation content 331 in the embodiment is located can generally include other hardware according to the actual function of the computer device, which will not be described in detail.

[0160] As shown in FIG4 , FIG4 is a block diagram of a device 40 for generating conversation content according to an exemplary embodiment of the present disclosure. The device includes an acquisition module 41 , an input module 42 and a reply module 43 .

[0161] The acquisition module 41 is used to: acquire conversation data in response to a conversation initiated by a user; wherein, if the current round of conversation is the first round of conversation, the conversation data includes the user's current question in the current round of conversation; if the current round of conversation is not the first round of conversation, the conversation data includes the user's current question in the current round of conversation and the user's historical questions in each round of historical conversation in at least one round of historical conversation before the current round of conversation.

[0162] The input module 42 is used to input the conversation data into the independent question conversion model, so that the independent question conversion model converts the conversation data into an independent question that represents the user intention in the current round of conversation; wherein the independent question conversion model is obtained by training a preset language model using a training data set including multiple training samples, each training sample in the training data set includes multiple rounds of conversation samples and independent question samples, and the independent question sample indicates the user intention corresponding to the latest round of conversation samples in the multiple rounds of conversation samples.

[0163] The answering module 43 is used to input the converted independent questions into the multi-round dialogue system, so that the multi-round dialogue system generates answer content for the current round of dialogue based on the independent questions.

[0164] The implementation process of the functions and effects of each module in the above-mentioned conversation content generation device is specifically described in the implementation process of the corresponding steps in the above-mentioned conversation content generation method, and will not be repeated here.

[0165] Accordingly, the embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned embodiment of the method for generating dialogue content / the method for constructing an independent question conversion model.

[0166] Accordingly, an embodiment of the present disclosure also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of an embodiment of a method for generating dialogue content / a method for constructing an independent question conversion model are implemented.

[0167] Accordingly, the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the embodiment of the method for generating dialogue content / the method for constructing an independent question conversion model are implemented.

[0168] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying any creative work.

[0169] The above embodiments can be applied to one or more computer devices. A computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0170] A computer device can be any electronic product that can interact with a user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, Internet Protocol Television (IPTV), smart wearable device, etc.

[0171] Computer equipment may also include network equipment and / or user equipment. Wherein, network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

[0172] The network where the computer device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0173] The foregoing description describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] The steps of the various methods above are divided only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0175] The phrases "specific examples" or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In the present disclosure, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0176] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the inventions claimed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosed embodiments that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0177] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0178] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for generating dialogue content, comprising: In response to a conversation initiated by a user, conversation data is acquired; wherein, if the current conversation is a first conversation, the conversation data includes the user's current question in the current conversation; if the current conversation is not the first conversation, the conversation data includes the user's current question in the current conversation and the user's historical questions in each historical conversation in at least one historical conversation before the current conversation; The dialogue data is input into an independent question conversion model, so that the independent question conversion model converts the dialogue data into an independent question representing the user intention in the current round of dialogue; wherein the independent question conversion model is obtained by training a preset language model using a training data set including a plurality of training samples, each training sample in the training data set includes a plurality of rounds of dialogue samples and an independent question sample, and the independent question sample indicates the user intention corresponding to the latest round of dialogue samples in the plurality of rounds of dialogue samples; The independent question obtained by conversion is input into a multi-round dialogue system, so that the multi-round dialogue system generates a response content for the current round of dialogue according to the independent question.

2. According to the method of claim 1, the multi-round dialogue samples are obtained by: Acquire a historical search word sequence, where the historical search word sequence includes one or more search words searched by the user in a historical time period, and each of the search words is used to search for a target object; Generate a first prompt text according to each search word in the historical search word sequence; The first prompt text is input into a first pre-trained language model so that the first pre-trained language model generates first multi-round dialogues according to the first prompt text; wherein the first multi-round dialogues are multi-round dialogues conducted in the identity of a user and an identity of a recommendation expert respectively, and the dialogue topics are related to each search term in the historical search term sequence.

3. According to the method of claim 2, the first prompt text includes one or more of the following information: role information indicating that the first pre-trained language model performs a conversation as a user and as a recommendation expert respectively; Instructing the first pre-trained language model to generate task information in which multiple rounds of dialogues are generated, the user's questions in each round of dialogues are related to one or more search terms in the historical search term sequence, and the answers of the recommended experts in each round of dialogues are related to target objects corresponding to one or more search terms in the historical search term sequence; Format information indicating the format of the multi-round dialogue output by the first pre-trained language model.

4. The method according to claim 2, wherein the historical search word sequence includes a plurality of search words arranged in chronological order of search time, and the overall correlation between the search words in the historical search word sequence satisfies a set correlation condition; The overall correlation between the search terms in the historical search term sequence is determined based on the average of the correlations between each pair of adjacent search terms in the historical search term sequence, and the correlation between each pair of adjacent search terms is determined based on the weighted value of the text similarity between the pair of search terms and the weighted value of the domain similarity between the pair of search terms.

5. According to the method of claim 4, the text similarity between a pair of adjacent search terms is the difference between the edit distance between the pair of adjacent search terms and the target string length, and the target string length refers to the string length of the search term with the larger string length in the pair of adjacent search terms.

6. According to the method of claim 4, the domain similarity between each pair of adjacent search terms is determined by the similarity between the categories to which the target objects corresponding to the pair of adjacent search terms belong.

7. The method according to claim 4, wherein the weight of the domain similarity is set to be higher than the weight of the text similarity.

8. A method for constructing an independent question conversion model, the independent question conversion model is used to generate independent questions based on dialogue data to characterize the user intention in the latest round of dialogue of the dialogue data, the method comprising: Acquire a training data set including a plurality of training samples, wherein each training sample in the training data set includes a plurality of rounds of dialogue samples and an independent question sample, wherein the independent question sample indicates a user intention corresponding to a latest round of dialogue sample in the plurality of rounds of dialogue samples; Inputting the multi-round conversation samples contained in each training sample in the training data set into a preset language model, so that the language model predicts an independent question representing the user intention of the latest round of conversation samples among the multi-round conversation samples based on the input multi-round conversation samples, and training the language model with minimizing the error between the predicted independent question and the corresponding independent question sample as the optimization goal; and Based on the trained language model, the independent question conversion model is obtained.

9. A method for processing conversation data, comprising: In response to a conversation initiated by a user, conversation data is acquired; wherein, if the current conversation is a first conversation, the conversation data includes the user's current question in the current conversation; if the current conversation is not the first conversation, the conversation data includes the user's current question in the current conversation and the user's historical questions in each historical conversation in at least one historical conversation before the current conversation; Inputting the dialogue data into an independent question conversion model, so that the independent question conversion model converts the dialogue data into independent questions representing the user intention in the current round of dialogue; The independent question conversion model is obtained by training a preset language model using a training data set including multiple training samples, each training sample in the training data set contains multiple rounds of dialogue samples and independent question samples, and the independent question sample indicates the user intention corresponding to the latest round of dialogue samples in the multiple rounds of dialogue samples.

10. A device for generating dialogue content, comprising: An acquisition module, configured to: acquire conversation data in response to a conversation initiated by a user; wherein, if the current conversation is the first conversation, the conversation data includes the user's current question in the current conversation; if the current conversation is not the first conversation, the conversation data includes the user's current question in the current conversation and the user's historical question in each historical conversation in at least one historical conversation before the current conversation; An input module, used to: input the dialogue data into an independent question conversion model, so that the independent question conversion model converts the dialogue data into an independent question representing the user intention in the current round of dialogue; wherein the independent question conversion model is obtained by training a preset language model using a training data set including a plurality of training samples, each training sample in the training data set includes a plurality of rounds of dialogue samples and an independent question sample, and the independent question sample indicates the user intention corresponding to the latest round of dialogue sample in the plurality of rounds of dialogue samples; The answering module is used to input the converted independent question into the multi-round dialogue system, so that the multi-round dialogue system generates the answer content for the current round of dialogue according to the independent question.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the methods of claims 1 to 9.

Images