Data construction method, device, apparatus, medium, and product

Abstract

The application relates to the technical field of artificial intelligence, and provides a data construction method, device, equipment, medium and product. The method comprises the following steps: obtaining a user query instruction; inputting the user query instruction into a context construction model to obtain a dialogue context corresponding to the user query instruction; interacting with an assistant model based on the user query instruction and the dialogue context to obtain first multi-round dialogue data, and interacting with the assistant model based on the user query instruction to obtain second multi-round dialogue data, thereby generating multi-round preference data pairs for training a large language model; wherein the first multi-round dialogue data comprises reply content generated by the assistant model according to what the user thinks and what the user says, and the second multi-round dialogue data comprises reply content generated by the assistant model according to what the user says. The data construction method, device, equipment, medium and product provided by the application can improve the dynamic adaptation capability of the large language model to individualized contexts.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data construction method, apparatus, device, medium and product. Background Technology

[0002] Currently, large language models have demonstrated remarkable capabilities, enabling them to align with universal human values ​​through post-training. However, when dealing with personalized users, current intelligent assistants based on large language models struggle to adapt to individual preferences while maintaining alignment with universal values, making it difficult to meet the diverse backgrounds and needs of users. To achieve personalized alignment, related technologies primarily enhance the model's ability to respond personally by constructing personalized datasets. However, most of these personalized datasets are single-turn data, and even those datasets that are multi-turn dialogues still involve step-by-step customization of personalized preferences.

[0003] In real-world personalized scenarios, users typically don't immediately reveal all their preferences when asking a question. Instead, they gradually inform the model of their preferences based on the model's responses, aiming to arrive at a response that meets their needs. This process is usually multi-step, and the model should prioritize satisfying the user's overall preferences. Single-round preference data only provides information on which response is better. Model optimization based on this type of data is based on immediate rewards and cannot achieve the goal of maximizing global rewards, resulting in large language models lacking dynamic adaptability to personalized situations. Summary of the Invention

[0004] This application provides a data construction method, apparatus, device, medium, and product to address the technical problem that large language models lack the ability to dynamically adapt to personalized contexts.

[0005] In a first aspect, embodiments of this application provide a data construction method, including: Retrieve user query instructions; The user query command is input into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The system interacts with the assistant model based on user query commands and dialogue context to obtain the first multi-round dialogue data, and interacts with the assistant model based on user query commands to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0006] In one embodiment, the context building model and the real-time response model are built based on a hierarchical probabilistic hybrid model, which includes a first network layer and a second network layer. The first network layer is used to dynamically match the distribution of user dialogue categories, and the second network layer is used to guide the behavioral details of user dialogue.

[0007] In one embodiment, the process of obtaining user conversation categories includes: Acquire multiple sets of multi-turn dialogue data; The dialogue flow logic features in each group of multi-turn dialogue data are extracted using a large language model; Based on the extracted dialogue flow logic features, multiple sets of multi-turn dialogue data are clustered to obtain multiple clusters; User dialogue categories are generated based on the dialogue flow logic features corresponding to the clusters.

[0008] In one embodiment, the training process of the context-building model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The first round of user input in the sample dialogue data is identified as the first input feature, and the sample dialogue data is identified as the first output label. The parameters of the hierarchical probabilistic mixture model are optimized based on the first input features and the first output label to obtain the context construction model.

[0009] In one embodiment, the training process of the real-time response model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The context settings, dialogue history, and assistant model responses for each round are concatenated to obtain the second input feature, and the user feedback corresponding to each round in the sample dialogue data is determined as the second output label. The parameters of the hierarchical probabilistic mixture model are optimized based on the second input feature and the second output label to obtain the real-time response model.

[0010] In one embodiment, obtaining sample dialogue data includes: Retrieve conversation log data; Filter out initial multi-turn dialogue data with coherent dialogue logic from the dialogue log data; By adding contextual settings and user thoughts for each round to the initial multi-turn dialogue data using a large language model, sample dialogue data is obtained.

[0011] In one embodiment, each round of the sample dialogue data is labeled with a user dialogue category. During the parameter optimization process, the second network layer guides the behavioral details of the user dialogue in each round based on the user dialogue category corresponding to each round.

[0012] Secondly, embodiments of this application provide a data construction apparatus, comprising: The acquisition module is used to acquire user query commands; The processing module is used to input user query commands into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The processing module is also used to interact with the assistant model based on user query commands and dialogue context to obtain the first multi-round dialogue data, and to interact with the assistant model based on user query commands to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0013] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the steps of the data construction method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the data construction method described in the first aspect.

[0015] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the data construction method described in the first aspect.

[0016] The data construction method, apparatus, device, medium, and product provided in this application embodiment obtain a corresponding dialogue context by inputting user query commands into a context construction model. The dialogue context represents the scene and user expectations corresponding to the user query command, providing a globally applicable personalized questioning background for multi-turn interactions. This allows the constructed dialogue data to present the personalized context behind the same user query command. By inputting the dialogue context, dialogue history, and the current response content of the assistant model into a real-time response model, simulated user feedback containing the user's thoughts and statements is obtained. The user's thoughts represent the user's inner activities in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. Thus, by combining observable and unobservable information, the real-time feedback of real users in multi-turn interactions is completely simulated, restoring the multi-step process in real scenarios where users gradually reflect their own thoughts and preferences based on the model's response.

[0017] Furthermore, under the same user query command and dialogue context, the first multi-turn dialogue data includes the response content generated by the assistant model based on the user's thoughts and words, while the second multi-turn dialogue data includes the response content generated by the assistant model based solely on the user's words. Since the assistant model additionally acquires unobservable information about the user's inner thoughts when generating the first multi-turn dialogue data, its response content is more in line with the user's global goals and personalized preferences. Therefore, the preference relationship between the first and second multi-turn dialogue data is based on information difference and has an objective basis. Reliable multi-turn preference data pairs can be generated without relying on additional manual annotation, thus ensuring the credibility of the preference data.

[0018] Based on this, the embodiments of this application construct multi-turn preference data pairs for training a large language model at the granularity of the entire multi-turn dialogue. This enables the large language model trained based on the multi-turn preference data pairs to be optimized to meet the user's global preferences. This overcomes the deficiency that single-turn preference data can only achieve immediate reward optimization, and maximizes the global reward. This effectively improves the dynamic adaptability of the large language model to personalized user scenarios in multi-turn interactions and improves the user experience in personalized scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the data construction method provided in the embodiments of this application.

[0021] Figure 2 This is the second flowchart illustrating the data construction method provided in the embodiments of this application.

[0022] Figure 3 This is the third flowchart illustrating the data construction method provided in the embodiments of this application.

[0023] Figure 4 This is the fourth flowchart illustrating the data construction method provided in the embodiments of this application.

[0024] Figure 5 This is the fifth flowchart illustrating the data construction method provided in the embodiments of this application.

[0025] Figure 6 This is the sixth flowchart illustrating the data construction method provided in the embodiments of this application.

[0026] Figure 7 This is a schematic diagram of the data construction device provided in the embodiments of this application.

[0027] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] To facilitate understanding of this application, the following will be introduced first.

[0030] Large Language Model (LLM) refers to a deep neural network model that is pre-trained on massive text corpora and has the ability to understand and generate natural language.

[0031] Supervised fine-tuning (SFT) refers to the process of supervising the training of a pre-trained model using labeled instruction data.

[0032] Direct Multi-Turn Preference Optimization (DMPO) is an algorithm that optimizes the preference alignment of a model based on preference data at the multi-turn dialogue level.

[0033] Multi-turn preference data refers to data obtained by ranking preferences across different multi-turn dialogues based on the overall performance of the model in the same personalized context and user environment. Multi-turn preference data uses the overall multi-turn dialogue as the ranking granularity, rather than the response of a single turn.

[0034] Currently, large language models have demonstrated remarkable capabilities, enabling them to align with universal human values ​​through post-training. However, when dealing with personalized users, current intelligent assistants based on large language models struggle to adapt to individual preferences while maintaining alignment with universal values, making it difficult to meet the diverse backgrounds and needs of users. To achieve personalized alignment, related technologies primarily enhance the model's ability to respond personally by constructing personalized datasets. However, most of these personalized datasets are single-turn data, and even those datasets that are multi-turn dialogues still involve step-by-step customization of personalized preferences.

[0035] In real-world personalized scenarios, users typically don't immediately reveal all their preferences when they first ask a question. Instead, they gradually inform the model of their preferences based on the model's responses, ensuring the final response meets their needs. This process is usually multi-step, and the model should aim to satisfy the user's overall preferences. Single-round preference data only provides information on which response is better, representing model optimization based on immediate rewards. It cannot achieve the goal of maximizing global rewards and therefore cannot enable the model to dynamically adapt to personalized situations.

[0036] Based on this, embodiments of this application provide a data construction method that can build a global personalized user environment for user query commands, obtain multiple sets of multi-turn dialogues through multiple interactions between the user environment and the assistant model, and construct multi-turn preference data pairs for the same question context based on the information difference between observable and unobservable information, thereby enhancing the personalized alignment capability of the large language model at the multi-turn dialogue level and improving its dynamic adaptability to personalized user contexts.

[0037] The following combination Figures 1-6 The data construction method provided in the embodiments of this application will be introduced.

[0038] Figure 1 This is one of the flowcharts illustrating the data construction method provided in the embodiments of this application. (Refer to...) Figure 1 This application provides a data construction method, which may include: Step 110: Obtain the user's query command.

[0039] In some embodiments, user queries can be obtained from a set of user query commands, such as a command dataset used for training large language models. Each command in the command dataset can serve as a user query command, i.e., as a dialogue topic for subsequent multi-turn interactions between the user environment and the assistant model.

[0040] Optionally, the user's query instruction can be a natural language question or request from any field or task type, such as an information consultation question, a solution design request, a code writing request, etc. This application embodiment does not limit this.

[0041] Step 120: Input the user's query command into the context building model to obtain the dialogue context corresponding to the user's query command.

[0042] The dialogue context is used to characterize the scenario and user expectations corresponding to the user's query. Specifically, the dialogue context sets the situation or background of the dialogue, including a description of the scenario in which the user makes the query and the user's overall goal (i.e., user expectations). The dialogue context is globally applicable to the entire multi-turn dialogue and reflects the personalized questioning context of the user's request.

[0043] It's important to note that the same user query may elicit different responses depending on the personalized context. For example, a user query asking for a fitness plan recommendation might come from a beginner or an experienced fitness enthusiast, with goals ranging from fat loss to muscle gain. This personalized background information is often not reflected in the initial query. Therefore, it's necessary to supplement the query with a dialogue context, providing a personalized question background.

[0044] In one embodiment, to accurately represent the personalized user environment, the user environment can be set to include three parts: the dialogue context (or context setting), the user's thoughts, and the user's spoken words. The main function of the user environment is to provide an interactive environment for the assistant model, including background information about the scenario in which the user raises the query command, and providing feedback on the assistant model's response in each round.

[0045] The dialogue context can include a description of the scenario in which the user makes the query and the user's overall goal, and is information that is globally applicable to multi-turn dialogues.

[0046] What the user wants to hear is the content of the assistant model's response in each round, including the user's unexpressed inner thoughts, including the user's emotions and the user's motivation for saying so, which belongs to the assistant model's unobservable information.

[0047] What the user says refers to what the user tells the assistant model, which is observable information for the assistant model. It should be noted that in the final round of the conversation, since the user terminates the conversation, there is no "what the user says" in the final round, only "what the user thinks," and this "what the user thinks" must include the reason for ending the conversation.

[0048] By setting up scenarios to present the personalized questioning background behind the same user request, and by revealing the user's thoughts and opinions to present the unobservable inner activities such as user thinking and opinions, a complete user environment is presented in three parts: scenario setting, user thoughts, and user statements, so as to further realize the construction of credible multi-turn preference data.

[0049] In one embodiment, the acquired user query instruction can be input into a pre-trained context construction model. The context construction model can output a dialogue context corresponding to the user query instruction. The training process of the context construction model can be referred to the following... Figure 3 I understand, so I won't go into details here.

[0050] Step 130: Interact with the assistant model based on the user's query instructions and dialogue context to obtain the first multi-round dialogue data, and interact with the assistant model based on the user's query instructions to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model.

[0051] The interaction process with the assistant model involves inputting the dialogue context, dialogue history, and the assistant model's current response into a real-time response model to obtain simulated user feedback. This simulated user feedback includes the user's thoughts and statements. The user's thoughts represent their internal thoughts in response to the assistant model's current response, while their statements represent their next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0052] In some embodiments, the interaction with the assistant model is implemented through multi-turn interactions between the user agent (or user environment) and the assistant model. Specifically, this includes inputting the dialogue context, dialogue history, and the assistant model's current response into a real-time response model to obtain simulated user feedback. This simulated user feedback includes the user's thoughts and words. The user's thoughts represent their internal thoughts in response to the assistant model's current response, such as their emotions and motivations. The user's words represent their next question to the assistant model.

[0053] As an example, the process of constructing a single multi-turn dialogue may include: using a user query command as the first round of user input, and the assistant model providing a response based on the first round of user input; the real-time response model providing simulated user feedback, including the user's thoughts and statements, based on the input dialogue context, dialogue history, and the assistant model's current response; determining whether the user's thoughts contain information to terminate the dialogue; if the user's thoughts contain information to terminate the dialogue, the dialogue stops, and the construction of this multi-turn dialogue is completed; if the user's thoughts do not contain information to terminate the dialogue, the assistant model continues to provide the response for the next round based on the simulated user feedback, and returns to the real-time response model to continue the next round of interaction.

[0054] In other words, the embodiments of this application determine whether the dialogue should end based on the information contained in the user's thoughts. Since the user's thoughts record the user's true attitude towards the assistant model's response, such as the reason for stopping the dialogue if the need has been met or if the user gives up asking further questions, this can be used as the basis for determining whether the dialogue should end. This can ensure the continuity and diversity of the constructed multi-turn dialogue and avoid the dialogue being cut off at an unreasonable point or continuing indefinitely.

[0055] In some embodiments, during the generation of multi-turn preference data pairs, in order to ensure the credibility of the data, observable and unobservable environmental information is provided to the assistant model simultaneously to obtain multi-turn dialogues that better match the user's goals and preferences. Specifically, this includes constructing a first multi-turn dialogue data, constructing a second multi-turn dialogue data, and combining them to obtain multi-turn preference data pairs.

[0056] When constructing the first multi-turn dialogue data, during the interaction, the assistant model provides responses based on the user's thoughts and statements in the next round. That is, when generating responses, the assistant model can capture both the observable statements and the unobservable thoughts of the user. Since the user's thoughts include unexpressed emotions, motivations, and other inner activities, responses generated by the assistant model that capture this information are closer to the user's overall goals and true preferences. Therefore, the first multi-turn dialogue data serves as the preferred dialogue among the multi-turn preference data pairs.

[0057] When constructing the second round of dialogue data, under the same user query command and the same user environment determined by the same dialogue context, the assistant model in the next round can provide a response based on what the user says. That is, the assistant model only acquires observable information, which is consistent with the information acquisition conditions of the assistant model in real deployment scenarios. The second round of dialogue data serves as the secondary dialogue in the multi-round preference data pair.

[0058] Furthermore, the first and second rounds of dialogue data under the same questioning context can be combined to obtain a set of multi-round preference data pairs.

[0059] It should be noted that the user's thoughts in the real-time response belong to unobservable information of the model, while the user's words belong to observable information. Constructing preference pairs based on the information difference between observable and unobservable information allows the preference relationship between preferred and secondary dialogues to have an objective and interpretable basis, without relying on additional manual annotation or scoring models for ranking, thus ensuring the credibility of multi-turn preference data pairs.

[0060] In some embodiments, after obtaining multi-turn preference data pairs, a direct multi-turn preference optimization algorithm can be used to train the supervised fine-tuned large language model based on these data pairs. This further enhances the large language model's personalized alignment capability at the multi-turn dialogue level. Since the multi-turn preference data pairs are obtained by ranking different dialogues based on the model's overall performance in multi-turn dialogues, the supervision signals they provide are geared towards the global objective of multi-turn interactions. Therefore, the model trained based on this data can optimize to satisfy the user's global preferences, maximizing global rewards and overcoming the limitation that single-turn preference data can only achieve immediate reward optimization.

[0061] In one embodiment, the context building model and the real-time response model are constructed based on a hierarchical probabilistic hybrid model. The hierarchical probabilistic hybrid model includes a first network layer and a second network layer. The first network layer is used to dynamically match the distribution of user dialogue categories. The second network layer is used to guide the behavioral details of the user dialogue.

[0062] Specifically, considering the diversity and complexity of real-world multi-turn dialogues in terms of context, preferences, and dialogue styles, directly using a single model for user simulation makes it difficult to reproduce the distributional characteristics of real-world human dialogues. To effectively simulate personalized user environments in real-world multi-turn dialogues, the context construction model and real-time response model are built based on a hierarchical probabilistic mixture model. Thus, a hierarchical probabilistic mixture model based on real-world multi-turn dialogue examples can be combined to simulate personalized environments for specific user query commands, incorporating the realism and diversity of human multi-turn dialogues into the model.

[0063] Furthermore, the hierarchical probabilistic hybrid model includes a first network layer and a second network layer: the first network layer is used to dynamically match the distribution of user dialogue categories, and the second network layer is used to guide the behavioral details of user dialogue.

[0064] As an example, given an input context x, the probability of obtaining the output y from a population P can be decomposed into group-based probabilities. The probability of is shown in formula (1): (1).

[0065] Where k is the index of the group. This represents the k-th group, i.e., the k-th user dialogue category. Let represent the gating weight of the k-th group output by the first network layer, and satisfy . . This represents the output probability of the large language model. Formula (1) reflects the role of the first network layer, namely, through gating the weights. Dynamically match the distribution of input context x across different user dialogue categories.

[0066] Furthermore, for modeling To integrate it into the behavioral details of the output sample, the probability of the output y of the two-layer probabilistic hybrid model can be expressed as formula (2): (2).

[0067] in, Example ( , The index is provided, and the example is a real multi-turn dialogue data example. This represents the output of the second network layer, specifically the k-th user dialogue category. The gating weight of the j-th sample under the given condition satisfies the following for any group k. In other words, the first network layer uses gating weights. Distribution matching is performed across different user dialogue categories. The second network layer, given a user dialogue category, uses gating weights... By selecting from a range of examples associated with the category, the generated user dialogue is precisely guided by the real human dialogue behaviors contained in the examples, thus reflecting a hierarchical structure of first matching the category and then guiding the details.

[0068] In some embodiments, each example is assigned a user dialogue category and corresponding group based on its labeled user dialogue category. The set of samples associated with the k-th group is denoted as . To obtain the gating parameters π and A pre-trained embedding model (such as the Qwen3 embedding model, or other embedding models with text semantic encoding capabilities) is used to encode the input context x, sample e, and group g respectively. The encoding function is expressed as follows: π and π were obtained through similarity calculation. As shown in formulas (3) and (4): (3).

[0069] (4) in, , , Let K be the gating network parameters to be optimized, and K be the total number of groups. Equation (4) represents the expression for a given number of groups. The set of samples associated with this group Within this group, the similarity between the encoding vector of the input context and the encoding vector of each example is normalized using softmax to obtain the sample gating weight distribution under this condition. This ensures that for any group k, there is .

[0070] The objective function for gating network optimization is expressed as the maximum log-likelihood of the total observations D, and the log-likelihood function is shown in formula (5): (5).

[0071] in, This represents the parameters of the gated network to be optimized. , Let be the i-th sample in the observation dataset D. The parameters of the gated network are optimized by maximizing the log-likelihood function shown in formula (5).

[0072] Furthermore, in some embodiments, to conserve computational resources, a sparse gating mechanism can be employed, that is, for any input x, the large language model is estimated to have the highest probability of... The log probabilities of the first m group and example pairs are used, where m is a preset positive integer, thus avoiding the need to iterate through all combinations of groups and examples, significantly reducing training overhead. Furthermore, since the data samples for estimating the gated network parameters and the example samples come from the same dataset, the samples themselves should be avoided during training to prevent model degradation due to information leakage.

[0073] Through the above-mentioned hierarchical probabilistic hybrid model, the first network layer ensures that the simulated user environment is consistent with real human dialogue in terms of macro-category distribution, and the second network layer ensures that the simulated user environment is close to real human dialogue examples in terms of micro-behavioral details, thereby effectively improving the authenticity, diversity and complexity of the constructed personalized context and preferences.

[0074] Figure 2 This is a second schematic flowchart of the data construction method provided in the embodiments of this application. (Refer to...) Figure 2 When implementing the process of obtaining user dialogue categories, this application embodiment provides an optional implementation method, which may include: Step 210: Obtain multiple sets of multi-turn dialogue data.

[0075] The multi-turn dialogue data can be real multi-turn dialogue data, such as multi-turn dialogue data between users and AI assistants obtained from real dialogue logs. Considering that real human multi-turn dialogues exhibit significant diversity in dialogue flow logic, this application embodiment clusters the multi-turn dialogue data based on dialogue flow logic to obtain a set of dialogue flow types for personalized user environment simulation.

[0076] Step 220: Extract the dialogue flow logic features from each group of multi-turn dialogue data using a large language model.

[0077] Specifically, the logical features of dialogue flow in real-world multi-turn dialogues can be summarized using a large language model with prompts. For example, the prompt could be: Task Description: Given a dialogue between a user and an AI assistant, summarize the characteristics of the dialogue flow logic, including the evolution of user intent, the relevance of user questions, the driving force of dialogue flow, and the dialogue structure. Important Notes: The information summarized must be inferred from the dialogue; it cannot be fabricated. ##Example {Dialogue Content and Flow Characteristics Example}### Please complete the following based on the above content, strictly adhering to the example format: {Multi-turn Dialogue to be Summarized}.

[0078] In other words, the extracted dialogue flow logic features can include features from dimensions such as user intent evolution, user question association, dialogue flow dynamics, and dialogue structure. Furthermore, by setting precautions in the prompts and providing examples (few-shots), the accuracy and format consistency of the extracted features are ensured.

[0079] Step 230: Based on the extracted dialogue flow logic features, cluster multiple sets of multi-turn dialogue data to obtain multiple clusters.

[0080] Specifically, based on the dialogue flow logic characteristics summarized in step S220, multiple sets of multi-turn dialogue data are clustered. Clustering methods can employ clustering algorithms. For example, the dialogue flow logic features can first be encoded into feature vectors using an embedding model, and then K-means clustering, hierarchical clustering, or density-based clustering can be performed based on the feature vectors. This application embodiment does not limit the specific clustering algorithm.

[0081] Step 240: Generate user dialogue categories based on the dialogue flow logic features corresponding to the clusters.

[0082] Specifically, for each cluster, a category name and the core features of that category are summarized to obtain user dialogue categories. For example, the obtained user dialogue categories may include: The progressive deepening category is characterized by the gradual concretization or focus of intent, hierarchical dependence of issues or logical chains, user-driven discussion to evolve to a deeper level, and linear or step-by-step dialogue structure. The constraint-driven category is characterized by the continuous addition of technical or real-world constraints, with the solution iteratively adjusted under these constraints. Users take the constraints as the core driving force, and the structure becomes progressively more complex as constraints are added. The verification feedback category is characterized by users providing test feedback or raising questions, which drives the iteration and correction of the solution. The driving force is based on the results of practical verification, and the structure presents a closed loop of problem, feedback, and optimization.

[0083] The categories described above are merely examples; in actual implementation, other numbers and contents of user dialogue categories can be obtained based on the clustering results. The generated user dialogue categories and their core features serve as the groups in the first network layer of the hierarchical probabilistic mixture model, used to dynamically match the distribution of user dialogue categories.

[0084] As mentioned above, the user environment includes scenario settings, user thoughts, and user statements. User thoughts and statements can be collectively referred to as user feedback, which is the simulated output of the user. In this embodiment, both scenario settings and user feedback are obtained based on the aforementioned hierarchical probabilistic mixture model, corresponding to the scenario construction model and the real-time response model, respectively.

[0085] Figure 3 This is the third flowchart illustrating the data construction method provided in this application's embodiments. (Refer to...) Figure 3 When implementing the training process of the context construction model, this application embodiment provides an optional implementation method, which may include: Step 310: Obtain sample dialogue data.

[0086] The sample dialogue data includes multi-turn dialogue data with added dialogue context and user thoughts, i.e., real multi-turn dialogue data completed with user context. The specific methods for obtaining this data are as follows. Figure 5 I understand, and will not elaborate further here.

[0087] Step 320: Determine the first round of user input in the sample dialogue data as the first input feature and the sample dialogue data as the first output label.

[0088] Specifically, the first round of user input in the real dialogue that completes the user environment is used as the input x (i.e., the first input feature) of the training sample, and the whole real dialogue that completes the user environment (including the context setting, the user's thoughts in each round, the user's words, and the assistant's replies) is used as the output y (i.e., the first output label).

[0089] The reason for constructing the training data in this way is that the application scenario of the context-building model is to infer the reasonable personalized question context behind a user query given only one user query command during the data construction stage. By training with the first round of user input as input and the complete dialogue with context as output, the model learns the mapping relationship from an initial query to a complete personalized environment, thereby generating a matching dialogue context for any user query command during the inference stage.

[0090] Step 330: Optimize the parameters of the hierarchical probability mixture model based on the first input features and the first output label to obtain the context construction model.

[0091] Specifically, the first input feature is used as the first input feature in formula (5). The first output label is used as the first output label in formula (5). By maximizing the log-likelihood objective function shown in formula (5), the gating network parameters of the hierarchical probability mixture model are optimized, and the context construction model is obtained after training. During the training process, the aforementioned sparse gating mechanism can also be used to save computational resources and avoid using the samples themselves as examples.

[0092] Figure 4 This is the fourth flowchart illustrating the data construction method provided in the embodiments of this application. (Refer to...) Figure 4 When implementing the training process of a real-time response model, this application embodiment provides an optional implementation method, which may include: Step 410: Obtain sample dialogue data.

[0093] The sample dialogue data includes multi-turn dialogue data with added dialogue context and user thoughts. Similar to the training of the context building model, the sample dialogue data is also multi-turn dialogue data with added dialogue context and user thoughts. It can share the same batch of sample dialogue data with the training of the context building model, or it can be obtained separately. This application embodiment does not limit this.

[0094] Step 420: Concatenate the context settings, dialogue history and assistant model response content corresponding to each round to obtain the second input feature, and determine the user feedback corresponding to each round in the sample dialogue data as the second output label.

[0095] In some embodiments, unlike the training data for the context-building model, the real-time response model uses each round of a real dialogue that completes the user's environment as a training sample. Specifically, for any round in the sample dialogue data, the text obtained by concatenating the context setting corresponding to that round, the dialogue history up to that round, and the assistant model's response content in that round is used as input x (i.e., the second input feature); the user's feedback in the real dialogue that completes the user's environment (i.e., what the user thinks and what the user says in that round) is used as output y (the second output label). If it is the last round, the output y includes the user's thoughts containing the reason for stopping the dialogue.

[0096] The reason for constructing the training data in this way is that the application scenario of the real-time response model is to simulate real-time feedback from a real user for each round of responses from the assistant model during multi-round interactions. The model is trained using context settings, dialogue history, and the current round of assistant responses as input, and the current round of user feedback as output. This allows the model to learn the patterns of how real users react to assistant responses in specific situations and dialogue contexts, including both internal thoughts (what the user thinks) and external expressions (what the user says).

[0097] Step 430: Optimize the parameters of the hierarchical probabilistic hybrid model based on the second input features and the second output labels to obtain the real-time response model.

[0098] Specifically, the second input feature and the second output label are respectively used as the second input feature and the second output label in formula (5). and The parameters of the hierarchical probabilistic mixture model are optimized by maximizing the log-likelihood objective function, resulting in a real-time response model after training. Similarly, a sparse gating mechanism can be used during training, and the samples themselves should be avoided as examples.

[0099] Considering that constructing diverse and complex situations, preferences, and dialogue styles that more closely resemble real humans requires a foundation of authentic multi-turn dialogues, but obtaining authentic multi-turn dialogue scenarios is costly, and authentic multi-turn dialogues only contain the interaction content of each turn. Since different users have different speaking habits, it is impossible to obtain the user's opinion and satisfaction with the assistant model's responses solely from what the user says. Therefore, this application proposes to supplement the corresponding user environment based on authentic multi-turn dialogues, providing data examples for user simulation in subsequent steps.

[0100] Figure 5 This is the fifth flowchart illustrating the data construction method provided in the embodiments of this application. (Refer to...) Figure 5 When acquiring sample dialogue data, this application embodiment provides an optional implementation method, which may include: Step 510: Obtain dialogue log data.

[0101] Specifically, this involves collecting existing real-world conversation log data, such as conversation logs between users and assistants accumulated during the actual operation of intelligent assistant products. It is understood that the collection of conversation log data should be conducted with user authorization and in compliance with relevant regulations, and sensitive information in the logs should be anonymized.

[0102] Step 520: Filter out initial multi-turn dialogue data with coherent dialogue logic from the dialogue log data.

[0103] Specifically, multi-turn dialogue data with coherent dialogue logic can be filtered out using techniques such as keyword recognition. For example, it's possible to identify whether the dialogue contains keywords that demonstrate logical connections between turns, such as "therefore," "then," and "however," which indicate causal, sequential, or contrastive relationships. Multi-turn dialogues containing such keywords typically have coherent dialogue logic between turns and are suitable as foundational data for user context completion. This filtering step can eliminate dialogue data where turns are independent and lack logical connections, ensuring the quality of the sample dialogue data.

[0104] Step 530: Add contextual settings and user thoughts for each round to the initial multi-turn dialogue data using a large language model to obtain sample dialogue data.

[0105] Specifically, by using a large language model to prompt, the context setting (including scene description, user motivation or user goal) and the user's thoughts in each round are expanded for each group of multi-turn dialogues.

[0106] An example prompt could be to complete the dialogue context, user role, and user thoughts for a given multi-turn dialogue, following these rules: 1. Dialogue context refers to the background information of the dialogue; user role includes the user's goal. The dialogue context and user role are globally applicable across multiple turns of the dialogue; 2. User thoughts refer to the response content of each round of the assistant model, including the user's unexpressed inner thoughts, including the user's emotions and motivations, and should be distinguished from the user's spoken content using square brackets; 3. You should carefully consider the user's thoughts and simulate a human user as closely as possible; 4. After the assistant's final response, complete the user's thoughts, including the reason for stopping the dialogue. ##Input Example: {Input Sample}##Output Example: {Output Sample} Next, please complete the dialogue context, user role, and user thoughts for the following dialogue, outputting them in English. {Dialogue to be Completed}.

[0107] Through the rules constrained by the aforementioned prompts, the supplemented sample dialogue data's dialogue context and user roles are globally applicable across multiple rounds of dialogue. Furthermore, each round's user thoughts include unexpressed emotions and motivations, distinguished from spoken words by formatting elements such as square brackets. The final round includes supplementary user thoughts containing the reason for ending the dialogue. The resulting sample dialogue data is multi-round dialogue data with added dialogue context and user thoughts. It can be used simultaneously for training context-building and real-time response models, and can also serve as a real-world dialogue example referenced in the second network layer of a hierarchical probabilistic hybrid model.

[0108] Furthermore, the category labels for the aforementioned user dialogues are obtained at the dialogue level. However, not every user in every round of a dialogue belonging to a certain category will perform the corresponding action. For example, in a group of dialogues collectively labeled as the verification feedback category, only some rounds of users may be providing test feedback, while other rounds of users may be supplementing requirements or confirming information. If the category labels at the dialogue level are directly used in round-level user simulations, it will lead to inaccurate behavioral guidance.

[0109] Based on this, in one embodiment, each round of the sample dialogue data is labeled with a user dialogue category, that is, each round of the dialogue is labeled with a category tag for user simulation in subsequent stages. Round-level labeling can be achieved, for example, by using a cue-based large language model, referring to the core features of each user dialogue category, to determine and label the category to which the user behavior belongs in each round.

[0110] Accordingly, during parameter optimization, the second network layer guides the behavioral details of the user dialogue in each round based on the user dialogue category corresponding to each round. Specifically, during the training and inference of the real-time response model, for the input context of each round, the first network layer matches the category distribution based on the user dialogue category corresponding to that round, and the second network layer selects the most matching real dialogue sample from the sample set associated with that user dialogue category based on gating weights, guiding the behavioral details of the user dialogue in that round. This ensures that the generated user feedback is consistent with the actual dialogue flow logic in that round in terms of behavioral details, further improving the realism and accuracy of user simulation at the round level.

[0111] Figure 6 This is the sixth flowchart illustrating the data construction method provided in this application. (Refer to...) Figure 6 The data construction method provided in this application involves three parts: user environment, assistant model, and multi-round preference data.

[0112] The user environment provides the interaction environment for the assistant model, comprising two parts: context setting and real-time response. Context setting includes a description of the scenario in which the user submits their query and the user's overall goal. It applies globally to the entire multi-turn dialogue and represents the personalized questioning context behind the same user query. Real-time response is generated by the real-time response model for each round of responses from the assistant model, including the user's "thoughts" and "sayings": The user's thoughts are unobservable information, representing the user's unexpressed inner thoughts in response to the assistant model's current response, including the user's emotions and motivations, and in the final round, the reason for ending the dialogue; the user's sayings are observable information, representing the user's next question to the assistant model, i.e., what the user actually says to the assistant model.

[0113] like Figure 6 As shown, the user environment and the assistant model interact through two paths. First, both the user's thoughts and spoken words are provided to the assistant model for interaction. In this case, the assistant model generates the next round of responses based on both unobservable and observable information. Because it understands the user's unexpressed inner thoughts, the generated responses are more aligned with the user's overall goals and personalized preferences. Second, only the user's spoken words are provided to the assistant model for interaction. In this case, the assistant model generates the next round of responses based solely on observable information, consistent with the information acquisition conditions of the assistant model in real-world deployment scenarios. In each round of interaction, the real-time response model provides a real-time response based on the context settings, dialogue history, and the assistant model's current response. If the user's thoughts contain information indicating the end of the dialogue, the dialogue stops; otherwise, the next round of interaction continues.

[0114] Based on the above interactions, two sets of multi-turn dialogues can be obtained under the same contextual settings, thereby generating multi-turn preference data. Specifically, the multi-turn dialogues obtained by the assistant model based on the user's thoughts and statements (i.e., the first multi-turn dialogue data) are selected as the preferred multi-turn dialogues, while the multi-turn dialogues obtained by the assistant model based solely on the user's statements (i.e., the second multi-turn dialogue data) are selected as the secondary multi-turn dialogues. Furthermore, the preferred and secondary multi-turn dialogues constitute a set of multi-turn preference data pairs, which are used for multi-turn preference alignment training of the large language model.

[0115] Based on this, a preference ranking is constructed based on the information difference between observable and unobservable information, so that the preference relationship between the best and second best choices has an objective basis, ensuring the credibility of multi-round preference data and helping to improve the dynamic adaptability of large language models to personalized situations.

[0116] In the above embodiments of this application, compared with the single-turn personalized alignment scheme in related technologies, it is possible to construct multi-turn preference data for personalized scenarios. The constructed multi-turn preference data pairs are ranked with the overall performance of multi-turn dialogue as the sorting granularity and are oriented towards the user's global preference goal. This helps to improve the dynamic adaptability of the large language model to personalized scenarios and achieve the optimization goal of maximizing global rewards.

[0117] Furthermore, the model constructs a personalized environment based on what users think and say, and uses the information difference between observable and unobservable information to construct preferred and secondary dialogues, so that the preference ranking has an objective basis and helps to ensure the credibility of preference data.

[0118] Furthermore, by using a hierarchical probabilistic hybrid model to construct user environments by combining dialogue flow features from real dialogue data (first network layer) and real multi-turn data examples (second network layer), the authenticity, diversity, and complexity of the constructed scenarios and preferences can be improved.

[0119] Furthermore, by determining whether a dialogue should end based on the information contained in the user's thoughts, the coherence and diversity of the constructed multi-turn dialogues can be achieved.

[0120] The data construction apparatus provided in the embodiments of this application will be described below. The data construction apparatus described below can be referred to in correspondence with the data construction method described above.

[0121] Figure 7 This is a schematic diagram of the data construction apparatus provided in an embodiment of this application. (Refer to...) Figure 7 The data construction apparatus provided in this application includes: Module 710 is used to obtain user query commands; Processing module 720 is used to input user query commands into the context building model to obtain a dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The processing module 720 is also used to interact with the assistant model based on the user query command and dialogue context to obtain the first multi-round dialogue data, and to interact with the assistant model based on the user query command to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0122] In one embodiment, the context building model and the real-time response model are built based on a hierarchical probabilistic hybrid model, which includes a first network layer and a second network layer. The first network layer is used to dynamically match the distribution of user dialogue categories, and the second network layer is used to guide the behavioral details of user dialogue.

[0123] In one embodiment, the process of obtaining the user dialogue category includes: Acquire multiple sets of multi-turn dialogue data; The dialogue flow logic features in each group of multi-turn dialogue data are extracted using a large language model; Based on the extracted dialogue flow logic features, multiple sets of multi-turn dialogue data are clustered to obtain multiple clusters; User dialogue categories are generated based on the dialogue flow logic features corresponding to the clusters.

[0124] In one embodiment, the training process of the context construction model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The first round of user input in the sample dialogue data is identified as the first input feature, and the sample dialogue data is identified as the first output label. The parameters of the hierarchical probabilistic mixture model are optimized based on the first input features and the first output label to obtain the context construction model.

[0125] In one embodiment, the training process of the real-time response model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The context settings, dialogue history, and assistant model responses for each round are concatenated to obtain the second input feature, and the user feedback corresponding to each round in the sample dialogue data is determined as the second output label. The parameters of the hierarchical probabilistic mixture model are optimized based on the second input feature and the second output label to obtain the real-time response model.

[0126] In one embodiment, the acquisition module 710 is specifically used for: Retrieve conversation log data; Filter out initial multi-turn dialogue data with coherent dialogue logic from the dialogue log data; By adding contextual settings and user thoughts for each round to the initial multi-turn dialogue data using a large language model, sample dialogue data is obtained.

[0127] In one embodiment, each round of the sample dialogue data is labeled with a user dialogue category. During the parameter optimization process, the second network layer guides the behavioral details of the user dialogue in each round based on the user dialogue category corresponding to each round.

[0128] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call a computer program stored in the memory 830 to execute steps of a data construction method, such as: Retrieve user query instructions; The user query command is input into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The system interacts with the assistant model based on user query commands and dialogue context to obtain the first multi-round dialogue data, and interacts with the assistant model based on user query commands to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0129] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0130] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the data construction methods provided in the above embodiments, such as including: Retrieve user query instructions; The user query command is input into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The system interacts with the assistant model based on user query commands and dialogue context to obtain the first multi-round dialogue data, and interacts with the assistant model based on user query commands to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0131] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing a processor to execute the data construction method steps of the methods provided in the above embodiments, such as including: Retrieve user query instructions; The user query command is input into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. The system interacts with the assistant model based on user query commands and dialogue context to obtain the first multi-round dialogue data, and interacts with the assistant model based on user query commands to obtain the second multi-round dialogue data, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into the real-time response model to obtain simulated user feedback. The simulated user feedback includes the user's thoughts and the user's statements. The user's thoughts represent the user's inner thoughts in response to the current response of the assistant model, and the user's statements represent the user's next question to the assistant model. The first multi-turn dialogue data includes the response generated by the assistant model based on the user's thoughts and statements, and the second multi-turn dialogue data includes the response generated by the assistant model based on the user's statements.

[0132] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0133] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0134] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data construction method, characterized in that, include: Retrieve user query instructions; The user query command is input into the context building model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to represent the scenario and user expectations corresponding to the user query command. Based on the user query command and the dialogue context, the assistant model interacts to obtain the first multi-round dialogue data, and based on the user query command and the assistant model, the second multi-round dialogue data is obtained, generating multi-round preference data pairs for training the large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into a real-time response model to obtain simulated user feedback; the simulated user feedback includes the user's thoughts and the user's statements, the user's thoughts are used to characterize the user's inner thoughts in response to the current response of the assistant model, and the user's statements are used to characterize the user's next question to the assistant model; The first multi-turn dialogue data includes the response content generated by the assistant model based on the user's thoughts and words, and the second multi-turn dialogue data includes the response content generated by the assistant model based on the user's words.

2. The data construction method according to claim 1, characterized in that, The context building model and the real-time response model are built based on a hierarchical probabilistic hybrid model, which includes a first network layer and a second network layer. The first network layer is used to dynamically match the distribution of user dialogue categories, and the second network layer is used to guide the behavioral details of user dialogue.

3. The data construction method according to claim 2, characterized in that, The process of obtaining the user dialogue category includes: Acquire multiple sets of multi-turn dialogue data; The dialogue flow logic features in each group of multi-turn dialogue data are extracted using a large language model; Based on the extracted dialogue flow logic features, the multiple sets of multi-turn dialogue data are clustered to obtain multiple clusters; Based on the dialogue flow logic features corresponding to the clusters, the user dialogue categories are generated.

4. The data construction method according to claim 2, characterized in that, The training process of the scenario-building model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The first round of user input in the sample dialogue data is identified as the first input feature, and the sample dialogue data is identified as the first output label. The parameters of the hierarchical probability mixture model are optimized based on the first input features and the first output label to obtain the context construction model.

5. The data construction method according to claim 3, characterized in that, The training process of the real-time response model includes: Acquire sample dialogue data, which includes multi-turn dialogue data with added dialogue context and user thoughts; The context settings, dialogue history, and assistant model responses for each round are concatenated to obtain the second input feature, and the user feedback corresponding to each round in the sample dialogue data is determined as the second output label. The parameters of the hierarchical probability mixture model are optimized based on the second input feature and the second output label to obtain the real-time response model.

6. The data construction method according to claim 4 or 5, characterized in that, The acquisition of sample dialogue data includes: Retrieve conversation log data; Filter out initial multi-turn dialogue data with coherent dialogue logic from the dialogue log data; By adding contextual settings and user thoughts for each round to the initial multi-turn dialogue data using a large language model, the sample dialogue data is obtained.

7. The data construction method according to claim 4 or 5, characterized in that, Each round of the sample dialogue data is labeled with a user dialogue category. During the parameter optimization process, the second network layer guides the behavioral details of the user dialogue in each round based on the user dialogue category corresponding to each round.

8. A data construction apparatus, characterized in that, include: The acquisition module is used to acquire user query commands; The processing module is used to input the user query command into the context construction model to obtain the dialogue context corresponding to the user query command. The dialogue context is used to characterize the scenario and user expectations corresponding to the user query command. The processing module is further configured to interact with the assistant model based on the user query command and the dialogue context to obtain first multi-round dialogue data, and interact with the assistant model based on the user query command to obtain second multi-round dialogue data, generating multi-round preference data pairs for training a large language model; The interaction process with the assistant model includes inputting the dialogue context, dialogue history, and the current response of the assistant model into a real-time response model to obtain simulated user feedback; the simulated user feedback includes the user's thoughts and the user's statements, the user's thoughts are used to characterize the user's inner thoughts in response to the current response of the assistant model, and the user's statements are used to characterize the user's next question to the assistant model; The first multi-turn dialogue data includes the response content generated by the assistant model based on the user's thoughts and words, and the second multi-turn dialogue data includes the response content generated by the assistant model based on the user's words.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the data construction method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the data construction method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the data construction method according to any one of claims 1 to 7.

Images