Human-computer interaction method and device

Abstract

The embodiment of the invention discloses a man-machine interaction method and device, and the method comprises the steps: obtaining a voice request which is a request for commanding a vehicle-mounted terminal to execute an instruction, or a request for making a question to the vehicle-mounted terminal; reply information for the voice request is generated according to the voice request and a big language model, the big language model is obtained through training based on target sample data in the sample data, and the target sample data is sample data meeting the data quality requirement and the data diversity requirement, the data diversity demand means that the type number of the included sample data is greater than or equal to a type threshold value. The question answering accuracy of the intelligent vehicle machine can be improved.

Description

Technical Field

[0001] This application relates to intelligent vehicle system technology, including but not limited to a human-computer interaction method and device. Background Technology

[0002] During operation, intelligent vehicle systems typically need to generate responses that address user inquiries. These intelligent vehicle systems are artificial intelligence robots installed within the vehicle, such as those embedded in the in-vehicle terminal.

[0003] In related technologies, the responses from intelligent vehicle systems may be inaccurate or unreasonable, which may result in the inability to meet user inquiries and lead to poor accuracy in the responses. Summary of the Invention

[0004] In view of this, the human-computer interaction method and apparatus provided in this application can improve the accuracy of intelligent vehicle systems in answering questions. The human-computer interaction method and apparatus provided in this application are implemented as follows: One aspect of this application provides a human-computer interaction method applied to an in-vehicle terminal, the method comprising: Obtain a voice request, which is a request to instruct the vehicle terminal to execute an instruction, or a request to ask a question to the vehicle terminal; The system generates response information for the voice request based on the voice request and the large language model. The large language model is trained on target sample data in the sample data. The target sample data is sample data that meets the data quality requirements and data diversity requirements. The data diversity requirement refers to the number of types of sample data included being greater than or equal to the type threshold.

[0005] Another aspect of the embodiments of this application provides a human-computer interaction device applied to an in-vehicle terminal, the device including: an acquisition module and a response module; The acquisition module is used to acquire voice requests, which are requests to instruct the vehicle terminal to execute instructions or requests to ask questions to the vehicle terminal. The response module is used to generate response information for the voice request based on the voice request and the large language model. The large language model is trained based on the target sample data in the sample data. The target sample data is sample data that meets the data quality requirements and data diversity requirements. The data diversity requirement refers to the number of types of sample data included being greater than or equal to the type threshold.

[0006] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement the method of this application.

[0007] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0008] The human-computer interaction method and apparatus provided in this application embodiment can acquire voice requests and generate response information for the voice requests based on the voice requests and a large language model. The large language model is trained on target sample data from the sample data. The target sample data is sample data that meets both data quality and data diversity requirements. Since the target sample data used meets both data quality and data diversity requirements, these data are those with good training results. Training the large language model based on this data not only improves training efficiency but also, because it meets the data diversity requirement, allows training on the large language model based on various types of sample data, thus improving the comprehensiveness of the training and enhancing the training effect of the large language model. This allows the large language model to output more accurate and reasonable response information. During the training process, the large language model tends to generate correct response information, further improving the accuracy of intelligent vehicle systems in answering questions. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram illustrating the application scenario provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the human-computer interaction method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the large language model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the process for training a large language model provided in the embodiments of this application; Figure 5 This is a schematic diagram of the process for filtering target sample data provided in the embodiments of this application; Figure 6 This is a schematic diagram of the diversity screening process provided in the embodiments of this application; Figure 7 This is a schematic diagram of the target space mapping structure provided in the embodiments of this application; Figure 8This is another schematic diagram of the process for filtering target sample data provided in the embodiments of this application; Figure 9 This is a schematic diagram of the iterative training process for a large language model provided in the embodiments of this application; Figure 10 This is a schematic diagram of the SFT iteration process provided in the embodiments of this application; Figure 11 This is a schematic diagram of the DPO iteration process provided in the embodiments of this application; Figure 12 This is a schematic diagram of the sample data determination provided in the embodiments of this application; Figure 13 This is a schematic diagram of the human-computer interaction device provided in the embodiments of this application; Figure 14 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0014] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0015] To more clearly explain the human-computer interaction method provided in the embodiments of this application, a possible application scenario in the embodiments of this application will be explained in detail below.

[0016] Figure 1 This is a schematic diagram of the application scenario provided in the embodiments of this application. Please refer to it. Figure 1This scenario may include: an in-vehicle terminal 100, which may be, for example, a smart vehicle system. The in-vehicle terminal 100 is installed in the smart cockpit and can have intelligent dialogue with the user in the smart cockpit. For example, it can generate corresponding answers based on the user's questions, or execute corresponding tasks based on the user's instructions.

[0017] For example, users can ask the vehicle terminal 100 questions, such as: the vehicle's remaining battery / fuel level, whether the doors are closed, and the temperature inside and outside the vehicle; users can also instruct the vehicle terminal 100 to perform tasks, such as: commanding the vehicle terminal 100 to open the windows, commanding the vehicle terminal 100 to turn on the air conditioning, etc.

[0018] The vehicle terminal 100 can respond to user questions or commands. For example, for the above questions, the vehicle terminal can respond with the vehicle's actual remaining battery / fuel level, door open / close status, and interior / exterior temperatures. The vehicle terminal can also respond after executing a command, such as: windows are open, air conditioning is on.

[0019] In this scenario, one user or multiple users can interact with the vehicle terminal 100. There are no specific restrictions. For example, the user in the driver's seat can interact with the vehicle terminal 100, and the user in the passenger seat can also interact with the vehicle terminal 100.

[0020] During the interaction between the user and the vehicle terminal 100, communication can be achieved through language. For example, the user can ask a question, the vehicle terminal 100 can collect the corresponding audio signal, perform speech recognition, determine the user's intent, and then generate corresponding response content based on a large language model to reply to the user. The method of replying to the user is not limited to text reply, voice reply, or image reply, and no specific restrictions are made here.

[0021] For text replies, the vehicle terminal 100 can be equipped with a corresponding electronic screen, which can display the text reply. For voice replies, the vehicle terminal can be equipped with a corresponding speaker, which can play the voice reply. For image replies, the image reply can be displayed on the electronic screen.

[0022] In one embodiment, during the response process of the vehicle terminal, any one of the three response methods described above can be used, or any two or three methods can be combined to generate the corresponding response, without any specific limitations.

[0023] In the embodiments of this application, taking the in-vehicle terminal 100 as an intelligent vehicle system as an example, the intelligent vehicle system can typically generate a response that satisfies the user's question during operation. For example, after a user asks a question, the system analyzes the content of the question and generates a reasonable response.

[0024] However, in related technologies, the responses from intelligent vehicle systems may be inaccurate or unreasonable, which leads to... Figure 1 The in-vehicle terminal 100 shown cannot meet the user's questions, resulting in poor accuracy in answering questions by the intelligent vehicle system.

[0025] For example, when a user asks "What's the weather like today?", they are essentially asking for climate information. However, if the in-vehicle terminal answers "The temperature inside the car is 24 degrees Celsius today," it only answers the temperature inside the car and does not specifically answer the weather outside the car, such as whether it is sunny or rainy. Such an answer may not satisfy the user's question.

[0026] To address the aforementioned problems in related technologies, this application provides a human-computer interaction method. The following explains a feasible implementation process of this human-computer interaction method.

[0027] Figure 2 This is a flowchart illustrating the human-computer interaction method provided in the embodiments of this application. Please refer to... Figure 2 The method includes: S210: Obtain voice request.

[0028] It should be noted that the subject executing this method can be the aforementioned vehicle terminal, such as a smart vehicle system.

[0029] Optionally, the voice request is a request to instruct the vehicle terminal to execute an instruction, or a request to ask a question to the vehicle terminal.

[0030] The voice request can be a sentence spoken by the user, a question asked by the user as explained above, or a command from the user.

[0031] For user questions, it could be the aforementioned request to ask a question to the vehicle terminal; for user instructions, it could be the aforementioned request to instruct the vehicle terminal to execute an instruction.

[0032] The vehicle terminal can obtain the above voice request and generate a corresponding response based on the content of the voice request, that is, the reply information for the voice request.

[0033] S220: Generate response information for the voice request based on the voice request and the large language model.

[0034] Among them, the large language model is trained based on the target sample data in the sample data. The target sample data is sample data that meets the requirements of data quality and data diversity.

[0035] In the actual training process, not all sample data is used. Target sample data can be determined from the sample data, and the large language model is trained based on the target sample data.

[0036] In one embodiment, the target sample data can be sample data that meets both data quality requirements and data diversity requirements. The data quality requirement means that the target sample data needs to be candidate sample data, and the sample data can be screened for quality to determine the candidate sample data. The data diversity requirement means that the target sample data needs to be diverse sample data, and the sample data can be screened for diversity to determine the diverse sample data.

[0037] It should be noted that the data diversity requirement refers to the number of types of sample data included being greater than or equal to the type threshold.

[0038] The type threshold can be a preset threshold, which is not specifically limited here. It can be set according to actual needs, such as hundreds or thousands of types.

[0039] For multiple sample data, the more types of these sample data there are, the greater the differences between these sample data, and the more diverse these sample data will be, which means that the above-mentioned data diversity requirement is met.

[0040] Optionally, the candidate sample data can be data that significantly improves the model accuracy during training; diverse data can refer to multiple data with large differences.

[0041] In one embodiment, the target sample data includes sample voice requests and sample response information. The sample voice requests are determined from historical voice requests, and the sample response information is obtained by modifying the historical response information.

[0042] The sample response information is a reply based on the sample voice request.

[0043] For example, the type of sample data can specifically refer to the type of sample voice request. The type of sample voice request can include, for example, the type of request that needs to be performed, such as the request to open the car door or open the car window; and the type of request that needs to be answered, such as the request to inquire about weather or traffic. In the process of screening sample data, multiple different types of sample data can be selected as the above-mentioned target sample data to obtain target sample data that meets the requirements of data diversity.

[0044] It should be noted that a large language model can be a neural network model that is trained multiple times. For example, the large language model can be trained once at regular intervals, and there are no specific restrictions here.

[0045] In each training session, the sample data used can include sample voice requests and sample response information. The sample data used in each training session can be determined from historical data prior to this training session; for example, sample voice requests can be determined from historical voice requests, and sample response information can be obtained by correcting historical response information.

[0046] It should be noted that since large language models can be trained multiple times, the data generated in the actual work process before each training can be used as historical data. That is to say, the sample data used in each training process can be different, and can all be historical data generated in the actual application process in the previous period.

[0047] Among them, historical voice requests can be voice requests initiated by users during actual work, and historical response information can be response information generated by the large language model in response to user requests during actual work. If the historical response information is correct, it can be used as sample response information; if the historical response information is incorrect, it can be corrected to obtain sample response information.

[0048] The correctness of the response can be determined based on the user's reaction. For example, if the user requests a new response after the response is generated, or if the user's tone is detected as angry, it can be determined that the response is incorrect and needs to be corrected. The correction can be done manually.

[0049] During training, sample voice requests can be identified from historical voice requests, and sample response information can be obtained after correcting historical response information. This yields sample data, which is then used to train a large language model. The trained large language model can then be used in practical applications to generate response information based on voice requests.

[0050] It should be noted that during the use of the large language model, training can be performed periodically, and the model can be used again after training is completed. The data generated during the use can be used as new historical data. In the next training process, new sample data can be generated based on this historical data, thereby realizing the training of the model. After multiple training sessions, a more suitable large language model can be obtained. Based on this large language model, response information that meets user needs can be generated.

[0051] For example, after obtaining a trained large language model, a voice request can be input into the large language model, which will then generate and output the corresponding response information, which can be presented to the user in the form of voice, text, or image, thereby providing a reply to the user.

[0052] The human-computer interaction method provided in this application embodiment can acquire voice requests and generate response information for the voice requests based on the voice requests and a large language model. The large language model is trained on target sample data from the sample data. The target sample data is sample data that meets both data quality and data diversity requirements. Since the target sample data used meets both data quality and data diversity requirements, these data are those with good training results. Training the large language model based on this data not only improves training efficiency but also, because it meets the data diversity requirement, allows training on the large language model based on various types of sample data, thus improving the comprehensiveness of the training and enhancing the training effect of the large language model. This allows the large language model to output more accurate and reasonable response information. During the training process, the large language model tends to generate correct response information, further improving the accuracy of intelligent vehicle systems in answering questions.

[0053] The specific structural relationships of the large language model provided in the embodiments of this application will be explained below.

[0054] Figure 3 This is a schematic diagram of the structure of the large language model provided in the embodiments of this application. Please refer to... Figure 3 The input data for this large language model can be the aforementioned voice request, and the output data for this large language model can be the aforementioned response information.

[0055] The voice request can be, for example, a piece of speech, or feature information or text information derived from a piece of speech. There are no specific restrictions here. If it is a piece of speech, a corresponding speech conversion module can be added to the large language model to convert the speech into feature information or text information. If it is feature information or text information, the large language model can directly perform the corresponding processing without speech conversion.

[0056] The response can be a voice message or a text message. If it is a voice message, the in-vehicle terminal can play the voice message after receiving it. If it is a text message, the in-vehicle terminal can display the text message, or the in-vehicle terminal can convert the text message into a voice message based on the voice broadcast module in the in-vehicle terminal and play the converted voice message.

[0057] It should be noted that, in the application of the above-mentioned large language model, a response to a voice request can be obtained and output after the large language model processes the request. In actual implementation, the large language model not only needs to be applied, but also needs to be trained.

[0058] In one embodiment, Figure 3 The structure shown is one feasible structure of a large language model. In actual implementation, the structure of the above large language model can be adjusted according to actual needs. For example, in addition to the above voice request, the input can also include historical data and perceptual information.

[0059] Historical data can include user input information from previous rounds and output information from the large language model. Additionally, it can also record the corresponding user's identity information. Perceptual information can include the content displayed on the current screen.

[0060] The output response information can be obtained by combining information. For example, the output of a large language model can include: an output interface, multiple output contents, and preset standard response instructions. The above response information can be obtained by combining the above multiple output information. The response information can be a complete and reasonable statement.

[0061] The following explains one of the feasible training processes for the aforementioned large language model provided in the embodiments of this application.

[0062] Figure 4 This is a schematic diagram illustrating the process of training a large language model as provided in the embodiments of this application. Please refer to... Figure 4 The steps for training a large language model are as follows: S410: Filter out target sample data from the sample data.

[0063] It should be noted that, as explained above, the sample data is selected from historical data. In the actual training process, not all sample data is used. Target sample data can be determined from the sample data, and the large language model is trained based on the target sample data.

[0064] In one embodiment, the target sample data can be sample data that meets both data quality requirements and data diversity requirements. The data quality requirement means that the target sample data needs to be high-quality sample data, and the sample data can be screened for quality to determine the candidate sample data. The data diversity requirement means that the target sample data needs to be diverse sample data, and the sample data can be screened for diversity to determine the diverse sample data.

[0065] Alternatively, high-quality data can be data that significantly improves the model's accuracy during training; diverse data can refer to multiple data sets with significant differences.

[0066] S420: Based on the target sample data, perform supervised fine-tuning SFT iteration and direct preference optimization DPO iteration on the large language model to obtain the trained large language model.

[0067] It should be noted that after obtaining multiple target sample data through the above screening process, the large language model can be trained based on the target sample data. The training can include supervised fine-tuning SFT iteration and direct preference optimization (DPO) iteration.

[0068] In one embodiment, during the supervised fine-tuning of the SFT iteration: manually labeled input-output pairs are used as training data, allowing the model to learn the patterns and rules of these samples, thereby adjusting the model's parameters so that the model's output is closer to the expected output represented by the labeled data, ultimately enabling the model to better adapt to a specific task.

[0069] For the SFT (Supervised Fine-Tuning) training process, a large amount of labeled data is first prepared. This data is then input into a pre-trained language model, and the difference between the model's output and the labeled output is calculated. This difference is typically measured using a loss function such as cross-entropy. Next, the model's parameters are updated using backpropagation algorithms and optimizers such as Adam to minimize the loss function. This process is repeated multiple times until the model's performance on the validation set reaches a satisfactory level or stabilizes.

[0070] In one embodiment, during the Direct Preference Optimization (DPO) iteration, human preference data on the model's output can be used to guide training through a reward model or preference model, enabling the model to generate results that better align with user preferences. It optimizes the model directly based on preference data, without requiring complex reinforcement learning steps as in traditional reinforcement learning human feedback (RLHF) methods.

[0071] The DPO (Direct Preference Optimization) training process begins by collecting human preference data, typically presented as comparison pairs. For example, for the same input, there are multiple different outputs, some of which better align with human preferences. This preference data is then used to train a reward model or preference model that scores the model's output, representing its consistency with human preferences. In each iteration, the parameters of the original language model are adjusted based on the reward model's score to maximize the match between the model's output and human preferences.

[0072] In the human-computer interaction method provided in this application embodiment, target sample data can be selected from the sample data; supervised fine-tuning SFT iteration and direct preference optimization (DPO) iteration are performed on the large language model based on the target sample data to obtain the trained large language model. Since the target sample data meets the requirements for data quality and data diversity, training the model based on this data can more accurately and quickly improve the model's accuracy, thereby obtaining more accurate and reasonable response information based on the trained large language model.

[0073] It should be noted that in the process of screening target sample data, high-quality screening can be performed first, followed by diversity screening, to obtain target sample data that meets the requirements. The following explains a feasible implementation process for screening target sample data provided in the embodiments of this application.

[0074] Figure 5 This is a schematic diagram of the process for filtering target sample data provided in the embodiments of this application. Please refer to... Figure 5 Select target sample data from the sample data, including: S510: Select candidate sample data from the sample data.

[0075] It should be noted that in the actual implementation process, candidate sample data can be selected from the sample data first. The candidate sample data can be sample data used as prompt information in application models where the instruction following difficulty is greater than or equal to the difficulty threshold, and / or sample data used as prompt information in application models where the loss value is greater than or equal to the loss threshold.

[0076] The instruction following difficulty refers to the degree of challenge the model faces when following specific instructions to perform a task. The specific instructions are the prompts provided by the application model, which are also the sample data used as prompts.

[0077] Optionally, the application model can be a neural network model used to filter data during the model training phase, or it can be a large language model, for example, a large language model that has not yet been trained. The application model needs to execute specific instructions during training, and these specific instructions can serve as prompts. In actual implementation, each sample data can be used as a prompt to allow the application model to execute.

[0078] It should be noted that the application model can use each sample data as a prompt for execution. During the execution process, the instruction following difficulty of the application model based on each prompt can be calculated. If the instruction following difficulty is greater than or equal to the difficulty threshold, the sample data used as prompts in the application model can be used as the target sample data.

[0079] The loss value refers to the overall loss of the test set during the application model's execution of the test set data. The test set data can be data used to test the application model.

[0080] It should be noted that the application model can use each sample data as a prompt message for execution. During the execution process, the loss value of the application model based on each prompt message can be calculated. If the loss value is greater than or equal to the loss threshold, the sample data used as prompt messages in the application model can be used as the target sample data.

[0081] Optionally, in the actual implementation process, at least one of the above two methods can be used to determine the candidate sample data. For example, the candidate sample data can be determined by the instruction following difficulty screening method, or by the loss value screening method. Alternatively, the above two methods can be used to select the candidate sample data by using the intersection or union method.

[0082] After filtering the candidate sample data using the above methods, it is possible to filter sample data with diversity.

[0083] S520: Based on the cumulative sample data, select the target sample data from the candidate sample data.

[0084] Among them, the distance between the location of the target sample data mapped in the target space and the location of the cumulative sample data mapped in the target space is greater than or equal to the distance threshold.

[0085] It should be noted that the accumulated sample data can be a type of cold start data, which refers to the training data that has already been accumulated in the application model.

[0086] It can filter out data of different types from the existing cumulative sample data from the candidate sample data based on the existing cumulative sample data, thereby achieving diverse filtering.

[0087] It should be noted that a spatial mapping method can be used for diversity screening. Each cumulative sample data and candidate sample data can be treated as a multi-dimensional vector data. For example, if it is a two-dimensional vector data, it can be mapped to a two-dimensional space; if it is a three-dimensional vector data, it can be mapped to a three-dimensional space; if it is an N-dimensional vector data, it can be mapped to an N-dimensional space, where N can be a positive integer greater than or equal to 1. The diversity is determined based on the position of these sample data in the target space.

[0088] Among them, data that is far away from each cumulative sample data can be selected from the candidate sample data as the above target sample data. That is to say, the distance between the position of the target sample data mapped in the target space and the position of any cumulative sample data mapped in the target space is greater than or equal to the distance threshold.

[0089] In the human-computer interaction method provided in this application embodiment, candidate sample data can be selected from the sample data; target sample data can be selected from the candidate sample data based on the cumulative sample data. The candidate sample data can be sample data used as prompts in application models where the instruction following difficulty is greater than or equal to a difficulty threshold, and / or sample data used as prompts in application models where the loss value is greater than or equal to a loss threshold; the distance between the position of the target sample data mapped in the target space and the position of any cumulative sample data mapped in the target space is greater than or equal to a distance threshold. Through the above-described high-quality selection process and diversity selection process, target sample data that balances high quality and diversity can be obtained.

[0090] The following sections will explain the process of selecting candidate sample data and screening data with diversity.

[0091] In one embodiment, selecting candidate sample data from the sample data includes: using each sample data as a prompt for the application model, and selecting candidate sample data from the sample data based on the confusion level of the application model with different prompts; and / or, using each sample data as a prompt for the application model, inputting test set data into the application model with different prompts, and selecting candidate sample data from the sample data based on the loss value of each application model.

[0092] The instruction following difficulty can be calculated based on the confusion level. The instruction following difficulty of each application model with different prompts can be determined by calculating the confusion level of the application model, and the candidate sample data can be screened based on the instruction following difficulty.

[0093] Optionally, the loss value can be calculated based on the test set data. The loss value of each application model with different prompts can be determined by inputting the test set data, and the candidate sample data can be filtered based on the loss value.

[0094] In one embodiment, candidate sample data is selected from the sample data based on the confusion level of the application models with different prompts, including: determining the instruction following difficulty of the application models with different prompts based on the ratio of the first confusion level and the second confusion level; and using the prompts corresponding to the application models with instruction following difficulty greater than or equal to the difficulty threshold as candidate sample data.

[0095] It should be noted that for different application models with different prompts, the first and second perplexity of the application model can be calculated separately.

[0096] The first level of perplexity is the perplexity of the application model in generating the target response information based on the target request, and the second level of perplexity is the perplexity of the application model in generating the target response information.

[0097] In other words, the first level of perplexity is the perplexity of the application model generating the target response information based on a specific target request, while the second level of perplexity is the perplexity of the application model generating the target response information under any circumstances.

[0098] It's important to note that perplexity (PPL) is a crucial metric for evaluating the performance of large language models. PPL measures the model's uncertainty regarding textual data. For example, lower perplexity indicates more accurate predictions and better text modeling; higher perplexity indicates greater confusion and poorer predictive ability.

[0099] For example, low-perplexity models are able to predict the next word in text more accurately, demonstrating stronger text generation and comprehension capabilities. For instance, in machine translation tasks, lower-perplexity models can more accurately generate sentences in the required language, making the translation results more consistent with natural language expression habits.

[0100] The formula for calculating the difficulty of instruction following is as follows: IFDθ(Q,A)=PPLθ(A|Q) / PPLθ(A); Where IFDθ(Q, A) refers to the instruction following difficulty of the model, PPLθ(A|Q) is the first perplexity mentioned above, and PPLθ(A) is the second perplexity mentioned above; A refers to the target response information, and Q refers to the target request.

[0101] It should be noted that after determining the instruction following difficulty using the above calculation formula, the relationship between the instruction following difficulty and the difficulty threshold can be used to determine whether the prompt information corresponding to the application model can be used as the above candidate sample data.

[0102] In the human-computer interaction method provided in this application embodiment, the instruction following difficulty of the application model with different prompts can be determined based on the ratio of the first confusion degree and the second confusion degree of the application model with different prompts; the prompts corresponding to the application models with instruction following difficulty greater than or equal to the difficulty threshold are used as candidate sample data. Specifically, by calculating the confusion degree, the instruction following difficulty of the application model with different prompts can be calculated more quickly and accurately, thereby enabling more accurate screening of candidate sample data.

[0103] The above process explains the selection of candidate sample data based on instruction following difficulty. The following explains the selection of candidate sample data based on loss value provided in the embodiments of this application.

[0104] In one embodiment, selecting candidate sample data from the sample data based on the loss value of each application model includes: using the prompt information corresponding to the application model whose loss value is greater than or equal to the loss threshold as candidate sample data.

[0105] It should be noted that after obtaining application models with different prompts, test set data can be input into each application model to obtain the corresponding loss value. The prompts of the application models with loss values ​​greater than or equal to the loss threshold can be used as candidate sample data.

[0106] The loss value can be a percentage between 0 and 1. The closer it is to 1, the smaller the loss; the closer it is to 0, the larger the loss. In other words, the larger the loss value, the smaller the loss of the applied model, and vice versa.

[0107] It should be noted that the loss values ​​can be compared directly using the methods described above, or the difference in loss can be compared, as detailed below: In one embodiment, selecting candidate sample data from the sample data based on the loss value of each application model includes: inputting test set data into the application model without prompts, determining the loss difference between the loss value of the application model with prompts and the loss value of the application model without prompts, and using the prompts corresponding to the application models whose loss difference is greater than or equal to the difference threshold as candidate sample data.

[0108] It should be noted that the test set data can be first input into the application model without prompts to obtain a loss value. Alternatively, the test set data can be input into each application model with prompts to obtain a loss value. Based on these two loss values, the corresponding loss difference can be obtained.

[0109] For example, if the loss value of an application model without prompts is 0.33, and the loss value of an application model with prompts is 1, then the loss difference can be the difference between the loss value of the application model with prompts and the loss value of the application model without prompts, which is 0.67.

[0110] Assuming the difference threshold is 0.5, then 0.67 is greater than 0.5, so the sample data of the application model with prompt information can be used as the candidate sample data.

[0111] The specific formula is as follows: case score=one shot score-zero shot score; Here, the case score is the loss difference mentioned above, the one-shot score can be the loss value of any application model with prompting information, and the zero-shot score can be the loss value of an application model without prompting information.

[0112] In the human-computer interaction method provided in this application embodiment, the prompt information corresponding to the application model with a loss value greater than or equal to a loss threshold can be used as candidate sample data. Alternatively, test set data can be input into the application model without prompt information, and the loss difference between the loss value of the application model with prompt information and the loss value of the application model without prompt information can be determined; the prompt information corresponding to the application model with a loss difference greater than or equal to the difference threshold can be used as candidate sample data. By comparing the loss value or loss difference, the loss situation of each application model with prompt information can be accurately calculated, thereby allowing the sample data used as prompt information in the application model that meets the loss value requirement or loss difference requirement to be used as candidate sample data, improving the accuracy of determining candidate sample data.

[0113] In one embodiment, any one or more of the above methods can be used to determine candidate sample data from the sample data, and then diversity screening can be further performed based on the candidate sample data to obtain target sample data.

[0114] The following explains one feasible implementation process for data diversity screening provided in the embodiments of this application.

[0115] Figure 6 This is a schematic diagram of the diversity screening process provided in the embodiments of this application. Please refer to... Figure 6 Based on the cumulative sample data, target sample data is selected from the candidate sample data, including: S610: Based on the mapping model, the cumulative sample data and the candidate sample data are mapped to the target space.

[0116] The mapping model is a model obtained by training on accumulated sample data. As explained above, the accumulated sample data is cold-start data, that is, the training data that has been accumulated so far. A large language model that has not been trained can be trained on this data to obtain the above mapping model. In other words, the mapping model can also be a large language model, and its structure can be similar to the aforementioned application model and large language model.

[0117] This mapping model can map the cumulative sample data and the candidate sample data to the target space. Taking two-dimensional data as an example, if both the cumulative sample data and the candidate sample data are two-dimensional data, then these data can be mapped to the corresponding positions in the two-dimensional space. Each sample data can be a two-dimensional vector in the two-dimensional space.

[0118] Correspondingly, if the data is three-dimensional or higher, it can be mapped to the corresponding dimension of space to determine the location.

[0119] S620: Determine the target distance between the position of each candidate sample data in the target space and the position of each cumulative sample data in the target space.

[0120] It should be noted that after mapping the target space, the position of each candidate sample data in the target space can be determined, as can the position of each cumulative sample data in the target space. For each candidate sample data, the distance between the position of the candidate sample data in the target space and the position of each cumulative sample data in the target space can be calculated.

[0121] For example, in a two-dimensional space, the target distance can be calculated based on the distance between two vectors in the two-dimensional space; in a multi-dimensional space, the target distance can be calculated based on the formula for calculating the distance between two vectors in the corresponding dimension.

[0122] S630: If the distance to each target is greater than or equal to the distance threshold, the corresponding candidate sample data will be used as the target sample data.

[0123] It should be noted that, assuming there are 10 cumulative sample data, there can be 10 target distances for each candidate sample data. These target distances represent the distances from the candidate sample data to each cumulative sample data in the target space.

[0124] If the distance to each target is greater than or equal to the distance threshold, then the candidate sample data can be used as target sample data with diversity. Multiple target distances for each candidate sample data can be calculated in the above manner, thereby filtering out multiple target sample data.

[0125] It should be noted that distance in the target space can represent the similarity of sample data. The closer the sample data is, the higher the similarity. The distance can be used to filter out candidate sample data with large differences in similarity, so as to obtain candidate sample data with diversity, which is the target sample data. The target sample data has the characteristics of both high quality and diversity.

[0126] The human-computer interaction method provided in this application embodiment can map cumulative sample data and candidate sample data into a target space based on a mapping model; determine the target distance between the position of each candidate sample data in the target space and the position of each cumulative sample data in the target space; and, if each target distance is greater than or equal to a distance threshold, use the corresponding candidate sample data as the target sample data. By mapping to the target space, the positions of different sample data in the target space can be accurately calculated, thereby determining diverse sample data based on the target distance, achieving the determination of sample data that is both diverse and of high quality.

[0127] The following diagram illustrates how to map and display sample data in the target space, using a two-dimensional spatial mapping as an example.

[0128] Figure 7 This is a schematic diagram of the target space mapping structure provided in the embodiments of this application. Please refer to... Figure 7 , Figure 7Each circle shown can be the location of a sample data point. The sample data within the range of D0 can be the aforementioned candidate sample data, and the data within the range of Q0 can be the aforementioned cumulative sample data. The distance between each candidate sample data point and each cumulative sample data point can be determined, thereby filtering out the target sample data.

[0129] It should be noted that, Figure 6 The method shown uses the candidate sample data as the target sample data when the distance to each target is greater than or equal to the distance threshold. In actual implementation, it can also use the candidate sample data as the target sample data when more than a certain number of target distances are greater than or equal to the distance threshold.

[0130] pass Figure 7 The method shown can calculate the target distance between any candidate sample data and any cumulative sample data, and then the target sample data can be selected from the candidate sample data based on the corresponding target distance.

[0131] To more clearly explain the process of filtering target data provided in the embodiments of this application, the process of filtering target sample data from sample data is explained below with schematic diagrams.

[0132] Figure 8 This is another flowchart illustrating the process of filtering target sample data provided in this application embodiment. Please refer to... Figure 8 For sample data, selection of candidate sample data can be obtained by using the instruction following difficulty filtering method and / or loss value filtering method.

[0133] After obtaining the candidate sample data, the target distance between each candidate sample data and each cumulative sample data can be determined by mapping them to the cumulative sample data. By comparing the target distances, specific and diverse sample data can be further filtered out from the candidate sample data, which is the aforementioned target sample data.

[0134] After obtaining the target sample data through the above method, the large language model can be trained based on the target sample data. The following explains one feasible implementation process for actually training the large language model.

[0135] Figure 9 This is a schematic diagram illustrating the iterative training process of a large language model provided in the embodiments of this application. Please refer to... Figure 9 Based on the target sample data, the large language model is subjected to supervised fine-tuning SFT iteration and direct preference optimization DPO iteration to obtain the trained large language model, including: S910: Supervised fine-tuning of the large language model using the first target sample data in the target sample data through SFT iteration.

[0136] It should be noted that after obtaining the target sample data, the first target sample data can be selected from the target sample data, and the large language model can be trained by SFT iteration using the first target sample data.

[0137] The first target sample data is the target sample data selected from the target sample data to construct the STF data.

[0138] Optionally, after obtaining the target sample data, sampling can be performed through the "M + ICL" neural network layer in the large language model to generate multiple candidate expectations. Then, the optimal expectation is obtained by filtering based on the "LLM-as-a-Judge (M layer can be reused)" neural network layer in the large language model to construct the SFT data of step T0, which is the first target sample data mentioned above. Then, multiple rounds of SFT iterative training can be performed until the SFT iteration termination condition is met.

[0139] It should be noted that there are multiple ways to terminate SFT. For example, you can set a fixed number of SFT iterations to constrain it, or you can constrain it by the proportion of the first target sample data, or you can constrain it by the difference in the number of the first target sample data in two adjacent SFT training rounds, or you can constrain it by the difference in accuracy in the entire large language model. There are no specific restrictions here. You can choose one of the constraint methods to set the termination condition according to the actual needs.

[0140] S920: After the SFT iteration termination condition is met, the large language model is subjected to direct preference optimization (DPO) iteration based on the second target sample data in the target sample data.

[0141] Optionally, if the SFT iteration termination condition is not met, the iteration can be repeated until the SFT iteration termination condition is met; after the SFT iteration termination condition is met, DPO iteration can be performed.

[0142] It should be noted that second target sample data can be selected from the target sample data, and the large language model can be trained using DPO iterative training with the second target sample data.

[0143] The second target sample data is the target sample data selected from the target sample data to serve as the seed set for DPO iteration.

[0144] Optionally, after completing the SFT iteration, target sample data with higher training gain can be selected as the seed set for the T-th iteration of DPO, which is the second target sample data mentioned above. This data can be sampled using the "M + ICL" neural network layer in the large language model to generate multiple candidate expectations. Then, these expectations are selected based on the "LLM-as-a-Judge" neural network layer in the large language model to construct...<chosen, rejected> In the pair data, where chosen refers to the correct result and rejected refers to the incorrect result, the pair data can be processed using the DPO technique, which provides both a correct and an incorrect answer. Through DPO, the large language model can learn to select the correct result with a high probability by contrastive learning. Furthermore, reinforcement learning can be performed through DPO to further enhance the ability of the large language model trained by SFT to understand some speech requests that are difficult to comprehend. Then, multiple rounds of DPO iterative training can be performed until the DPO iteration termination condition is met.

[0145] It should be noted that the termination condition for DPO iteration can be constrained by the difference in the number of second target sample data in two adjacent DPO training rounds, or by the difference in accuracy in the entire large language model. No specific restrictions are imposed here, and one of the constraint methods can be selected to set the termination condition according to actual needs.

[0146] In one embodiment, the Direct Preference Optimization (DPO) iteration of the large language model is performed based on the second target sample data in the target sample data, including: generating multiple candidate sample data by sampling based on the second target sample data and the large language model; selecting desired sample data from the candidate sample data; and performing DPO reinforcement learning based on the desired sample data, wherein the desired sample data is candidate sample data used to improve the accuracy of the model's response.

[0147] The candidate sample data is as described above.<chosen, rejected> Pair data, the expected sample data is the data that is more difficult for the large language model to understand in the candidate sample data. This data can be labeled after screening, which can improve the accuracy of the model's answer.

[0148] By sampling to generate candidate sample data and filtering expected sample data, the ability of large language models to understand some difficult speech requests can be further enhanced, thereby improving the accuracy of the response information generated by large language models.

[0149] S930: After satisfying the DPO iteration termination condition, the trained large language model is obtained.

[0150] Optionally, if the DPO iteration termination condition is not met, the iteration can be repeated until the DPO iteration termination condition is met; after the DPO iteration termination condition is met, the training can be completed and a trained large language model can be obtained.

[0151] In the human-computer interaction method provided in this application embodiment, a large language model can be subjected to supervised fine-tuning SFT iteration based on the first target sample data in the target sample data; after the SFT iteration termination condition is met, the large language model can be subjected to direct preference optimization (DPO) iteration based on the second target sample data in the target sample data; after the DPO iteration termination condition is met, the trained large language model is obtained. Specifically, by sequentially performing SFT iteration and DPO iteration on the target sample data, the large language model can be trained. During the training process, especially during the DPO iteration, reinforcement learning can be performed on the parts of the data that the large language model finds difficult to understand, thereby improving the understanding ability of the large language model and the accuracy of the response information output by the large language model.

[0152] It should be noted that in this embodiment, a data flywheel approach (i.e., a data-driven automated closed-loop process) can be used to train the large language model. After performing SFT iteration and DPO iteration in sequence, the trained large language model can be obtained.

[0153] The SFT iteration process and DPO iteration process provided in the embodiments of this application will be explained below.

[0154] Figure 10 This is a schematic diagram of the SFT iteration process provided in the embodiments of this application. Please refer to... Figure 10 The training process for SFT can include: The aforementioned target sample data can be used as prompt information and input into the initial mt model. It can then pass through neural network layers such as SFT-fail-Vt, SFT-fail-repre-Vt, ICL, and M in sequence to achieve sampling at the neural network level and generate multiple candidate expectations. Then, it can be filtered based on the "LLM-as-a-Judge" neural network layer in the large language model to obtain the first target sample data. After obtaining the first target sample data, it can be input into the mt model again for the next round of iterative training. This process is repeated until the SFT iteration termination condition is met.

[0155] Here, mt can be the large language model in each iteration, where t refers to the t-th iteration of SFT, and M can be a neural network layer, specifically a neural network layer with a large language model structure.

[0156] Figure 11 This is a schematic diagram of the DPO iteration process provided in the embodiments of this application. Please refer to... Figure 11 The training process for DPO can include: The aforementioned target sample data can be input as prompt information into the mt' model, and then sequentially passed through neural network layers such as DPO-fail-Vt, DPO-fail-repre-Vt, ICL, and M. This allows for sampling at the neural network level and the generation of multiple candidate expectations. These expectations can then be filtered based on the "LLM-as-a-Judge" neural network layer in the large language model to obtain the candidate sample data, which is the aforementioned...<chosen, rejected> The pair data can be used to input the candidate sample data into the mt' model again for the next round of iterative training. This process is repeated until the DPO iteration termination condition is met.

[0157] Here, mt' can be the large language model in each iteration, where t' refers to the t'th iteration of DPO.

[0158] It should be noted that in the process of selecting sample data, not all historical data is used as sample data. Sample data can be obtained by further targeted filtering from historical data. For example, data that is frequently asked and data that is answered incorrectly can be used as sample data.

[0159] In one embodiment, the method further includes: using historical voice requests that occur more than or equal to a frequency threshold as sample voice requests; and / or using historical voice requests that include negative sentiment tags as sample voice requests, wherein the negative sentiment tags are used to characterize that the historical response information corresponding to the historical voice request does not meet the user's needs.

[0160] Figure 12 This is a schematic diagram illustrating the determination of sample data provided in the embodiments of this application. Please refer to... Figure 12 The method includes: S1210: Use historical voice requests that appear more than or equal to the number of times as sample voice requests.

[0161] The frequency threshold can be, for example, 5, and historical voice requests that appear more than or equal to 5 times can be used as sample voice requests.

[0162] S1220: Use historical voice requests that include negative sentiment labels as sample voice requests.

[0163] Negative sentiment labels can be generated based on user responses. For example, if the user instructs to repeat the response after the in-vehicle terminal generates a response, or if the user's tone is poor, it can be determined that the historical response information corresponding to the historical voice request does not meet the user's needs. Such information can be labeled with negative sentiment labels, and historical voice requests with negative sentiment labels can be used as sample voice requests.

[0164] S1230: Correct the response information corresponding to each sample voice request to obtain sample response information.

[0165] The above method can be used to identify sample voice requests. After correcting the corresponding response information for these sample voice requests, sample response information can be obtained. Combining the sample voice requests and sample response information can yield sample data.

[0166] In the human-computer interaction method provided in this application embodiment, historical voice requests that occur more than or equal to a frequency threshold can be used as sample voice requests; and / or, historical voice requests including negative emotion tags can be used as sample voice requests. By selecting sample voice requests from historical voice requests, the large language model can be trained based on specific sample voice requests. This allows the large language model to be trained with emphasis on frequently occurring voice requests and voice requests corresponding to user-unsatisfactory responses, thereby improving the accuracy of the large language model.

[0167] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0168] Based on the foregoing embodiments, this application provides a human-computer interaction device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0169] Figure 13This is a schematic diagram of the human-computer interaction device provided in the embodiments of this application. Please refer to... Figure 13 In another aspect of the embodiments of this application, a human-computer interaction device is also provided, which is applied to an in-vehicle terminal. The device includes: an acquisition module 1310 and a response module 1320. The acquisition module 1310 is used to acquire voice requests, which are requests to instruct the vehicle terminal to execute instructions or requests to ask questions to the vehicle terminal. The response module 1320 is used to generate response information for the voice request based on the voice request and the large language model. The large language model is trained based on the target sample data in the sample data. The target sample data is sample data that meets the data quality requirements and data diversity requirements. The data diversity requirement refers to the number of types of sample data included being greater than or equal to the type threshold.

[0170] In one embodiment, the device further includes: a training module 1330, which is used to select target sample data from the sample data; and to perform supervised fine-tuning SFT iteration and direct preference optimization DPO iteration on the large language model based on the target sample data to obtain the trained large language model.

[0171] In one embodiment, the training module 1330 is specifically used to filter candidate sample data from the sample data. The candidate sample data is sample data used as prompt information in application models where the instruction following difficulty is greater than or equal to a difficulty threshold, and / or sample data used as prompt information in application models where the loss value is greater than or equal to a loss threshold. Based on the cumulative sample data, target sample data is filtered from the candidate sample data. The distance between the position of the target sample data mapped in the target space and the position of the cumulative sample data mapped in the target space is greater than or equal to a distance threshold.

[0172] In one embodiment, the training module 1330 is specifically used to use each sample data as a prompt for the application model, and to select candidate sample data from the sample data based on the perplexity of the application model with different prompts; and / or, to use each sample data as a prompt for the application model, and to input the test set data into the application model with different prompts, and to select candidate sample data from the sample data based on the loss value of each application model.

[0173] In one embodiment, the training module 1330 is specifically used to determine the instruction following difficulty of the application model with different prompts based on the ratio of the first perplexity and the second perplexity of the application model with different prompts. The first perplexity is the perplexity of the application model generating the target response information based on the target request, and the second perplexity is the perplexity of the application model generating the target response information. The prompts corresponding to the application models with instruction following difficulty greater than or equal to the difficulty threshold are used as candidate sample data.

[0174] In one embodiment, the training module 1330 is specifically used to use the prompt information corresponding to the application model with a loss value greater than or equal to the loss threshold as candidate sample data.

[0175] In one embodiment, the training module 1330 is specifically used to input test set data into the application model without prompt information, determine the loss difference between the loss value of the application model with prompt information and the loss value of the application model without prompt information, and use the prompt information corresponding to the application model whose loss difference is greater than or equal to the difference threshold as candidate sample data.

[0176] In one embodiment, the training module 1330 is specifically used to map the cumulative sample data and candidate sample data into the target space based on the mapping model, which is a model obtained after training based on the cumulative sample data; determine the target distance between the position of each candidate sample data in the target space and the position of each cumulative sample data in the target space; and, if each target distance is greater than or equal to a distance threshold, use the corresponding candidate sample data as the target sample data.

[0177] In one embodiment, the training module 1330 is specifically used to perform supervised fine-tuning SFT iteration on the large language model based on first target sample data in the target sample data, wherein the first target sample data is the target sample data selected from the target sample data for constructing SFT data; after the SFT iteration termination condition is met, the large language model is performed direct preference optimization (DPO) iteration based on second target sample data in the target sample data, wherein the second target sample data is the target sample data selected from the target sample data as the seed set for DPO iteration; after the DPO iteration termination condition is met, the trained large language model is obtained.

[0178] In one embodiment, the training module 1330 is specifically used to generate multiple candidate sample data based on the second target sample data and the large language model; to select the desired sample data from the candidate sample data; and to perform DPO reinforcement learning based on the desired sample data, wherein the desired sample data is candidate sample data used to improve the accuracy of the model's answer.

[0179] In one embodiment, the training module 1330 is further configured to use historical voice requests that occur more than or equal to a frequency threshold as sample voice requests; and / or to use historical voice requests that include negative sentiment labels as sample voice requests, wherein the negative sentiment labels are used to characterize that the historical response information corresponding to the historical voice request does not meet the user's needs.

[0180] The human-computer interaction device provided in this application embodiment can acquire voice requests and generate response information for the voice requests based on the voice requests and a large language model. The large language model is trained on target sample data from the sample data. The target sample data is sample data that meets both data quality and data diversity requirements. Since the target sample data used meets both data quality and data diversity requirements, these data are those with good training results. Training the large language model based on this data not only improves training efficiency but also, because it meets the data diversity requirement, allows training on the large language model based on various types of sample data, thus improving the comprehensiveness of the training and enhancing the training effect of the large language model. This allows the large language model to output more accurate and reasonable response information. During the training process, the large language model tends to generate correct response information, further improving the accuracy of intelligent vehicle systems in answering questions.

[0181] The description of the above device embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0182] It should be noted that, in the embodiments of this application... Figure 13 The module division shown in the human-computer interaction device is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0183] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0184] Figure 14 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Please refer to... Figure 14 This application provides a computer device, which can be the aforementioned vehicle-mounted terminal, and its internal structure diagram can be as follows. Figure 14 As shown, the computer device includes a processor 1420, memory, and a network interface 1440 connected via a system bus 1410. The processor 1420 provides computing and control capabilities. The memory includes a non-volatile storage medium 1431 and internal memory 1432. The non-volatile storage medium 1431 stores an operating system, computer programs, and a database. The internal memory 1432 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 1431. The database is used to store data. The network interface 1440 is used to communicate with external terminals via a network connection. When the computer program is executed by the processor 1420, it implements the methods described above.

[0185] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0186] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0187] Those skilled in the art will understand that Figure 14 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In one embodiment, the human-computer interaction device provided in this application can be implemented as a computer program, and the computer program can be implemented in the form of, for example, Figure 14 The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0189] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0190] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0191] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0192] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0193] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0194] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0195] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0196] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0197] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0198] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0199] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0200] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0201] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A human-computer interaction method, characterized in that, Applied to vehicle-mounted terminals, the method includes: Obtain a voice request, which is either a request to instruct the vehicle terminal to execute an instruction or a request to ask a question to the vehicle terminal. The system generates a response to the voice request based on the voice request and a large language model. The large language model is trained on target sample data in the sample data. The target sample data is sample data that meets the data quality requirements and data diversity requirements. The data diversity requirement refers to the number of types of sample data included being greater than or equal to a type threshold.

2. The method according to claim 1, characterized in that, The training steps for the large language model are as follows: Select target sample data from the sample data; Based on the target sample data, the large language model is subjected to supervised fine-tuning SFT iteration and direct preference optimization (DPO) iteration to obtain the trained large language model.

3. The method according to claim 2, characterized in that, The step of filtering target sample data from the sample data includes: Select candidate sample data from the sample data. The candidate sample data are sample data used as prompt information in application models where the instruction following difficulty is greater than or equal to the difficulty threshold, and / or sample data used as prompt information in application models where the loss value is greater than or equal to the loss threshold. Based on the cumulative sample data, target sample data is selected from the candidate sample data, and the distance between the position of the target sample data mapped in the target space and the position of the cumulative sample data mapped in the target space is greater than or equal to a distance threshold.

4. The method according to claim 3, characterized in that, The process of selecting candidate sample data from the sample data includes: Each sample data point is used as a prompt for the application model, and candidate sample data is selected from the sample data based on the confusion level of the application model for different prompts; and / or, Each sample data is used as a prompt for the application model, and the test set data is input into the application model with different prompts. Based on the loss value of each application model, candidate sample data is selected from the sample data.

5. The method according to claim 4, characterized in that, The process of selecting candidate sample data from the sample data based on the confusion level of the application model with different prompts includes: The instruction following difficulty of the application model with different prompts is determined by the ratio of the first confusion degree and the second confusion degree of the application model with different prompts. The first confusion degree is the confusion degree of the application model in generating the target response information based on the target request, and the second confusion degree is the confusion degree of the application model in generating the target response information. The prompt information corresponding to the application model whose instruction following difficulty is greater than or equal to the difficulty threshold is used as the candidate sample data.

6. The method according to claim 4, characterized in that, The step of selecting candidate sample data from the sample data based on the loss value of each application model includes: The prompt information corresponding to the application model whose loss value is greater than or equal to the loss threshold is used as the candidate sample data.

7. The method according to claim 4, characterized in that, The step of selecting candidate sample data from the sample data based on the loss value of each application model includes: The test set data is input into the application model without prompts, and the loss difference between the application model with prompts and the application model without prompts is determined. The prompt information corresponding to the application model whose loss difference is greater than or equal to the difference threshold is used as the candidate sample data.

8. The method according to claim 3, characterized in that, The step of selecting target sample data from the candidate sample data based on cumulative sample data includes: The cumulative sample data and the candidate sample data are mapped to the target space based on the mapping model, which is a model obtained after training based on the cumulative sample data. Determine the target distance between the position of each candidate sample data in the target space and the position of each cumulative sample data in the target space; If each target distance is greater than or equal to a distance threshold, the corresponding candidate sample data will be used as the target sample data.

9. The method according to claim 2, characterized in that, The process of performing supervised fine-tuning SFT iterations and direct preference optimization (DPO) iterations on the large language model based on the target sample data to obtain the trained large language model includes: The large language model is subjected to supervised fine-tuning SFT iteration based on the first target sample data in the target sample data, wherein the first target sample data is the target sample data selected from the target sample data to construct the SFT data; After the SFT iteration termination condition is met, the large language model is subjected to Direct Preference Optimization (DPO) iteration based on the second target sample data in the target sample data. The second target sample data is the target sample data selected from the target sample data as the seed set for DPO iteration. After the DPO iteration termination condition is met, the trained large language model is obtained.

10. The method according to claim 9, characterized in that, The step of performing Direct Preference Optimization (DPO) iteration on the large language model based on the second target sample data in the target sample data includes: Multiple candidate sample data are generated by sampling based on the second target sample data and the large language model; Desired sample data is selected from the candidate sample data, and DPO reinforcement learning is performed based on the desired sample data, which is candidate sample data used to improve the accuracy of the model's answer.

11. The method according to claim 1, characterized in that, The method further includes: Historical voice requests that occur more than or equal to a certain number of times will be used as sample voice requests in the sample data; and / or, Historical voice requests, including those with negative sentiment labels, are used as sample voice requests in the sample data. The negative sentiment labels are used to indicate that the historical response information corresponding to the historical voice request does not meet the user's needs.

12. A human-computer interaction device, characterized in that, The device, used in vehicle-mounted terminals, includes: an acquisition module and a response module; The acquisition module is used to acquire voice requests, which are requests to instruct the vehicle terminal to execute instructions or requests to ask questions to the vehicle terminal. The response module is used to generate response information for the voice request based on the voice request and the large language model. The large language model is trained based on target sample data in the sample data. The target sample data is sample data that meets the data quality requirements and data diversity requirements. The data diversity requirements refer to the number of types of sample data included being greater than or equal to a type threshold.

Images