Instruction screening method and device, equipment and storage medium
By selecting high-quality and diverse target sample instructions from the sample instructions, the problem of poor model training effect was solved, and the accuracy of model output was improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies often result in poor model training performance, and well-trained models often fail to meet user needs.
High-quality sample instructions are selected from the sample instructions based on the difficulty of instruction following and the loss value. The target sample instructions are then selected for model training by combining the positional distance of the accumulated sample instructions.
This improves the effectiveness of model training, enabling the trained model to output more accurate results.
Smart Images
Figure CN121859969A_ABST
Abstract
Description
Technical Field
[0001] This application relates to model training technology, and includes, but is not limited to, an instruction screening method, apparatus, device, and storage medium. Background Technology
[0002] During the training process of a model, it is usually necessary to use corresponding sample instructions to train the model.
[0003] In related technologies, model training is based on arbitrary sample instructions, which leads to poor model training results, and the trained model cannot meet the user's needs. Summary of the Invention
[0004] In view of this, the instruction filtering method, apparatus, device, and storage medium provided in the embodiments of this application can improve the training effect of the model and enable the trained model to output more accurate results. The instruction filtering method, apparatus, device, and storage medium provided in the embodiments of this application are implemented as follows: One aspect of this application provides an instruction filtering method, including: High-quality sample instructions are selected from the sample instructions. High-quality sample instructions are those used as prompts in application models where the instruction following difficulty meets the following difficulty threshold, and / or those used as prompts in application models where the loss value meets the loss threshold. Based on the accumulated sample instructions, target sample instructions are selected from the high-quality sample instructions. The target sample instructions are mapped to the target space and the target distance between the target sample instructions and the accumulated sample instructions in the target space meets the distance threshold. The target sample instructions are used to train the target model.
[0005] Another aspect of the embodiments of this application also provides an instruction screening device, including: a quality screening module and a diversity screening module; The quality screening module is used to filter out high-quality sample instructions from the sample instructions. High-quality sample instructions are those that are used as prompts in application models where the instruction following difficulty meets the following difficulty threshold, and / or those that are used as prompts in application models where the loss value meets the loss threshold. The diversity filtering module is used to filter target sample instructions from high-quality sample instructions based on the cumulative sample instructions. The target sample instructions are mapped to the target space and the target distance between the target sample instructions and the cumulative sample instructions in the target space meets a distance threshold. The target sample instructions are used to train the target model.
[0006] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement the method of this application.
[0007] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.
[0008] The instruction filtering method, apparatus, device, and storage medium provided in this application embodiment can filter high-quality sample instructions from sample instructions. High-quality sample instructions are those used as prompts in application models where the instruction following difficulty meets a following difficulty threshold, and / or those used as prompts in application models where the loss value meets a loss threshold. Based on accumulated sample instructions, target sample instructions are filtered from high-quality sample instructions. The target distance between the position of the target sample instruction mapped in the target space and the position of the accumulated sample instructions mapped in the target space meets a distance threshold. Specifically, filtering instructions based on instruction following difficulty and / or loss value can obtain high-quality sample instructions, while filtering based on distance in the target space can obtain diverse target sample instructions. Based on the above filtering methods, target sample instructions with both high quality and diversity can be obtained. During the training of the target model based on target sample instructions, the training effect of the model can be improved, allowing the trained model to output more accurate results. Attached Figure Description
[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0010] Figure 1 This is a schematic diagram illustrating the application scenario provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the instruction filtering method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the process for screening high-quality sample instructions provided in the embodiments of this application; Figure 4 This is a flowchart illustrating the process of filtering high-quality sample instructions based on instruction following difficulty, as provided in the embodiments of this application. Figure 5 This is a flowchart illustrating the instructions for screening high-quality samples based on loss difference provided in the embodiments of this application; Figure 6 This is a schematic diagram of the diversity screening process provided in the embodiments of this application; Figure 7 This is a flowchart illustrating various methods for implementing diversity filtering provided in the embodiments of this application; Figure 8 This is a schematic diagram of the target space mapping structure provided in the embodiments of this application; Figure 9 This is a flowchart illustrating the instruction for filtering target samples provided in the embodiments of this application; Figure 10 This is a schematic diagram of the instruction filtering device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation
[0011] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.
[0012] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0013] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0014] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0015] To more clearly explain the instruction filtering method provided in the embodiments of this application, a possible application scenario in the embodiments of this application will be explained in detail below.
[0016] Figure 1 This is a schematic diagram of the application scenario provided in the embodiments of this application. Please refer to it. Figure 1 This scenario may include: an in-vehicle terminal 100, which may be, for example, a smart vehicle system. The in-vehicle terminal 100 is installed in the smart cockpit and can have intelligent dialogue with the user in the smart cockpit. For example, it can generate corresponding answers based on the user's questions, or execute corresponding tasks based on the user's instructions.
[0017] For example, users can ask the vehicle terminal 100 questions, such as: the vehicle's remaining battery / fuel level, whether the doors are closed, and the temperature inside and outside the vehicle; users can also instruct the vehicle terminal 100 to perform tasks, such as: commanding the vehicle terminal 100 to open the windows, commanding the vehicle terminal 100 to turn on the air conditioning, etc.
[0018] The vehicle terminal 100 can respond to user questions or commands. For example, for the above questions, the vehicle terminal can respond with the vehicle's actual remaining battery / fuel level, door open / close status, and interior / exterior temperature. The vehicle terminal can also respond after executing a command, such as: the window is open, the air conditioner is on.
[0019] In this scenario, one user or multiple users can interact with the vehicle terminal 100. There are no specific restrictions. For example, the user in the driver's seat can interact with the vehicle terminal 100, and the user in the passenger seat can also interact with the vehicle terminal 100.
[0020] During the interaction between the user and the vehicle terminal 100, communication can be achieved through language. For example, the user can ask a question, the vehicle terminal 100 can collect the corresponding audio signal, perform speech recognition, determine the user's intent, and then generate corresponding response content based on a large language model to reply to the user. The method of replying to the user is not limited to text reply, voice reply, or image reply, and no specific restrictions are made here.
[0021] For text replies, the vehicle terminal 100 can be equipped with a corresponding electronic screen, which can display the text reply. For voice replies, the vehicle terminal can be equipped with a corresponding speaker, which can play the voice reply. For image replies, the image reply can be displayed on the electronic screen.
[0022] In one embodiment, during the response process of the vehicle terminal, any one of the three response methods described above can be used, or any two or three methods can be combined to generate the corresponding response, without any specific limitations.
[0023] In the embodiments of this application, taking the in-vehicle terminal 100 as an intelligent vehicle system as an example, the intelligent vehicle system can typically generate a response that satisfies the user's question during operation. For example, after a user asks a question, the system analyzes the content of the question and generates a reasonable response.
[0024] In related technologies, model training is based on arbitrary sample instructions, which leads to poor model training results, and the trained model cannot meet the user's needs.
[0025] Therefore, in order to improve the accuracy of intelligent vehicle systems in answering questions, it is necessary to train the target model in the intelligent vehicle system. Target sample instructions can be selected, and then the intelligent vehicle system can be trained based on the target sample instructions.
[0026] It should be noted that the aforementioned target model can be, for example, a large language model set in the in-vehicle terminal 100, that is, the intelligent vehicle system.
[0027] To address the aforementioned problems in related technologies and enable the trained model to output more accurate results, this application provides an instruction filtering method. This method selects high-quality and diverse target sample instructions from sample instructions through instruction filtering, thereby enabling model training. The following explains one feasible implementation process of the instruction filtering method provided in this application.
[0028] Figure 2 This is a flowchart illustrating the instruction filtering method provided in the embodiments of this application. Please refer to... Figure 2 Instruction filtering methods include: S210: Filter out high-quality sample instructions from the sample instructions.
[0029] It should be noted that the subject executing this method can be the aforementioned vehicle terminal, such as a smart vehicle system.
[0030] It should be noted that in the actual implementation process, high-quality sample instructions can be selected from the sample instructions first. High-quality sample instructions are those used as prompts in application models where the instruction following difficulty meets the following difficulty threshold, and / or those used as prompts in application models where the loss value meets the loss threshold.
[0031] Among them, satisfying the following difficulty threshold means that the instruction following difficulty is greater than or equal to the following difficulty threshold, and satisfying the loss threshold means that the loss value is greater than or equal to the loss threshold.
[0032] The instruction following difficulty refers to the degree of challenge the model faces when following specific instructions to perform a task. The specific instructions mentioned above are the prompts provided by the application model, which are sample instructions that serve as prompts.
[0033] Optionally, the application model can be a neural network model used for data filtering during the model training phase. This model can be similar to the target model, both being large language models. For example, it can be a large language model that has not yet been trained. During training, the application model needs to execute specific instructions, which can serve as prompts. In actual implementation, each sample instruction can be used as a prompt for the application model to execute.
[0034] It should be noted that the application model can execute each sample instruction as a prompt. During execution, the instruction following difficulty of the application model based on each prompt can be calculated. If the instruction following difficulty is greater than or equal to the difficulty threshold, the sample instruction used as a prompt in the application model can be used as the target sample instruction.
[0035] The loss value refers to the overall loss of the test set during the application model's execution of the test set data. The test set data can be data used to test the application model.
[0036] It should be noted that the application model can execute each sample instruction as a prompt. During the execution process, the loss value of the application model based on each prompt can be calculated. If the loss value is greater than or equal to the loss threshold, the sample instruction used as a prompt in the application model can be used as the target sample instruction.
[0037] Optionally, in the actual implementation process, at least one of the above two methods can be used to determine high-quality sample instructions. For example, high-quality sample instructions can be determined by using the instruction following difficulty screening method, or by using the loss value screening method. Alternatively, the intersection or union of the above two methods can be used to select sample instructions that satisfy both methods or any one of them as high-quality sample instructions.
[0038] After filtering high-quality sample instructions using the methods described above, it is possible to filter sample instructions with greater diversity.
[0039] S220: Based on the accumulated sample instructions, select the target sample instructions from the high-quality sample instructions.
[0040] Among them, the target distance between the location of the target sample instruction mapped in the target space and the location of the cumulative sample instruction mapped in the target space satisfies the distance threshold.
[0041] Among them, satisfying the distance threshold means that the target distance is greater than or equal to the distance threshold.
[0042] It should be noted that the accumulated sample instruction can be a type of cold start data, which refers to the training data that has already been accumulated in the application model.
[0043] It can filter out data of different types from existing cumulative sample instructions from high-quality sample instructions, thereby achieving diverse filtering.
[0044] It should be noted that a spatial mapping method can be used for diversity filtering. Each cumulative sample instruction and high-quality sample instruction can be treated as a multi-dimensional vector data. For example, if it is a two-dimensional vector data, these sample instructions can be mapped to a two-dimensional space; if it is a three-dimensional vector data, these sample instructions can be mapped to a three-dimensional space; if it is an N-dimensional vector data, these sample instructions can be mapped to an N-dimensional space, where N can be a positive integer greater than or equal to 1. The diversity is determined based on the position of these sample instructions in the target space.
[0045] Among these, data that is far from the cumulative sample instructions can be selected from the high-quality sample instructions as the target sample instructions.
[0046] For example, the distance between the location of the target sample instruction mapped in the target space and the location of any cumulative sample instruction mapped in the target space is greater than or equal to a distance threshold.
[0047] In one embodiment, after performing the above step S220, the target model can also be trained based on the target sample instructions.
[0048] Since the target sample instructions have both high quality and diversity, training the target model based on these target sample instructions can improve the accuracy of the target model.
[0049] The instruction filtering method provided in this application embodiment can filter high-quality sample instructions from sample instructions. High-quality sample instructions are those used as prompts in application models where the instruction following difficulty meets a following difficulty threshold, and / or those used as prompts in application models where the loss value meets a loss threshold. Based on cumulative sample instructions, target sample instructions are filtered from high-quality sample instructions. The target distance between the position of the target sample instruction mapped in the target space and the position of the cumulative sample instructions mapped in the target space meets a distance threshold. Specifically, filtering instructions based on instruction following difficulty and / or loss value yields high-quality sample instructions, while filtering based on distance in the target space yields diverse target sample instructions. Based on these filtering methods, target sample instructions that combine high quality and diversity can be obtained. During the training of the target model based on these target sample instructions, the training effect of the model can be improved, allowing the trained model to output more accurate results.
[0050] The following explains one feasible implementation method for filtering high-quality sample instructions provided in the embodiments of this application.
[0051] Figure 3 This is a flowchart illustrating the high-quality sample instruction screening process provided in this application embodiment. Please refer to... Figure 3 High-quality sample instructions are selected from the sample instructions, including: S310: Each sample instruction is used as a prompt for the application model, and high-quality sample instructions that meet the following difficulty threshold are selected from the sample instructions based on the instruction following difficulty of the application model with different prompts.
[0052] The instruction following difficulty can be calculated based on the perplexity level. The instruction following difficulty of each application model with different prompts can be determined by calculating the perplexity level of the application model, and high-quality sample instructions can be selected based on the instruction following difficulty.
[0053] It should be noted that the prompt information can be information that gives the application model a correct instruction. In the actual implementation process, each sample instruction can be used as the prompt information of the application model, and the confusion level can be determined according to the application model with different prompt information, thereby determining the corresponding instruction following difficulty. In this way, high-quality sample instructions can be selected based on the instruction following difficulty.
[0054] In one embodiment, perplexity (PPL) is an important metric for evaluating the performance of large language models. Perplexity is a standard for assessing the quality of large language models; it measures the model's uncertainty about text data. For example, the lower the perplexity, the more accurate the model's predictions are and the better its text modeling performance; the higher the perplexity, the greater the model's confusion about the text and the worse its predictive ability.
[0055] For example, low-perplexity models are able to predict the next word in text more accurately, demonstrating stronger text generation and comprehension capabilities. For instance, in machine translation tasks, lower-perplexity models can more accurately generate sentences in the required language, making the translation results more consistent with natural language expression habits.
[0056] The difficulty of following the above instructions can be calculated using perplexity, thereby enabling the screening process of high-quality sample instructions.
[0057] S320: Each sample instruction is used as a prompt for the application model, and the test set instructions are input into the application models with different prompts. Based on the loss value of each application model, high-quality sample instructions that meet the loss threshold are selected from the sample instructions.
[0058] Optionally, the loss value can be calculated based on the test set data. The loss value of each application model with different prompts can be determined by inputting the test set data, and high-quality sample instructions can be selected based on the loss value.
[0059] The loss value can be a percentage between 0 and 1. The closer it is to 1, the smaller the loss; the closer it is to 0, the larger the loss. In other words, the larger the loss value, the smaller the loss of the applied model, and vice versa.
[0060] The instruction filtering method provided in this application embodiment can use each sample instruction as a prompt for the application model, and filter high-quality sample instructions that meet the following difficulty threshold from the sample instructions based on the instruction following difficulty of the application models with different prompts. Alternatively, each sample instruction can be used as a prompt for the application model, and test set instructions can be input into application models with different prompts. High-quality sample instructions that meet the loss threshold can be filtered from the sample instructions based on the loss value of each application model. High-quality sample instructions can be determined by determining the instruction following difficulty, and / or by determining the loss value. Either of these methods can accurately filter high-quality sample instructions, thereby achieving the training of the target model.
[0061] The implementation process of the instructions for determining high-quality samples provided in the embodiments of this application will be explained in detail below.
[0062] Figure 4 This is a flowchart illustrating the process of filtering high-quality sample instructions based on instruction following difficulty, as provided in the embodiments of this application. Please refer to... Figure 4 Based on the confusion level of the application model with different prompts, high-quality sample instructions are selected from the sample instructions, including: S410: Determine the instruction following difficulty of the application model with different prompts based on the ratio of the first confusion degree to the second confusion degree of the application model with different prompts.
[0063] It should be noted that for different application models with different prompts, the first and second perplexity of the application model can be calculated separately.
[0064] The first level of perplexity is the perplexity of the application model in generating the target response information based on the target request, and the second level of perplexity is the perplexity of the application model in generating the target response information.
[0065] In other words, the first level of perplexity is the perplexity of the application model generating the target response information based on a specific target request, while the second level of perplexity is the perplexity of the application model generating the target response information under any circumstances.
[0066] S420: Use the prompt information corresponding to the application model whose instruction following difficulty is greater than or equal to the difficulty threshold as a high-quality sample instruction.
[0067] The formula for calculating the difficulty of instruction following is as follows: IFDθ(Q,A)=PPLθ(A|Q) / PPLθ(A); Where IFDθ(Q, A) refers to the instruction following difficulty of the model, PPLθ(A|Q) is the first perplexity mentioned above, and PPLθ(A) is the second perplexity mentioned above; A refers to the target response information, and Q refers to the target request.
[0068] It should be noted that after determining the instruction following difficulty using the above calculation formula, the relationship between the instruction following difficulty and the difficulty threshold can be used to determine whether the prompt information corresponding to the application model can be used as the high-quality sample instruction.
[0069] For example, if the difficulty of following an instruction is greater than or equal to the difficulty threshold, the prompt information corresponding to that application model can be used as a high-quality sample instruction; if the difficulty of following an instruction is less than the difficulty threshold, the prompt information corresponding to that application model does not need to be used as a high-quality sample instruction.
[0070] In the instruction filtering method provided in this application embodiment, the instruction following difficulty of the application model with different prompts can be determined based on the ratio of the first perplexity and the second perplexity of the application model with different prompts. Prompts corresponding to application models with instruction following difficulty greater than or equal to a difficulty threshold are considered high-quality sample instructions. Specifically, by calculating perplexity, the instruction following difficulty of the application model with different prompts can be calculated more quickly and accurately, thereby enabling more accurate filtering of high-quality sample instructions.
[0071] The above process explains the selection of high-quality sample instructions based on instruction following difficulty. The following explains the selection of high-quality sample instructions based on loss value provided in the embodiments of this application.
[0072] Figure 5 This is a flowchart illustrating the instructions for screening high-quality samples based on loss difference provided in this application embodiment. Please refer to... Figure 5 High-quality sample instructions are selected from the sample instructions based on the loss value of each application model, including: S510: Input the test set instructions into the application model without prompts, and determine the loss difference between the loss value of the application model with prompts and the loss value of the application model without prompts.
[0073] It should be noted that the test set data can be first input into the application model without prompts to obtain a loss value. Alternatively, the test set data can be input into each application model with prompts to obtain a loss value. Based on these two loss values, the corresponding loss difference can be obtained.
[0074] For example, if the loss value of an application model without prompts is 0.33, and the loss value of an application model with prompts is 1, then the loss difference can be the difference between the loss value of the application model with prompts and the loss value of the application model without prompts, which is 0.67.
[0075] Assuming the difference threshold is 0.5, then 0.67 is greater than 0.5, so the sample instruction with the prompt information of the application model can be used as a high-quality sample instruction.
[0076] The specific formula is as follows: case score=one shot score-zero shot score; Here, the case score is the loss difference mentioned above, the one-shot score can be the loss value of any application model with prompting information, and the zero-shot score can be the loss value of an application model without prompting information.
[0077] S520: Use the prompt information corresponding to the application model whose loss difference is greater than or equal to the difference threshold as a high-quality sample instruction.
[0078] It should be noted that after determining the loss difference using the above calculation formula, the relationship between the loss difference and the difference threshold can be used to determine whether the prompt information corresponding to the application model can be used as the above high-quality sample instruction.
[0079] For example, if the loss difference is greater than or equal to the difference threshold, the prompt message corresponding to the application model can be used as a high-quality sample instruction; if the loss difference is less than the threshold, the prompt message corresponding to the application model does not need to be used as a high-quality sample instruction.
[0080] It should be noted that, in addition to comparing loss differences, comparisons can also be made directly based on loss values, as follows: In one embodiment, high-quality sample instructions are selected from the sample instructions based on the loss value of each application model, including: using the prompt information corresponding to the application model whose loss value is greater than or equal to the loss threshold as high-quality sample instructions.
[0081] It should be noted that after obtaining application models with different prompts, test set data can be input into each application model to obtain the corresponding loss value. The prompts of application models with loss values greater than or equal to the loss threshold can be used as high-quality sample instructions.
[0082] After calculating the loss value for each application model, the relationship between the loss value and the loss threshold can be used to determine whether the prompt information corresponding to that application model can be used as the aforementioned high-quality sample instruction.
[0083] For example, if the loss value is greater than or equal to the loss threshold, the prompt message corresponding to the application model can be used as a high-quality sample instruction; if the loss value is less than the loss threshold, the prompt message corresponding to the application model does not need to be used as a high-quality sample instruction.
[0084] In the instruction filtering method provided in this application embodiment, the prompt information corresponding to the application model with a loss value greater than or equal to a loss threshold can be used as high-quality sample instructions. Alternatively, test set data can be input into the application model without prompt information, and the loss difference between the loss value of the application model with prompt information and the loss value of the application model without prompt information can be determined; the prompt information corresponding to the application model with a loss difference greater than or equal to the difference threshold can be used as high-quality sample instructions. By comparing the loss value or loss difference, the loss situation of each application model with prompt information can be accurately calculated, thereby allowing sample instructions that serve as prompt information in application models that meet the loss value requirement or loss difference requirement to be used as high-quality sample instructions, improving the accuracy of determining high-quality sample instructions.
[0085] In one embodiment, any one or more of the above methods can be used to determine high-quality sample instructions from the sample instructions, and then diversity screening can be further performed based on the high-quality sample instructions to obtain target sample instructions.
[0086] The following explains one feasible implementation process for data diversity screening provided in the embodiments of this application.
[0087] Figure 6 This is a schematic diagram of the diversity screening process provided in the embodiments of this application. Please refer to... Figure 6 Based on the accumulated sample instructions, target sample instructions are selected from high-quality sample instructions, including: S610: Based on the mapping model, cumulative sample instructions and high-quality sample instructions are mapped to the target space.
[0088] The mapping model is a model obtained after training based on accumulated sample instructions. As explained above, the accumulated sample instructions are cold-start data, that is, the training data that has been accumulated so far. A large language model that has not been trained can be trained based on this data to obtain the above mapping model. In other words, the mapping model can also be a large language model, and its structure can be similar to the aforementioned application model and target model.
[0089] This mapping model can process cumulative sample instructions and high-quality sample instructions, and can map these data into the target space. Taking two-dimensional data as an example, if both cumulative sample instructions and high-quality sample instructions are two-dimensional data, then these data can be mapped to the corresponding positions in the two-dimensional space. Each sample instruction can be a two-dimensional vector in the two-dimensional space.
[0090] Correspondingly, if the data is three-dimensional or higher, it can be mapped to the corresponding dimension of space to determine the location.
[0091] S620: Determine the target distance between the position of each high-quality sample instruction in the target space and the position of each cumulative sample instruction in the target space.
[0092] It should be noted that after mapping the target space, the position of each high-quality sample instruction in the target space can be determined, as can the position of each cumulative sample instruction in the target space. For each high-quality sample instruction, the distance between the position of the high-quality sample instruction in the target space and the position of each cumulative sample instruction in the target space can be calculated.
[0093] For example, in a two-dimensional space, the target distance can be calculated based on the distance between two vectors in the two-dimensional space; in a multi-dimensional space, the target distance can be calculated based on the formula for calculating the distance between two vectors in the corresponding dimension.
[0094] S630: Based on the multiple target distances of each high-quality sample instruction, filter out the target sample instructions that meet the distance threshold from the high-quality sample instructions.
[0095] It should be noted that, assuming there are 10 cumulative sample instructions, there can be 10 target distances for each high-quality sample instruction. These target distances represent the distances from the high-quality sample instruction to each cumulative sample instruction in the target space.
[0096] In actual implementation, these target distances can be used to determine whether these high-quality sample instructions can be used as target sample instructions.
[0097] For example, all or part of the target distance can be compared with a distance threshold, and the comparison result can be used to determine whether the corresponding high-quality sample instruction can be used as the target sample instruction.
[0098] The instruction filtering method provided in this application embodiment can map cumulative sample instructions and high-quality sample instructions into a target space based on a mapping model; determine the target distance between the position of each high-quality sample instruction in the target space and the position of each cumulative sample instruction in the target space; and filter target sample instructions that meet the distance threshold from the high-quality sample instructions based on multiple target distances for each high-quality sample instruction. By mapping to the target space, the positions of different sample instructions in the target space can be accurately calculated, thereby determining diverse sample instructions based on the target distance, achieving the determination of sample instructions that possess both diversity and high quality.
[0099] The following explains several feasible implementation processes for diversity screening provided in the embodiments of this application.
[0100] Figure 7 For a flowchart illustrating various methods of diversity screening provided in the embodiments of this application, please refer to... Figure 7 Based on multiple target distances for each high-quality sample instruction, target sample instructions are selected from the high-quality sample instructions, including: S710: If the distance to each target is greater than or equal to the distance threshold, the corresponding high-quality sample instruction will be used as the target sample instruction.
[0101] It should be noted that if the distance to each target is greater than or equal to the distance threshold, then the high-quality sample instruction can be regarded as a target sample instruction with diversity. The above method can be used to calculate multiple target distances for each high-quality sample instruction, thereby filtering out multiple target sample instructions.
[0102] S720: When the average distance is greater than or equal to the distance threshold, the corresponding high-quality sample instruction is used as the target sample instruction.
[0103] The average distance is the average of the distances between multiple targets for each high-quality sample instruction.
[0104] It should be noted that in actual implementation, since the cumulative sample instructions may be scattered and relatively discrete, it is impossible to guarantee that the distance to each target is greater than or equal to the distance threshold. To solve this problem, the average distance method can be used.
[0105] If the average distance is greater than or equal to the distance threshold, the high-quality sample instruction can be used as a target sample instruction with diversity. Multiple target distances for each high-quality sample instruction can be calculated using the above method, thereby allowing multiple target sample instructions to be selected.
[0106] S730: If the target distance of the number of sample instructions exceeds the distance threshold, the corresponding high-quality sample instruction will be used as the target sample instruction.
[0107] It should be noted that in actual implementation, if the target distance of the number of sample instructions exceeds the distance threshold, the corresponding high-quality sample instruction can be used as the target sample instruction.
[0108] The quantity threshold can be a specific numerical value or a preset percentage value in all cumulative sample instructions; no specific restrictions are imposed here.
[0109] The instruction filtering method provided in this application embodiment can select high-quality sample instructions as target sample instructions when the distance to each target is greater than or equal to a distance threshold. Alternatively, it can select high-quality sample instructions as target sample instructions when the average distance is greater than or equal to a distance threshold; or, it can select high-quality sample instructions as target sample instructions when the distance to more than a certain number of sample instructions is greater than or equal to a distance threshold. These multiple methods can be used to filter suitable instructions with diversity, thereby obtaining target sample instructions that combine high quality and diversity, improving the accuracy of training the target model.
[0110] The following diagram illustrates how to map and display sample instructions in the target space using a two-dimensional spatial mapping as an example.
[0111] Figure 8 This is a schematic diagram of the target space mapping structure provided in the embodiments of this application. Please refer to... Figure 7 , Figure 8 Each circle shown can be the location of a sample instruction. The sample instructions within the D0 range can be the high-quality sample instructions mentioned above, and the data within the Q0 range can be the cumulative sample instructions mentioned above. The distance between each high-quality sample instruction and each cumulative sample instruction can be determined, thereby filtering out the target sample instructions.
[0112] pass Figure 8 The method shown can calculate the target distance between any high-quality sample instruction and any cumulative sample instruction, and then the target sample instruction can be selected from the high-quality sample instructions based on the corresponding target distance.
[0113] To more clearly explain the process of filtering target data provided in the embodiments of this application, the process of filtering target sample instructions from sample instructions is explained below with schematic diagrams.
[0114] Figure 9 This is a flowchart illustrating the instruction for filtering target samples provided in this embodiment of the application. Please refer to... Figure 9 For sample instructions, high-quality sample instructions can be obtained by using a screening method based on instruction following difficulty and / or a screening method based on loss value.
[0115] After obtaining high-quality sample instructions, the target distance between each high-quality sample instruction and each cumulative sample instruction can be determined by mapping them to cumulative sample instructions. By comparing the target distances, specific and diverse sample instructions, namely the aforementioned target sample instructions, can be further filtered from the high-quality sample instructions.
[0116] After receiving the target sample instructions, the target model can be trained.
[0117] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0118] Based on the foregoing embodiments, this application provides an instruction filtering device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.
[0119] Figure 10 This is a schematic diagram of the instruction filtering device provided in the embodiments of this application. Please refer to... Figure 10 The instruction screening device includes: a quality screening module 1010 and a diversity screening module 1020; The quality screening module 1010 is used to screen out high-quality sample instructions from the sample instructions. High-quality sample instructions are sample instructions used as prompt information in application models where the instruction following difficulty meets the following difficulty threshold, and / or sample instructions used as prompt information in application models where the loss value meets the loss threshold. The diversity filtering module 1020 is used to filter target sample instructions from high-quality sample instructions based on the cumulative sample instructions. The target sample instructions are mapped to the target space and the target distance between the target sample instructions and the cumulative sample instructions is mapped to the target space meets the distance threshold. The target sample instructions are used to train the target model.
[0120] In one embodiment, the quality screening module 1010 is specifically used to treat each sample instruction as prompt information for the application model, and to select high-quality sample instructions that meet the following difficulty threshold from the sample instructions according to the instruction following difficulty of the application model with different prompt information; and / or, to treat each sample instruction as prompt information for the application model, and to input the test set instructions into the application model with different prompt information, and to select high-quality sample instructions that meet the loss threshold from the sample instructions according to the loss value of each application model.
[0121] In one embodiment, the quality screening module 1010 is specifically used to determine the instruction following difficulty of the application model with different prompts based on the ratio of the first perplexity and the second perplexity of the application model with different prompts. The first perplexity is the perplexity of the application model generating the target response information based on the target request, and the second perplexity is the perplexity of the application model generating the target response information. The prompts corresponding to the application models with instruction following difficulty greater than or equal to the difficulty threshold are used as high-quality sample instructions.
[0122] In one embodiment, the quality screening module 1010 is specifically used to use the prompt information corresponding to the application model with a loss value greater than or equal to the loss threshold as a high-quality sample instruction.
[0123] In one embodiment, the quality screening module 1010 is specifically used to input test set instructions into application models without prompt information, determine the loss difference between the loss value of each application model with prompt information and the loss value of the application model without prompt information, and use the prompt information corresponding to the application model whose loss difference is greater than or equal to the difference threshold as a high-quality sample instruction.
[0124] In one embodiment, the diversity screening module 1020 is specifically used to map cumulative sample instructions and high-quality sample instructions into a target space based on a mapping model, wherein the mapping model is a model obtained after training based on the cumulative sample instructions; determine the target distance between the position of each high-quality sample instruction in the target space and the position of each cumulative sample instruction in the target space; and screen out target sample instructions that meet the distance threshold from the high-quality sample instructions based on the multiple target distances of each high-quality sample instruction.
[0125] In one embodiment, the diversity screening module 1020 is specifically used to: 1) use the corresponding high-quality sample instruction as the target sample instruction when each target distance is greater than or equal to a distance threshold; 2) use the corresponding high-quality sample instruction as the target sample instruction when the average distance is greater than or equal to a distance threshold, wherein the average distance is the average of multiple target distances for each high-quality sample instruction; 3) use the corresponding high-quality sample instruction as the target sample instruction when the target distance of more than a number of sample instructions is greater than or equal to a distance threshold.
[0126] The instruction filtering device provided in this application embodiment can filter high-quality sample instructions from sample instructions. High-quality sample instructions are those used as prompts in application models where the instruction following difficulty meets a following difficulty threshold, and / or those used as prompts in application models where the loss value meets a loss threshold. Based on accumulated sample instructions, target sample instructions are filtered from high-quality sample instructions. The target distance between the position of the target sample instruction mapped in the target space and the position of the accumulated sample instructions mapped in the target space meets a distance threshold. Specifically, filtering instructions based on instruction following difficulty and / or loss value yields high-quality sample instructions, while filtering based on distance in the target space yields diverse target sample instructions. Based on these filtering methods, target sample instructions that combine high quality and diversity can be obtained. During the training of the target model based on these target sample instructions, the training effect of the model can be improved, allowing the trained model to output more accurate results.
[0127] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0128] It should be noted that, in the embodiments of this application... Figure 10 The instruction filtering device shown is illustrative of the module division, representing only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical entities, or be integrated into one unit with two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.
[0129] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.
[0130] Figure 11 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Please refer to... Figure 11 This application provides a computer device, which can be the aforementioned vehicle-mounted terminal, and its internal structure diagram can be as follows. Figure 11 As shown. The computer device includes a processor 1120, memory, and a network interface 1140 connected via a system bus 1110. The processor 1120 provides computing and control capabilities. The memory includes a non-volatile storage medium 1131 and internal memory 1132. The non-volatile storage medium 1131 stores an operating system, computer programs, and a database. The internal memory 1132 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 1131. The database is used to store data. The network interface 1140 is used to communicate with external terminals via a network connection. When the computer program is executed by the processor 1120, it implements the aforementioned methods.
[0131] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.
[0132] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.
[0133] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0134] In one embodiment, the instruction filtering device provided in this application can be implemented as a computer program, and the computer program can be implemented as follows: Figure 11 The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.
[0135] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.
[0136] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.
[0137] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.
[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0139] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.
[0140] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.
[0141] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.
[0142] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.
[0143] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0144] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0145] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0146] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.
[0147] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for filtering instructions, characterized in that, include: High-quality sample instructions are selected from the sample instructions. The high-quality sample instructions are those that serve as prompts in application models where the instruction following difficulty meets the following difficulty threshold, and / or those that serve as prompts in application models where the loss value meets the loss threshold. Based on the accumulated sample instructions, target sample instructions are selected from the high-quality sample instructions. The target sample instructions are mapped to a position in the target space, and the target distance between the position of the accumulated sample instructions and the position of the accumulated sample instructions in the target space satisfies a distance threshold. The target sample instructions are used to train the target model.
2. The method according to claim 1, characterized in that, The process of filtering high-quality sample instructions from sample instructions includes: Each sample instruction is used as a prompt for the application model, and high-quality sample instructions that meet the following difficulty threshold are selected from the sample instructions based on the instruction following difficulty of the application model with different prompts; and / or, Each sample instruction is used as a prompt for the application model, and the test set instructions are input into the application models with different prompts. Based on the loss value of each application model, high-quality sample instructions that meet the loss threshold are selected from the sample instructions.
3. The method according to claim 2, characterized in that, The step of selecting high-quality sample instructions that meet the following difficulty threshold from the sample instructions based on the instruction following difficulty of the application model according to different prompt information includes: The instruction following difficulty of the application model with different prompts is determined by the ratio of the first confusion degree and the second confusion degree of the application model with different prompts. The first confusion degree is the confusion degree of the application model in generating the target response information based on the target request, and the second confusion degree is the confusion degree of the application model in generating the target response information. The prompt information corresponding to the application model whose difficulty is greater than or equal to the difficulty threshold is used as the high-quality sample instruction.
4. The method according to claim 2, characterized in that, The step of selecting high-quality sample instructions that meet the loss threshold from the sample instructions based on the loss value of each application model includes: The prompt information corresponding to the application model whose loss value is greater than or equal to the loss threshold is used as the high-quality sample instruction.
5. The method according to claim 2, characterized in that, The step of selecting high-quality sample instructions that meet the loss threshold from the sample instructions based on the loss value of each application model includes: The test set instructions are input into the application model without prompts, and the loss difference between the application model with prompts and the application model without prompts is determined. The prompt information corresponding to the application model whose loss difference is greater than or equal to the difference threshold is used as the high-quality sample instruction.
6. The method according to claim 1, characterized in that, The step of filtering target sample instructions from the high-quality sample instructions based on accumulated sample instructions includes: The cumulative sample instructions and the high-quality sample instructions are mapped to the target space based on the mapping model, which is a model obtained after training based on the cumulative sample instructions. Determine the target distance between the location of each high-quality sample instruction in the target space and the location of each cumulative sample instruction in the target space; Based on the multiple target distances of each high-quality sample instruction, target sample instructions that meet the distance threshold are selected from the high-quality sample instructions.
7. The method according to claim 6, characterized in that, The step of filtering target sample instructions that meet the distance threshold from the high-quality sample instructions based on multiple target distances for each high-quality sample instruction includes: If each target distance is greater than or equal to a distance threshold, the corresponding high-quality sample instruction is used as the target sample instruction; or, If the average distance is greater than or equal to a distance threshold, the corresponding high-quality sample instruction is used as the target sample instruction, where the average distance is the average of multiple target distances for each high-quality sample instruction; or, If the target distance of a number of sample instructions exceeds a distance threshold, the corresponding high-quality sample instruction will be used as the target sample instruction.
8. An instruction filtering device, characterized in that, include: Quality screening module, diversity screening module; The quality screening module is used to screen out high-quality sample instructions from the sample instructions. The high-quality sample instructions are sample instructions used as prompt information in application models where the instruction following difficulty meets the following difficulty threshold, and / or sample instructions used as prompt information in application models where the loss value meets the loss threshold. The diversity filtering module is used to filter target sample instructions from the high-quality sample instructions based on the cumulative sample instructions. The target sample instructions are mapped to a target space and the target distance between the target sample instructions and the cumulative sample instructions is mapped to a target space satisfies a distance threshold. The target sample instructions are used to train the target model.
9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.