Data set construction method, electronic equipment and storage medium
By building high-quality supervised data sets, using preset open source models and original data sets, the problem of insufficient mathematical reasoning capabilities of large language models is solved, and the robustness and generalization capabilities of the model are improved.
Patent Information
- Application Number
- CN202410563556.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-08
- Publication Date
- 2025-02-07
AI Technical Summary
The existing technology is difficult to effectively stimulate the mathematical reasoning ability of large language models, especially in building high-quality supervised data sets, which have problems with low data quality and high computational cost.
By obtaining the original data set corresponding to the preset open source model and multiple problem types to be solved, the initial supervision data set is determined, and a high-quality target supervision data set is constructed through steps such as cleaning, deduplication and kernel sampling.
It realizes high-quality supervised data set construction, improves the robustness and generalization ability of the problem-solving model, and can effectively solve mathematical problems and natural language problems.
Smart Images
Figure CN119398162A_ABST
Abstract
Description
Technical Field
[0001] The present application generally relates to the field of artificial intelligence technology. More specifically, the present application relates to a data set construction method, an electronic device and a storage medium. Background Art
[0002] With the emergence of emergent capabilities in large language models (LLMs), it has become possible to use large language models to solve mathematical problems. However, compared with other natural language understanding tasks, it is still relatively difficult to stimulate the mathematical reasoning ability of large language models to solve mathematical problems. Currently, methods for activating mathematical reasoning capabilities in large language models can be divided into prompt engineering, contextual learning, or supervised fine-tuning (SFT). Among them, prompt engineering methods are relatively incapable, and contextual learning methods face computational costs and stability issues. It is possible to enhance the capabilities of open source large language models through supervised fine-tuning to build a mathematical problem-solving system.
[0003] Supervised data is crucial for supervised fine-tuning. Regarding the format of supervised data, supervised data produced by mathematical problems generally consists of questions and answers, where the questions maintain the original question form and the answers use various complete reasoning steps. However, large language models are good at understanding mathematical problems in natural language, but are not good at stable symbolic reasoning and numerical calculations. In addition, the quantity and quality of supervised data are also critical for supervised fine-tuning. On the one hand, the existing methods of obtaining high-quality mathematical problems and annotating them are not easy to implement. On the other hand, the existing methods of obtaining data such as mathematical problems do not consider data quality and directly use or simply filter all the data produced.
[0004] In view of this, there is an urgent need to provide a dataset construction method so as to form high-quality supervised data for supervised fine-tuning and improve the robustness and generalization ability of the problem-solving model. Summary of the invention
[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a data set construction method, an electronic device and a storage medium in multiple aspects. The data set construction method can form high-quality supervised data for supervised fine-tuning, thereby improving the robustness and generalization ability of the problem-solving model.
[0006] In a first aspect, the present application provides a dataset construction method, comprising: obtaining a plurality of original datasets corresponding to a preset open source model and a plurality of types of problems to be solved; determining a plurality of initial supervised datasets corresponding to the plurality of original datasets and a plurality of question word vector representations corresponding to the plurality of original datasets based on the plurality of original datasets and the preset open source model; performing cleaning processing on the plurality of initial supervised datasets respectively to obtain a plurality of intermediate supervised datasets; determining a plurality of datasets to be mixed in the plurality of intermediate supervised datasets based on the plurality of question word vector representations; and merging the plurality of datasets to be mixed to obtain a target supervised dataset.
[0007] In some embodiments, determining multiple initial supervised data sets corresponding to multiple original data sets based on multiple original data sets and a preset open source model includes: inputting the problems to be solved corresponding to the multiple original data sets into the preset open source model respectively, and obtaining multiple problem result codes output by the preset open source model; executing the multiple problem result codes respectively, and obtaining multiple code execution results; retaining the correct execution results among the multiple code execution results; inputting the correct execution results, the problems to be solved corresponding to the correct execution results, and the problem result codes corresponding to the correct execution results into the preset open source model, and obtaining the target problem results output by the preset open source model; comparing the target problem results with the original problem results of the original data sets, and obtaining the problem reasoning paths corresponding to the target problem results with the same result comparison; determining the multiple initial supervised data sets corresponding to the multiple original data sets based on the problems to be solved corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison.
[0008] In some embodiments, before determining multiple initial supervised data sets corresponding to multiple original data sets based on the problems to be solved corresponding to the target problem results with the same result comparison and the problem reasoning path corresponding to the target problem results with the same result comparison, the method also includes: repeatedly executing the steps of inputting the problems to be solved corresponding to the multiple original data sets into the preset open source model respectively to the steps of comparing the target problem results with the original problem results of the original data sets to obtain the problem reasoning path corresponding to the target problem results with the same result comparison, and performing kernel sampling to obtain the problem reasoning path after quantitative enhancement.
[0009] In some embodiments, performing the nuclear sampling includes: performing the nuclear sampling based on a preset temperature parameter and a preset sampling quantity.
[0010] In some embodiments, determining multiple initial supervised data sets corresponding to multiple original data sets based on the problems to be solved corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison includes: merging the problems to be solved corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison to obtain multiple initial supervised data sets corresponding to the multiple original data sets.
[0011] In some embodiments, cleaning processes are respectively performed on multiple initial supervised data sets, including: extracting code blocks from the initial supervised data sets; parsing the code blocks into a first syntax tree; normalizing the first syntax tree to obtain a second syntax tree; normalizing the second syntax tree to obtain normalized code; deduplicating the normalized code to obtain deduplicated normalized code; and restoring the deduplicated normalized code to deduplicated supervisory data.
[0012] In some embodiments, after restoring the deduplication specification code to deduplication supervision data, the method also includes: screening and extracting reasoning paths of problems of the same type as the problems to be solved in the deduplication supervision data based on a preset number of reasoning paths to obtain an intermediate supervision data set; wherein the absolute value of the difference between the preset number of reasoning paths and the actual number of solutions to the problems to be solved is less than m, where m is a positive integer and m is less than or equal to 2.
[0013] In some embodiments, determining multiple data sets to be mixed among multiple intermediate supervised data sets based on multiple question word vector representations includes: visualizing the question word vector representations corresponding to the multiple intermediate supervised data sets into a two-dimensional space to obtain a data set distribution map; and determining in the data set distribution map that the intermediate supervised data sets with inconsistent distribution positions are the data sets to be mixed.
[0014] In a second aspect, the present application provides an electronic device, comprising: a processor; and a memory, on which a program code for constructing a data set is stored, and when the program code is executed by the processor, the electronic device implements the method as described above.
[0015] In a third aspect, the present application provides a non-transitory machine-readable storage medium having stored thereon a program code for constructing a data set, and when the program code is executed by a processor, the method described above can be implemented.
[0016] The technical solution provided by this application may have the following beneficial effects:
[0017] The data set construction method, electronic device and storage medium provided by the present application obtain multiple original data sets corresponding to a preset open source model and multiple types of problems to be solved, and then determine multiple initial supervised data sets corresponding to the multiple original data sets based on the multiple original data sets and the preset open source model, so as to expand the reasoning path of each problem to be solved, thereby effectively enhancing the data of the multiple original data sets. Then, the multiple initial supervised data sets are cleaned respectively to obtain multiple intermediate supervised data sets, so as to improve the data quality of the data sets.
[0018] Furthermore, the present application can also determine multiple question word vector representations corresponding to multiple original data sets based on multiple original data sets and preset open source models, and determine multiple data sets to be mixed in multiple intermediate supervised data sets based on multiple question word vector representations, and then merge the multiple data sets to be mixed to obtain a target supervised data set. The problem-solving model obtained by supervised fine-tuning of the large language model using the target supervised data set can solve problems similar to the problems to be solved corresponding to the original data sets, and can also solve problems of different types from the problems to be solved corresponding to the original data sets, and has high robustness and generalization capabilities. In practical applications such as solving math problems and answering natural language problems, solving questions or problems through this problem-solving model can improve the efficiency and quality of problem solving.
[0019] In general, this application can form high-quality supervised data for supervised fine-tuning, thereby improving the robustness and generalization ability of the problem-solving model. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0021] Figure 1 An exemplary flow chart showing a method for constructing a data set according to some embodiments of the present application is shown;
[0022] Figure 2 An exemplary flow chart showing a method for constructing a data set according to other embodiments of the present application is shown;
[0023] Figure 3 An exemplary flow chart showing a method for constructing a data set according to some other embodiments of the present application is shown;
[0024] Figure 4 A schematic diagram of the structure of an electronic device shown in an embodiment of the present application is shown;
[0025] Figure 5A schematic diagram of a deduplication algorithm in a data set construction method according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. For the simplicity and clarity of the description, the figure marks may be repeated in the drawings to indicate corresponding or similar elements when deemed appropriate. In addition, the present application sets forth many specific details in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practiced without these specific details. In other cases, well-known methods, processes, and components are not described in detail to avoid blurring the embodiments described herein. Moreover, the description should not be regarded as limiting the scope of the embodiments described herein. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.
[0027] It should be understood that the possible terms "first" or "second" etc. in the claims, specifications and drawings disclosed in this application are used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of this application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.
[0028] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.
[0029] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0030] It becomes possible to build a mathematical problem-solving system by enhancing the capabilities of open source large language models through supervised fine-tuning. Supervised data is crucial for supervised fine-tuning. As for the format of supervised data, supervised data produced by mathematical problems generally consists of questions and answers, where the questions maintain the original question form and the answers use various complete reasoning steps. However, large language models are good at understanding mathematical problems in natural language, but are not good at stable symbolic reasoning and numerical calculations. In addition, the quantity and quality of supervised data are also critical for supervised fine-tuning. On the one hand, the existing methods of obtaining high-quality mathematical problems and annotating them are not easy to implement. On the other hand, the existing methods of obtaining data such as mathematical problems do not consider the data quality and directly use or simply filter all the data produced.
[0031] In view of this, there is an urgent need to provide a dataset construction method so as to form high-quality supervised data for supervised fine-tuning and improve the robustness and generalization ability of the problem-solving model.
[0032] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.
[0033] Figure 1 An exemplary flowchart showing a method for constructing a data set in some embodiments of the present application is shown in FIG. Figure 1 , the data set construction method shown in the embodiment of the present application may include:
[0034] In step S101, a preset open source model and multiple original data sets corresponding to multiple types of problems to be solved are obtained. In an embodiment of the present application, the aforementioned preset open source model may be one or more, and the preset open source model may include but is not limited to Llama2 7B / 13B / 70B, Codellama 7B / 13B / 34B, ToRA-CODE 7B / 13B / 34B and ToRA70B. It can be understood that the types of preset open source models are diverse. In actual applications, it is necessary to select a suitable type of open source model as the preset open source model according to the actual application situation. The present application does not impose any restrictions in this regard.
[0035] In addition, each of the above-mentioned original data sets will correspond to a type of problem to be solved, and a type of problem to be solved may contain one or more problems to be solved. The multiple original data sets may include but are not limited to GSM8K, MATH, GSM-Hard, SVAMP, ASDiv and TAL-SCQ5K. It can be understood that the types of original data sets are diverse. In practical applications, it is necessary to select the appropriate type of original data set according to the actual application situation. This application does not impose any restrictions in this regard.
[0036] In step S102, multiple initial supervised data sets corresponding to multiple original data sets and multiple problem word vector representations corresponding to multiple original data sets are determined based on multiple original data sets and preset open source models. In an embodiment of the present application, the original data set and the preset open source model can be used to expand the reasoning path of each problem to be solved, so as to achieve effective data enhancement for multiple original data sets. In addition, the preset open source model can also be used to determine the multiple problem word vector representations of multiple problems to be solved corresponding to multiple original data sets. Among them, word vector (Word embedding) is a general term for a group of language modeling and feature learning technologies in word (Word) embedded natural language processing (NLP), and words or phrases from the vocabulary will be mapped to vectors of real numbers. It involves mathematical embedding from a one-dimensional space of each word to a continuous vector space with a lower dimension. The method for generating the aforementioned mapping may include generating using a neural network, generating using the dimensionality reduction of the word co-occurrence matrix, and the like. The problem word vector representation of the present application is a word vector representation generated for the problem to be solved.
[0037] Exemplarily, when determining multiple problem word vector representations of multiple problems to be solved corresponding to multiple original data sets, the preset open source model can adopt the Codellama34B model, and then use the Codellama34B model to generate problem word vector representations of GSM8K, MATH, GSM-Hard, SVAMP, ASDiv and TAL-SCQ5K respectively.
[0038] In step S103, the multiple initial supervised data sets are cleaned to obtain multiple intermediate supervised data sets. In the embodiment of the present application, the aforementioned cleaning process may include but is not limited to data deduplication and quantity restriction, which is intended to improve the data quality of the data set.
[0039] In step S104, multiple data sets to be mixed in multiple intermediate supervised data sets are determined based on multiple question word vector representations. It can be understood that in order for the problem-solving model obtained by supervised fine-tuning of the large language model to have the ability to solve problems similar to the problems to be solved corresponding to the original data set, and to solve problems of different types of problems to be solved corresponding to the original data set, it is necessary to select data sets for different types of problems to be solved from multiple intermediate supervised data sets for mixing. Different types of problems to be solved can be distinguished by different question word vector representations, thereby achieving the purpose of selecting multiple data sets to be mixed from multiple intermediate supervised data sets.
[0040] In step S105, multiple data sets to be mixed are merged to obtain a target supervised data set. In the embodiment of the present application, after obtaining the target supervised data set, the large language model can be fine-tuned in a supervised manner using the target supervised data set on the base model of the Llama2 7B / 13B / 70B and Codellama 7B / 13B / 34B series, thereby obtaining a problem solving model based on the large language model.
[0041] This application obtains multiple original data sets corresponding to a preset open source model and multiple types of problems to be solved, and then determines multiple initial supervised data sets corresponding to the multiple original data sets based on the multiple original data sets and the preset open source model, so as to expand the reasoning path of each problem to be solved, thereby effectively enhancing the data of the multiple original data sets. Then, the multiple initial supervised data sets are cleaned separately to obtain multiple intermediate supervised data sets, so as to improve the data quality of the data sets.
[0042] Furthermore, the present application can also determine multiple question word vector representations corresponding to multiple original data sets based on multiple original data sets and preset open source models, and determine multiple data sets to be mixed in multiple intermediate supervised data sets based on multiple question word vector representations, and then merge multiple data sets to be mixed to obtain a target supervised data set. The problem-solving model obtained by supervised fine-tuning of a large language model using a target supervised data set can solve problems similar to those to be solved corresponding to the original data set, and can also solve problems of different types from those to be solved corresponding to the original data set, and has high robustness and generalization capabilities.
[0043] In general, this application can form high-quality supervised data for supervised fine-tuning, thereby improving the robustness and generalization ability of the problem-solving model.
[0044] In some embodiments, the data in the original data set can be input into a preset open source model, and the results output by the preset open source model can be processed to construct an initial supervised data set. Figure 2 Let’s describe the construction process of the initial supervised dataset in detail. Figure 2 See the exemplary flowchart of the data set construction method of some other embodiments of the present application. Figure 2 , the data set construction method shown in the embodiment of the present application may include:
[0045] In step S201, the problems to be solved corresponding to the multiple original data sets are respectively input into the preset open source model to obtain multiple problem result codes output by the preset open source model. In an embodiment of the present application, when obtaining multiple problem result codes, the preset open source model can select ToRA-CODE 7B / 13B / 34B and / or ToRA 70B. It can be understood that the model type adopted by the preset open source model when obtaining multiple problem result codes is diverse. In practical applications, it is necessary to select according to the actual application situation. The present application does not impose any restrictions in this regard. After the problems to be solved corresponding to the multiple original data sets are input into the preset open source model, the preset open source model can output multiple answers to the questions. There may be a code part in each question answer. After the code part is extracted, multiple problem result codes corresponding to the multiple question answers are obtained.
[0046] In step S202, multiple problem result codes are executed respectively to obtain multiple code execution results, and the correct execution results in the multiple code execution results are retained. In an embodiment of the present application, the aforementioned problem result code can be executed by the Python interpreter to obtain the code execution result. During the execution of multiple problem result codes, some problem result codes may not be correctly executed. The code execution results corresponding to the problem result codes that cannot be correctly executed need to be eliminated, and the correct execution results in the multiple code execution results are finally retained.
[0047] It is understandable that there are various execution methods for the problem result code. In practical applications, it is necessary to select a suitable execution method to execute the problem result code according to the actual application situation. This application does not impose any restrictions in this regard.
[0048] In step S203, the correct execution result, the problem to be solved corresponding to the correct execution result, and the problem result code corresponding to the correct execution result are input into the preset open source model to obtain the target problem result output by the preset open source model. In an embodiment of the present application, the correct execution result, the problem to be solved corresponding to the correct execution result, and the problem result code corresponding to the correct execution result can be spliced and input into a preset open source model (e.g., ToRA-CODE 7B / 13B / 34B and / or ToRA 70B) to obtain the target problem result output by the preset open source model.
[0049] In step S204, the target problem result is compared with the original problem result of the original data set to obtain the problem reasoning path corresponding to the target problem result with the same result comparison. In the embodiment of the present application, if the target problem result is the same as the original problem result of the original data set, it means that the problem reasoning path at this time is correct, and the problem reasoning path at this time can deduce the correct answer to the current problem to be solved. It is necessary to retain this problem reasoning path, otherwise the problem reasoning path is eliminated. Finally, the problem reasoning path corresponding to the target problem result with the same result comparison is retained.
[0050] In step S205, the original data is enhanced by expanding the problem reasoning path through kernel sampling. In the embodiment of the present application, specifically, the steps of inputting the to-be-solved problems corresponding to the multiple original data sets into the preset open source model respectively to the step of comparing the target problem result with the original problem result of the original data set to obtain the problem reasoning path corresponding to the target problem result with the same result comparison can be repeatedly performed, and kernel sampling is performed to obtain the problem reasoning path after quantitative enhancement.
[0051] Among them, nucleus sampling, also known as Top-p sampling, is a text sampling strategy used in natural language generation, that is, a strategy for selecting one of a batch of possible text words predicted as the final output. Specifically, nucleus sampling can be performed based on a preset temperature parameter and a preset sampling number. The aforementioned preset temperature parameter can be used to adjust the probability distribution of each word before sampling. The lower the preset temperature parameter, the greater the probability distribution gap, and the easier it is to sample a word with a high probability; on the contrary, the higher the preset temperature parameter, the smaller the probability distribution gap, which increases the chance of low-probability words being sampled. In an embodiment of the present application, the preset temperature parameter can be set to 0.9. It can be understood that in actual applications, the setting method of the preset temperature parameter is diverse and needs to be set according to the actual application situation. The present application does not impose any restrictions in this regard. In addition, the preset sampling number can be set to 100, and the nucleus sampling can be stopped when the sampling number meets the preset sampling number. It can also be understood that in actual applications, the setting method of the preset sampling number is diverse and needs to be set according to the actual application situation. The present application does not impose any restrictions in this regard.
[0052] In step S206, multiple initial supervised data sets corresponding to multiple original data sets are determined based on the unsolved problems corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison. In an embodiment of the present application, the unsolved problems corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison can be merged to obtain multiple initial supervised data sets corresponding to the multiple original data sets.
[0053] In some embodiments, multiple initial supervised data sets may be cleaned to obtain multiple high-quality intermediate supervised data sets, and appropriate intermediate supervised data sets may be combined to construct a target supervised data set. Figure 3 Let’s explain in detail the process of constructing a target supervised dataset based on multiple initial supervised datasets. Figure 3 An exemplary flow chart showing a method for constructing a data set according to some other embodiments of the present application is shown. Figure 5 A schematic diagram of a deduplication algorithm in a data set construction method according to an embodiment of the present application is shown. Figure 3 and Figure 5 , the data set construction method shown in the embodiment of the present application may include:
[0054] In step S301, multiple initial supervised data sets are cleaned to obtain multiple intermediate supervised data sets. In the embodiment of the present application, the cleaning process may include performing a deduplication algorithm on the initial supervised data set and limiting the upper limit of the inference path. Figure 3 As shown, the steps of deduplication algorithm processing can be specifically as follows: first, extract the code block c in the initial supervision data set i , where the code block c i Can contain query data q i , response data a i and source data i The original data of i The code block c can then be transformed into an abstract syntax tree (AST) i Parse to the first syntax tree t i Then we can do the first syntax tree t i Perform normalization processing to obtain the second syntax tree t′ i , wherein the normalization process may be to replace the variable name v with a lowercase letter and the function name f with an uppercase letter, thereby modifying the first syntax tree t i Then the second syntax tree t′ can be standardized. i Perform normalization conversion to obtain the normalized code c′ i Then, the normalized code c′ can be matched by plain text i Perform deduplication processing to obtain the deduplication canonical code c′. Finally, the deduplication-free supervised data d′ can be restored by using the deduplication-free canonical code c′, query data q, response data a, and source data s. Thus, all data d in the initial supervised data set can be iteratively processed through the above steps.
[0055] Furthermore, in an embodiment of the present application, in order to construct an intermediate supervision data set with higher quality, the problem reasoning paths of the same type of problems to be solved in the deduplicated supervision data d′ can also be screened and extracted based on the preset number of reasoning paths to obtain an intermediate supervision data set, the purpose of which is to implement random screening of problem reasoning paths of the same type of problem to be solved to control the upper limit of the number of problem reasoning paths of the same type of problem to be solved. And the absolute value of the difference between the preset number of reasoning paths and the actual number of solutions to the problem to be solved is less than m, where m is a positive integer and m is less than or equal to 2, so as to control the problem reasoning paths of the same type of problem to be solved to basically meet the actual number of solutions to the problem to be solved.
[0056] It will be understood that the above description of deduplication algorithm processing and reasoning path upper limit limit is merely exemplary. In actual applications, the specific execution steps of deduplication algorithm processing and reasoning path upper limit limit need to be determined according to the actual application situation. This application does not impose any restrictions in this regard.
[0057] In step S302, the question word vector representations corresponding to the multiple intermediate supervision data sets are visualized in two-dimensional space to obtain a data set distribution map. In an embodiment of the present application, t-SNE (t-distributed stochastic neighbor embedding) can be used to visualize the question word vector representations corresponding to the multiple intermediate supervision data sets in two-dimensional space. t-SNE is an unsupervised nonlinear technology, mainly used for data exploration and visualization of high-dimensional data.
[0058] In step S303, the intermediate supervision data set with inconsistent distribution positions is determined as the data set to be mixed in the data set distribution map. For example, assuming that in the data set distribution map, the distribution positions of the question word vector representations of GSM8K, MATH and TAL-SCQ5K respectively generated by the Codellama34B model are inconsistent with the distribution positions of the other three GSM-Hard, SVAMP and ASDiv, then it can be determined that the intermediate supervision data sets corresponding to GSM8K, MATH and TAL-SCQ5K are the data sets to be mixed.
[0059] In step S304, multiple data sets to be mixed are merged to obtain a target supervised data set. The target supervised data set is obtained by merging multiple data sets to be mixed, and then the problem-solving model obtained by supervised fine-tuning the large language model using the target supervised data set can solve problems similar to the problems to be solved corresponding to the original data set, and can also solve problems of different types from the problems to be solved corresponding to the original data set, and has high robustness and generalization ability.
[0060] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides an electronic device for executing a data set construction method and corresponding embodiments.
[0061] Figure 4 FIG. 4 is a block diagram showing the hardware configuration of an electronic device 400 that can implement the data set construction method of an embodiment of the present application. Figure 4 As shown, the electronic device 400 may include a processor 410 and a memory 420. Figure 4 In the electronic device 400, only the components related to this embodiment are shown. Therefore, it is obvious to those skilled in the art that the electronic device 400 may also include components related to this embodiment. Figure 4 The components shown in the figure are different from the common components. For example: fixed-point arithmetic units.
[0062] The electronic device 400 may correspond to a computing device having various processing functions, for example, a function for generating a neural network, training or learning a neural network, quantizing a floating-point neural network to a fixed-point neural network, or retraining a neural network. For example, the electronic device 400 may be implemented as various types of devices, such as a personal computer (PC), a server device, a mobile device, etc.
[0063] The processor 410 controls all functions of the electronic device 400. For example, the processor 410 controls all functions of the electronic device 400 by executing a program stored in the memory 420 on the electronic device 400. The processor 410 may be implemented by a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an artificial intelligence processor chip (IPU), etc. provided in the electronic device 400. However, the present application is not limited thereto.
[0064] In some embodiments, the processor 410 may include an input / output (I / O) unit 411 and a computing unit 412. The I / O unit 411 may be used to receive various data, such as a preset open source model and multiple original data sets corresponding to multiple types of problems to be solved. Exemplarily, the computing unit 412 may be used to determine multiple initial supervision data sets corresponding to multiple original data sets and multiple question word vector representations corresponding to multiple original data sets based on multiple original data sets received via the I / O unit 411 and the preset open source model; and then clean the multiple initial supervision data sets respectively to obtain multiple intermediate supervision data sets; and then determine multiple data sets to be mixed in the multiple intermediate supervision data sets based on the multiple question word vector representations; and then merge the multiple data sets to be mixed to obtain a target supervision data set. This target supervision data set may be output by the I / O unit 411, for example. The output data may be provided to the memory 420 for reading and use by other devices (not shown), or may be directly provided to other devices for use.
[0065] The memory 420 is hardware for storing various data processed in the electronic device 400. For example, the memory 420 can store processed data and data to be processed in the electronic device 400. The memory 420 can store data sets involved in the data set construction method process that has been processed or to be processed by the processor 410, for example, a preset open source model and multiple original data sets corresponding to multiple types of problems to be solved, etc. In addition, the memory 420 can store applications, drivers, etc. to be driven by the electronic device 400. For example: the memory 420 can store various programs related to the data set construction method to be executed by the processor 410. The memory 420 can be a DRAM, but the present application is not limited thereto. The memory 420 can include at least one of a volatile memory or a non-volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FRAM), etc. The volatile memory may include dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), PRAM, MRAM, RRAM, ferroelectric RAM (FeRAM), etc. In an embodiment, the memory 420 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), a high-density flash memory (CF), a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (xD) card, caches, or a memory stick.
[0066] In summary, the specific functions implemented by the memory 420 and the processor 410 of the electronic device 400 provided in the implementation mode of this specification can be explained in comparison with the aforementioned implementation modes in this specification, and can achieve the technical effects of the aforementioned implementation modes, and will not be repeated here.
[0067] In this embodiment, the processor 410 can be implemented in any suitable manner. For example, the processor 410 can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc.
[0068] It should also be understood that any module, unit, component, server, computer, terminal or device that executes instructions exemplified herein may include or otherwise access computer-readable media, such as storage media, computer storage media or data storage devices (removable and / or non-removable) such as disks, optical disks or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules or other data.
[0069] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.
Claims
1. A method for constructing a data set, characterized in that: include: Obtain multiple original data sets corresponding to preset open source models and multiple types of problems to be solved; Determine, based on the multiple original data sets and the preset open source model, multiple initial supervised data sets corresponding to the multiple original data sets and multiple question word vector representations corresponding to the multiple original data sets; Cleaning the multiple initial supervised data sets respectively to obtain multiple intermediate supervised data sets; Determine a plurality of data sets to be mixed among the plurality of intermediate supervision data sets based on the plurality of question word vector representations; The multiple data sets to be mixed are merged to obtain a target supervision data set.
2. The method for constructing a data set according to claim 1, characterized in that: The determining, based on the multiple original data sets and the preset open source model, multiple initial supervised data sets corresponding to the multiple original data sets comprises: Inputting the to-be-solved problems corresponding to the multiple original data sets into the preset open source model respectively, and obtaining multiple problem result codes output by the preset open source model; Execute the multiple problem result codes respectively to obtain multiple code execution results; Preserve the correct execution results among multiple code execution results; Inputting the correct execution result, the problem to be solved corresponding to the correct execution result, and the problem result code corresponding to the correct execution result into the preset open source model to obtain the target problem result output by the preset open source model; Comparing the target problem result with the original problem result of the original data set to obtain a problem reasoning path corresponding to the target problem result with the same result comparison; Based on the to-be-solved problems corresponding to the target problem results with the same result comparison and the problem reasoning paths corresponding to the target problem results with the same result comparison, multiple initial supervised data sets corresponding to the multiple original data sets are determined.
3. The method for constructing a data set according to claim 2, characterized in that: Before determining the multiple initial supervised data sets corresponding to the multiple original data sets based on the to-be-solved problems corresponding to the target problem results having the same result comparison and the problem reasoning paths corresponding to the target problem results having the same result comparison, the method further includes: Repeat the steps of inputting the problems to be solved corresponding to the multiple original data sets into the preset open source model respectively to the step of comparing the target problem result with the original problem result of the original data set to obtain a problem reasoning path corresponding to the target problem result with the same result comparison, and perform kernel sampling to obtain a problem reasoning path after quantitative enhancement.
4. The method for constructing a data set according to claim 3, characterized in that: The performing nuclear sampling includes: performing nuclear sampling based on a preset temperature parameter and a preset sampling quantity.
5. The method for constructing a data set according to claim 2, characterized in that: The method of determining the multiple initial supervised data sets corresponding to the multiple original data sets based on the to-be-solved problems corresponding to the target problem results having the same result comparison and the problem reasoning paths corresponding to the target problem results having the same result comparison comprises: The to-be-solved problems corresponding to the target problem results having the same result comparison are merged with the problem reasoning paths corresponding to the target problem results having the same result comparison, so as to obtain a plurality of initial supervised data sets corresponding to the plurality of original data sets.
6. The method for constructing a data set according to claim 1, characterized in that: The cleaning process of the multiple initial supervision data sets respectively comprises: Extracting code chunks from the initial supervised dataset; Parsing the code block into a first syntax tree; Normalizing the first syntax tree to obtain a second syntax tree; Performing normalization conversion on the second syntax tree to obtain normalized code; Performing deduplication processing on the standardized code to obtain a deduplication standardized code; The deduplication specification code is restored to deduplication supervision data.
7. The method for constructing a data set according to claim 6, characterized in that: After restoring the deduplication specification code to deduplication supervision data, the method further includes: Based on a preset number of reasoning paths, the problem reasoning paths of the same type of problems to be solved in the deduplicated supervision data are screened and extracted to obtain the intermediate supervision data set; The absolute value of the difference between the number of preset reasoning paths and the number of actual solutions to the problem to be solved is less than m, where m is a positive integer and m is less than or equal to 2.
8. The method for constructing a data set according to claim 1, characterized in that: The step of determining a plurality of to-be-mixed data sets in the plurality of intermediate supervision data sets based on the plurality of question word vector representations comprises: Visualize the question word vector representations corresponding to the multiple intermediate supervision data sets into a two-dimensional space to obtain a data set distribution map; The intermediate supervision data set whose distribution position is inconsistent is determined in the data set distribution map as the data set to be mixed.
9. An electronic device, characterized in that: include: processor; as well as A memory having program codes for constructing a data set stored thereon, wherein when the program codes are executed by the processor, the electronic device implements the method as claimed in any one of claims 1 to 8.
10. A non-transitory machine-readable storage medium having stored thereon a program code for constructing a data set, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 8 is implemented.