A large model fine-tuning data matching method and related device

Abstract

The present application discloses a large model fine-tuning data matching method and related devices, which relate to the field of artificial intelligence. The scene model specially trained for each scene is verified by using the verification data set of each scene, and the first evaluation result set obtained is used as the optimal evaluation effect of each scene. At the same time, the initial general scene model is verified and evaluated using each of the verification data sets to obtain the second evaluation result of each of the verification data sets. With the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, each of the initial supervised fine-tuning data sets is updated according to the preset learning rate according to the gradient descent algorithm. The general scene model is updated using the updated merged data set, and then the updated general scene model is verified and updated using each verification data set. The above process is repeated continuously to obtain the optimal data set matching. While improving the efficiency of data matching optimization, the accuracy of the optimal matching is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large model fine-tuning data matching method and related devices. Background Art

[0002] Large model fine-tuning refers to the process of further training a pre-trained large deep learning model using a new, task-specific dataset. The main purpose of this fine-tuning technique is to adapt the model to new, specific tasks or domains without having to train a completely new model from scratch.

[0003] Currently, when matching data for fine-tuning large models, we rely on manually pre-set matching strategies, and need to find the best matching strategy through repeated trials. Since the manually set matching strategy depends on the subjective experience of the developer, it is difficult to accurately capture the optimal matching strategy, and it is time-consuming and inefficient. Summary of the invention

[0004] In view of the above problems, the present application provides a large model fine-tuning data matching method and related devices to achieve the purpose of improving the accuracy and matching efficiency of data matching. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for fine-tuning data matching of a large model, comprising:

[0006] S1. The scenario model corresponding to each application scenario is evaluated and verified using the verification data set corresponding to the application scenario to obtain a first evaluation result of the verification data set. The scenario model corresponding to each application scenario is obtained by training a general model with the initial supervised fine-tuning data set corresponding to each application scenario.

[0007] S2, using each of the verification data sets to perform verification evaluation on the initial general scenario model, to obtain a second evaluation result of each of the verification data sets, wherein the initial general scenario model is obtained by training the general model using an initial merged data set, and the initial merged data set is obtained by merging each of the initial supervised fine-tuning data sets;

[0008] S3, with the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, update each of the initial supervised fine-tuning data sets according to the gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set, wherein the first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results;

[0009] S4, using the updated initial merged data set to update the initial universal scene model to obtain an updated initial universal scene model;

[0010] S5, using each of the verification data sets to perform verification evaluation on the updated initial general scenario model to obtain an updated second evaluation result set, and using the updated second evaluation result set as the second evaluation result set;

[0011] S6. Repeat steps S3 to S5 until the distance is no greater than a first preset value or the gradient change is no greater than a second preset value.

[0012] In a possible implementation, before the scenario model corresponding to each application scenario is verified and evaluated using the verification data set corresponding to the application scenario to obtain the first evaluation result of the verification data set, the method further includes:

[0013] The general model is reduced to obtain a reduced general model, and the reduced general model is used to replace the general model to execute steps S1 to S6.

[0014] In a possible implementation, the large model fine-tuning data matching method further includes:

[0015] After step S6 is completed, the general model is trained according to the most recently updated initial merged data set, and each of the verification data sets is used to perform verification and evaluation on the trained general model.

[0016] In a possible implementation, the reducing operation on the general model to obtain the reduced general model includes:

[0017] A model having the same base model as the general model and having a smaller parameter scale than the general model is used as the reduced general model.

[0018] In a possible implementation, the method takes minimizing the distance between the first evaluation result set and the second evaluation result set as the goal, updates each of the initial supervised fine-tuning data sets according to a preset learning rate according to a gradient descent algorithm to obtain an updated initial merged data set, including:

[0019] Determine a loss function according to a preset distance calculation function;

[0020] Determine a gradient vector between the first evaluation result set and the second evaluation result set according to the loss function;

[0021] Each of the initial supervised fine-tuning datasets is updated according to the gradient vector and the preset learning rate.

[0022] In a possible implementation, the preset learning rate includes: a quantity or a ratio adjusted each time.

[0023] The second aspect of the present application provides a large model fine-tuning data matching device, comprising:

[0024] An application scenario evaluation module is used to execute S1, to perform verification evaluation on the scenario model corresponding to each application scenario using the verification data set corresponding to the application scenario, and obtain a first evaluation result of the verification data set, wherein the scenario model corresponding to each application scenario is obtained by training the general model with the initial supervised fine-tuning data set corresponding to each application scenario;

[0025] A general scenario evaluation module is used to execute S2, to perform verification evaluation on the initial general scenario model using each of the verification data sets, to obtain a second evaluation result of each of the verification data sets, wherein the initial general scenario model is obtained by training the general model using an initial merged data set, and the initial merged data set is obtained by merging each of the initial supervised fine-tuning data sets;

[0026] A data set adjustment module, used to execute S3, with the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, update each of the initial supervised fine-tuning data sets according to a gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set, wherein the first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results;

[0027] A general scene updating module, used to execute S4, update the initial general scene model using the updated initial merged data set to obtain an updated initial general scene model;

[0028] A general scenario evaluation update module, used to execute S5, perform verification evaluation on the updated initial general scenario model using each of the verification data sets to obtain an updated second evaluation result set, and use the updated second evaluation result set as the second evaluation result set; and

[0029] The general scenario loop evaluation module is used to make the data set adjustment module repeat step S3, the general scenario update module repeat step S4, and the general scenario evaluation update module repeat step S5 until the distance is no greater than a first preset value or the gradient change is no greater than a second preset value.

[0030] A third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the large model fine-tuning data matching method of the first aspect or any implementation of the first aspect.

[0031] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0032] The memory is used to store computer programs;

[0033] The processor is used to execute the computer program so that the electronic device can implement the large model fine-tuning data matching method according to the first aspect or any implementation method of the first aspect.

[0034] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the large model fine-tuning data ratio method of the above-mentioned first aspect or any implementation method of the first aspect.

[0035] By means of the above technical scheme, the method for fine-tuning data ratio of a large model provided by the present application verifies the scene model specially trained for each scene by using the verification data set of each scene, and obtains the first evaluation result set as the optimal evaluation effect of each scene. At the same time, the initial general scene model is verified and evaluated using each of the verification data sets to obtain the second evaluation result of each of the verification data sets. With the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, each of the initial supervised fine-tuning data sets is updated according to the preset learning rate according to the gradient descent algorithm. The general scene model is updated using the updated merged data set, and then the updated general scene model is verified and updated using each verification data set to obtain the updated second evaluation result. The ratio calculated according to the gradient is updated and the above process is continuously repeated until the distance is not greater than the first preset value or the gradient change is not greater than the second preset value, and the optimal data set ratio is obtained. Compared with the prior art, this scheme realizes the automatic optimization of the data ratio of large model fine-tuning based on the gradient algorithm, while improving the efficiency of data ratio optimization, the accuracy of the optimal ratio is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0037] Figure 1 A structural diagram of a large model fine-tuning data matching system provided in this application;

[0038] Figure 2 A structural diagram of a terminal provided for this application;

[0039] Figure 3 A structural diagram of a server provided for this application;

[0040] Figure 4 A flowchart of the method for fine-tuning data matching for a large model provided in this application;

[0041] Figure 5 The application flow chart of the large model fine-tuning data matching method provided in this application;

[0042] Figure 6 Gradient diagram of the data set ratios for the three application scenarios provided in this application;

[0043] Figure 7 A structural diagram of a large model fine-tuning data matching device provided in this application;

[0044] Figure 8 This is a structural diagram of the electronic device provided for this application. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0046] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0047] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0048] See also Figure 1 , Figure 1 1 shows a schematic diagram of a large model fine-tuning data matching system architecture. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 In the example, a server is included, and the server 200 can provide the method provided in the embodiment of the present application for one or more terminals.

[0049] Among them, the terminal 100 can be installed with an application for fine-tuning data ratio control for a large model. The above application and web page can provide an interface. The terminal 100 can receive relevant parameters entered by the user on the parameter setting interface and send the above parameters to the server 200. The server 200 can obtain processing results based on the received parameters and return the processing results to the terminal 100.

[0050] It should be understood that in some optional implementations, the terminal 100 can also complete the action of obtaining the processing result based on the received parameters by itself without the cooperation of the server, and the embodiments of the present application are not limited to this.

[0051] Next describe Figure 1 The product form of the mid-terminal 100;

[0052] The terminal 100 in the embodiment of the present application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any limitation on this.

[0053] Figure 2 An optional hardware structure diagram of the terminal 100 is shown.

[0054] refer to Figure 2 As shown, the terminal 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), an earphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190 and other components. Those skilled in the art will appreciate that Figure 2 These are merely examples of terminals or multi-function devices and do not constitute limitations on the terminals or multi-function devices, which may include more or fewer components than those shown in the figures, or combinations of certain components, or different components.

[0055] The input unit 130 can be used to receive input digital or character information, and generate key signal input related to the user settings and function control of the portable multifunctional device. Specifically, the input unit 130 may include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect the user's touch operations on or near it (such as the user's operation on or near the touch screen using any suitable object such as fingers, joints, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch screen can detect the user's touch action on the touch screen, convert the touch action into a touch signal and send it to the processor 170, and can receive and execute the command sent by the processor 170; the touch signal at least includes the touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented using multiple types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0056] Among them, the input device 132 can receive input data and the like.

[0057] The display unit 140 can be used to display information input by the user or provided to the user, various menus of the terminal 100, interactive interfaces, file display and / or playback of any multimedia file. In the embodiment of the present application, the display unit 140 can be used to display the interface of parameter setting, processing results, etc.

[0058] The memory 120 can be used to store instructions and data. The memory 120 can mainly include an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files, texts, etc.; the instruction storage area can store software units such as operating systems, applications, instructions required for at least one function, or their subsets and extensions. It can also include a non-volatile random access memory; provide the processor 170 with hardware, software and data resources including management of computing and processing equipment, and support control software and applications. It is also used for the storage of multimedia files, and the storage of running programs and applications.

[0059] The processor 170 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal 100. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it executes various functions of the terminal 100 and processes data, thereby controlling the terminal device as a whole. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 170. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips. The processor 170 may also be used to generate corresponding operation control signals, send them to corresponding components of the computing and processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0060] Among them, the memory 120 can be used to store software codes related to the large model fine-tuning data matching method, the processor 170 can execute the steps of the large model fine-tuning data matching method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve corresponding functions.

[0061] The radio frequency unit 110 (optional) can be used for receiving and sending information or receiving and sending signals during a call, for example, after receiving the downlink information of the base station, it is sent to the processor 170 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the radio frequency unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile communication (Global System of Mobile communication, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Long Term Evolution (Long Term Evolution, LTE), email, Short Messaging Service (SMS), etc.

[0062] In this embodiment of the present application, the radio frequency unit 110 can send each initial supervised fine-tuning data set to the server 200, and receive the processing result of the large model fine-tuning data ratio sent by the server 200.

[0063] It should be understood that the radio frequency unit 110 is optional and can be replaced by other communication interfaces, such as a network port.

[0064] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0065] The terminal 100 further includes an external interface 180 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal 100 to communicate with other devices, or to connect a charger to charge the terminal 100 .

[0066] Although not shown, the terminal 100 may also include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which are not described in detail here. Some or all of the methods described below may be applied in the following embodiments. Figure 2 In the terminal 100 shown.

[0067] Next describe Figure 1 The product form of the server 200;

[0068] Figure 3 A structural diagram of a server 200 is provided, such as Figure 3 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0069] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0070] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0071] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard drive (HDD), or a solid state drive (SSD).

[0072] Among them, the memory 204 can be used to store software codes related to the large model fine-tuning data matching method, the processor 202 can execute the steps of the large model fine-tuning data matching method of the chip, and can also schedule other units to realize corresponding functions.

[0073] It should be understood that the above-mentioned terminal 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned terminal 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0074] Currently, when fine-tuning large pre-trained models, we rely on manually pre-set ratio strategies and find the best ratio through repeated experiments. Although this method can optimize model performance to a certain extent, it also has some significant limitations and shortcomings.

[0075] First, the method of manually setting the ratio strategy relies on the subjective experience of the developer, which may lead to the inability to accurately capture the optimal data ratio. For example, in mixed sequential training, the ratio of domain data to general data may require very fine adjustments, such as 1:1, 1:2, 1:3, etc., while in two-stage mixed fine-tuning, the ratio of general data to specific domain data is set to 1:k, and k is a series of preset values. If the optimal ratio happens to fall outside this preset range, such as 2:3 or 3:4, it is difficult to achieve optimal performance by manual preset alone.

[0076] Second, existing ratio selection methods usually rely on traversal experiments, which means that all preset ratio strategies need to be tested and then the one with the best performance is selected. This method is extremely time-consuming, especially when there are a large number of ratio strategies. Each ratio needs to be trained or fine-tuned, which undoubtedly consumes a lot of time and computing resources. On the contrary, if fewer ratio strategies are set, although time can be saved, the best performance may be missed.

[0077] In order to solve the above problems, the embodiment of the present application provides a method for fine-tuning data ratio of a large model. The method for fine-tuning data ratio of a large model of the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0078] Reference Figure 4 , Figure 4A flow chart of a method for fine-tuning data matching for a large model provided in an embodiment of the present application is shown as follows: Figure 4 As shown, the large model fine-tuning data ratio method provided in the embodiment of the present application may include steps S1 to S6, and these steps are described in detail below.

[0079] S1. The scenario model corresponding to each application scenario is verified and evaluated using the verification data set corresponding to the application scenario to obtain the first evaluation result of the verification data set. The scenario model corresponding to each application scenario is obtained by training the general model with the initial supervised fine-tuning data set corresponding to each application scenario.

[0080] Specifically, refer to Figure 5 As shown in the middle right part, when performing model training tasks, especially for large models, one of the core goals is to ensure that the model can adapt to and support multiple different downstream application scenarios. In order to achieve this goal, it is necessary to prepare various types of training data and corresponding verification data sets according to the specific task requirements of these downstream scenarios. For example, in the training process of a large programming model, the model should be able to handle multiple tasks such as code generation annotations, code translation, annotation generation code, code safety checks, etc. For these different scenarios, corresponding supervised fine-tuning data sets need to be prepared.

[0081] For example, there are n different scenarios, and the initial supervised fine-tuning datasets used for these scenarios can be labeled as Tdata_1, Tdata_2, ..., Tdata_n, respectively, and the corresponding verification datasets are labeled as Vdata_1, Vdata_2, ..., Vdata_n, respectively.

[0082] In the initial training stage of the model, which refers to steps 4 to 6, a general model, namely the LLM_S model, is used to perform separate supervised fine-tuning on the initial supervised fine-tuning datasets of the n scenes, thereby obtaining n models adapted to different scenes, which can be marked as LLM_S_1, LLM_S_2, ..., LLM_S_n. After fine-tuning, the performance of these models is evaluated using the validation datasets corresponding to each scene, and the initial evaluation results, namely the first evaluation results, are obtained. The initial evaluation results of the n scenes are marked as R_1, R_2, ..., R_n.

[0083] It should be noted here that the data contained in each initial supervised fine-tuning dataset is adapted to the application scenario and can include datasets of one or more scenarios. For example, if the general large model is to support both general scenarios and certain domain scenarios, the fine-tuning dataset must consist of at least two parts: the dataset of the domain scenario and the dataset of the general scenario. For example, the original general large model performs well in general scenarios, but performs poorly in customer service scenarios and development scenarios in the communications industry. If the original large model is to support these two scenarios and general scenarios, it is necessary to prepare customer service scenario datasets, development scenario datasets, and general scenario datasets.

[0084] S2. Use each verification data set to verify and evaluate the initial general scenario model to obtain a second evaluation result of each verification data set. The initial general scenario model is obtained by training the general model using the initial merged data set, and the initial merged data set is obtained by merging each initial supervised fine-tuning data set.

[0085] Specifically, in the iterative training phase of the model, Figure 5 As shown in steps 9 to 12 in , the training data of these n scenarios need to be merged according to different data ratios in order to train a unified model that can support multiple scenarios. The general scenario model to be obtained can be recorded as LLM_S_U. After each iterative training, the validation data set is used again to evaluate the performance of the model, and the iterative evaluation results are obtained, marked as R_U_1, R_U_2, ..., R_U_n, and the model is further adjusted and optimized based on these results. In this way, it is ensured that the large model can show good performance and adaptability in multiple downstream scenarios.

[0086] S3. With the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, each initial supervised fine-tuning data set is updated according to the gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set. The first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results.

[0087] S4. Use the updated initial merged data set to update the initial universal scene model to obtain an updated initial universal scene model.

[0088] S5. Perform verification evaluation on the updated initial general scenario model using each verification data set to obtain an updated second evaluation result set, and use the updated second evaluation result set as the second evaluation result set.

[0089] S6. Repeat steps S3 to S5 until the distance is no greater than a first preset value or the gradient change is no greater than a second preset value.

[0090] Specifically, in the above steps 9 to 12, the purpose is to correct the data ratio by comparing the distance and distance change between the first evaluation result set (R_1, R_2, ..., R_n) and the iterative evaluation result set (i.e., the second evaluation result set) (R_U_1, R_U_2, ..., R_U_n), so as to determine the best data set ratio.

[0091] Because the verification results obtained by the models LLM_S_1, LLM_S_2, ..., LLM_S_n of the above scenarios represent the best results that can be achieved in different scenarios, for example:

[0092] The verification results of LLM_S_1 on the verification data of all scenarios are (R_1_1, R_1_2, …, R_1_n), the verification results of LLM_S_2 on the verification data of all scenarios are (R_2_1, R_2_2, …, R_2_n), and the verification results of LLM_S_n on the verification data of all scenarios are (R_n_1, R_n_2, …, R_n_n). Considering the extreme ratio situation, only the data of one scenario is selected for training, and the performance is the best in this scenario, while the performance in other scenarios is not as good as the corresponding effect of using only one scenario. Therefore, R_1_1, R_2_2, …, R_n_n (that is, the above R_1, R_2, …, R_n) are the best results that can be achieved for each scenario.

[0093] The problem solved by different ratios is that in a certain ratio, the result obtained performs best in all scenarios, that is, the theoretical best effect is the closest to R_1_1, R_2_2, ..., R_n_n in all scenarios. Translated into a geometric problem, if there are two scenes, the theoretical best effect is the center of the two points. If there are three scenes, the theoretical best effect is the "Fermat point" of the triangle composed of three points. And so on. In multiple scenarios, in high-dimensional space, the theoretical best point can be calculated. For example, refer to Figure 6 As shown in the figure, it is a principle diagram for finding the data set ratio with the best effect for the three scenes. The distance is calculated by using a preset distance algorithm, and then it is converted into a gradient descent problem. The derivative of the loss function is taken to find a method to minimize the loss function and determine the best ratio.

[0094] From the above, we can see that the large model fine-tuning data ratio method, by combining the goal-oriented optimization strategy and the data set ratio update strategy based on gradient calculation, greatly improves the efficiency and accuracy of finding the optimal data ratio.

[0095] In a possible implementation, the above steps aim to minimize the distance between the first evaluation result set and the second evaluation result set, and update each initial supervised fine-tuning data set according to the preset learning rate according to the gradient descent algorithm to obtain an updated initial merged data set. That is, the specific process of updating the data set ratio according to the gradient calculation includes:

[0096] Step 1: Determine the loss function based on the preset distance calculation function.

[0097] Step 2: Determine the gradient vector between the first evaluation result set and the second evaluation result set based on the loss function.

[0098] Step 3: Update each initial supervised fine-tuning data set according to the gradient vector and the preset learning rate, wherein the preset learning rate may include: the amount or proportion of each adjustment.

[0099] Specifically, with Figure 6 The determination method is similar to that in , marking the theoretical best point as R_max (R_max_1, R_max_2, ..., R_max_n), and the initialization effect point as R_ proportioning (R_ proportioning_1, R_proportioning_2, ..., R_ proportioning_n), then the distance between the two is marked as f(R_max, R_proportioning), and the distance is calculated using the Euclidean distance method. It is converted into a gradient descent problem, and the method to minimize the objective function is found by finding the derivative of the objective function.

[0100] in,

[0101] f(R_max, R_ proportioning) =

[0102] The loss function is:

[0103]

[0104] Calculate the gradient. The gradient is the slope of the loss function at the current point, pointing to the direction where the loss increases fastest. For the square of the Euclidean distance, the gradient is:

[0105]

[0106] Calculate the partial derivatives and get:

[0107]

[0108]

[0109] …

[0110]

[0111] The gradient vector is:

[0112]

[0113] Update the coordinates of the current point. In order to move the current point toward the target point, the coordinates of the current point need to be updated in the reverse direction of the gradient. First, update the data ratio:

[0114]

[0115]

[0116] …

[0117]

[0118] Then use the updated dataset Data, via Figure 5 In steps 10 to 12, we obtain the new R_proportioning (R_proportioning_1, R_proportioning_2, …, R_proportioning_n). For the learning rate, you can increase 1k data each time or increase it by 10% proportionally.

[0119] Then, the above steps are repeated until the current point is close enough to the target point, or the gradient change is very small, indicating that it is close to the minimum value. The first preset value and the second preset value used for comparison can be selected and adjusted by those skilled in the art as needed, and are not limited here.

[0120] To further improve the speed of model training, in a possible implementation, before performing step S1, using the verification data set corresponding to the application scenario to perform verification evaluation on the scenario model corresponding to each application scenario, and obtaining the first evaluation result of the verification data set, the following is further included:

[0121] The general model is reduced to obtain a reduced general model, and the reduced general model is used to replace the general model to execute steps S1 to S6.

[0122] After step S6 is completed, the general model is trained according to the most recently updated initial merged data set, and each verification data set is used to perform verification and evaluation on the trained general model.

[0123] Specifically, when reducing the size of the model, a model having the same base model as the general model and having a smaller parameter scale than the general model may be used as the reduced general model.

[0124] In the specific implementation, you can choose the appropriate large model according to the scenario problem you are expected to solve, such as CodeLLaMA-70B for code scenarios and Qwen-70B for Chinese question-answering scenarios. Scale the model and use a small parameter scale model of the same base model. Figure 5 As shown in steps 1 to 3 in , the scaled large model LLM_S is used as the fine-tuning training and iterative updating model for task 2 (i.e., steps 4 to 6). The large model LLM_L before scaling is used as the fine-tuning training model for task 4 (i.e., the process from steps 15 to 19).

[0125] In Task 4, use LLM_L as the basic general model, modify the training data Tdata_1, Tdata_2, …, Tdata_n according to the ratio obtained in Task 3 (i.e., steps 7 to 14), input LLM_L, and complete fine-tuning training. The resulting model is LLM_L_END, and use the verification data Vdata_1, Vdata_2, …, Vdata_n for evaluation. The evaluation results are obtained and the training is completed.

[0126] The above introduces a large model fine-tuning data matching method provided by an embodiment of the present application. The following will introduce a device for executing the above large model fine-tuning data matching method.

[0127] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a large model fine-tuning data matching device provided in an embodiment of the present application. Figure 7 As shown, the large model fine-tuning data matching device includes:

[0128] The application scenario evaluation module 701 is used to execute S1, to perform verification evaluation on the scenario model corresponding to each application scenario using the verification data set corresponding to the application scenario, and obtain the first evaluation result of the verification data set, wherein the scenario model corresponding to each application scenario is obtained by training the general model with the initial supervised fine-tuning data set corresponding to each application scenario;

[0129] A general scenario evaluation module 702 is used to execute S2, to perform verification evaluation on the initial general scenario model using each of the verification data sets, to obtain a second evaluation result of each of the verification data sets, wherein the initial general scenario model is obtained by training the general model using an initial merged data set, and the initial merged data set is obtained by merging each of the initial supervised fine-tuning data sets;

[0130] The data set adjustment module 703 is used to execute S3, with the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, and to update each of the initial supervised fine-tuning data sets according to the gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set, wherein the first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results;

[0131] A general scene updating module 704 is used to execute S4, update the initial general scene model using the updated initial merged data set to obtain an updated initial general scene model;

[0132] The general scenario evaluation update module 705 is used to execute S5, perform verification evaluation on the updated initial general scenario model using each of the verification data sets to obtain an updated second evaluation result set, and use the updated second evaluation result set as the second evaluation result set; and

[0133] The general scenario loop evaluation module 706 is used to make the data set adjustment module 703 repeat step S3, the general scenario update module 704 repeat step S4, and the general scenario evaluation update module 705 repeat step S5 until the distance is no greater than the first preset value or the gradient change is no greater than the second preset value.

[0134] In a possible implementation, it further includes: a model scaling module, which is used to perform a scaling operation on the general model to obtain a scaled general model, and use the scaled general model to replace the general model to execute steps S1 to S6.

[0135] In a possible implementation, it also includes: an original model training module, which is used to train the general model according to the most recently updated initial merged data set after step S6 is executed, and use each verification data set to verify and evaluate the trained general model.

[0136] In a possible implementation, the process in which the model scaling module performs a scaling operation on the general model to obtain the scaled general model includes:

[0137] A model having the same base model as the general model and having a smaller parameter scale than the general model is used as the reduced general model.

[0138] In a possible implementation, the data set adjustment module 703 takes the minimum distance between the first evaluation result set and the second evaluation result set as the goal, updates each of the initial supervised fine-tuning data sets according to the gradient descent algorithm according to the preset learning rate, and obtains the updated initial merged data set, including:

[0139] Determine a loss function according to a preset distance calculation function;

[0140] Determine a gradient vector between the first evaluation result set and the second evaluation result set according to the loss function;

[0141] Each of the initial supervised fine-tuning datasets is updated according to the gradient vector and the preset learning rate.

[0142] In a possible implementation, the preset learning rate used in the data set adjustment module 703 includes: the amount or ratio of each adjustment.

[0143] The present application also provides an electronic device in an embodiment. Figure 8 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0144] like Figure 8 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0145] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a memory card, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0146] Also provided in an embodiment of the present application is a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any large model fine-tuning data matching method provided in an embodiment of the present application.

[0147] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any large model fine-tuning data matching method provided in an embodiment of the present application.

[0148] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0149] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0150] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0151] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for fine-tuning data ratio of a large model, characterized in that: include: S1. The scenario model corresponding to each application scenario is evaluated and verified using the verification data set corresponding to the application scenario to obtain a first evaluation result of the verification data set. The scenario model corresponding to each application scenario is obtained by training the general model with the initial supervised fine-tuning data set corresponding to each application scenario. S2, using each of the verification data sets to perform verification evaluation on the initial general scenario model, to obtain a second evaluation result of each of the verification data sets, wherein the initial general scenario model is obtained by training the general model using an initial merged data set, and the initial merged data set is obtained by merging each of the initial supervised fine-tuning data sets; S3, with the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, update each of the initial supervised fine-tuning data sets according to the gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set, wherein the first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results; S4, using the updated initial merged data set to update the initial universal scene model to obtain an updated initial universal scene model; S5, using each of the verification data sets to perform verification evaluation on the updated initial general scenario model to obtain an updated second evaluation result set, and using the updated second evaluation result set as the second evaluation result set; S6. Repeat steps S3 to S5 until the distance is no greater than a first preset value or the gradient change is no greater than a second preset value.

2. The method for fine-tuning data ratio of a large model according to claim 1, characterized in that: Before the scenario model corresponding to each application scenario is verified and evaluated using the verification data set corresponding to the application scenario to obtain the first evaluation result of the verification data set, the method further includes: The general model is reduced to obtain a reduced general model, and the reduced general model is used to replace the general model to execute steps S1 to S6.

3. The method for fine-tuning data ratio of a large model according to claim 2, characterized in that: Also includes: After step S6 is completed, the general model is trained according to the most recently updated initial merged data set, and each of the verification data sets is used to perform verification and evaluation on the trained general model.

4. The method for fine-tuning data ratio of a large model according to claim 2, characterized in that: The step of performing a reduction operation on the general model to obtain a reduced general model includes: A model having the same base model as the general model and having a smaller parameter scale than the general model is used as the reduced general model.

5. The method for fine-tuning data ratio of a large model according to claim 1, characterized in that: The method aims to minimize the distance between the first evaluation result set and the second evaluation result set, and updates each of the initial supervised fine-tuning data sets according to a preset learning rate according to a gradient descent algorithm to obtain an updated initial merged data set, including: Determine a loss function according to a preset distance calculation function; Determine a gradient vector between the first evaluation result set and the second evaluation result set according to the loss function; Each of the initial supervised fine-tuning datasets is updated according to the gradient vector and the preset learning rate.

6. The large model fine-tuning data matching method according to any one of claims 1 to 5, characterized in that: The preset learning rate includes: the amount or ratio of each adjustment.

7. A large model fine-tuning data matching device, characterized in that: include: An application scenario evaluation module is used to execute S1, to perform verification evaluation on the scenario model corresponding to each application scenario using the verification data set corresponding to the application scenario, and obtain a first evaluation result of the verification data set, wherein the scenario model corresponding to each application scenario is obtained by training the general model with the initial supervised fine-tuning data set corresponding to each application scenario; A general scenario evaluation module is used to execute S2, to perform verification evaluation on the initial general scenario model using each of the verification data sets, to obtain a second evaluation result of each of the verification data sets, wherein the initial general scenario model is obtained by training the general model using an initial merged data set, and the initial merged data set is obtained by merging each of the initial supervised fine-tuning data sets; A data set adjustment module, used to execute S3, with the goal of minimizing the distance between the first evaluation result set and the second evaluation result set, update each of the initial supervised fine-tuning data sets according to a gradient descent algorithm at a preset learning rate to obtain an updated initial merged data set, wherein the first evaluation result set includes all first evaluation results, and the second evaluation result set includes all second evaluation results; A general scene updating module, used to execute S4, update the initial general scene model using the updated initial merged data set to obtain an updated initial general scene model; A general scenario evaluation update module, used to execute S5, perform verification evaluation on the updated initial general scenario model using each of the verification data sets to obtain an updated second evaluation result set, and use the updated second evaluation result set as the second evaluation result set; as well as, The general scenario loop evaluation module is used to make the data set adjustment module repeat step S3, the general scenario update module repeat step S4, and the general scenario evaluation update module repeat step S5 until the distance is no greater than a first preset value or the gradient change is no greater than a second preset value.

8. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the large model fine-tuning data matching method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the large model fine-tuning data matching method as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the large model fine-tuning data matching method as described in any one of claims 1 to 6.

Images