Model fine tuning method and device, electronic equipment, storage medium and computer program product

Abstract

The embodiment of the invention provides a model fine tuning method, a model fine tuning device, electronic equipment, a chip, a storage medium and a computer program product, and the method comprises the steps: executing one or more rounds of first iterative training until a first stop condition is met, and obtaining a fine-tuned basic model; wherein each round of the first iterative training process comprises the following steps: dividing the plurality of computing nodes into N groups according to the model capability information of each computing node in the plurality of computing nodes; generating N group global models according to the first model and the model structure corresponding to each group in the N groups; the model structure corresponding to each group is determined according to the computing node with the worst model capability in each group; training a group global model corresponding to each group through each computing node of each group to obtain a trained group global model corresponding to each group; and updating the first model parameter based on the trained group global model parameter corresponding to each group.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of large model technology, and specifically to a model fine-tuning method, a model fine-tuning device, an electronic device, a chip, a storage medium, and a computer program product. Background Art

[0002] During the federated training of the model, the global model parameters are sent to each institution for training. Each institution uses the same model for training. However, the computing resources of large and small institutions are different, so there is a risk of long training waiting time and high consumption of training resources. Summary of the invention

[0003] Embodiments of the present application provide a model fine-tuning method, a model fine-tuning device, an electronic device, a chip, a storage medium, and a computer program product.

[0004] The model fine-tuning method provided in the embodiment of the present application includes:

[0005] One or more rounds of first iteration training are performed until the first stop condition is met to obtain a fine-tuned basic model; wherein each round of first iteration training process includes:

[0006] According to model capability information of each computing node in the multiple computing nodes, the multiple computing nodes are divided into N groups;

[0007] Generate N group global models according to the first model and the model structures corresponding to each of the N groups; the N group global models correspond to the N groups one by one; the model structures corresponding to each group are determined according to the computing nodes with the worst model capabilities in each group;

[0008] The group global model corresponding to each group is trained through each computing node of each group to obtain the trained group global model corresponding to each group;

[0009] The first model parameters are updated based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

[0010] The model fine-tuning device provided in the embodiment of the present application includes:

[0011] Fine-tuning module: used to perform one or more rounds of first iteration training until the first stop condition is met to obtain a fine-tuned basic model; wherein each round of first iteration training process includes:

[0012] According to model capability information of each computing node in the multiple computing nodes, the multiple computing nodes are divided into N groups;

[0013] Generate N group global models according to the first model and the model structures corresponding to each of the N groups; the N group global models correspond to the N groups one by one; the model structures corresponding to each group are determined according to the computing nodes with the worst model capabilities in each group;

[0014] The group global model corresponding to each group is trained through each computing node of each group to obtain the trained group global model corresponding to each group;

[0015] The first model parameters are updated based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

[0016] The electronic device provided in the embodiment of the present application includes: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute any model fine-tuning method provided in the embodiment of the present application.

[0017] The chip provided in the embodiment of the present application includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes any model fine-tuning method provided in the embodiment of the present application.

[0018] The storage medium provided in the embodiments of the present application is used to store a computer program, and the computer program enables a computer to execute any model fine-tuning method provided in the embodiments of the present application.

[0019] The computer program product provided in the embodiments of the present application includes a computer program, which, when executed by a processor, implements any model fine-tuning method provided in the embodiments of the present application.

[0020] Through the model fine-tuning method provided in the embodiment of the present application, the resources of each node participating in the training are grouped according to the model capability information of each node, and the node with the worst model capability in the group is selected to determine the structure of the group's global model. According to the basic model and the model structure corresponding to each group, the corresponding group global model is generated for each group, so that each node can train a model that matches its own capability, fully utilize the resources of each node, and improve training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 Schematic diagram of the implementation process of the model fine-tuning method provided in the embodiment of the present application Figure 1 ;

[0023] Figure 2 A functional structure diagram of a large-model federated fine-tuning system provided for an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the process of fine-tuning the model provided in the embodiment of the present application;

[0025] Figure 4 A schematic structural diagram of a model fine-tuning device 400 provided in an embodiment of the present application;

[0026] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0027] Figure 6 A schematic structural diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be noted that in the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the embodiments of the present application, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0030] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between two items, or an association relationship between the two items, or a relationship between indication and being indicated, configuration and being configured, and the like.

[0031] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0032] With the development of big data technology and digital economy, data elements, as a new type of production factor, have received extensive attention from all sectors of society. The circulation and sharing of data elements while ensuring security issues will help release the value of data elements. Various industries and institutions have accumulated a large amount of industry data, but due to privacy requirements, production and operation restrictions and other reasons, data cannot be directly aggregated and used on demand, resulting in the phenomenon of "data islands", which hinders the circulation and use of data.

[0033] The data circulation sharing service network is a next-generation intensive and efficient data circulation infrastructure that is cross-industry, cross-regional, cross-field, and cross-subject, providing a low-cost, high-efficiency, and reliable circulation environment for data circulation transactions. It is mainly composed of a delivery service platform (DSP), a data service node (DSN), and a private network. The DSP and each node device can be connected through an Internet Protocol (IP) private network to ensure the security of data transmission.

[0034] Large model fine-tuning refers to fine-tuning the pre-trained ultra-large-scale parameter base model for specific downstream tasks to improve performance; model training (supervised learning) and model performance optimization are usually performed on labeled data. Pre-training is performed on large-scale public datasets through unsupervised or self-supervised learning methods. Through pre-training, the model can learn rich feature representations and potential structures in the data, but the model effect is not good for specific fields. Improving the effect of models for specific industries requires collecting labeled industry datasets for model training and effect optimization.

[0035] Federated learning refers to a distributed machine learning paradigm that enables clients (devices or organizations) to collaboratively train machine learning models without exposing client data. Unlike traditional centralized machine learning techniques, data is fixed locally instead of being collected on a central server to reduce systemic privacy risks. Federated learning is an effective way to address data isolation challenges.

[0036] Medical, legal, financial and other industry data are usually stored in various institutions in a scattered manner. Subject to privacy protection requirements, how to effectively integrate and utilize these distributed data resources while avoiding privacy risks and processing efficiency issues brought about by data centralization has become a technical problem that needs to be solved urgently. At the same time, the existing methods of independently processing and analyzing data in one institution have limitations in performance and effect. Generating industry applications through industry large models can greatly enhance and enrich business capabilities. Therefore, the data of multiple institutions in the same industry are jointly fine-tuned for large models, while meeting the needs of data privacy protection, and there is a practical need for the federated fine-tuning of industry large models. The current method is to generate teacher models and student models separately through the knowledge distillation method for large and small institutions to conduct federated training separately. However, this method does not take into account the differences in computing power and data resources between small institutions. All small institutions use the same student model for training, which has the risk of long training waiting time and high training resource consumption.

[0037] Figure 1 Schematic diagram of the implementation process of the model fine-tuning method provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown, the embodiment of the present application provides a model fine-tuning method, which includes the following steps:

[0038] Step 101: performing one or more rounds of first iterative training until a first stop condition is met to obtain a fine-tuned basic model; wherein each round of first iterative training process includes:

[0039] According to model capability information of each computing node in the multiple computing nodes, the multiple computing nodes are divided into N groups;

[0040] Generate N group global models according to the first model and the model structures corresponding to each of the N groups; the N group global models correspond to the N groups one by one; the model structures corresponding to each group are determined according to the computing nodes with the worst model capabilities in each group;

[0041] The group global model corresponding to each group is trained through each computing node of each group to obtain the trained group global model corresponding to each group;

[0042] The first model parameters are updated based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

[0043] In an embodiment of the present application, the first stop condition includes one or more of the following: convergence of the basic model after fine-tuning, reaching a first preset number of training rounds, and reaching a first preset training time.

[0044] In an embodiment of the present application, the model capability information of the node includes: resource status and training data volume; wherein the resource status includes the node graphics processing unit (GPU) peak computing power C, GPU utilization R, and node training data volume D; the model capability of the node can be characterized by the model carrying capacity of the node, and the model carrying capacity of each node is calculated by the real-time available computing power resources and training data volume of each node. Exemplarily, the model carrying capacity P = T*n*C*(1-R), wherein T is the specified computing time, representing the maximum acceptable time for the node in a round of training, and the value of T can be set according to actual conditions, and C is the number of floating point operations (FLOPs) of the total training computing amount; it can be understood that the larger the model carrying capacity value, the stronger the model capability of the node, and the smaller the model carrying capacity value, the worse the model capability of the node. According to actual needs, one or more model carrying capacity thresholds are designed to divide multiple computing nodes into N groups.

[0045] In the embodiment of the present application, the computing power requirement of the model can be characterized by the model parameter scale M and the node operation data volume. For example, the computing power requirement of the model W = 8*M*D; the maximum optional model parameter scale M of the node max =P / (8*D)=[T*n*C*(1-R)] / (8*D), calculate the maximum optional model parameter scale of each computing node in the group, take the computing node with the smallest maximum optional model parameter scale in the group as the baseline, and design the model structure of this group. It can be understood that after generating the group global model corresponding to this group according to the basic model and the model structure of this group, each computing node in this group collaboratively trains the group global model corresponding to this group.

[0046] In an embodiment of the present application, a time interval can also be configured for each computing node, and each computing node reports model capability information according to the configured time interval. In this way, before the first iteration of training in each round starts, real-time model capability information of each computing node can be obtained to ensure the consistency of the grouping of computing nodes and fully utilize the resources of each computing node.

[0047] Based on this, in an optional implementation manner of the present application, before each round of the first iterative training starts, the following further includes:

[0048] Receive model capability information reported by each of the multiple computing nodes.

[0049] In the embodiment of the present application, for each of the N groups, one or more rounds of second iterative training are performed to train the global model corresponding to the group, wherein each round of the second iterative training process includes sending the group global model corresponding to the group to each computing node in the group, and each computing node in the group trains the group global model through its own data, and collaboratively updates the group global model parameters after the training of each group. The second iterative training is stopped until the second stop condition is met, and the trained group global model corresponding to the group is obtained.

[0050] Based on this, in an optional implementation manner of the present application, the group global model corresponding to each group is trained by each computing node of each group to obtain the group global model after the training of each group, including:

[0051] One or more rounds of second iterative training are performed until the second stop condition is met, and the group global model after each group training is obtained; wherein each round of second iterative training process includes:

[0052] Sending the second model to each computing node in the group so that each computing node trains the second model;

[0053] Receive the trained second model parameters fed back by each computing node of each group, and update the parameters of the second model corresponding to each group based on the trained second model parameters fed back by each computing node of each group; wherein, in the first round of training of the second iterative training, the second model is the group global model corresponding to each group, and in other rounds of training of the second iterative training except the first round, the second model is the group global model corresponding to each group after the parameters of the previous round are updated.

[0054] In the embodiment of the present application, the second stop condition includes one or more of the following: reaching a second preset number of training rounds, reaching a second preset training time.

[0055] In an embodiment of the present application, for each computing node, the training time or number of trainings in each round of second iterative training can be pre-configured. When the pre-configured training time or number of trainings is reached, the second iterative training of this round is stopped. After each computing node in the group has reached the pre-configured training time or number of trainings, the group global model corresponding to the group is collaboratively updated through the updated group global model parameters corresponding to each computing node.

[0056] In an embodiment of the present application, in order to reduce the amount of model parameters without affecting the model effect, knowledge distillation can be used to obtain N group global models. Specifically, according to the model structure corresponding to each group, the basic model knowledge is distilled into N teacher models of different sizes. The N teacher models of different sizes correspond one-to-one to the N groups. The teacher models corresponding to each group are then subjected to knowledge distillation to obtain the group global model corresponding to each group. During the model training process, the teacher model and the group global model are collaboratively updated. After each round of the second iterative training process is completed, the teacher model and the group global model are collaboratively updated once until the second iterative training process is stopped. The basic model is collaboratively updated using the parameters of each teacher model to complete a round of the first iterative training process.

[0057] In the embodiment of the present application, during the model training process, if the computing node is resource-constrained or other situations occur, resulting in the computing node being unable to complete the model training, the computing node can send and report the first information, indicating that the computing node cannot complete the model training. At this time, the first iteration training process of the current round is stopped, the model parameters generated by the first iteration training process of the current round are deleted, and the first iteration training process of the next round is restarted. This is because during the model training process, one or more computing nodes cannot complete the training task, resulting in an imbalance in the entire model training process, affecting the model training effect.

[0058] Based on this, in an optional implementation manner of the present application, the model fine-tuning method further includes:

[0059] When the first information sent by the first computing node is received, the first iterative training process of the current round is stopped and the first iterative training process of the next round is performed; the first information indicates that the first computing node cannot complete the model training.

[0060] In the embodiment of the present application, in the case where the computing power of the computing nodes to complete the fine-tuning of the large model is limited, the backbone architecture of the group's global model can be frozen, and the fine-tuning module can be updated to complete the training. The group's global model parameters can be updated by aggregating the fine-tuning module parameters to achieve federated learning fine-tuning and achieve the same model effect.

[0061] Based on this, in an optional implementation manner of the present application, the group global model includes a fine-tuning module and a backbone architecture, and before sending the second model to each computing node in the group, further includes:

[0062] The backbone architecture of the group global model is frozen and simplified into a simulator.

[0063] In the embodiment of the present application, the fine-tuning module may adopt an adapter module and / or a prompt module. In actual application, other modules may also be adopted, and the embodiment of the present application does not limit this.

[0064] Based on this, in an optional implementation manner of the present application, the fine-tuning module includes: an adapter module and / or a prompt module.

[0065] The model fine-tuning method provided in the embodiment of the present application can be applied to the DSP in the data circulation network, where the computing node is DSN, and the reference Figure 2 , Figure 2 The functional structure diagram of the large model federated fine-tuning system provided in the embodiment of the present application, the relevant modules in the DSP are the node status collection module, the node grouping registration module and the aggregation calculation module, the node status collection module is used to complete the collection of the computing power resource status of the organization node during the calculation process, that is, to collect the model capability information of the DSN, the node grouping registration module is used to group the DSN nodes, the aggregation calculation module is used for model knowledge distillation and model parameter aggregation, the relevant modules in the organization node DSN are the node resource monitoring module and the local operation module, the node detection module is used to complete the collection and reporting of the computing power resources of this node, and the local operation module is used to accept the group global model issued by the DSP aggregation calculation module to connect to the data source of this node to complete the node model update operation, such as Figure 2 As shown, DSN1 and DSN2 are a group, DSN3 is a group, and DSNn is a group.

[0066] The model fine-tuning method provided in the embodiment of the present application can be applied to other networks suitable for federated learning in addition to the data circulation network. The embodiment of the present application does not limit this. Taking the fine-tuning module as the adapter module as an example, refer to Figure 3 , Figure 3 The following is a flow chart of the model fine-tuning process provided in an embodiment of the present application. In this embodiment, the model fine-tuning method is applied to a data circulation network, including the following steps:

[0067] Step 301: Each industry organization selects a basic large model, each organization node prepares its own data source, and completes data alignment based on the data circulation network.

[0068] Step 302: Each organization node DSN reports the node's current resource status and the node's training data volume.

[0069] Step 303: The DSP receives the resource status and training data volume of each node reported by the DSN node resource monitoring module within a specified time interval. The resource status includes the node GPU peak computing power, GPU utilization R, and the node training data volume D.

[0070] Step 304: DSP calculates the model scale carrying capacity of the node based on the real-time computing power status of each node and the amount of training data. According to the different model carrying capacities, thresholds are set and nodes are divided into N groups for registration.

[0071] Step 305: DSP selects the device with the smallest resources in the group as the group baseline, designs the training model structure of this group, and distills the basic model into N teacher models of different sizes through the knowledge distillation method. After performing knowledge distillation on the teacher models of each group, N group global models are obtained. The group global model includes an aggregation adapter and a backbone architecture. The backbone architecture of the group global model is frozen and simplified into a simulator and then sent to each DSN node of this group. Multiple DSN local operation modules in each of the N groups are locally trained and collaboratively update the aggregation adapter. In the embodiment of the present application, there is an aggregation adapter before and after the backbone architecture.

[0072] Step 306: Each group performs K rounds of second-iteration training on the group global model. The DSP collects the update results of the global models of N groups of different sizes, extracts the knowledge between the N teacher models and the aggregated group global model, and co-evolves the models of both parties. The global model is updated collaboratively using the N teacher model parameters to complete a round of first-iteration training.

[0073] Step 307: When a single node encounters resource constraints during training, it is dynamically reported to the DSP, and the DSP adjusts the next round of training groups. Otherwise, after reaching the preset number of training rounds of the second iterative training process, the DSP dynamically updates the node grouping according to the node status reported by the DSN node resource monitoring.

[0074] Step 308: Repeat steps 302-307 until the model converges or reaches the preset number of training rounds of the first iteration training, stop training, DSP outputs the industry large model, and distributes it to each structural node for reasoning.

[0075] The model fine-tuning method provided in the embodiment of the present application realizes the federation fine-tuning of the large model between different institutions in the same industry based on the data circulation network without leaving the local area to ensure privacy of the data of each institution, and at the same time considers the dynamic changes in the computing power resources of the participating nodes, and adaptively adjusts the training model structure of each institution node. The adjustment method is that each institution node reports the resource availability after a set time interval, and the server side selects the nodes participating in the current round of pre-training and groups them according to the resource availability. The device with the smallest resources in the group is selected as the group baseline, and the training model structure of this group is designed, and the basic model knowledge is distilled into group global models of different sizes. The backbone architecture of the group global model is frozen and simplified to a simulator. The nodes of each institution in the group use their own data to collaboratively train two global adapters that adapt to the group global model, co-evolve the group global model and the global model that updates the parameters, and converge the global model parameters to improve the generalization of the model. In this way, the self-adaptive completion of the federation fine-tuning of the industry large model is achieved without the institutional data being transmitted out of the node, and the privacy and efficient training of the industry large model is achieved.

[0076] The present application embodiment also provides a model fine-tuning device 400, referring to Figure 4 The model fine-tuning device 400 in this embodiment includes:

[0077] Fine-tuning module: used to perform one or more rounds of first iterative training until the first stopping condition is met to obtain a fine-tuned basic model; wherein each round of first iterative training process includes: dividing multiple computing nodes into N groups according to the model capability information of each computing node in the multiple computing nodes; generating N group global models according to the first model and the model structure corresponding to each group in the N groups; the N group global models correspond to the N groups one by one; the model structure corresponding to each group is determined according to the computing node with the worst model capability in each group; training the group global model corresponding to each group through each computing node of each group to obtain the trained group global model corresponding to each group; updating the first model parameters based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

[0078] In an embodiment of the present application, the fine-tuning module is used to perform one or more rounds of second iterative training until the second stop condition is met to obtain a group global model after each group is trained; wherein each round of the second iterative training process includes: sending the second model to each computing node in the group; so that each computing node trains the second model; receiving the trained second model parameters fed back by each computing node of each group, and updating the parameters of the second model corresponding to each group based on the trained second model parameters fed back by each computing node of each group; wherein, in the first round of training of the second iterative training, the second model is the group global model corresponding to each group, and in other rounds of training of the second iterative training out of the first round, the second model is the group global model corresponding to each group after the parameters of the previous round are updated.

[0079] In an embodiment of the present application, the fine-tuning module is used to receive model capability information reported by each computing node among multiple computing nodes.

[0080] In an embodiment of the present application, the first stopping condition includes one or more of the following: convergence of the basic model after fine-tuning, reaching a first preset number of training rounds, and reaching a first preset training time; the second stopping condition includes one or more of the following: reaching a second preset number of training rounds and reaching a second preset training time.

[0081] In an embodiment of the present application, the fine-tuning module is used to stop the first iterative training process of the current round and perform the first iterative training process of the next round when receiving the first information sent by the first computing node; the first information indicates that the first computing node cannot complete the model training.

[0082] In an embodiment of the present application, the group global model includes a fine-tuning module and a backbone architecture. Before sending the second model to each computing node within the group, the fine-tuning module is used to freeze the backbone architecture of the group global model and simplify the backbone architecture of the group global model into a simulator.

[0083] In the embodiment of the present application, the fine-tuning module includes: an adapter module and / or a prompt module.

[0084] Those skilled in the art should understand that Figure 4 The implementation functions of each module in the model fine-tuning device 400 shown can be understood by referring to the relevant description of the aforementioned method. Figure 4 The functions of the modules in the model fine-tuning device 400 shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0085] Figure 5 It is a schematic structural diagram of an electronic device 500 provided in an embodiment of the present application. Figure 5 The electronic device 500 shown includes a processor 510, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0086] Alternatively, if Figure 5 As shown, the electronic device 500 may further include a memory 520. The processor 510 may call and run a computer program from the memory 520 to implement the method in the embodiment of the present application.

[0087] The memory 520 may be a separate device independent of the processor 510 , or may be integrated into the processor 510 .

[0088] Alternatively, if Figure 5 As shown, the electronic device 500 may further include a transceiver 530, and the processor 510 may control the transceiver 530 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0089] The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of the antennas may be one or more.

[0090] The electronic device 500 may specifically be the model fine-tuning device 400 of the embodiment of the present application, and the electronic device 500 may implement the corresponding processes implemented by the model fine-tuning device 400 in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0091] Illustratively, an embodiment of the present application further provides a computer program product, including a computer program, which can be executed by the processor 510 of the electronic device 500 to complete the steps of any of the aforementioned methods.

[0092] Figure 6 It is a schematic structural diagram of the chip of an embodiment of the present application. Figure 6 The chip 600 shown includes a processor 610, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0093] Alternatively, if Figure 6 As shown, the chip 600 may further include a memory 620. The processor 610 may call and run a computer program from the memory 620 to implement the method in the embodiment of the present application.

[0094] The memory 620 may be a separate device independent of the processor 610 , or may be integrated into the processor 610 .

[0095] Optionally, the chip 600 may further include an input interface 630. The processor 610 may control the input interface 630 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0096] Optionally, the chip 600 may further include an output interface 640. The processor 610 may control the output interface 640 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0097] The chip can be applied to the electronic device 500 in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the electronic device 500 in the various methods of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0098] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0099] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0100] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0101] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0102] The embodiment of the present application further provides a storage medium for storing a computer program. The storage medium can be applied to the electronic device 500 in the embodiment of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the electronic device 500 in each method of the embodiment of the present application, which will not be described in detail for the sake of brevity.

[0103] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0104] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0105] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0106] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0108] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or an electronic device 500, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0109] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model fine-tuning method, characterized in that: include: One or more rounds of first iteration training are performed until the first stop condition is met to obtain a fine-tuned basic model; wherein each round of first iteration training process includes: According to model capability information of each computing node in the multiple computing nodes, the multiple computing nodes are divided into N groups; Generate N group global models according to the first model and the model structures corresponding to each of the N groups; the N group global models correspond to the N groups one by one; the model structures corresponding to each group are determined according to the computing nodes with the worst model capabilities in each group; The group global model corresponding to each group is trained through each computing node of each group to obtain the trained group global model corresponding to each group; The first model parameters are updated based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

2. The method according to claim 1, characterized in that The group global model corresponding to each group is trained by each computing node of each group to obtain the group global model after the training of each group, including: One or more rounds of second iterative training are performed until the second stop condition is met, and the group global model after each group training is obtained; wherein each round of second iterative training process includes: Sending the second model to each computing node in the group so that each computing node trains the second model; Receive the trained second model parameters fed back by each computing node of each group, and update the parameters of the second model corresponding to each group based on the trained second model parameters fed back by each computing node of each group; wherein, in the first round of training of the second iterative training, the second model is the group global model corresponding to each group, and in other rounds of training of the second iterative training except the first round, the second model is the group global model corresponding to each group after the parameters of the previous round are updated.

3. The method according to claim 2, characterized in that The group global model includes a fine-tuning module and a backbone architecture, and before sending the second model to each computing node in the group, further includes: The backbone architecture of the group global model is frozen and simplified into a simulator.

4. The method according to claim 3, characterized in that: The fine-tuning module includes: an adapter module and / or a prompt module.

5. The method according to any one of claims 1 to 4, characterized in that Before each round of the first iterative training begins, the method further includes: Receive model capability information reported by each of the multiple computing nodes.

6. The method according to any one of claims 1 to 4, characterized in that Also includes: When receiving the first information sent by the first computing node, stopping the first iterative training process of the current round and performing the first iterative training process of the next round; The first information indicates that the first computing node cannot complete model training.

7. The method according to any one of claims 2 to 4, characterized in that The first stop condition includes one or more of the following: convergence of the fine-tuned basic model, reaching a first preset number of training rounds, and reaching a first preset training time; The second stop condition includes one or more of the following: reaching a second preset number of training rounds, reaching a second preset training time.

8. A model fine-tuning device, characterized in that: include: Fine-tuning module: used to perform one or more rounds of first iteration training until the first stop condition is met to obtain a fine-tuned basic model; wherein each round of first iteration training process includes: According to model capability information of each computing node in the multiple computing nodes, the multiple computing nodes are divided into N groups; Generate N group global models according to the first model and the model structures corresponding to each of the N groups; the N group global models correspond to the N groups one by one; the model structures corresponding to each group are determined according to the computing nodes with the worst model capabilities in each group; The group global model corresponding to each group is trained through each computing node of each group to obtain the trained group global model corresponding to each group; The first model parameters are updated based on the trained group global model parameters corresponding to each group; wherein, in the first round of iteration of the first iterative training, the first model is the basic model, and in other rounds of iteration of the first iterative training except the first round, the first model is the basic model after the parameters of the previous round are updated.

9. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the model fine-tuning method as described in any one of claims 1 to 7.

10. A chip, characterized in that: include: A processor, used to call and run a computer program from a memory, so that a device equipped with the chip executes the model fine-tuning method described in any one of claims 1 to 7.

11. A storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the model fine-tuning method according to any one of claims 1 to 7.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the model fine-tuning method according to any one of claims 1 to 7.

Images