Data processing and model fine-tuning methods, apparatuses, electronic devices and storage media

Abstract

This disclosure provides data processing and model fine-tuning methods, apparatuses, electronic devices, and storage media, relating to artificial intelligence fields such as computer vision, deep learning, and image processing. The data processing method may include: acquiring data to be processed corresponding to a predetermined downstream task; processing the data to be processed according to a task processing model corresponding to the downstream task to obtain a data processing result. The task processing model is obtained by fine-tuning a pre-trained model based on training data corresponding to the downstream task. During fine-tuning, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters in the pre-trained model, and each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variables and the corresponding target parameters is used as the fine-tuned target parameters. Applying the scheme described in this disclosure can improve the accuracy of data processing results, etc.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to data processing and model fine-tuning methods, apparatuses, electronic devices and storage media in the fields of computer vision, deep learning and image processing. Background Technology

[0002] Currently, when performing image classification, the image classification model is usually obtained by using a model pre-training + model fine-tuning mode. The model pre-training + model fine-tuning mode means that the model is first pre-trained on a very large dataset to obtain a pre-trained model, and then fine-tuned for downstream tasks according to the actual application scenario, such as obtaining the required image classification model through fine-tuning. Summary of the Invention

[0003] This disclosure provides methods, apparatus, electronic devices, and storage media for data processing and model fine-tuning.

[0004] A data processing method, comprising:

[0005] Obtain the data to be processed corresponding to the scheduled downstream tasks;

[0006] According to the task processing model corresponding to the downstream task, the data to be processed is processed to obtain the data processing result. The task processing model is obtained by fine-tuning a pre-trained model based on the training data corresponding to the downstream task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model, and each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variables and the corresponding target parameters is used as the fine-tuned target parameters.

[0007] A model fine-tuning method includes:

[0008] Obtain a pre-trained model, determine at least one of the model parameters of the pre-trained model as a target parameter, and set corresponding residual variables for each target parameter;

[0009] Obtain training data corresponding to the predetermined downstream task, and fine-tune the pre-trained model using the training data to obtain the task processing model corresponding to the downstream task. During the fine-tuning process, the target parameter remains unchanged, the residual variable corresponding to the target parameter is updated, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

[0010] A data processing device includes: a data acquisition module and a data processing module;

[0011] The data acquisition module is used to acquire the data to be processed corresponding to the predetermined downstream tasks;

[0012] The data processing module is used to process the data to be processed according to the task processing model corresponding to the downstream task to obtain the data processing result. The task processing model is obtained by fine-tuning a pre-trained model based on the training data corresponding to the downstream task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model, and each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variables and the corresponding target parameters is used as the fine-tuned target parameters.

[0013] A model fine-tuning device includes: a parameter processing module and a model fine-tuning module;

[0014] The parameter processing module is used to obtain a pre-trained model, determine at least one of the model parameters of the pre-trained model as a target parameter, and set corresponding residual variables for each target parameter.

[0015] The model fine-tuning module is used to acquire training data corresponding to a predetermined downstream task, and to fine-tune the pre-trained model using the training data to obtain a task processing model corresponding to the downstream task. During the fine-tuning process, the target parameter remains unchanged, the residual variable corresponding to the target parameter is updated, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

[0016] An electronic device, comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described above.

[0020] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0021] A computer program product includes a computer program / instructions that, when executed by a processor, implement the method described above.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0024] Figure 1 This is a flowchart of an embodiment of the data processing method described in this disclosure;

[0025] Figure 2 This is a flowchart of an embodiment of the model fine-tuning method described in this disclosure;

[0026] Figure 3 This is a schematic diagram of the target parameters and corresponding residual variables described in this disclosure;

[0027] Figure 4 This is a schematic diagram of the low-rank matrix decomposition method for ΔW as described in this disclosure;

[0028] Figure 5 This is a schematic diagram of the composition structure of Embodiment 500 of the data processing apparatus described in this disclosure;

[0029] Figure 6 This is a schematic diagram of the composition structure of Embodiment 600 of the model fine-tuning device described in this disclosure;

[0030] Figure 7 A schematic block diagram of an electronic device 700 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0031] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0032] Furthermore, it should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0033] Figure 1 This is a flowchart illustrating an embodiment of the data processing method described in this disclosure. Figure 1As shown, the specific implementation methods are as follows.

[0034] In step 101, the data to be processed corresponding to the predetermined downstream task is obtained.

[0035] In step 102, the data to be processed is processed according to the task processing model corresponding to the downstream task to obtain the data processing result. The task processing model is obtained by fine-tuning the pre-trained model based on the training data corresponding to the downstream task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model, and each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variables and the corresponding target parameters is used as the fine-tuned target parameters.

[0036] Traditional model fine-tuning methods typically involve full parameter tuning of the pre-trained model. However, this approach can easily cause the model to forget the knowledge learned during the pre-training phase, thereby affecting the model's transfer performance and the performance of the fine-tuned model. For example, it can affect the performance of the fine-tuned image classification model, thus impacting the accuracy of the image classification results.

[0037] The scheme described in the above method embodiment can replace the update of the original pre-trained model parameters with the update of residual variables, thereby effectively preserving the knowledge learned in the pre-training stage, reducing the destruction of pre-trained knowledge, and enabling the rich knowledge learned in the pre-training stage to be efficiently transferred to downstream tasks, thereby improving the model fine-tuning effect and the performance of the fine-tuned model. Correspondingly, using the fine-tuned model to process the data to be processed can improve the accuracy of the data processing results.

[0038] Preferably, the downstream task includes an image classification task, and each training data includes: an image and the corresponding image classification result. Obtaining the data to be processed includes: obtaining the image to be classified. The step of processing the data to be processed according to the task processing model corresponding to the downstream task to obtain the data processing result includes: classifying the image to be classified according to the image classification model corresponding to the image classification task to obtain the image classification result.

[0039] In other words, the scheme described in this disclosure can be applied to image classification scenarios (i.e., image classification tasks). After obtaining a pre-trained model, the pre-trained model can be fine-tuned based on the training data corresponding to the image classification task to obtain an image classification model. Each piece of training data can include an image and the image classification result for that image, such as manually annotated image classification results.

[0040] Accordingly, during the inference phase, the image to be classified can be acquired and used as input to the image classification model to obtain the output image classification result.

[0041] Since the performance of the image classification model obtained according to the scheme described in this disclosure is improved, the accuracy of the image classification results can be improved by using it for image classification. Moreover, there is no increase in any additional parameters during the inference stage, thereby achieving zero increase in time consumption and zero increase in memory usage.

[0042] As can be seen, the implementation of the above data processing method depends on the pre-trained task processing model. The following explains how to obtain this model.

[0043] Figure 2 This is a flowchart of an embodiment of the model fine-tuning method described in this disclosure. Figure 2 As shown, the specific implementation methods are as follows.

[0044] In step 201, a pre-trained model is obtained, at least one of the model parameters of the pre-trained model is determined as the target parameter, and corresponding residual variables are set for each target parameter.

[0045] In step 202, training data corresponding to the predetermined downstream task is obtained, and the pre-trained model is fine-tuned using the training data to obtain the task processing model corresponding to the downstream task. During the fine-tuning process, the target parameter remains unchanged, the residual variable corresponding to the target parameter is updated, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

[0046] By adopting the scheme described in the above method embodiments, the update of residual variables can be used instead of the update of the original pre-trained model parameters, thereby effectively preserving the knowledge learned in the pre-training stage, that is, reducing the destruction of pre-trained knowledge, so that the rich knowledge learned in the pre-training stage can be efficiently transferred to downstream tasks, thereby improving the model fine-tuning effect and the performance of the fine-tuned model. Correspondingly, using the fine-tuned model to process the data to be processed can improve the accuracy of the data processing results.

[0047] There are no restrictions on how to obtain the pre-trained model. For example, a pre-trained model can be obtained by pre-training on a very large dataset. In addition, there are no restrictions on what kind of model the pre-trained model is, which can be determined according to actual needs.

[0048] A pre-trained model includes multiple model parameters. At least one of these model parameters can be identified as the target parameter, and corresponding residual variables can be set for each target parameter.

[0049] Preferably, all model parameters in the pre-trained model can be determined as target parameters, and corresponding residual variables can be set for each target parameter. In addition, the initial value of each residual variable can be set to 0.

[0050] The specific number of model parameters to be determined as target parameters can be determined according to actual needs. Generally speaking, the more target parameters there are, the better the performance of the task processing model after fine-tuning. Therefore, preferably, all model parameters in the pre-trained model can be determined as target parameters.

[0051] In addition, the initial value of each residual variable can be assigned to 0, that is, each residual variable is initialized to 0 at the beginning of fine-tuning, and has a zero mean property, which can make the training more stable.

[0052] Furthermore, for the pre-trained model, training data corresponding to the predetermined downstream task can be obtained, and the pre-trained model can be fine-tuned using the obtained training data to obtain the task processing model corresponding to the downstream task. During fine-tuning, the target parameters can remain unchanged, and only the residual variables corresponding to the target parameters are updated. Of course, if not all model parameters in the pre-trained model are used as target parameters, i.e., if non-target parameters exist, they can be updated in the traditional way. Alternatively, the sum of the fine-tuned residual variables and the corresponding target parameters can be used as the fine-tuned target parameters.

[0053] Figure 3 This is a schematic diagram illustrating the target parameters and corresponding residual variables described in this disclosure. Figure 3 As shown, assuming a certain objective parameter is W, R represents a real number, d represents the dimension, and compared to the traditional method of directly updating W during the fine-tuning stage, the scheme described in this disclosure introduces a residual variable. During the fine-tuning phase, only ΔW is updated. Once the fine-tuning is complete, W' = W + ΔW can be calculated to obtain the fine-tuned target parameter W'.

[0054] Suppose that the pre-trained model contains 10 model parameters (this is just an example; the actual number may be much larger), and assume that all 10 model parameters are used as target parameters, resulting in target parameters 1 to 10. Then the model parameters in the task processing model include the fine-tuned target parameters 1 to 10. Suppose that 8 model parameters in the pre-trained model are used as target parameters, and the remaining 2 model parameters are used as non-target parameters. Then the model parameters in the task processing model include the fine-tuned target parameters 1 to 8, the fine-tuned non-target parameter 9, and the fine-tuned non-target parameter 10.

[0055] Compared to the pre-trained model, the fine-tuned task processing model has no change in structure and no additional parameters are added.

[0056] Preferably, the residual variable can also be subjected to low-rank matrix decomposition to obtain a first decomposition result and a second decomposition result. Accordingly, during the fine-tuning process, the first decomposition result and the second decomposition result can be updated respectively.

[0057] Taking ΔW as an example again, Figure 4 This is a schematic diagram illustrating the low-rank matrix decomposition method for ΔW described in this disclosure. Figure 4 As shown, through low-rank matrix decomposition, we can obtain the first decomposition result U and the second decomposition result V, where ΔW = UV, and... r << d, where r is a hyperparameter and its specific value can be determined according to actual needs.

[0058] Through the above processing, the high-dimensional residual variables are decomposed into two low-dimensional subspaces, which allows for optimization of the variables in the low-dimensional subspaces. This reduces the model optimization cost, accelerates model convergence, improves model fine-tuning efficiency, reduces resource consumption, and further minimizes the disruption to pre-trained knowledge.

[0059] In addition, preferably, after fine-tuning is completed, fine-tuned residual variables can be generated based on the fine-tuned first decomposition results and the fine-tuned second decomposition results, and the sum of the fine-tuned residual variables and the corresponding target parameters can be used as the fine-tuned target parameters.

[0060] The fine-tuned residual variables can be determined based on the first and second decomposition results, and then the fine-tuned target parameters can be determined based on the fine-tuned residual variables and the original target parameters.

[0061] Preferably, the downstream task includes an image classification task, wherein each training data includes an image and the corresponding image classification result, and obtaining the task processing model corresponding to the downstream task includes obtaining the image classification model corresponding to the image classification task.

[0062] In other words, the solution described in this disclosure can be applied to image classification scenarios. After obtaining a pre-trained model, it can be fine-tuned based on the training data corresponding to the image classification task to obtain an image classification model. Each piece of training data can include an image and its image classification result, such as manually labeled image classification results.

[0063] Accordingly, during the inference phase, the image to be classified can be acquired and used as input to the image classification model to obtain the output image classification result.

[0064] Since the performance of the image classification model obtained according to the scheme described in this disclosure is improved, the accuracy of the image classification results can be improved by using it for image classification.

[0065] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this disclosure. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure. Furthermore, for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0066] In summary, the scheme described in the embodiments of this disclosure can replace the update of the original pre-trained model parameters with the update of residual variables, thereby effectively preserving the knowledge learned in the pre-training stage, reducing the destruction of pre-trained knowledge, and enabling the rich knowledge learned in the pre-training stage to be efficiently transferred to downstream tasks, thus improving the model fine-tuning effect and the performance of the fine-tuned model. In addition, the residual variables can be subjected to low-rank matrix decomposition, thereby optimizing the variables in low-dimensional subspace, reducing the model optimization cost, accelerating the model convergence speed, improving the model fine-tuning efficiency, reducing resource consumption, and further reducing the destruction of pre-trained knowledge. Furthermore, the fine-tuned model can be used to process the data to be processed, such as performing image classification on images to be classified, which can improve the accuracy of image classification results. Moreover, there is no addition of any additional parameters in the inference stage, and the cost of later model maintenance and upgrades is reduced, enabling rapid iterative updates of the model.

[0067] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0068] Figure 5 This is a schematic diagram of the structural composition of Embodiment 500 of the data processing apparatus described in this disclosure. Figure 5 As shown, it includes: a data acquisition module 501 and a data processing module 502.

[0069] The data acquisition module 501 is used to acquire the data to be processed corresponding to the predetermined downstream tasks.

[0070] The data processing module 502 is used to process the data to be processed according to the task processing model corresponding to the downstream task, and obtain the data processing result. The task processing model is obtained by fine-tuning the pre-trained model according to the training data corresponding to the downstream task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model, and each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variables and the corresponding target parameters is used as the fine-tuned target parameters.

[0071] By adopting the scheme described in the above-described device embodiment, the update of residual variables can be used instead of the update of the original pre-trained model parameters, thereby effectively preserving the knowledge learned in the pre-training stage, reducing the destruction of pre-trained knowledge, and enabling the rich knowledge learned in the pre-training stage to be efficiently transferred to downstream tasks, thereby improving the model fine-tuning effect and the performance of the fine-tuned model. Correspondingly, using the fine-tuned model to process the data to be processed can improve the accuracy of the data processing results.

[0072] Preferably, the downstream task includes an image classification task, and each training data includes an image and the corresponding image classification result. Accordingly, the data acquisition module 501 acquires the data to be processed by acquiring the image to be classified, and the data processing module 502 processes the data to be processed according to the task processing model corresponding to the downstream task to obtain the data processing result by classifying the image to be classified according to the image classification model corresponding to the image classification task to obtain the image classification result.

[0073] Figure 6 This is a schematic diagram of the structural composition of Embodiment 600 of the model fine-tuning device described in this disclosure. Figure 6 As shown, it includes: parameter processing module 601 and model fine-tuning module 602.

[0074] The parameter processing module 601 is used to obtain the pre-trained model, determine at least one of the model parameters of the pre-trained model as the target parameter, and set the corresponding residual variables for each target parameter.

[0075] The model fine-tuning module 602 is used to obtain training data corresponding to the predetermined downstream task, fine-tune the pre-trained model using the training data, and obtain the task processing model corresponding to the downstream task. During the fine-tuning process, the target parameter remains unchanged, the residual variable corresponding to the target parameter is updated, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

[0076] By adopting the scheme described in the above-described device embodiment, the update of residual variables can be used instead of the update of the original pre-trained model parameters, thereby effectively preserving the knowledge learned in the pre-training stage, reducing the destruction of pre-trained knowledge, and enabling the rich knowledge learned in the pre-training stage to be efficiently transferred to downstream tasks, thereby improving the model fine-tuning effect and the performance of the fine-tuned model. Correspondingly, using the fine-tuned model to process the data to be processed can improve the accuracy of the data processing results.

[0077] The pre-trained model includes multiple model parameters. The parameter processing module 601 can determine at least one of the model parameters as the target parameter and set the corresponding residual variables for each target parameter.

[0078] Preferably, the parameter processing module 601 can determine all model parameters in the pre-trained model as target parameters, and can set corresponding residual variables for each target parameter. In addition, the initial value of each residual variable can be set to 0.

[0079] Furthermore, for the pre-trained model, the model fine-tuning module 602 can acquire training data corresponding to the predetermined downstream task and use the acquired training data to fine-tune the pre-trained model, thereby obtaining the task processing model corresponding to the downstream task. During fine-tuning, the target parameters can remain unchanged; only the residual variables corresponding to the target parameters are updated. Of course, if not all model parameters in the pre-trained model are used as target parameters—that is, if non-target parameters exist—then the non-target parameters can be updated in the traditional way. Additionally, the sum of the fine-tuned residual variables and the corresponding target parameters can be used as the fine-tuned target parameters.

[0080] Preferably, the model fine-tuning module 602 can also perform low-rank matrix decomposition on the residual variables to obtain a first decomposition result and a second decomposition result. Accordingly, during the fine-tuning process, the first decomposition result and the second decomposition result can be updated respectively.

[0081] In addition, preferably, after the fine-tuning is completed, the model fine-tuning module 602 can also generate fine-tuned residual variables based on the fine-tuned first decomposition result and the fine-tuned second decomposition result, and can use the sum of the fine-tuned residual variables and the corresponding target parameters as the fine-tuned target parameters.

[0082] Preferably, the downstream task includes: an image classification task, wherein each training data includes: an image and the corresponding image classification result, and the model fine-tuning module 602 obtains the task processing model corresponding to the downstream task, including: obtaining the image classification model corresponding to the image classification task.

[0083] Figure 5 and Figure 6The specific workflow of the device embodiment shown can be found in the relevant descriptions in the foregoing method embodiments, and will not be repeated here.

[0084] In summary, the scheme described in the embodiments of this disclosure can replace the update of the original pre-trained model parameters with the update of residual variables, thereby effectively preserving the knowledge learned in the pre-training stage, reducing the destruction of pre-trained knowledge, and enabling the rich knowledge learned in the pre-training stage to be efficiently transferred to downstream tasks, thereby improving the model fine-tuning effect and the performance of the fine-tuned model. In addition, the residual variables can be subjected to low-rank matrix decomposition, thereby optimizing the variables in low-dimensional subspace, reducing the model optimization cost, accelerating the model convergence speed, improving the model fine-tuning efficiency, reducing resource consumption, and further reducing the destruction of pre-trained knowledge. Furthermore, the fine-tuned model can be used to process the data to be processed, such as performing image classification on images to be classified, which can improve the accuracy of image classification results. Moreover, there is no addition of any additional parameters in the inference stage, and the cost of later model maintenance and upgrades is reduced, enabling rapid iterative updates of the model.

[0085] The solutions described in this disclosure can be applied to the field of artificial intelligence, particularly computer vision, deep learning, and image processing. Artificial intelligence is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It involves both hardware and software technologies. Artificial intelligence hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0086] Furthermore, the data to be processed in the embodiments described in this disclosure is not targeted at any specific user and does not reflect the personal information of any specific user. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions of this disclosure all comply with relevant laws and regulations and do not violate public order and good morals.

[0087] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0088] Figure 7A schematic block diagram of an electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0089] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 702 or a computer program loaded from storage unit 708 into random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0090] Multiple components in device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of monitors, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0091] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as those described in this disclosure. For example, in some embodiments, the methods described in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the methods described in this disclosure can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the methods described in this disclosure by any other suitable means (e.g., by means of firmware).

[0092] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0094] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0096] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0097] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0098] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0099] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A data processing method, comprising: Obtain the image to be classified corresponding to the image classification task; The image to be classified is classified according to the image classification model corresponding to the image classification task to obtain the image classification result. The image classification model is obtained by fine-tuning a pre-trained model based on the training data corresponding to the image classification task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model. Each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter. Each training data includes an image and the corresponding image classification result.

2. A model fine-tuning method, comprising: Obtain a pre-trained model, determine at least one of the model parameters of the pre-trained model as a target parameter, and set corresponding residual variables for each target parameter; Training data corresponding to an image classification task is obtained. Each training data includes an image and its corresponding image classification result. The pre-trained model is fine-tuned using the training data to obtain an image classification model corresponding to the image classification task. During the fine-tuning process, the target parameter remains unchanged, and the residual variable corresponding to the target parameter is updated. This includes performing a low-rank matrix decomposition on the residual variable to obtain a first decomposition result and a second decomposition result. The first decomposition result and the second decomposition result are updated respectively, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

3. The method according to claim 2, wherein, The step of determining at least one of the model parameters of the pre-trained model as target parameters and setting corresponding residual variables for each target parameter includes: All model parameters in the pre-trained model are determined as the target parameters. Corresponding residual variables are set for each target parameter, and the initial value of each residual variable is set to 0.

4. The method according to claim 2, wherein, The step of using the sum of the fine-tuned residual variable and the corresponding target parameter as the fine-tuned target parameter includes: The fine-tuned residual variable is generated based on the first and second decomposition results after fine-tuning, and the sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter.

5. A data processing apparatus, comprising: Data acquisition module and data processing module; The data acquisition module is used to acquire the image to be classified corresponding to the image classification task; The data processing module is used to classify the image to be classified according to the image classification model corresponding to the image classification task, and obtain the image classification result. The image classification model is obtained by fine-tuning a pre-trained model based on the training data corresponding to the image classification task. During the fine-tuning process, the target parameters in the pre-trained model remain unchanged, while the residual variables corresponding to the target parameters are updated. The target parameters are at least one of the model parameters of the pre-trained model. Each target parameter has a corresponding residual variable. The sum of the fine-tuned residual variable and the corresponding target parameter is used as the fine-tuned target parameter. Each training data includes an image and the corresponding image classification result.

6. A model fine-tuning device, comprising: Parameter processing module and model fine-tuning module; The parameter processing module is used to obtain a pre-trained model, determine at least one of the model parameters of the pre-trained model as a target parameter, and set corresponding residual variables for each target parameter. The model fine-tuning module is used to acquire training data corresponding to the image classification task. Each training data includes an image and the corresponding image classification result. The pre-trained model is fine-tuned using the training data to obtain the image classification model corresponding to the image classification task. During the fine-tuning process, the target parameter remains unchanged, and the residual variable corresponding to the target parameter is updated. This includes performing a low-rank matrix decomposition on the residual variable to obtain a first decomposition result and a second decomposition result, updating the first decomposition result and the second decomposition result respectively, and using the sum of the fine-tuned residual variable and the corresponding target parameter as the fine-tuned target parameter.

7. The apparatus according to claim 6, wherein, The parameter processing module determines all model parameters in the pre-trained model as the target parameters, sets corresponding residual variables for each target parameter, and assigns an initial value of 0 to each residual variable.

8. The apparatus according to claim 6, wherein, The model fine-tuning module generates the fine-tuned residual variable based on the fine-tuned first decomposition result and the fine-tuned second decomposition result, and uses the sum of the fine-tuned residual variable and the corresponding target parameter as the fine-tuned target parameter.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the method of any one of claims 1-4.

Images