Model fine-tuning method and device, electronic equipment and storage medium
Patent Information
- Application Number
- CN202211651346.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-21
- Publication Date
- 2026-08-21
- Estimated Expiration
- 2042-12-21
AI Technical Summary
[0004]虽然从数据量着手是比较有效的解决过拟合的办法,但是在实际任务中往往缺少相应微调数据,或者数据成本较高,获取难度大等情况;移除特征或者删除层等方式简化网络结构,往往会弱化网络的拟合能力,对分割效果产生影响;早停法可以防止过拟合,但也存在模型还没有训练到最优结果却被停止了训练,导致模型最终的分割效果不理想
[0031]本申请的实施例,将输入图像分别输入第一预训练模型和第二预训练模型,得到第一预训练模型输出的第一结果和第二预训练模型输出的第二结果,并获得第二结果与输入图像的标签之间的第一损失、第二结果与第一结果之间的第二损失,最后根据第一损失和第二损失,对第二预训练模型进行模型微调。由此,通过模型和标签直接的损失以及模型和冻结的预训练模型的输出之间计算的先验语义保留损失,使得微调的模型能够保留预训练模型的先验语义特征,从而避免模型在少量微调数据集上过拟合,可以在不增加模型大小,不改变模型的复杂度,不增加模型的推理耗时的同时避免模型微调时过拟合,有效提升分割类任务中模型的鲁棒性,保证模型的分割质量。
Smart Images

Figure CN116188892B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image recognition technology, and more specifically, to a method, apparatus, electronic device, and storage medium for fine-tuning a model. Background Technology
[0002] Deep learning technology is gradually being integrated into all aspects of social life. Segmentation algorithms, as an important branch of deep learning, are also applied in various fields, such as medical image segmentation, video portrait segmentation, instance segmentation, and head portrait segmentation. Segmentation algorithms often need to be fine-tuned on existing pre-trained models using a small amount of data, but overfitting methods are often encountered during the fine-tuning process.
[0003] To address the overfitting problem caused by fine-tuning segmentation task models, there are several main solutions: one is to increase the amount of data used during fine-tuning; another is to simplify the network structure by removing features or deleting layers to avoid overfitting; and yet another is to use early stopping, which involves monitoring the loss during training and stopping training when an upward trend is observed to prevent overfitting.
[0004] While addressing overfitting by increasing data volume is a relatively effective approach, in real-world tasks, there is often a lack of corresponding fine-tuning data, or the data is costly and difficult to obtain. Simplifying the network structure by removing features or deleting layers often weakens the network's fitting ability and affects the segmentation results. Early stopping can prevent overfitting, but it can also lead to the model being stopped before it reaches its optimal training result, resulting in an unsatisfactory final segmentation performance. Summary of the Invention
[0005] To address the problems existing in the prior art, embodiments of this application provide a model fine-tuning method, apparatus, electronic device, and storage medium, which can avoid overfitting during model fine-tuning without increasing model size, changing model complexity, or increasing model inference time, effectively improving the robustness of the model in segmentation tasks and ensuring the segmentation quality of the model.
[0006] In a first aspect, embodiments of this application provide a method for fine-tuning a model, including:
[0007] Obtain a fine-tuning dataset, which includes the input image;
[0008] The input image is input into a first pre-trained model and a second pre-trained model respectively to obtain a first result output by the first pre-trained model and a second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights.
[0009] Obtain a first loss between the second result and the label of the input image, and a second loss between the second result and the first result;
[0010] Based on the first loss and the second loss, the second pre-trained model is fine-tuned.
[0011] Further, obtaining the first loss between the second result and the label of the input image, and the second loss between the second result and the first result, includes:
[0012] Calculate the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result;
[0013] The first loss is obtained based on the first structural similarity, and the second loss is obtained based on the second structural similarity.
[0014] Furthermore, the first pre-trained model and the second pre-trained model are segmentation models, which are used to segment the input target image to obtain a mask image corresponding to the target image.
[0015] Further, the step of fine-tuning the second pre-trained model based on the first loss and the second loss includes:
[0016] Based on the first loss and the second loss, the parameters of each layer in the second pre-trained model are adjusted to achieve model fine-tuning of the second pre-trained model.
[0017] Secondly, embodiments of this application also provide a model fine-tuning device, comprising:
[0018] The acquisition module is used to obtain the fine-tuning dataset, which includes the input image;
[0019] The model output result acquisition module is used to input the input image into the first pre-trained model and the second pre-trained model respectively to obtain the first result output by the first pre-trained model and the second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights.
[0020] The loss calculation module is used to obtain a first loss between the second result and the label of the input image, and a second loss between the second result and the first result;
[0021] The fine-tuning module is used to fine-tune the second pre-trained model based on the first loss and the second loss.
[0022] Furthermore, the loss calculation module is specifically used for:
[0023] Calculate the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result;
[0024] The first loss is obtained based on the first structural similarity, and the second loss is obtained based on the second structural similarity.
[0025] Furthermore, the fine-tuning module is specifically used for:
[0026] Based on the first loss and the second loss, the parameters of each layer in the second pre-trained model are adjusted to achieve model fine-tuning of the second pre-trained model.
[0027] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement a fine-tuning method for the model described in the first aspect above when executing the program.
[0028] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, the computer program being used to implement the fine-tuning method of the model according to the first aspect described above.
[0029] Fifthly, embodiments of this application also provide a computer program product having a computer program stored thereon, the computer program being used to implement the fine-tuning method of the model according to the first aspect described above.
[0030] The embodiments of this application bring the following beneficial effects:
[0031] In embodiments of this application, the input image is input into a first pre-trained model and a second pre-trained model, respectively, to obtain a first result output by the first pre-trained model and a second result output by the second pre-trained model. A first loss between the second result and the label of the input image, and a second loss between the second result and the first result are also obtained. Finally, the second pre-trained model is fine-tuned based on the first and second losses. Thus, by using the loss between the model and the label, and the prior semantic preservation loss calculated between the model and the output of the frozen pre-trained model, the fine-tuned model can retain the prior semantic features of the pre-trained model. This avoids overfitting on a small amount of fine-tuning data, and prevents overfitting during model fine-tuning without increasing model size, complexity, or inference time. This effectively improves the robustness of the model in segmentation tasks and ensures the segmentation quality. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0033] Figure 1 A flowchart illustrating a model fine-tuning method provided in an embodiment of this application;
[0034] Figure 2 A schematic diagram illustrating a model fine-tuning method provided in an embodiment of this application;
[0035] Figure 3 A structural block diagram of a model fine-tuning device provided in an embodiment of this application;
[0036] Figure 4 This is a schematic diagram of an embodiment of an electronic device provided in this application.
[0037] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.
[0039] In the specification, claims, and accompanying drawings of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.
[0040] The following describes, in conjunction with the accompanying drawings, a method, apparatus, electronic device, and storage medium for fine-tuning a model according to embodiments of this application.
[0041] The model fine-tuning method of this invention can be applied to fine-tuning many types of models. In a specific application scenario, such as a model for image segmentation tasks, this method is used for...
[0042] This method involves fine-tuning models for segmentation tasks to address the overfitting problem that arises during model fine-tuning in such tasks. This approach can be implemented without increasing model size, complexity, or performance.
[0043] While reducing the time consumption of inference, this approach avoids overfitting during model fine-tuning, effectively improving the robustness of the model in segmentation tasks and ensuring the segmentation quality. Segmentation tasks include, for example, medical image segmentation, video portrait segmentation, instance segmentation, and avatar segmentation.
[0044] Figure 1 This is a flowchart of a method for fine-tuning a model according to an embodiment of this application. Figure 1 As shown, the fine-tuning method of the model in this application embodiment includes the following steps:
[0045] S101: Obtain the fine-tuning dataset, which includes the input image. That is, prepare a small amount of data for fine-tuning the model. For example, in a foreground segmentation task, the input image could be an image containing the target person, labeled with a mask that segments the target person.
[0046] 5S102: Input the input image into the first pre-trained model and the second pre-trained model respectively to obtain the...
[0047] The first result output by a pre-trained model and the second result output by a second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights.
[0048] Taking the segmentation model as an example of the segmentation task mentioned above, the first pre-trained model and the second pre-trained model are segmentation models. The segmentation model is used to segment the target image into a mask image corresponding to the target image.
[0049] As a concrete example, such as Figure 2 As shown, the input image (i.e., the input picture) is an image that includes the target person. First, the input image is fed into a second pre-trained model (such as...). Figure 2 The first pre-trained model (with weight updates) generates a mask result (a mask image) and then uses another frozen pre-trained model (the first pre-trained model). Figure 2 The pre-trained model (marked with weight freeze) is used to input the same image into the first pre-trained model, which generates the corresponding mask image, i.e., the first result (another mask image).
[0050] The outputs of the first and second pre-trained models, i.e., the mask images, are determined by the models themselves. That is, the function of the first and second pre-trained models is to segment the input image into corresponding mask images. It is understood that the outputs of the first and second pre-trained models are merely illustrative; in other examples, they could also be used to segment other results (non-mask images). Figure 2 The model (the mask image shown in the image).
[0051] S103: Obtain the first loss between the second result and the label of the input image, and the second loss between the second result and the first result.
[0052] In a specific example, obtaining the first loss between the second result and the label of the input image, and the second loss between the second result and the first result, includes: calculating the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result; obtaining the first loss based on the first structural similarity, and obtaining the second loss based on the second structural similarity.
[0053] Combination Figure 2 As shown, the second pre-trained model (such as Figure 2 The structural similarity between the mask output (i.e., the second result) of the pre-trained model (with weight updates) and the label of the input image is calculated to determine the loss between the output and the label. Figure 2 As shown, this loss is denoted as the consistency loss. Furthermore, the second result is also related to the first result, namely the first pre-trained model (such as...). Figure 2 The structural similarity of the masked output of the pre-trained model (marked with weight freeze) is calculated, and then the loss between its output and the output of the first pre-trained model is determined. Figure 2 As shown, this loss is denoted as the prior semantic preservation loss.
[0054] In the above description, both the first loss and the second loss can be calculated using methods such as SSIM. SSIM (Structural Similarity) is a metric that measures the similarity between two images.
[0055] S104: Fine-tune the second pre-trained model based on the first loss and the second loss.
[0056] Specifically, the second pre-trained model is fine-tuned based on the first loss and the second loss, including adjusting the parameters of each layer in the second pre-trained model according to the first loss and the second loss to achieve model fine-tuning. By using the loss between the model and labels, and the prior semantic preservation loss calculated between the model and the output of the frozen pre-trained model, the fine-tuned model can retain the prior semantic features of the pre-trained model, thereby avoiding overfitting on a small amount of fine-tuning dataset.
[0057] According to the model fine-tuning method of the present invention, the input image is input into a first pre-trained model and a second pre-trained model respectively, to obtain a first result output by the first pre-trained model and a second result output by the second pre-trained model. A first loss between the second result and the label of the input image, and a second loss between the second result and the first result are obtained. Finally, the second pre-trained model is fine-tuned based on the first and second losses. Thus, by using the loss between the model and the label, and the prior semantic preservation loss calculated between the model and the output of the frozen pre-trained model, the fine-tuned model can retain the prior semantic features of the pre-trained model, thereby avoiding overfitting on a small fine-tuning dataset. This method avoids overfitting during model fine-tuning without increasing model size, complexity, or inference time, effectively improving the robustness of the model in segmentation tasks and ensuring the segmentation quality.
[0058] Figure 3 This is a structural block diagram of the fine-tuning device for a model according to an embodiment of this application. Figure 3 As shown, a model fine-tuning device according to an embodiment of this application specifically includes: an acquisition module 310, a model output result acquisition module 320, a loss calculation module 330, and a fine-tuning module 340, wherein:
[0059] The acquisition module 310 is used to obtain a fine-tuning dataset, which includes the input image;
[0060] The model output result acquisition module 320 is used to input the input image into the first pre-trained model and the second pre-trained model respectively to obtain the first result output by the first pre-trained model and the second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights.
[0061] The loss calculation module 330 is used to obtain a first loss between the second result and the label of the input image, and a second loss between the second result and the first result;
[0062] Fine-tuning module 340 is used to fine-tune the second pre-trained model based on the first loss and the second loss.
[0063] In one embodiment of the present invention, the loss calculation module 330 is specifically used for:
[0064] Calculate the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result;
[0065] The first loss is obtained based on the first structural similarity, and the second loss is obtained based on the second structural similarity.
[0066] In one embodiment of the present invention, the fine-tuning module 340 is specifically used for:
[0067] Based on the first loss and the second loss, the parameters of each layer in the second pre-trained model are adjusted to achieve model fine-tuning of the second pre-trained model.
[0068] According to the model fine-tuning device of the present invention, an input image is input into a first pre-trained model and a second pre-trained model respectively, to obtain a first result output by the first pre-trained model and a second result output by the second pre-trained model. A first loss between the second result and the label of the input image, and a second loss between the second result and the first result are obtained. Finally, the second pre-trained model is fine-tuned based on the first and second losses. Thus, by using the loss between the model and the label, and the prior semantic preservation loss calculated between the model and the output of the frozen pre-trained model, the fine-tuned model can retain the prior semantic features of the pre-trained model, thereby avoiding overfitting on a small fine-tuning dataset. This avoids overfitting during model fine-tuning without increasing model size, complexity, or inference time, effectively improving the robustness of the model in segmentation tasks and ensuring the segmentation quality.
[0069] It should be noted that the specific implementation of the fine-tuning device of the model in this application embodiment is similar to the specific implementation of the fine-tuning method of the model in this application embodiment. Please refer to the description in the method section for details, which will not be repeated here.
[0070] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.
[0071] like Figure 4As shown, the electronic device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from the storage section 702 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0072] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.
[0073] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a machine-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined in the electronic device of this application.
[0074] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electronic device, apparatus, or device that is electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an electronic device, apparatus, or device that executes instructions. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in connection with an electronic device, apparatus, or device whose instructions are executed. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0075] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of processing and receiving devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based electronic device that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0076] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor, which executes the program to implement a model fine-tuning method: obtaining a fine-tuning dataset, the fine-tuning dataset including an input image; inputting the input image into a first pre-trained model and a second pre-trained model respectively, obtaining a first result output by the first pre-trained model and a second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights; obtaining a first loss between the second result and the label of the input image, and a second loss between the second result and the first result; and fine-tuning the second pre-trained model based on the first loss and the second loss.
[0077] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute a fine-tuning method for the model described in this application: obtaining a fine-tuning dataset, the fine-tuning dataset including an input image; inputting the input image into a first pre-trained model and a second pre-trained model respectively, obtaining a first result output by the first pre-trained model and a second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights; obtaining a first loss between the second result and the label of the input image, and a second loss between the second result and the first result; and fine-tuning the second pre-trained model based on the first loss and the second loss.
[0078] In another aspect, this application also provides a computer program product, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer program product stores one or more programs that, when used by one or more processors, execute a fine-tuning method for the model described in this application: obtaining a fine-tuning dataset, the fine-tuning dataset including an input image; inputting the input image into a first pre-trained model and a second pre-trained model respectively, obtaining a first result output by the first pre-trained model and a second result output by the second pre-trained model, wherein the first pre-trained model is a pre-trained model with frozen weights; obtaining a first loss between the second result and the label of the input image, and a second loss between the second result and the first result; and fine-tuning the second pre-trained model based on the first loss and the second loss.
[0079] The above description is merely a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural transformations made based on the content of this application's specification and drawings under the concept of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.
Claims
1. A method for fine-tuning a model, characterized in that, include: Obtain a fine-tuning dataset, which includes an input image. The fine-tuning dataset is applied to an image segmentation task, which includes at least one of medical image segmentation, video portrait segmentation, instance segmentation, or head portrait segmentation. The input image is simultaneously input into the first pre-trained model and the second pre-trained model to obtain the first result output by the first pre-trained model and the second result output by the second pre-trained model. The first pre-trained model and the second pre-trained model are both image segmentation models. The image segmentation model is used to output a corresponding pixel-level mask image based on the input image. The first pre-trained model is a pre-trained segmentation model with completely frozen weights, which participates in the fine-tuning process of the second pre-trained model as a prior semantic reference model. Obtain a first loss between the second result and the label of the input image, and a second loss between the second result and the first result. The first loss is a consistency loss, used to ensure pixel-level matching between the segmentation result and the true label. The second loss is a priori semantic preservation loss, used to constrain the second pre-trained model to retain the mature semantic features of the pre-trained segmentation model. Based on the first loss and the second loss, the second pre-trained model is fine-tuned.
2. The fine-tuning method for the model according to claim 1, characterized in that, The process of obtaining the first loss between the second result and the label of the input image, and the second loss between the second result and the first result, includes: Calculate the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result; The first loss is obtained based on the first structural similarity, and the second loss is obtained based on the second structural similarity.
3. The method for fine-tuning the model according to claim 1, characterized in that, The first pre-trained model and the second pre-trained model are segmentation models, which are used to segment the input target image to obtain a mask image corresponding to the target image.
4. The method for fine-tuning the model according to any one of claims 1-3, characterized in that, The step of fine-tuning the second pre-trained model based on the first loss and the second loss includes: Based on the first loss and the second loss, the parameters of each layer in the second pre-trained model are adjusted to achieve model fine-tuning of the second pre-trained model.
5. A fine-tuning device for a model, characterized in that, include: The acquisition module is used to obtain a fine-tuning dataset, which includes an input image. The fine-tuning dataset is applied to an image segmentation task, which includes at least one of medical image segmentation, video portrait segmentation, instance segmentation, or avatar segmentation. The model output result acquisition module is used to synchronously input the input image into the first pre-trained model and the second pre-trained model to obtain the first result output by the first pre-trained model and the second result output by the second pre-trained model. The first pre-trained model and the second pre-trained model are both image segmentation models. The image segmentation model is used to output a corresponding pixel-level mask image based on the input image. The first pre-trained model is a pre-trained segmentation model with completely frozen weights, which participates in the fine-tuning process of the second pre-trained model as a prior semantic reference model. The loss calculation module is used to obtain a first loss between the second result and the label of the input image, and a second loss between the second result and the first result. The first loss is a consistency loss, used to ensure pixel-level matching between the segmentation result and the real label. The second loss is a priori semantic preservation loss, used to constrain the second pre-trained model to retain the mature semantic features of the pre-trained segmentation model. The fine-tuning module is used to fine-tune the second pre-trained model based on the first loss and the second loss.
6. The fine-tuning device for the model according to claim 5, characterized in that, The loss calculation module is specifically used for: Calculate the first structural similarity between the second result and the label of the input image, and the second structural similarity between the second result and the first result; The first loss is obtained based on the first structural similarity, and the second loss is obtained based on the second structural similarity.
7. The fine-tuning device for the model according to claim 5 or 6, characterized in that, The fine-tuning module is specifically used for: Based on the first loss and the second loss, the parameters of each layer in the second pre-trained model are adjusted to achieve model fine-tuning of the second pre-trained model.
8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is used to implement the fine-tuning method of the model according to any one of claims 1-4 when executing the program.
9. A computer-readable storage medium having a computer program stored thereon for implementing a method for fine-tuning a model according to any one of claims 1-4.
10. A computer program product having a computer program stored thereon, the computer program being used to implement a fine-tuning method for a model according to any one of claims 1-4.
Citation Information
Patent Citations
Pedestrian re-identification model training method and device, computing equipment and storage medium
CN113936302A