Model training method, electronic device and storage medium

Abstract

The present application provides a model training method, an electronic device and a storage medium, the method comprising: inputting the first training sample data into the model to be trained to obtain the first output feature of the first training sample data; performing the first stage comparative learning training on the model to be trained according to the first output feature and the historical output feature of the first training sample data to obtain the intermediate model corresponding to the model to be trained; inputting the second training sample data into the intermediate model to obtain the second output feature of the second training sample data; performing the second stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data; the third output feature is obtained by inputting the second training sample data into the comparative learning model, and the comparative learning model is the intermediate model obtained by the first stage comparative learning training. The embodiments of the present application can make the features of the trained model have both discriminability and transferability.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a model training method, electronic equipment and storage medium. Background Art

[0002] With the continuous development of artificial intelligence technology, more and more scenarios use neural network models to perform tasks, such as using ResNet18 network to perform image retrieval tasks, image clustering tasks, etc. For a network model, it is expected that it can not only perform the current task, but also be transferred to perform other tasks, that is, it has both discriminability and transferability. However, during the model training process, the discriminability and transferability of the model are incompatible, especially in the later stages of training, the discriminability of features will increase, but at the same time, the transferability will drop sharply. Summary of the invention

[0003] In view of the above problems, embodiments of the present application are proposed to provide a model training method, electronic device and storage medium that overcome the above problems or at least partially solve the above problems.

[0004] According to a first aspect of an embodiment of the present application, a model training method is provided, comprising:

[0005] Inputting first training sample data into the model to be trained to obtain a first output feature of the first training sample data;

[0006] According to the first output feature and the historical output feature corresponding to the first training sample data, the first stage comparative learning training is performed on the model to be trained to obtain an intermediate model corresponding to the model to be trained; wherein the historical output feature is the output feature of the first training sample data in a training round before the current training round;

[0007] Inputting the second training sample data into the intermediate model to obtain a second output feature of the second training sample data;

[0008] According to the second output feature and the third output feature corresponding to the second training sample data, the intermediate model is subjected to second-stage contrastive learning training; wherein the third output feature is obtained by inputting the second training sample data into the contrastive learning model, and the contrastive learning model is the intermediate model obtained by the first-stage contrastive learning training.

[0009] According to a second aspect of an embodiment of the present application, a model training device is provided, comprising:

[0010] A training model processing module, used for inputting the first training sample data into the training model to obtain the first output feature of the first training sample data;

[0011] a first-stage training module, configured to perform first-stage comparative learning training on the model to be trained according to the first output feature and the historical output feature corresponding to the first training sample data, so as to obtain an intermediate model corresponding to the model to be trained; wherein the historical output feature is the output feature of the first training sample data in a training round before the current training round;

[0012] An intermediate model processing module, used for inputting the second training sample data into the intermediate model to obtain a second output feature of the second training sample data;

[0013] The second stage training module is used to perform second stage contrastive learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data; wherein the third output feature is obtained by inputting the second training sample data into the contrastive learning model, and the contrastive learning model is the intermediate model obtained by the first stage contrastive learning training.

[0014] According to the third aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the model training method described in the first aspect is implemented.

[0015] According to the fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the model training method described in the first aspect is implemented.

[0016] According to the fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program or computer instructions, which, when executed by a processor, implements the model training method described in the first aspect.

[0017] The model training method, electronic device and storage medium provided in the embodiments of the present application obtain a first output feature by inputting first training sample data into the model to be trained, and then performing first-stage comparative learning training on the model to be trained according to the first output feature and the historical output feature of the first training sample data to obtain an intermediate model corresponding to the model to be trained, and then inputting second training sample data into the intermediate model to obtain a second output feature of the second training sample data, and then determining a third output feature corresponding to the second training sample data through the comparative learning model, wherein the comparative learning model is an intermediate model obtained by the first-stage comparative learning training, and then performing second-stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data. Since the model to be trained is trained in two stages, the discriminability of the model features can be improved based on the comparative learning training in the first stage, and in the second stage of training, the intermediate model is subjected to comparative learning training based on the real-time output of the intermediate model and the output of the comparative learning model with fixed parameters, so that the feature transferability can be improved on the basis of ensuring the discriminability of the model features, so that the features of the target model obtained by training can have both discriminability and transferability.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application.

[0020] Figure 1 It is a flowchart of the steps of a model training method provided in an embodiment of the present application;

[0021] Figure 2 This is a flowchart of performing the first phase of comparative learning training on the training model in an embodiment of the present application;

[0022] Figure 3 is a flow chart of performing the second stage comparative learning training on the intermediate model in an embodiment of the present application;

[0023] Figure 4 It is a flowchart of the steps of a model training method provided in an embodiment of the present application;

[0024] Figure 5 It is a structural block diagram of a model training device provided in an embodiment of the present application;

[0025] Figure 6 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0027] In recent years, research on computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence has made important progress. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving many types of technologies such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks, etc. Computer vision, as an important branch of artificial intelligence, specifically allows machines to recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security control, urban management, traffic management, building management, park management, face access, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, automatic driving, smart medical care, face payment, face unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live webcasting, beauty, beauty, medical beauty, intelligent temperature measurement, etc. The embodiment of the present application also involves computer vision technology, specifically a model training method. The model based on computer vision technology can be trained using the model training method in the embodiment of the present application, so that the features of the trained model can be both discriminable and transferable. The specific scheme is as follows:

[0028] Figure 1 is a flowchart of the steps of a model training method provided in an embodiment of the present application. The method can be applied to the training of models such as image recognition models and image retrieval models. Figure 1 As shown, the method may include:

[0029] Step 110: input the first training sample data into the model to be trained to obtain the first output feature of the first training sample data.

[0030] The model to be trained may be an image recognition model or an image retrieval model, and the first training sample data may be a first sample image.

[0031] In the embodiment of the present application, the training of the training model is divided into two stages, and comparative learning training is performed in each stage, which can improve the transferability while ensuring the discriminability of model features.

[0032] The first training sample data is input into the model to be trained, the first training sample data is processed by the model to be trained, the features of the first training sample data are extracted, and the first output features corresponding to the first training sample data are obtained.

[0033] Step 120: Perform a first-stage comparative learning training on the model to be trained based on the first output feature and the historical output feature corresponding to the first training sample data to obtain an intermediate model corresponding to the model to be trained.

[0034] The historical output features are output features of the first training sample data in a training round before a current training round.

[0035] The first stage of training is mainly to improve the discriminability of model features, which can also be called the Information Aggregation Stage. Discriminability refers to the ability to gain benefits in the current discrimination task, that is, high discriminability in the current discrimination task.

[0036] When the first phase of comparative learning training is performed on the training model, a memory bank can be introduced to store the historical output features of each first training sample data. After obtaining the first output feature of the first training sample data, the first output feature can be saved in the memory bank storing output features, and used as the historical output feature corresponding to the first training sample data in subsequent training rounds.

[0037] After obtaining the first output feature of the first training sample data through the model to be trained, the historical output feature corresponding to the first training sample data is obtained from the memory bank, and the contrastive learning loss is determined based on the first output feature and the historical output feature, and the parameters of the model to be trained are adjusted based on the contrastive learning loss to implement contrastive learning training of the model to be trained, and the training process is iteratively executed. When the end condition of the first stage training is met, the contrastive learning training of the first stage ends, and the intermediate model corresponding to the model to be trained is obtained, and the intermediate model with fixed parameters is saved as the contrastive learning model. Among them, the intermediate model continues the contrastive learning training of the second stage; the fixed parameters of the contrastive learning model, which can also be called the information bank, keep the parameters unchanged during the second stage training, and are used to provide the output features of contrastive learning during the second stage training to ensure the discriminability of the model features.

[0038] During the first stage of comparative learning training, the same first training sample data set is used for training according to multiple training rounds. In each round, the first training sample data set is divided into multiple training batches for training. For each first training sample data, it is processed by the model to be trained in the training rounds before the current training round to obtain the corresponding output features. The output features are used as the historical output features of the first training sample data in this round of training.

[0039] Step 130: input the second training sample data into the intermediate model to obtain the second output features of the second training sample data.

[0040] Wherein, when the model to be trained is an image processing model or an image retrieval model, the second training sample data is a sample image.

[0041] The second training sample data is input into the intermediate model, the second training sample data is processed by the intermediate model, the features of the second training sample data are extracted, and the second output features corresponding to the second training sample data are obtained.

[0042] Step 140, performing a second-stage contrastive learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data; wherein the third output feature is obtained by inputting the second training sample data into the contrastive learning model, and the contrastive learning model is the intermediate model obtained by the first-stage contrastive learning training.

[0043] The second stage of training is to improve the transferability of the model while ensuring the discriminability of the model. This stage can also be called the Information Preservation Stage. Transferability means that better fine-tuning effects can be obtained on other tasks.

[0044] The second training sample data is input into the contrastive learning model, and the sample data is processed by the contrastive learning model to obtain the third output feature corresponding to the second training sample data. Based on the second output feature and the third output feature, the contrastive learning loss is determined, and the parameters of the intermediate model are adjusted based on the contrastive learning loss to implement contrastive learning training of the intermediate model. The training process is iteratively executed. When the loss value converges, the training is completed and the target model is obtained. The trained target model can be used to process data, for example, images can be processed for image recognition, or images can be processed for image retrieval and recall of similar images.

[0045] The model training method provided in the embodiment of the present application obtains a first output feature by inputting first training sample data into a model to be trained, performs a first-stage comparative learning training on the model to be trained according to the first output feature and the historical output feature of the first training sample data, obtains an intermediate model corresponding to the model to be trained, inputs second training sample data into the intermediate model, obtains a second output feature of the second training sample data, determines a third output feature corresponding to the second training sample data through the comparative learning model, the comparative learning model is an intermediate model obtained by the first-stage comparative learning training, performs a second-stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data, and because the model to be trained is trained in two stages, the discriminability of the model features can be improved based on the comparative learning training in the first stage, and in the second stage of training, the intermediate model is subjected to comparative learning training based on the real-time output of the intermediate model and the output of the comparative learning model with fixed parameters, which can improve the feature transferability on the basis of ensuring the discriminability of the model features, so that the features of the target model obtained by training can have both discriminability and transferability.

[0046] Figure 2 is a flowchart of performing the first phase of comparative learning training on the training model in the embodiment of the present application, such as Figure 2 As shown, according to the first output feature and the historical output feature corresponding to the first training sample data, the first stage comparative learning training is performed on the model to be trained to obtain the intermediate model corresponding to the model to be trained, including:

[0047] Step 121: input the first training sample data into the model to be trained to obtain the first output feature of the first training sample data.

[0048] The model to be trained may be a model with initialized parameters, or a model with adjusted parameters based on training of a previous batch or a previous sample data.

[0049] When the first stage of training is performed on the model to be trained, the first training sample data is input into the model to be trained, the first training sample data is processed by the model to be trained, and features in the first training sample data are extracted to obtain first output features. After obtaining the first output feature of the first training sample data, the first output feature can be saved in a memory bank storing output features, and used as a historical output feature for subsequent training rounds.

[0050] It should be noted that step 121 is step 110 in the above embodiment. When performing the second phase of contrastive learning training, step 110 needs to be executed iteratively.

[0051] Step 122, determine the first cross entropy loss function value of the model to be trained based on the feature annotation corresponding to the first output feature and the first training sample data, and determine the first comparative learning loss function value based on the first output feature, the historical output feature corresponding to the first training sample data, and the historical output features corresponding to other sample data in the same batch.

[0052] Substitute the feature annotations corresponding to the first output feature and the first training sample data into the cross entropy loss function formula to obtain the first cross entropy loss function value of the model to be trained. Obtain the historical output features of the first training sample data from the memory bank storing the historical output features, and obtain the historical output features corresponding to other samples in the same training batch of the first training sample data, substitute the first output feature, the historical output features corresponding to the first training sample data, and the historical output features corresponding to other sample data into the comparative learning loss function formula to obtain the first comparative learning loss function value. The comparative learning loss function can shorten the distance between the first output feature and the historical output features corresponding to the first training sample data, and push the distance between the first output feature and the historical output features corresponding to other sample data away.

[0053] In one embodiment of the present application, a first comparative learning loss function value is determined based on the first output feature, the historical output feature corresponding to the first training sample data, and the historical output feature corresponding to other sample data in the same batch, including: determining the average feature of the first output feature and the historical output feature corresponding to the first training sample data, and determining the distance between the first output feature and the average feature as the first distance; determining the historical average feature of the historical output feature corresponding to each other sample data in the same batch, and determining the distance between the first output feature and the historical average feature of the other sample data as the second distance; determining the ratio of the first distance to the sum of the second distances in the same batch as the first ratio, and determining the negative logarithm of the first ratio as the first comparative learning loss function value.

[0054] Calculate the average feature of the first output feature and the historical output feature corresponding to the first training sample data, determine the distance between the first output feature and the average feature, and obtain the first distance; obtain the historical output features corresponding to other sample data in the same batch, and calculate the historical average feature of the historical output features corresponding to each other sample data, respectively determine the distance between the first output feature and the historical average feature of each other sample data, and obtain the second distance, determine the ratio of the first distance to the sum of the second distances in the same batch as the first ratio, and determine the negative logarithm of the first ratio as the first contrastive learning loss function value. That is, the first contrastive learning loss function value is determined by the following formula:

[0055]

[0056] Among them, L IAS is the first contrastive learning loss function value, represents the first output feature, v i represents the average feature, i represents the sequence number of the current first training sample data in the first training sample data set of the same batch, represents the first distance, v j represents the historical average characteristics corresponding to the other sample data, represents the second distance, and N represents the number of other sample data in the same batch.

[0057] By determining the first comparative learning loss function value according to the first output feature, the average feature of the historical output features corresponding to the first training sample data, and the historical average feature of the historical output features corresponding to other sample data in the same batch, and performing the first stage of comparative learning training on the training model based on the first comparative learning loss function value, the distance between the first output feature and the average feature of the same sample data can be shortened, and the distance between the first output feature and the historical average feature of other sample data can be extended, thereby making the model converge quickly.

[0058] In one embodiment of the present application, determining the average feature of the first output feature and the historical output feature corresponding to the first training sample data includes: taking a weighted average of the first output feature and the historical output feature corresponding to the first training sample data based on the weight of the first output feature and the weight of the historical output feature corresponding to the first training sample data to obtain the average feature.

[0059] Weights are assigned to the first output feature and the historical output feature corresponding to the current first training sample data. The weights can be assigned to the first output feature and the historical output feature corresponding to the current first training sample data in order from large to small weights, that is, the weight of the first output feature is the largest, the weight of the first historical output feature before the first output feature is smaller than the weight of the first output feature and larger than the weight of the second historical output feature before the first output feature, and so on. Weights are assigned to the first output feature and each historical output feature, and then based on the weight of the first output feature and the weight of the historical output feature of the first training sample data, the first output feature and the historical output feature of the first training sample data are weighted averaged to obtain an average feature. The average feature of the first output feature and the historical output feature corresponding to the first training sample data is determined by weighted average, so that the average feature obtained is closer to the first output feature, which is beneficial to the training of the model.

[0060] Step 123: Adjust the parameters of the model to be trained according to the first cross entropy loss function value and the first contrastive learning loss function value.

[0061] According to the first cross entropy loss function value and the first contrastive learning loss function value, the parameters of the model to be trained are adjusted through back propagation.

[0062] Step 124, determine whether the first stage training end condition is met, if not, iteratively execute steps 121 to 124, if yes, execute step 125.

[0063] The first stage training end condition is a pre-set first stage training end condition, which may be that the number of training rounds reaches a preset number of rounds, or that the loss value converges, etc.

[0064] Determine whether the first-stage training end condition is currently met. When the first-stage training end condition is that the number of training rounds reaches the preset number of rounds, determine whether the current number of training rounds reaches the preset number of rounds. If not, it is determined that the first-stage training end condition is not met. If yes, it is determined that the first-stage training end condition is met. When the first-stage training end condition is the loss value convergence, determine whether the sum of the first cross entropy loss function value and the first contrastive learning loss function value converges. If yes, it is determined that the first-stage training end condition is met. If not, it is determined that the first-stage training end condition is not met.

[0065] If the first stage training end condition is not met, step 121 to step 124 are iteratively executed; if the first stage training end condition is met, the first stage training is terminated and step 125 is executed.

[0066] Step 125, ending the first phase of comparative learning training, and obtaining an intermediate model corresponding to the model to be trained.

[0067] After the first stage of training is completed, the trained model to be trained is determined as the intermediate model, and the parameters of the intermediate model continue to be adjusted during the subsequent second stage of training; the parameters of the intermediate model are fixed, and the intermediate model with fixed parameters is determined as the contrastive learning model. The parameters of the contrastive learning model remain fixed in the second stage of training, and are used to provide output features of contrastive learning in the second stage of training.

[0068] By performing the first stage of training on the to-be-trained model based on the first cross entropy loss function value and the first contrastive learning loss function value, the discriminability of the model features can be quickly improved.

[0069] Figure 3 is a flow chart of performing the second stage comparative learning training on the intermediate model in the embodiment of the present application, such as Figure 3 As shown, according to the second output feature and the third output feature corresponding to the second training sample data, the intermediate model is subjected to second-stage comparative learning training, including:

[0070] Step 141: input the second training sample data into the intermediate model to obtain the second output features of the second training sample data.

[0071] The intermediate model is the model obtained at the end of the first phase of comparative learning training, or it may be the intermediate model after parameter adjustment based on the training of the previous batch or the previous sample data during the second phase of comparative learning training.

[0072] When the second stage contrastive learning training is performed on the intermediate model, the second training sample data is input into the intermediate model, the second training sample data is processed by the intermediate model, the features in the second training sample data are extracted, and the second output features are obtained. At the same time, the second training sample data is input into the contrastive learning model with fixed parameters, the second training sample data is processed by the contrastive learning model, the features in the second training sample data are extracted, and the third output features are obtained.

[0073] It should be noted that step 141 is step 130 in the above embodiment. When performing the second phase of contrastive learning training, step 130 needs to be executed iteratively.

[0074] Step 142, determine the second cross entropy loss function value of the intermediate model based on the feature annotation corresponding to the second output feature and the second training sample data, and determine the second comparative learning loss function value based on the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch.

[0075] Substitute the second output feature and the feature annotation corresponding to the second training sample data into the cross entropy loss function formula to obtain the second cross entropy loss function value of the intermediate model. Substitute the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch into the contrastive learning loss function formula to obtain the second contrastive learning loss function value. Among them, the third output feature corresponding to other sample data in the same batch is the output feature obtained by processing other sample data through the contrastive learning model.

[0076] By comparing the learning loss function, the distance between the second output feature and the third output feature of the second training sample data can be shortened, and the distance between the second output feature and the third output feature corresponding to other sample data can be extended.

[0077] In one embodiment of the present application, a second comparative learning loss function value is determined based on the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch, including: determining the distance between the second output feature and the third output feature corresponding to the second training sample data as the third distance; determining the distance between the second output feature and the third output feature corresponding to the other sample data as the fourth distance; determining the ratio of the third distance to the sum of the fourth distances in the same batch as the second ratio, and determining the negative logarithm of the second ratio as the second comparative learning loss function value.

[0078] Determine the distance between the second output feature and the third output feature corresponding to the second training sample data, and use the distance as the third distance; respectively determine the distance between the second output feature and the third output feature corresponding to each other sample data in the same batch, and use the distance as the fourth distance; determine the ratio of the third distance to the sum of the fourth distances in the same batch as the second ratio, and determine the negative logarithm of the second ratio as the second contrastive learning loss function value. That is, the second contrastive learning loss function value is determined by the following formula:

[0079]

[0080] Among them, L IRS is the second contrastive learning loss function value, represents the second output feature, represents the third output feature corresponding to the second training sample data, j represents the sequence number of the current second training sample data in the second training sample data set of the same batch, represents the third distance, represents the second output feature corresponding to the other sample data, represents the second distance, and N represents the number of other sample data in the same batch.

[0081] By determining the second contrastive learning loss function value according to the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch, and training the intermediate model based on the second contrastive learning loss function value, the distance between the second output feature and the third output feature of the same sample data can be shortened, and the distance between the second output feature and the third output feature corresponding to other sample data can be increased, thereby making the model converge quickly.

[0082] Step 143: Adjust the parameters of the intermediate model according to the second cross entropy loss function value and the second contrast learning loss function value.

[0083] According to the second cross entropy loss function value and the second contrastive learning loss function value, the parameters of the intermediate model are adjusted through back propagation.

[0084] Step 144, determine whether the second stage training conditions are met, if not, iteratively execute steps 141 to 144, if yes, execute step 145.

[0085] Among them, the second stage training condition can be that the second cross entropy loss function value and the second contrastive learning loss function value converge.

[0086] Determine whether the second stage training end condition is currently met. When the second stage training end condition is that the second cross entropy loss function value and the second contrast learning loss function value converge, determine whether the sum of the second cross entropy loss function value and the second contrast learning loss function value converges. If so, it is determined that the second stage training end condition is met. If not, it is determined that the second stage training end condition is not met.

[0087] If the second-stage training end condition is not met, step 141 to step 144 are iteratively executed; if the second-stage training end condition is met, step 145 is executed.

[0088] Step 145, ending the second phase of contrastive learning training and obtaining the target model.

[0089] After the second phase of contrastive learning training is completed, the trained intermediate model is used as the target model, which can be used to process the data.

[0090] By performing the second stage of training the intermediate model based on the second cross entropy loss function value and the second contrastive learning loss function value, the transferability of the model features can be improved while ensuring the discriminability of the model features.

[0091] Figure 4 is a flowchart of the steps of a model training method provided in an embodiment of the present application. The method is illustrated by taking the training of an image recognition model as an example. Figure 4 As shown, the method may include:

[0092] Step 410: input a first sample image into an image recognition model to be trained to obtain a first output feature of the first sample image.

[0093] The first sample image is input into the image recognition model to be trained, the first sample image is processed by the image recognition model to be trained, the image feature of the first sample image is extracted, and the first output feature of the first sample image is obtained.

[0094] Step 420: Perform a first-stage comparative learning training on the image recognition model to be trained based on the first output feature and the historical output feature corresponding to the first sample image, so as to obtain an intermediate model corresponding to the image recognition model to be trained.

[0095] The historical output features are output features of the first sample image in a training round before a current training round.

[0096] At the same time, when the image recognition model to be trained is subjected to the first stage of contrastive learning training, a memory bank can be introduced to store the historical output features of each first sample image. After the first output feature of the first sample image is obtained by the image recognition model to be trained, the historical output feature corresponding to the first sample image is obtained from the memory bank, and the contrastive learning loss is determined based on the first output feature and the historical output feature, and the parameters of the image recognition model to be trained are adjusted based on the contrastive learning loss to implement the contrastive learning training of the image recognition model to be trained, and the training process is iteratively executed. When the first stage training end condition is met, the first stage of contrastive learning training ends, and the intermediate model corresponding to the image recognition model to be trained is obtained, and the parameters are fixed, and the intermediate model with fixed parameters is used as the contrastive learning model. Among them, the intermediate model continues to be trained in the second stage; the contrastive learning model has fixed parameters, which can also be called an information bank. The parameters are kept unchanged during the second stage training, and are used to provide the output features of contrastive learning during the second stage training to ensure the discriminability of the model features.

[0097] Among them, the specific process of performing the first stage comparative learning training on the image recognition model to be trained is the same as the process of performing the first stage comparative learning training on the model to be trained in the above embodiment. Specifically, the first training sample data in the above embodiment may be replaced with the first sample image to implement the first stage comparative learning training on the image recognition model to be trained. The specific process may be referred to the above embodiment and will not be repeated here.

[0098] Step 430: Input the second sample image into the intermediate model to obtain a second output feature of the second sample image.

[0099] The second sample image is input into the intermediate model, the second sample image is processed by the intermediate model, image features of the second sample image are extracted, and second output features are obtained.

[0100] Step 440, performing a second-stage contrastive learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second sample image; wherein the third output feature is obtained by inputting the second sample image into the contrastive learning model, and the contrastive learning model is the intermediate model obtained by the first-stage contrastive learning training.

[0101] The second sample image is input into the contrastive learning model, and the second sample image is processed by the contrastive learning model to extract the image features of the second sample image, and obtain the third output features corresponding to the second sample image. Based on the second output features and the third output features, the contrastive learning loss is determined, and the parameters of the intermediate model are adjusted based on the contrastive learning loss to implement contrastive learning training of the intermediate model. The training process is iteratively executed. When the loss value converges, the training is completed, and the trained intermediate model is determined as the target image recognition model. The trained target image recognition model can be used to recognize images, that is, to extract image features, and then recognize based on image features.

[0102] Among them, the specific process of performing the second stage contrastive learning training on the intermediate model is the same as the process of performing the second stage contrastive learning training on the intermediate model in the above embodiment. Specifically, the second training sample data in the above embodiment may be replaced with the second sample image to implement contrastive learning training on the intermediate model, thereby obtaining a trained target image recognition model. The specific process can be referred to the above embodiment and will not be repeated here.

[0103] The model training method provided in the embodiment of the present application inputs a first sample image into an image recognition model to be trained to obtain a first output feature of the first sample image, performs a first-stage comparative learning training on the image recognition model to be trained according to the first output feature and the historical output feature corresponding to the first sample image, obtains an intermediate model corresponding to the image recognition model to be trained, inputs a second sample image into the intermediate model to obtain a second output feature of the first sample image, determines a third output feature corresponding to the second sample image through the comparative learning model, the comparative learning model is an intermediate model obtained by the first-stage comparative learning training, performs a second-stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second sample image, and because the image recognition model to be trained is trained in two stages, the discriminability of the model features can be improved based on the comparative learning training in the first stage, and in the second stage of training, the intermediate model is subjected to comparative learning training based on the real-time output of the intermediate model and the output of the comparative learning model with fixed parameters, which can improve the feature transferability on the basis of ensuring the discriminability of the model features, so that the features of the target model obtained by training can have both discriminability and transferability.

[0104] Table 1 shows the performance (measurement of discriminability) of the training method in the embodiment of the present application on the CIFAR-100 and ImageNet datasets for supervised training:

[0105] Table 1. Comparison of discriminability

[0106]

[0107] Table 2 shows the performance (measurement of transferability) of the training method in the embodiment of the present application after “fine-tuning” on the CUB200 and Aircraft datasets:

[0108] Table 2 Comparison of portability

[0109]

[0110] It can be seen from Table 1 and Table 2 that, compared with the baseline (i.e., the training method in the prior art), the training method in the embodiment of the present application can not only gain benefits in the current discrimination task, but also obtain better fine-tuning effects in other tasks. Therefore, the training method in the embodiment of the present application can make the features both discriminable and transferable.

[0111] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0112] Figure 5 is a structural block diagram of a model training device provided in an embodiment of the present application, such as Figure 5 As shown, the model training device may include:

[0113] The to-be-trained model processing module 510 is used to input the first training sample data into the to-be-trained model to obtain the first output feature of the first training sample data;

[0114] The first stage training module 520 is used to perform a first stage comparative learning training on the model to be trained according to the first output feature and the historical output feature corresponding to the first training sample data, so as to obtain an intermediate model corresponding to the model to be trained; wherein the historical output feature is the output feature of the first training sample data in the training round before the current training round;

[0115] An intermediate model processing module 530, configured to input the second training sample data into the intermediate model to obtain a second output feature of the second training sample data;

[0116] The second stage training module 540 is used to perform second stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data; wherein the third output feature is obtained by inputting the second training sample data into the comparative learning model, and the comparative learning model is the intermediate model obtained by the first stage comparative learning training.

[0117] Optionally, the first-stage training modules include:

[0118] A first loss value determination submodule, used to determine a first cross entropy loss function value of the model to be trained according to the first output feature and the feature annotation corresponding to the first training sample data, and to determine a first contrastive learning loss function value according to the first output feature, the historical output feature corresponding to the first training sample data, and the historical output features corresponding to other sample data in the same batch;

[0119] A first parameter adjustment submodule, used to adjust the parameters of the to-be-trained model according to the first cross entropy loss function value and the first contrastive learning loss function value;

[0120] The first iterative control submodule is used to iteratively execute the above-mentioned operations of determining the first cross entropy loss function value and the first contrast learning loss function value, and adjusting the parameters of the model to be trained until the end condition of the first stage training is met, thereby obtaining an intermediate model corresponding to the model to be trained.

[0121] Optionally, the first loss value determination submodule includes:

[0122] A first distance determining unit, configured to determine an average feature of the first output feature and the historical output features corresponding to the first training sample data, and determine a distance between the first output feature and the average feature as a first distance;

[0123] A second distance determination unit is used to determine a historical average feature of the historical output feature corresponding to each other sample data in the same batch, and determine a distance between the first output feature and the historical average feature of the other sample data as a second distance;

[0124] A first contrast loss value determining unit is used to determine a ratio of the first distance to the sum of the second distances in the same batch as a first ratio, and to determine a negative logarithm of the first ratio as the first contrast learning loss function value.

[0125] Optionally, the first distance determining unit includes:

[0126] The weighted average subunit is used to perform weighted averaging on the first output feature and the historical output feature corresponding to the first training sample data according to the weight of the first output feature and the weight of the historical output feature corresponding to the first training sample data to obtain the average feature.

[0127] Optionally, the second-stage training modules include:

[0128] A second loss value determination submodule, used to determine a second cross entropy loss function value of the intermediate model according to the second output feature and the feature annotation corresponding to the second training sample data, and to determine a second contrastive learning loss function value according to the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch;

[0129] A second parameter adjustment submodule, used for adjusting the parameters of the intermediate model according to the second cross entropy loss function value and the second contrastive learning loss function value;

[0130] The second iterative control submodule is used to iteratively execute the above-mentioned operations of determining the second cross entropy loss function value and the second contrastive learning loss function value, and adjusting the parameters of the intermediate model until the second stage training end condition is met to obtain the target model.

[0131] Optionally, the second loss value determination submodule includes:

[0132] A third distance determining unit, configured to determine a distance between the second output feature and a third output feature corresponding to the second training sample data as a third distance;

[0133] a fourth distance determining unit, configured to determine a distance between the second output feature and the third output feature corresponding to the other sample data as a fourth distance;

[0134] A second loss value determining unit is used to determine the ratio of the third distance to the sum of the fourth distances in the same batch as a second ratio, and determine the negative logarithm of the second ratio as the second contrastive learning loss function value.

[0135] Optionally, the model to be trained includes an image recognition model or an image retrieval model, the first training sample data is a first sample image, and the second training sample data is a second sample image.

[0136] For the specific implementation process of the functions corresponding to each module and unit in the device provided in the embodiment of the present application, please refer to Figure 1-Figure 4 The method embodiment shown here will not repeat the specific implementation process of the functions corresponding to each module and unit of the device.

[0137] The model training device provided in this embodiment obtains a first output feature by inputting first training sample data into a model to be trained, performs a first-stage comparative learning training on the model to be trained according to the first output feature and the historical output feature of the first training sample data, obtains an intermediate model corresponding to the model to be trained, inputs second training sample data into the intermediate model, obtains a second output feature of the second training sample data, determines a third output feature corresponding to the second training sample data through the comparative learning model, the comparative learning model is an intermediate model obtained by the first-stage comparative learning training, performs a second-stage comparative learning training on the intermediate model according to the second output feature and the third output feature corresponding to the second training sample data, and because the model to be trained is trained in two stages, the discriminability of the model features can be improved based on the comparative learning training in the first stage, and in the second stage of training, the intermediate model is subjected to comparative learning training based on the real-time output of the intermediate model and the output of the comparative learning model with fixed parameters, which can improve the feature transferability on the basis of ensuring the discriminability of the model features, so that the features of the target model obtained by training can have both discriminability and transferability.

[0138] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0139] Figure 6 is a structural block diagram of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device 600 may include one or more processors 610 and one or more memories 620 connected to the processor 610. The electronic device 600 may also include an input interface 630 and an output interface 640 for communicating with another device or system. The program code executed by the processor 610 may be stored in the memory 620.

[0140] The processor 610 in the electronic device 600 calls the program code stored in the memory 620 to execute the model training method in the above embodiment.

[0141] According to one embodiment of the present application, a computer-readable storage medium is also provided, and the computer-readable storage medium includes but is not limited to a disk storage, a CD-ROM, an optical storage, etc., and a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the model training method of the aforementioned embodiment is implemented.

[0142] According to one embodiment of the present application, a computer program product is also provided, including a computer program or computer instructions, which implement the model training method described in the above embodiment when executed by a processor.

[0143] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0144] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0149] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0150] The above is a detailed introduction to a model training method, electronic device and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model training method, characterized in that: include: Inputting first training sample data into a model to be trained to obtain a first output feature of the first training sample data, wherein the model to be trained includes an image recognition model or an image retrieval model, and the first training sample data is a first sample image; According to the first output feature and the historical output feature corresponding to the first training sample data, the first stage comparative learning training is performed on the model to be trained to obtain an intermediate model corresponding to the model to be trained; wherein the historical output feature is the output feature of the first training sample data in a training round before the current training round; Inputting second training sample data into the intermediate model to obtain second output features of the second training sample data, where the second training sample data is a second sample image; According to the second output feature and the third output feature corresponding to the second training sample data, the intermediate model is subjected to second-stage contrastive learning training; wherein the third output feature is obtained by inputting the second training sample data into the contrastive learning model, and the contrastive learning model is the intermediate model obtained by the first-stage contrastive learning training.

2. The method according to claim 1, characterized in that According to the first output feature and the historical output feature corresponding to the first training sample data, the first stage comparative learning training is performed on the model to be trained to obtain an intermediate model corresponding to the model to be trained, including: Determine a first cross entropy loss function value of the model to be trained according to the first output feature and the feature annotation corresponding to the first training sample data, and determine a first contrastive learning loss function value according to the first output feature, the historical output feature corresponding to the first training sample data, and the historical output features corresponding to other sample data in the same batch; Adjusting the parameters of the model to be trained according to the first cross entropy loss function value and the first contrastive learning loss function value; The above-mentioned operations of determining the first cross entropy loss function value and the first contrast learning loss function value, and adjusting the parameters of the model to be trained are iteratively performed until the end condition of the first stage training is met, and the intermediate model corresponding to the model to be trained is obtained.

3. The method according to claim 2, characterized in that Determining a first contrastive learning loss function value according to the first output feature, the historical output feature corresponding to the first training sample data, and the historical output features corresponding to other sample data in the same batch, includes: Determine an average feature of the first output feature and the historical output features corresponding to the first training sample data, and determine a distance between the first output feature and the average feature as a first distance; Determine a historical average feature of the historical output features corresponding to each other sample data in the same batch, and determine a distance between the first output feature and the historical average feature of the other sample data as a second distance; A ratio of the first distance to the sum of the second distances in the same batch is determined as a first ratio, and a negative logarithm of the first ratio is determined as the first contrastive learning loss function value.

4. The method according to claim 3, characterized in that Determining the first output feature and an average feature of historical output features corresponding to the first training sample data includes: According to the weight of the first output feature and the weight of the historical output feature corresponding to the first training sample data, a weighted average is performed on the first output feature and the historical output feature corresponding to the first training sample data to obtain the average feature.

5. The method according to any one of claims 1 to 4, characterized in that: According to the second output feature and the third output feature corresponding to the second training sample data, the intermediate model is subjected to second-stage contrastive learning training, including: Determine a second cross entropy loss function value of the intermediate model according to the second output feature and the feature annotation corresponding to the second training sample data, and determine a second contrastive learning loss function value according to the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch; Adjusting parameters of the intermediate model according to the second cross entropy loss function value and the second contrastive learning loss function value; The above-mentioned operations of determining the second cross entropy loss function value and the second contrastive learning loss function value, and adjusting the parameters of the intermediate model are iteratively performed until the second stage training end condition is met to obtain the target model.

6. The method according to claim 5, characterized in that Determining a second contrastive learning loss function value according to the second output feature, the third output feature corresponding to the second training sample data, and the third output feature corresponding to other sample data in the same batch, including: Determine a distance between the second output feature and a third output feature corresponding to the second training sample data as a third distance; Determine a distance between the second output feature and a third output feature corresponding to the other sample data as a fourth distance; A ratio of the third distance to the sum of the fourth distances in the same batch is determined as a second ratio, and a negative logarithm of the second ratio is determined as the second contrastive learning loss function value.

7. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the model training method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the model training method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that It includes a computer program or computer instructions, which, when executed by a processor, implements the model training method described in any one of claims 1 to 6.

Images