Image recognition model training method, recognition method and device

Abstract

The present application provides an image recognition model training method, a recognition method and a device. The method includes: obtaining a first training image set with labels and a second training image set without labels; obtaining a first model parameter and an F1 value of the first training model; updating the second training model according to the first model parameter and the F1 value; using the second training model to predict the training images in the second training image set to obtain the labels corresponding to the training images; using the first training image set and the second training image set and the labels corresponding to the training images to perform an iterative training on the first training model, and if the stop iteration condition is not met, then using the first model parameter and the F1 value after the iterative training to update the second training model again until the stop iteration condition is met, and a trained image recognition model is obtained. The present application improves the convergence speed in the model training process and the performance of the obtained model.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image recognition model training method, a recognition method and a device. Background Art

[0002] At present, artificial intelligence technology has been widely used in the field of image recognition. For example, deep learning networks are used for image classification. The prediction accuracy of deep learning networks is closely related to the amount and accuracy of sample data sets. Generally speaking, the larger the labeled sample set, the better the performance of the trained model, and the greater the labor cost of labeling the data set.

[0003] In practical applications, due to the limitation of the amount of labeled sample data, semi-supervised learning methods are gaining more and more attention in the industry. The model is trained using labeled samples, and then pseudo-labels are generated for unlabeled data using the trained model. The amount of labeled sample data is expanded using pseudo-labels, so that the model can be trained using labeled samples. However, the effectiveness of pseudo-labels in this method is low, resulting in low accuracy of the final model. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image recognition model training method, a recognition method and a device to improve the accuracy of the image recognition model obtained through training.

[0005] In a first aspect, an embodiment of the present application provides an image recognition model training method, comprising: obtaining a first labeled training image set and an unlabeled second training image set; obtaining first model parameters and an F1 value of the first training model; wherein the first training model is obtained by training using the first training image set, and the F1 value is obtained by calculating the precision and recall rate of the first training model; updating the second training model according to the first model parameters and the F1 value; predicting the training images in the second training image set using the second training model to obtain labels corresponding to the training images; iteratively training the first training model using the first training image set and the second training image set and the labels corresponding to the training images, and if a stop iteration condition is not met, updating the second training model again using the first model parameters and the F1 value after the iterative training until the stop iteration condition is met, thereby obtaining a trained image recognition model.

[0006] The embodiment of the present application updates the second training model by using the first model parameters and the F1 value, so that the pseudo labels predicted by the second training model have higher accuracy and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model.

[0007] In any embodiment, updating the second training model according to the first model parameters and the F1 value includes: calculating the second model parameters using a composite moving weighted average algorithm according to the first model parameters and the F1 value; and updating the parameters in the second training model using the second model parameters.

[0008] The embodiment of the present application updates the second training model by using the first model parameters and the F1 value, so that the pseudo labels predicted by the second training model have higher accuracy and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model.

[0009] In any embodiment, the method of calculating the second model parameter using a composite moving weighted average algorithm based on the first model parameter and the F1 value includes: Calculate the second model parameters; where θ i ' is the second model parameter calculated after the i-th round of iterative training of the first training model; α is the influence attenuation coefficient of the model parameter; f j is the F1 value after the jth round of iterative training of the first training model; θ j are the first model parameters after the j-th round of iterative training of the first training model.

[0010] The embodiment of the present application updates the second training model by using the first model parameters and the F1 value, so that the pseudo labels predicted by the second training model have higher accuracy and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model.

[0011] In any embodiment, the label corresponding to the training image is the image category score; the first training model is iteratively trained using the first training image set and the second training image set and the labels corresponding to the training images, including: inputting the first training image set and the second training image set into the first training model to obtain the image category score output by the first training model; calculating the loss value based on the first training image set, the second training image set, the image category score and the dynamic model weight; and optimizing the first model parameters in the first training model based on the loss value to complete one iterative training.

[0012] When performing enhanced training on the first training model, the embodiment of the present application uses the image category score as a parameter in the loss function, so that the loss function is more accurate, thereby improving the performance of the image recognition model obtained through training.

[0013] In any embodiment, the loss value is calculated according to the first training image set, the second training image set, the image category score and the dynamic model weight, including: according to the formula The loss value is calculated; where L is the loss value; M is the total number of image categories corresponding to the first training image set and the second training image set; N l is the number of samples in the first training image set; y kc is the label of the kth sample in the cth image category in the first training image set; p kc1 is the image category score of the cth image category predicted by the first training model for the kth sample in the first training image set; p kc2 N is the image category score predicted by the first training model for the kth sample in the second training image set belonging to the cth image category; u is the number of samples in the second training image set; s kc is the image classification score of the cth image category predicted by the second training model for the kth sample in the second training image set; β is the dynamic model weight.

[0014] The embodiment of the present application uses a dynamic weight method. As the number of iterative training increases, the proportion of unsupervised loss in the loss function gradually increases, thereby increasing the impact on the loss function and further improving the accuracy of the model obtained through training.

[0015] In any embodiment, the second training model has the same model structure as the first training model.

[0016] In a second aspect, an embodiment of the present application provides an image recognition method, comprising: obtaining an image to be recognized; inputting the image to be recognized into a pre-trained image recognition model to obtain a category label output by the image recognition model; wherein the image recognition model is trained according to the model training method of the first aspect.

[0017] In a third aspect, an embodiment of the present application provides an image recognition model training device, comprising: a first image acquisition module, used to acquire a first labeled training image set and an unlabeled second training image set; a model acquisition module, used to acquire a first model parameter and an F1 value of the first training model; wherein the first training model is obtained by training using the first training image set, and the F1 value is calculated using the precision and recall rate of the first training model; a model update module, used to update the second training model according to the first model parameter and the F1 value; a model training module, used to predict the training images in the second training image set using the second training model to obtain the labels corresponding to the training images; the first training model is iteratively trained once using the first training image set and the second training image set and the labels corresponding to the training images, and if the stop iteration condition is not met, the second training model is updated again using the first model parameter and the F1 value after the iterative training until the stop iteration condition is met, thereby obtaining a trained image recognition model.

[0018] In a fourth aspect, an embodiment of the present application provides an image recognition device, comprising: a second image acquisition module, used to acquire an image to be recognized; a recognition module, used to input the image to be recognized into a pre-trained image recognition model to obtain a category label output by the image recognition model; wherein the image recognition model is obtained by training according to the model training method of the first aspect.

[0019] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the processor and the memory communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method of the first aspect or the second aspect.

[0020] In a sixth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium, comprising: the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method of the first aspect or the second aspect.

[0021] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the image recognition model training method flow provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a first iteration process of the first training model provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of an image recognition method flow provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of the structure of an image recognition model training device provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of the structure of an image recognition device provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following embodiments of the technical solution of the present application are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0031] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0032] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0033] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0034] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0035] In the description of the embodiments of the present application, the technical terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the embodiments of the present application.

[0036] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0037] Semi-supervised learning is a key issue in the field of pattern recognition and machine learning. It is a learning method that combines supervised learning with unsupervised learning. It mainly considers how to use a small number of labeled samples and a large number of unlabeled samples for training and classification.

[0038] In order to solve the problem of low performance of the model obtained by training using the semi-supervised learning method in the prior art, the embodiment of the present application proposes an image recognition model training method. It can be understood that the image recognition model can be used to identify the type of goods in the image, the type of animals in the image, the type of crops in the image, etc., and the embodiment of the present application does not specifically limit the specific objects to be identified. In addition, in addition to being used to train image recognition models, the model training method can also be used to train deep learning models for other purposes, such as: air quality prediction models, defect non-destructive detection models, etc.

[0039] In addition, the model training method and image recognition method provided in the embodiments of the present application can be applied to electronic devices and servers; the electronic devices can be specifically smart phones, tablet computers, computers, personal digital assistants (PDAs), etc.; the servers can be specifically application servers or web servers. In addition, the model training method and image recognition method can be executed by the same electronic device or by different electronic devices.

[0040] To facilitate understanding of the technical solution provided in the embodiments of the present application, the application scenarios of the model training method and the image recognition method provided in the embodiments of the present application are introduced below, taking an electronic device as an example of the execution subject.

[0041] Figure 1 A flowchart of the image recognition model training method provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0042] Step 101: Obtain a first labeled training image set and a second unlabeled training image set.

[0043] The first training image set and the second training image set both include multiple training images, and the number of training images in the second training image set is greater than the number of training images in the first training image set. Each training image in the first training image set has a corresponding label, which is used to characterize the image category of the corresponding training image. The label can be obtained by manually annotating the training image, or it can be an existing annotated training image library obtained from the Internet. In different application scenarios, the labels of training images are different. Taking the identification of commodity categories as an example, the labels can be: beverages, milk, beer, liquor, laundry detergent, etc. Taking the identification of animal categories as an example, the labels can be: cats, dogs, sheep, cows, etc.

[0044] The training images in the second training image set are unlabeled images, that is, the training images have no corresponding labels. The second training image set can be acquired through an image acquisition device, and the image acquisition device can be built-in on the electronic device or can be an external device that is communicatively connected to the electronic device. Of course, the second training image set can also be composed of images acquired from the Internet, and the embodiment of the present application does not limit the source of the second training image set.

[0045] It should be noted that the training images in the first training image set and the second training image set both contain objects that need to be classified in corresponding scenes.

[0046] Step 102: Obtain first model parameters and F1 value of a first training model; wherein the first training model is obtained by training using a first training image set, and the F1 value is calculated using the precision and recall rate of the first training model.

[0047] Among them, after the electronic device obtains the first training image set, it uses the first training image set to perform supervised training on the initial first training model. That is, the first training model is iteratively trained multiple times using the first training image set, and when the preset number of iterations is reached or the loss value of the first training model tends to be stable, the training can be stopped to obtain the pre-trained first training model. It can be understood that the first training model can be constructed using a convolutional neural network or other deep learning networks, and the embodiments of the present application do not specifically limit this. The initial values ​​of the model parameters in the first training model can be obtained by random initialization.

[0048] After completing the pre-training of the first training model, the first model parameters and F1 (H-mean) value of the first training model can be obtained, wherein the F1 value can be calculated by the following formula (1):

[0049]

[0050] Wherein, f1 is F1; p1 is the precision of the first training model; r1 is the recall rate of the first training model. It can be understood that precision refers to the proportion of correctly predicted positives to all predicted positives. Recall rate refers to the proportion of correctly predicted positives to all actual positives. Both precision and recall rate can be test results obtained by testing the first training model with a test set. The first training image set can be divided into two parts, one part of the training images is used to train the first training model, and the other part is used as a test set to test the first training model after training.

[0051] Step 103: Update the second training model according to the first model parameters and the F1 value.

[0052] The model structure of the second training model is consistent with the model structure of the first training model. After obtaining the first model parameters and the F1 value, new model parameters are generated according to the first model parameters and the F1 value, and the new model parameters are used to replace the model parameters in the second training model to update the second training model.

[0053] Step 104: Use the second training model to predict the training images in the second training image set to obtain labels corresponding to the training images.

[0054] After obtaining the updated second training model, the training images in the second training image set are input into the second training model to obtain the labels corresponding to the training images output by the second training model. It can be understood that the labels output by the second training model are called pseudo labels of the training images.

[0055] Step 105: Perform an iterative training on the first training model using the first training image set and the second training image set and the labels corresponding to the training images. If the stop iteration condition is not met, update the second training model again using the first model parameters and F1 value after the iterative training until the stop iteration condition is met, thereby obtaining a trained image recognition model.

[0056] Among them, after obtaining the labels corresponding to the training images in the second training image set, the first training model is iteratively trained using the labeled first training image set and the labeled second training image set. The first training model that undergoes iterative training is temporarily referred to as an intermediate training model. After obtaining the intermediate training model, it can be determined whether the conditions for stopping iteration are met. If so, the intermediate training model is the trained image recognition model; if not, the first model parameters and F1 value corresponding to the intermediate training model are obtained, and the first model parameters and F1 value are used to loop through steps 103 to 105 until the conditions for stopping iteration are met.

[0057] The condition for stopping iteration includes at least one of the following:

[0058] The number of iterations reaches the preset number;

[0059] The F1 value of the first training model calculated in step 105 exceeds a preset threshold;

[0060] The deviation of the loss values ​​corresponding to the intermediate training models obtained between two adjacent iterative trainings is less than the preset deviation.

[0061] It can be understood that the above-mentioned preset number of times, preset thresholds and preset deviations can all be preset based on experience, and the embodiments of the present application do not impose any specific limitations on their specific values.

[0062] The embodiment of the present application updates the second training model by using the first model parameters and the F1 value, so that the pseudo labels predicted by the second training model have higher accuracy and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model.

[0063] Based on the above embodiment, updating the second training model according to the first model parameter and the F1 value includes:

[0064] According to the first model parameters and the F1 value, the second model parameters are calculated using a composite moving weighted average algorithm;

[0065] The parameters in the second training model are updated using the second model parameters.

[0066] In the specific implementation process, since the second model parameters corresponding to the second training model need to be updated once each time step 103-step 105 is executed, in the first iteration of step 103-step 105, the first model parameters and F1 values ​​are calculated in step 102; in the i-th (i is greater than 1) iteration of step 103-step 105, it is necessary to obtain the first model parameters and F1 values ​​corresponding to the 1st, 2nd,..., i-th iterations, respectively, and then use the composite moving weighted average algorithm to calculate the second model parameters based on the first model parameters and F1 values ​​obtained in previous iterations.

[0067] The embodiment of the present application uses the F1 value as an indicator of performance measurement, and adopts a composite moving weighted average algorithm to calculate the first model parameters and the F1 value, so that the obtained second model parameters are more accurate, thereby obtaining a second training model with better performance.

[0068] When specifically calculating the second model parameters, the following formula (2) may be used:

[0069]

[0070] Among them, θ i ' is the second model parameter calculated after the i-th round of iterative training of the first training model; α is the influence attenuation coefficient of the model parameter. It can be understood that the value of α can be obtained through multiple experiments in advance to obtain a better value, and 0<α<1; f j is the F1 value after the jth round of iterative training of the first training model, which can be calculated using formula (1); θ j is the first model parameter after the j-th round of iterative training of the first training model. i and j are both positive integers, and j is less than or equal to i.

[0071] The embodiment of the present application adds the influence attenuation coefficient and the F1 value of the first training model as performance metrics to the moving exponential average formula, so that the pseudo labels predicted by the second training model have higher accuracy and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model.

[0072] Based on the above embodiment, the label corresponding to the training image is the image category score; using the first training image set and the second training image set and the labels corresponding to the training images to perform an iterative training on the first training model, including:

[0073] Inputting the first training image set and the second training image set into the first training model to obtain image category scores output by the first training model;

[0074] Calculate the loss value according to the first training image set, the second training image set, the image category score, and the dynamic model weight;

[0075] The first model parameters in the first training model are optimized according to the loss value to complete one iterative training.

[0076] Based on the above embodiment, the image category score refers to the probability that the training image in the second training image set belongs to a certain category. The image category score can be a value between [0, 1], for example, it can be 0.8, which means that the probability that the training image belongs to a certain category is 0.8.

[0077] Figure 2 A schematic diagram of a first iteration process of the first training model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the iterative method includes:

[0078] Step 201: The training images in the first training image set and the second training image set are used as inputs for model training and input into the first training model. The first training model predicts each input training image and outputs the image category score corresponding to each training image. It can be understood that the classification categories are preset, assuming that there are 10 classification categories, and the image category scores corresponding to the 10 classification categories are output for each training image.

[0079] Step 202: Calculate the loss value; construct a loss function in advance, substitute the relevant information corresponding to the first training image set and the second training image set, as well as the image category score and the dynamic model weight into the loss function, and calculate the loss value. The value of the dynamic model weight is calculated based on a function that is determined in advance through experiments. For example, if the function is a linear function, the dynamic model weight increases linearly as the number of iterations increases; if the function is an exponential function, the dynamic model weight increases exponentially as the number of iterations increases.

[0080] Step 203: Optimize the first model parameters; after obtaining the loss value, calculate the gradient of the first model parameters according to the loss value, and optimize the first model parameters according to the gradient.

[0081] When performing enhanced training on the first training model, the embodiment of the present application uses the image category score as a parameter in the loss function, so that the loss function is more accurate, thereby improving the performance of the image recognition model obtained through training.

[0082] Based on the above embodiment, the loss function used when training the first training model is as shown in formula (3):

[0083]

[0084] Where L is the loss value. Substituting the parameters obtained after each iteration into formula (3), the loss value can be calculated.

[0085] M is the total number of image categories corresponding to the first training image set and the second training image set. Generally, the number of image categories corresponding to the first training image set and the second training image set are equal, and the corresponding image categories are also the same.

[0086] N l is the number of samples in the first training image set.

[0087] y kc is the label of the kth sample in the first training image set in the cth image category, and the label can be represented by 0 or 1, where 0 can represent that it does not belong to the cth image category, and 1 can represent that it belongs to the cth image category. It should be noted that the above 0 and 1 are only examples, and appropriate parameters can be used according to actual conditions to represent whether it belongs to the cth image category or not.

[0088] p kc1 is the image category score of the cth image category predicted by the first training model for the kth sample in the first training image set; p kc2 The image category score predicted by the first training model for the kth sample in the second training image set, belonging to the cth image category. The embodiment of the present application replaces the category label with the image category score, making the loss function more accurate, the model convergence faster, and the model performance improved to a certain extent.

[0089] N u is the number of samples in the second training image set. Usually, N u The value will be much larger than N l The value of .

[0090] s kc is the image classification score of the cth image category predicted by the second training model for the kth sample in the second training image set. The image classification score can be obtained by predicting the kth sample in the second training image set by the second training model in the current iteration.

[0091] β is the dynamic model weight, and β is a parameter that gradually increases from 0 to 1. Its specific increasing trend satisfies a certain function, such as a linear function, an exponential function, etc. In different application scenarios, the types of functions it satisfies are different. During the model training process, a better function can be selected through multiple experiments. The embodiment of the present application adopts a dynamic weight method. As the number of iterative training increases, the proportion of unsupervised loss in the loss function gradually increases. Because in the early stage of iteration, the performance of the second training model is poor, and the accuracy of the predicted pseudo-label is not high, it is necessary to reduce its impact on the loss function; however, in the later stage of iteration, the performance of the second training model is already high, and the accuracy of the predicted pseudo-label is already very high, and it is necessary to increase its impact on the loss function, so that the model can converge quickly during the training process and improve the performance of the obtained image recognition model.

[0092] Figure 3 A schematic diagram of an image recognition method flow provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method includes:

[0093] Step 301: Acquire the image to be recognized; It should be noted that in different application scenarios, the electronic device acquires the image to be recognized in different ways. The electronic device can acquire the image to be recognized through its own image acquisition device, or the electronic device can download the image to be recognized from the network, or receive the image to be recognized sent by other terminals.

[0094] Step 302: Input the image to be recognized into a pre-trained image recognition model to obtain the category label output by the image recognition model.

[0095] Among them, the image recognition model can be obtained by training through the model training method provided by the above embodiment, and the embodiments of the present application will not be repeated here. In addition, the type of image to be recognized should match the type that the image recognition model can recognize. For example: the image to be recognized is collected from a certain animal. To identify which type of animal the animal belongs to, the image to be recognized needs to be input into the image recognition model that identifies the animal category; if the image to be recognized is collected from a commodity in an unmanned vending machine, to identify the category of the commodity in the image, the image to be recognized needs to be input into the image recognition model that identifies the commodity category.

[0096] If multiple types of image recognition models are stored in the electronic device, the model identifier can be obtained when the image to be recognized is obtained. The model identifier is used to characterize the type of the image recognition model, and the model identifier can be input by the user. The image to be recognized is input into the image recognition model corresponding to the model identifier. Of course, the image to be recognized can be input into multiple types of image recognition models stored in the electronic device respectively. Whichever image recognition model can output the recognized category, that recognition result is used as the recognition result of the image to be recognized.

[0097] The embodiment of the present application updates the second training model by utilizing the first model parameters and the F1 value, so that the pseudo labels predicted by the second training model have higher precision and recall rate, thereby improving the convergence speed in the model training process and the performance of the obtained model, thereby improving the accuracy of image recognition.

[0098] Figure 4 This is a schematic diagram of the structure of an image recognition model training device provided in an embodiment of the present application. The device may be a module, program segment or code on an electronic device. It should be understood that the device is similar to the above-mentioned Figure 1 The method embodiment corresponds to and can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device can be found in the description above. To avoid repetition, detailed description is appropriately omitted here. The device includes: a first image acquisition module 401, a model acquisition module 402, a model update module 403 and a model training module 404, wherein:

[0099] The first image acquisition module 401 is used to acquire a first training image set with labels and a second training image set without labels;

[0100] The model acquisition module 402 is used to obtain the first model parameters and F1 value of the first training model; wherein the first training model is obtained by training using the first training image set, and the F1 value is calculated using the precision and recall rate of the first training model;

[0101] The model updating module 403 is used to update the second training model according to the first model parameters and the F1 value;

[0102] The model training module 404 is used to predict the training images in the second training image set using the second training model to obtain labels corresponding to the training images; and

[0103] The first training model is iteratively trained once using the first training image set and the second training image set and the labels corresponding to the training images. If the condition for stopping iteration is not met, the second training model is updated again using the first model parameters and F1 value after iterative training until the condition for stopping iteration is met, thereby obtaining a trained image recognition model.

[0104] Based on the above embodiment, the model updating module 403 is specifically used for:

[0105] According to the first model parameters and the F1 value, the second model parameters are calculated using a composite moving weighted average algorithm;

[0106] The parameters in the second training model are updated using the second model parameters.

[0107] Based on the above embodiment, the model updating module 403 is specifically used for:

[0108] According to the formula Calculating and obtaining a second model parameter;

[0109] Among them, θ i ' is the second model parameter calculated after the i-th round of iterative training of the first training model; α is the influence attenuation coefficient of the model parameter; f j is the F1 value after the jth round of iterative training of the first training model; θ j are the first model parameters after the j-th round of iterative training of the first training model.

[0110] Based on the above embodiment, the label corresponding to the training image is the image category score; the model training module 404 is specifically used for:

[0111] Inputting the first training image set and the second training image set into the first training model to obtain image category scores output by the first training model;

[0112] Calculate the loss value according to the first training image set, the second training image set, the image category score, and the dynamic model weight;

[0113] The first model parameters in the first training model are optimized according to the loss value to complete one iterative training.

[0114] Based on the above embodiment, the model training module 404 is specifically used for:

[0115] According to the formula Calculate and obtain the loss value;

[0116] Wherein, L is the loss value; M is the total number of image categories corresponding to the first training image set and the second training image set; N l is the number of samples in the first training image set; kc is the label of the kth sample in the first training image set on the cth image category; p kc1 is the image category score of the cth image category predicted by the first training model for the kth sample in the first training image set; p kc2N is the image category score predicted by the first training model for the kth sample in the second training image set belonging to the cth image category; u is the number of samples in the second training image set; kc is the image classification score of the cth image category predicted by the second training model for the kth sample in the second training image set; β is the dynamic model weight.

[0117] Figure 5 This is a schematic diagram of the structure of an image recognition device provided in an embodiment of the present application. The device may be a module, program segment or code on an electronic device. It should be understood that the device is similar to the above-mentioned Figure 3 The method embodiment corresponds to and can be executed Figure 3 The various steps involved in the method embodiment and the specific functions of the device can be found in the above description. To avoid repetition, detailed description is appropriately omitted here. The device includes: a second image acquisition module 501 and a recognition module 502, wherein:

[0118] The second image acquisition module 501 is used to acquire the image to be recognized;

[0119] The recognition module 502 is used to input the image to be recognized into a pre-trained image recognition model to obtain the category label output by the image recognition model;

[0120] Among them, the image recognition model is obtained by training according to the model training method provided in the above embodiment.

[0121] Figure 6 A schematic diagram of the physical structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device includes: a processor (processor) 601, a memory (memory) 602 and a bus 603; wherein,

[0122] The processor 601 and the memory 602 communicate with each other via the bus 603;

[0123] The processor 601 is used to call the program instructions in the memory 602 to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a first labeled training image set and a second unlabeled training image set; obtaining a first model parameter and an F1 value of a first training model; wherein the first training model is obtained by training with the first training image set, and the F1 value is obtained by calculating the precision and recall rate of the first training model; looping through the following steps to perform model training until the conditions for stopping iteration are met, thereby obtaining a trained image recognition model: updating the second training model according to the first model parameters and the F1 value; using the second training model to predict the training images in the second training image set to obtain the labels corresponding to the training images; performing an iterative training of the first training model using the first training image set and the second training image set and the labels corresponding to the training images, and if the conditions for stopping iteration are not met, updating the first model parameters and the F1 value. Or,

[0124] Obtain an image to be identified; input the image to be identified into a pre-trained image recognition model to obtain a category label output by the image recognition model; wherein the image recognition model is obtained by training according to the model training method provided in the above embodiment.

[0125] The processor 601 may be an integrated circuit chip with signal processing capabilities. The processor 601 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. It may implement or execute various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0126] The memory 602 may include but is not limited to random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), etc.

[0127] The present embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a first training image set with labels and a second training image set without labels; obtaining first model parameters and F1 value of a first training model; wherein the first training model is obtained by training with the first training image set, and the F1 value is obtained by calculating the precision and recall rate of the first training model; looping through the following steps to perform model training until the condition for stopping iteration is met, thereby obtaining a trained image recognition model: updating the second training model according to the first model parameters and F1 value; using the second training model to predict the training images in the second training image set to obtain the labels corresponding to the training images; performing an iterative training of the first training model using the first training image set and the second training image set and the labels corresponding to the training images, and if the condition for stopping iteration is not met, updating the first model parameters and F1 value. Or,

[0128] Obtain an image to be identified; input the image to be identified into a pre-trained image recognition model to obtain a category label output by the image recognition model; wherein the image recognition model is obtained by training according to the model training method provided in the above embodiment.

[0129] The present embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable the computer to execute the methods provided by the above-mentioned method embodiments, for example, including: obtaining a first training image set with labels and a second training image set without labels; obtaining first model parameters and F1 value of a first training model; wherein the first training model is obtained by training with the first training image set, and the F1 value is obtained by calculating the precision and recall rate of the first training model; looping through the following steps to perform model training until the condition for stopping iteration is met, and a trained image recognition model is obtained: updating the second training model according to the first model parameters and F1 value; using the second training model to predict the training images in the second training image set to obtain the labels corresponding to the training images; performing an iterative training of the first training model using the first training image set and the second training image set and the labels corresponding to the training images, and if the condition for stopping iteration is not met, updating the first model parameters and F1 value. Or,

[0130] Obtain an image to be identified; input the image to be identified into a pre-trained image recognition model to obtain a category label output by the image recognition model; wherein the image recognition model is obtained by training according to the model training method provided in the above embodiment.

[0131] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0132] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0134] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0135] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training an image recognition model, characterized in that: include: Obtain a first labeled training image set and a second unlabeled training image set; Obtaining first model parameters and an F1 value of a first training model; wherein the first training model is obtained by training using the first training image set, and the F1 value is obtained by calculating the precision and recall rate of the first training model; Update the second training model according to the first model parameters and the F1 value; Using the second training model to predict the training images in the second training image set, to obtain labels corresponding to the training images; Performing an iterative training on the first training model using the first training image set and the second training image set and labels corresponding to the training images; if a stop iteration condition is not met, updating the second training model again using the first model parameters and F1 value after the iterative training until the stop iteration condition is met, thereby obtaining a trained image recognition model; The updating of the second training model according to the first model parameter and the F1 value comprises: According to the first model parameters and the F1 value, a second model parameter is calculated using a composite moving weighted average algorithm; Updating parameters in the second training model using the second model parameters; The method of calculating the second model parameters by using a composite moving weighted average algorithm according to the first model parameters and the F1 value includes: According to the formula Calculating and obtaining the second model parameter; Among them, θ i ' is the second model parameter calculated after the i-th round of iterative training of the first training model; α is the influence attenuation coefficient of the model parameter; f j is the F1 value after the jth round of iterative training of the first training model; θ j are the first model parameters after the j-th round of iterative training of the first training model.

2. The method according to claim 1, characterized in that The label corresponding to the training image is an image category score; and the first training model is iterated once using the first training image set, the second training image set and the labels corresponding to the training images, including: Inputting the first training image set and the second training image set into the first training model to obtain image category scores output by the first training model; Calculate a loss value according to the first training image set, the second training image set, the image category score, and the dynamic model weight; The first model parameters in the first training model are optimized according to the loss value to complete one iterative training.

3. The method according to claim 2, characterized in that The calculating the loss value according to the first training image set, the second training image set, the image category score and the dynamic model weight includes: According to the formula Calculate and obtain the loss value; Wherein, L is the loss value; M is the total number of image categories corresponding to the first training image set and the second training image set; N l is the number of samples in the first training image set; kc is the label of the kth sample in the first training image set on the cth image category; p kc1 is the image category score of the cth image category predicted by the first training model for the kth sample in the first training image set; p kc2 N is the image category score predicted by the first training model for the kth sample in the second training image set belonging to the cth image category; u is the number of samples in the second training image set; kc is the image classification score of the cth image category predicted by the second training model for the kth sample in the second training image set; β is the dynamic model weight.

4. The method according to any one of claims 1 to 3, characterized in that: The second training model has the same model structure as the first training model.

5. An image recognition method, characterized in that: include: Obtain an image to be recognized; Inputting the image to be recognized into a pre-trained image recognition model to obtain a category label output by the image recognition model; Wherein, the image recognition model is obtained by training according to the model training method according to any one of claims 1-4.

6. An image recognition model training device, characterized in that: include: A first image acquisition module, used to acquire a first labeled training image set and a second unlabeled training image set; A model acquisition module, used to acquire first model parameters and an F1 value of a first training model; wherein the first training model is obtained by training using the first training image set, and the F1 value is calculated using the precision and recall rate of the first training model; A model updating module, used for updating the second training model according to the first model parameters and the F1 value; a model training module, configured to use the second training model to predict the training images in the second training image set to obtain labels corresponding to the training images; and Performing an iterative training on the first training model using the first training image set and the second training image set and labels corresponding to the training images; if a stop iteration condition is not met, updating the second training model again using the first model parameters and F1 value after the iterative training until the stop iteration condition is met, thereby obtaining a trained image recognition model; The model update module is specifically used for: According to the first model parameters and the F1 value, a second model parameter is calculated using a composite moving weighted average algorithm; Updating parameters in the second training model using the second model parameters; The method of calculating the second model parameters by using a composite moving weighted average algorithm according to the first model parameters and the F1 value includes: According to the formula Calculating and obtaining the second model parameter; Among them, θ i ' is the second model parameter calculated after the i-th round of iterative training of the first training model; α is the influence attenuation coefficient of the model parameter; f j is the F1 value after the jth round of iterative training of the first training model; θ j are the first model parameters after the j-th round of iterative training of the first training model.

7. An image recognition device, characterized in that: include: A second image acquisition module, used to acquire an image to be identified; A recognition module, used to input the image to be recognized into a pre-trained image recognition model to obtain a category label output by the image recognition model; Wherein, the image recognition model is obtained by training according to the model training method according to any one of claims 1-4.

8. An electronic device, characterized in that: include: processor, memory and bus, wherein, The processor and the memory communicate with each other via the bus; The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 5 by calling the program instructions.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which, when executed by a computer, enable the computer to perform the method according to any one of claims 1 to 5.

Images