Method and device for image recognition training and storage medium

Abstract

The invention relates to the field of image processing, in particular to a method and equipment for image recognition training and a storage medium. The method comprises the following steps: performing first data enhancement on image samples without effective identification marks to generate a first enhanced image sample set; using an artificial intelligence identification model and the first enhanced image sample set to carry out identification prediction on the image sample without the effective identification mark; labeling a pseudo label for the image sample without the effective identification mark by using a prediction result; and performing supervised training on the artificial intelligence identification model by using the image sample without the effective identification mark and the pseudo label. According to the method, clean data and noise data are utilized at the same time to train the model through semi-supervised learning, and dependence on data distribution and noise distribution assumption is broken.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method, device and storage medium for image recognition training. Background Art

[0002] When the training set for image classification contains noisy data, due to the strong fitting ability of neural networks, the noisy labels are usually overfitted during model training, resulting in a significant decrease in the generalization ability of the model on clean data sets. Therefore, it is particularly important to improve the robustness of the model to noisy label data.

[0003] Regarding the problem of image classification with noisy labels, the current mainstream methods are based on assumptions about noise distribution or data distribution, and are modeled on synthetic noise datasets (e.g., symmetric noise and asymmetric noise). However, in reality, there is a big difference between the real noise distribution and the synthetic noise. For example, real-world noise datasets usually contain multiple noises at the same time and these noises are fine-grained noises, while synthetic noise datasets usually contain only a single noise and assume that confusion occurs between similar large categories. Therefore, the robustness of traditional denoising methods that rely on assumptions about data distribution and noise distribution is not high. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a method, device and storage medium for image recognition training.

[0005] According to one aspect of the present disclosure, a method for image recognition training is provided, comprising:

[0006] Performing first data enhancement on image samples without valid identification marks to generate a first enhanced image sample set;

[0007] Using an artificial intelligence classification model and the first enhanced image sample set to perform recognition prediction on the image samples without valid recognition marks;

[0008] Using the prediction results to label the image samples without valid identification marks with pseudo labels; and

[0009] The artificial intelligence recognition model is supervised and trained using the image samples without valid identification marks and the pseudo labels.

[0010] According to an example of the present disclosure, using the image sample without valid identification mark and the pseudo label to supervise the artificial intelligence recognition model includes:

[0011] Performing second data enhancement on the image samples without valid identification marks to generate a second enhanced image sample set; and

[0012] Inputting the second enhanced image sample set and the pseudo label into the artificial intelligence recognition model for supervised training,

[0013] The second data enhancement is different from the first data enhancement.

[0014] According to an example of the present disclosure, using the prediction result to label the pseudo-label for the image sample without a valid identification mark includes: when the maximum probability in the prediction result is greater than or equal to a preset probability threshold, labeling the identification mark with the maximum probability as the pseudo-label of the image sample without a valid classification mark.

[0015] According to an example of the present disclosure, the method includes: before using the artificial intelligence recognition model and the first enhanced image sample set to perform recognition prediction on the image samples without valid recognition marks, using image samples with valid recognition marks and their recognition marks to perform supervised training on the artificial intelligence recognition model.

[0016] According to an example of the present disclosure, the method includes: dividing training image samples into noise image samples and clean image samples, the noise image samples serving as the image samples without valid identification marks, and the clean image samples serving as the image samples with valid identification marks.

[0017] According to an example of the present disclosure, dividing the training image samples into noisy image samples and clean image samples includes:

[0018] Performing third data enhancement on the training image samples to generate a third enhanced image sample set;

[0019] Using the artificial intelligence recognition model to perform recognition prediction on the third enhanced image sample set to determine whether each enhanced image sample in the third enhanced image sample set corresponds to the recognition mark of the training image sample;

[0020] Determining the proportion of the number of enhanced image samples corresponding to the target mark in the third enhanced image sample set; and

[0021] When the proportion is greater than or equal to a preset threshold, the training image sample is identified as the clean image sample; otherwise, the training image sample is identified as a noise image sample.

[0022] According to an example of the present disclosure, the method includes:

[0023] Extracting text features from target labeled text;

[0024] Extracting image features from image samples in the input image sample set; and

[0025] Based on the similarity between the text feature and the image feature, a valid image sample is determined from the input image sample set as the training image sample.

[0026] According to an example of the present disclosure, the first enhancement is a spatial change to the image sample, and the second enhancement includes a change to the image pixels themselves of the image sample.

[0027] According to another aspect of the present disclosure, a device for image recognition training is provided, comprising: a processor; a memory storing one or more computer program modules; wherein the one or more computer program modules are configured to execute the above-mentioned method for image recognition training when run by the processor.

[0028] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which computer instructions are stored, wherein the computer instructions, when executed by a processor, perform the above-mentioned method for image recognition training.

[0029] According to another aspect of the present disclosure, a computer program product is provided, comprising a medium for implementing a computer program for inputting into a computer and a computer program embodied in the medium, wherein the computer program is used to cause the computer to perform or execute the above-mentioned method for image recognition training.

[0030] In the above aspects of the present disclosure, the pre-trained model is trained through semi-supervised learning using both clean data and noisy data, breaking the dependence on the assumptions of data distribution and noise distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0032] Figure 1 A flowchart of a method for image recognition training according to an embodiment of the present disclosure is shown;

[0033] Figure 2 A flowchart of a method for image recognition training according to an embodiment of the present disclosure is shown;

[0034] Figure 3 A flowchart showing a process of screening training image samples from an input image sample set according to an embodiment of the present disclosure is shown;

[0035] Figure 4 A schematic diagram showing a process of screening training image samples from an input image sample set according to an example of an embodiment of the present disclosure;

[0036] Figure 5 An image example showing a process of screening training image samples from an input image sample set according to an embodiment of the present disclosure;

[0037] Figure 6 A flowchart of a process of dividing a training image sample into a noisy image sample and a clean image sample according to an embodiment of the present disclosure is shown;

[0038] Figure 7 A schematic diagram showing a process of dividing a training image sample into a noisy image sample and a clean image sample according to an example of an embodiment of the present disclosure is shown;

[0039] Figures 8a to 8d An example of how the consistency score affects the division of noisy image samples and clean image samples according to an embodiment of the present disclosure is shown;

[0040] Fig. 9 Shown based on Figure 5 An image example of the process of dividing the screened training image samples into noisy image samples and clean image samples;

[0041] Fig.10 A flowchart showing a process of training an image recognition model in a semi-supervised learning manner using noisy image samples and clean image samples according to an embodiment of the present disclosure;

[0042] Fig.11 A schematic diagram showing a process of training an image classification model in a semi-supervised learning manner using noisy image samples and clean image samples according to an example of an embodiment of the present disclosure;

[0043] Fig.12 A flowchart showing a process of correcting noise labels according to an embodiment of the present disclosure is shown;

[0044] Fig.13 Shows the correction Fig. 9 An image example of the process of determining noise labels for the noise image samples;

[0045] Fig.14 A functional block diagram of an image processing device according to an embodiment of the present disclosure is shown; and

[0046] Fig.15 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0048] Flowcharts are used in the present application to illustrate the steps of the method according to the embodiments of the present application. It should be understood that the preceding or following steps are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or a certain step or several steps may be removed from these processes.

[0049] First, refer to Figure 1 1 is a flowchart of a method 100 for image recognition training according to an embodiment of the present disclosure. Figure 1 As shown, the method 100 for image recognition training includes the following steps S102-S108.

[0050] In step S102, first data enhancement is performed on image samples without valid identification marks to generate a first enhanced image sample set.

[0051] In step S104, the artificial intelligence recognition model and the first enhanced image sample set are used to perform recognition prediction on the image samples without valid recognition marks.

[0052] In step S106, the prediction results are used to label the image samples without valid identification marks with pseudo labels.

[0053] In step S108, supervised training of the artificial intelligence classification model is performed using image samples without valid identification marks and pseudo labels.

[0054] Reference Fig.11 , taking image classification training as an example, the method 100 for image recognition training is further explained. Fig.11 FIG. 1 is a schematic diagram showing a process of using noisy image samples and clean image samples to train an image classification model in a semi-supervised learning manner according to an example of an embodiment of the present disclosure. Fig.11 Some image samples may be correctly labeled with categories (i.e., have valid classification labels) and can therefore be called clean image samples, while some image samples may be incorrectly labeled with categories and can therefore be called noisy image samples, and even some image samples may not be labeled with categories themselves (not shown). Such image samples that are incorrectly labeled with categories and have no labeled categories can be called image samples without valid classification labels.

[0055] exist Fig.11 In the example, image samples without valid classification labels (e.g., noise image samples) can be weakly enhanced to generate a weakly enhanced dataset of noise image samples. In some embodiments, weak enhancement can be referred to as first data enhancement. Then, the noise image samples can be classified and predicted using an artificial intelligence classification model and the weakly enhanced dataset of noise image samples to obtain the probability that the noise image samples belong to different categories. According to the probability that the noise image samples belong to different categories, the pseudo labels of the noise image samples can be obtained. That is, the category of the sample. Knowing the pseudo label of the noisy image sample Afterwards, we can use the noise image samples and their labels Perform supervised training of AI classification models.

[0056] exist Fig.11 In an example, the artificial intelligence classification model includes an image encoder 44 and a classifier 46, wherein the classifier 46 can be initialized using text features, and the image encoder 44 can be initialized using contrastive language-image pre-training (CLIP) parameters. In some embodiments, the image encoder 44 can extract image features from the image sample, and the classifier 46 can determine the probability that the image sample belongs to the category corresponding to the initialized text feature based on the correlation between the extracted image features and the initialized text features.

[0057] like Figure 1 The method 100 for image recognition training shown in the figure annotates pseudo-labels for image samples without valid identification marks so that image samples without valid identification marks can be used to train artificial intelligence recognition models, thereby breaking the traditional denoising method's reliance on data distribution and noise distribution assumptions and expanding the number of training samples.

[0058] Figure 2 FIG. 2 shows a flow chart of a method 200 for image recognition training according to an embodiment of the present disclosure. Figure 2 As shown, the method 200 for image recognition training includes the following three steps: S202, screening training image samples from an input image sample set; S204, dividing the training image samples into noisy image samples and clean image samples; S206, using the noisy image samples and the clean image samples to train the image recognition model in a semi-supervised learning manner.

[0059] Figure 3 FIG. 3 is a flowchart showing a process 300 of selecting training image samples from an input image sample set according to an embodiment of the present disclosure. The process 300 may correspond to Figure 2 Step S202 of the method shown. Figure 3 In the process 300 shown, text features can be extracted from the text of the target tag, S302; image features can be extracted from each image sample in the input image sample set, S304; then, based on the similarity between the text features and the image features, valid image samples are determined from the input image sample set as training image samples, S306.

[0060] Further, Figure 4 A schematic diagram of a process of screening training images from an input image sample set according to an example of an embodiment of the present disclosure is shown. Figure 4 Further explanation using classification prediction as an example Figure 3 The process shown in Figure 4 As shown, a text encoder 42 is used to extract text features from the text of the target category, and an image encoder 44 is used to extract image features from image samples in the input image sample set, wherein the text encoder 42 is associated with the image encoder 44. Then, the similarity between the text features and the image features can be compared to determine whether the input image sample is an out-of-domain (OOD) sample or an in-domain (ID) sample, and the out-of-domain sample can also be called an outlier.

[0061] In some embodiments, zero-shot classification based on the CLIP model can be used to identify out-of-domain samples. Specifically, in some embodiments, for all categories in the domain, the guide word prompt and the category name class_name (e.g., class c) can be used to construct the text text_c = prompt + class_name. For example, the guide word can be "This is a photo of...", and the category name can be "Indigo Bird", so the constructed text is "This is a photo of Indigo Bird". In some embodiments, the text encoder 42 (Text_encoder) can map the constructed text to a text vector T(c) = Text_encoder(text_c), such as Figure 4 The triangle in the dotted circle is shown, that is, the text features are extracted. The image encoder 44 (Image_encoder) can map all image samples belonging to the category in the domain into an image vector I (x c,i )=Image_encoder(x c,i ),like Figure 4 As shown in the circle in the dotted circle, that is, the image features are extracted. Then, the cosine similarity sim(x) of all image vectors belonging to the in-domain category and the text vector of each category in the in-domain category can be calculated according to the following formula (1): c,i ,c):

[0062]

[0063] Among them, sim(x c,i ,c) represents the cosine similarity between the image vector of the i-th image of class c and the text vector of class c. Then, the binary Gaussian mixture distribution W can be modeled based on the calculated cosine similarity between the image and each category c,i , and determine whether the image belongs to a category in the domain according to the preset threshold th. If it does not belong to any category in the domain, the image sample is determined as an out-of-domain sample and removed from the input image sample set to finally obtain the in-domain sample set. Specifically, the following formula (2) can be used to determine whether the image sample belongs to a category in the domain:

[0064]

[0065] Wherein, GMM is a Gaussian Mixture Model. In some embodiments, the in-domain samples may be valid image samples used as training image samples.

[0066] Figure 5 An image example is shown of a process of screening training image samples from an input image sample set according to an embodiment of the present disclosure. Figure 5 The image examples shown in the figure are based on the WebVision dataset and can be used in reference Figure 3 and Figure 4 The process described is obtained. Figure 5 As shown in the figure, the target category is "frilled lizard", the OOD of the filtered out-of-domain samples is quite different from the target category, while the ID of the in-domain samples is all lizard. By removing the out-of-domain samples, the significant noise data is reduced and the overfitting of significant noise is avoided. Figures 3 to 5 The process or reference described in the following section to select training image samples from a set of input image samples Figure 2 The described step S202 of screening training image samples from the input image sample set is an embodiment of a method for image recognition training. The method for image recognition training disclosed in the present invention can also be implemented without including the process and step S202 of screening training image samples from the input image sample set.

[0067] Figure 6 FIG. 6 is a flowchart showing a process 600 of dividing a training image sample into a noisy image sample and a clean image sample according to an embodiment of the present disclosure. The process 600 may correspond to Figure 2 Step S204 of the method shown. Figure 6In the process 600 shown, a third data enhancement may be performed on the training image samples to generate a third enhanced image sample set, S602; an artificial intelligence classification model may be used to perform recognition prediction on the third enhanced image sample set to determine whether each enhanced image sample in the third enhanced image sample set corresponds to an identification mark of the training image sample, S604; then, the proportion of the number of enhanced image samples corresponding to the target mark in the third enhanced image sample set is determined, S606; next, it is determined whether the proportion is greater than a preset threshold, S608; when the proportion is greater than or equal to the preset threshold, the training image sample is identified as a clean image sample, S610, otherwise, the training image sample is identified as a noise image sample, S612. In some embodiments, the third data enhancement of the training image sample may be a weak enhancement of the training image sample. In other embodiments, the third data enhancement of the training image sample may be a strong enhancement of the training image sample. In other embodiments, the third data enhancement of the training image sample may be a mixture of weak and strong enhancements of the training image sample. In addition, it should be understood that Figure 6 The process 600 shown in FIG. 6 is a process for one training image sample and can be performed on all training image samples in the image sample set. Figure 6 The process shown is used to divide the training image samples into noisy image samples and clean image samples.

[0068] Further, Figure 7 A schematic diagram of a process of dividing training image samples into noisy image samples and clean image samples according to an example of an embodiment of the present disclosure is shown. Figure 7 Further explanation using classification prediction as an example Figure 6 The process shown in Figure 7 As shown, the CLIP model can be used to predict the reference Figure 4 Whether the generated in-domain image sample belongs to the category marked by the image sample is used to determine whether the in-domain sample is a noise image sample or a clean image sample. Figure 7 In an embodiment of the present invention, the CLIP model may include an image encoder 44 and a classifier 46, wherein the classifier 46 may be initialized using text features and the image encoder 44 may be initialized using CLIP pre-trained parameters. In some embodiments, the classifier 46 may be similar to the reference Figure 4 The function of calculating the probability of an image sample belonging to each category by cosine similarity described above, and the image encoder 44 can be similar to the reference Figure 4 The image encoder 44 is described.

[0069] like Figure 7 As shown, first we can refer to Figure 4The generated in-domain image samples are weakly enhanced and / or strongly enhanced to generate a weakly enhanced image sample set, a strongly enhanced image sample set, or a mixed set of weakly enhanced image samples and strongly enhanced image samples. In some embodiments, weak enhancement may be a spatial change to the image sample, and strong enhancement may include a change to the image pixel itself of the image sample. In some embodiments, the spatial change to the image sample may include at least one of the following items: translation, inversion, rotation, cropping, and scaling. In some embodiments, the change to the image pixel itself of the sample may include at least one of the following items: a change in the color domain, sharpening, smoothing, and frequency domain filtering. In some embodiments, the third enhancement may be a weak enhancement; however, in other embodiments, the third enhancement may be a strong enhancement; further, in some embodiments, the third enhancement is a combination of weak enhancement and strong enhancement.

[0070] After the enhanced image sample set is generated, the image encoder 44 can be used to extract image features from the enhanced image samples in the enhanced image sample set, and the classifier 46 can determine the probability that the enhanced image sample belongs to the category corresponding to the initialized text feature (the category can correspond to the category marked by the training image sample) based on the correlation between the extracted image features and the text features of the initialized classifier 46. For example, for an enhanced data set consisting of K enhanced image samples, K probabilities can be generated. The consistency score score (x) of the sample can be calculated according to the following formula (3): i ):

[0071]

[0072] In other words, the proportion of enhanced image samples belonging to the category marked by the training image samples in the enhanced data set is calculated, and when the consistency score is greater than or equal to the preset threshold, the image sample is labeled with a clean label, and when the consistency score is less than the preset threshold, the image sample is labeled with a noisy label. In this article, consistency means that for clean samples, the model's prediction results for the same sample under different data augmentations should be the same and consistent with the category marked by the training image samples.

[0073] Figures 8a to 8d An example of how the consistency score affects the identification of clean / noisy labels according to an embodiment of the present disclosure is shown. Figures 8a to 8d The example in shows the original image, the data enhancement result of the image, and the prediction of the model. In some embodiments, the original image can be an in-domain image sample. Figure 8a It shows that the original image is an indigo bird, and the model predicts that all 10 data augmentation results are indigo birds, so Figure 8a The consistency score of the original image is 1. Figure 8b It shows that the model predicts 4 of the 10 data augmentation results as indigo birds, so Figure 8b The consistency score of the original image is 0.4. In addition, Figure 8c and Figure 8d It shows that none of the model's predictions for the 10 data augmentation results is the indigo bird, so Figure 8c and Figure 8d The consistency score of the original image is 0.

[0074] Fig. 9 Shown based on Figure 5 An example of the process of dividing the training image samples into noise image samples and clean image samples. Fig. 9 As shown in FIG, by identifying noisy image samples, the samples can be labeled with clean labels or noisy labels. However, during the labeling process, some clean image samples may be labeled with noisy labels. By dividing the training image samples into noisy image samples and clean image samples, the amount of computation for subsequent processing can be reduced.

[0075] Those skilled in the art will understand that Figure 6 and Figure 7 The process or reference for dividing training image samples into noisy image samples and clean image samples described Figure 2 The described step S204 of dividing the training image samples into noisy image samples and clean image samples is an embodiment of a method for image recognition training. The method for image recognition training disclosed in the present invention can also be implemented without including the process and step S204 of dividing the training image samples into noisy image samples and clean image samples.

[0076] Fig.10 FIG. 1 is a flowchart of a process 1000 for training an image recognition model in a semi-supervised learning manner using noisy image samples and clean image samples according to an embodiment of the present disclosure. The process 1000 may correspond to Figure 2 Step S206 of the method shown. Fig.10In the process 1000 shown, the first data enhancement and the second data enhancement may be performed on the image samples without valid identification marks to generate the first enhanced noise image set and the second enhanced noise image set, step S1002; the image samples with valid identification marks and their identification marks may be used to supervise the training of the artificial intelligence recognition model, step S1004. In some embodiments, the image samples with valid identification marks themselves may be used to train the artificial intelligence recognition model. In some embodiments, the image samples with valid identification marks may be data enhanced to generate an enhanced image sample set, so that there may be more image samples to train the artificial intelligence recognition model. In some embodiments, the data enhancement of the image samples with valid identification marks may be weak enhancement. However, in other embodiments, the data enhancement of the image samples with valid identification marks may be strong enhancement. By using each of the multiple image samples with valid identification marks to perform multiple rounds of training on the artificial intelligence recognition model, the artificial intelligence recognition model may have a good initial recognition effect for subsequent recognition prediction.

[0077] After step S1004, the image samples without valid identification marks can be used to perform recognition prediction on the image samples without valid identification marks using the image samples with valid identification marks and the trained artificial intelligence recognition model and the first enhanced image sample set, step S1006; then, pseudo labels are annotated for the image samples without valid identification marks using the prediction results, step S1008. Finally, supervised training of the artificial intelligence recognition model is performed using the image samples without valid identification marks and the pseudo labels, step S1010. In some embodiments, in step S1010, supervised training of the artificial intelligence recognition model can be performed using the image samples without valid identification marks themselves, their weakly enhanced image sample sets, and / or their strongly enhanced image sample sets. In addition, in some embodiments, supervised training of the artificial intelligence recognition model can be performed using the image samples with valid identification marks themselves, their weakly enhanced image sample sets, and / or their strongly enhanced image sample sets.

[0078] Fig.11 A schematic diagram of a process of using noisy image samples and clean image samples to train an image classification model in a semi-supervised learning manner according to an example of an embodiment of the present disclosure is shown. The process of using noisy image samples to train an image classification model in a semi-supervised learning manner has been described above and will not be repeated here. Fig.11 In the example, before classifying and predicting the noisy image samples, the clean image samples are weakly enhanced to generate a weakly enhanced image sample set of the clean image samples, and the weakly enhanced image sample set of the clean image samples and their classification labels are used to supervise the training of the artificial intelligence classification model. Fig.11 Shown is a strongly enhanced image sample set and pseudo labels using noisy image samples The artificial intelligence classification model is supervised and trained, but those skilled in the art should understand that the artificial intelligence classification model can be supervised and trained using the noise image samples themselves, the weakly enhanced image sample sets of the noise image samples, the clean image samples themselves, or the strongly enhanced image sample sets of the clean image samples. In some embodiments, a weakly enhanced image sample set different from the weakly enhanced image sample set used to predict the noise image samples can be used to supervise the classification model.

[0079] Fig.12 FIG. 1 is a flowchart of a process 1300 for correcting noise labels according to an embodiment of the present disclosure. Fig.12 In the process 1300 shown, in step S1302, the probability that the image samples without valid identification marks correspond to each identification mark can be predicted; then, it is determined whether the maximum probability is greater than or equal to the probability threshold, step S1304. When the maximum probability is greater than or equal to the preset probability threshold, the identification mark with the maximum probability is marked as a pseudo-label for the image sample without valid identification mark, step S1306; when the maximum probability is less than the preset probability threshold, the noise image sample is not used to train the recognition model, step S1308. The maximum probability is greater than a certain threshold, which ensures that the model has a high confidence in the prediction. The accuracy of the pseudo-labels involved in the training is improved by threshold screening, and as the model performance improves, more and more noise samples will participate in the training. It should be understood that Fig.12 The process 1300 shown in FIG. 1 only represents a training process. The noise image samples not used in step S1308 are not used in this training process, but may be used for subsequent training in subsequent training processes according to the probability of subsequent prediction. Through this robust semi-supervised learning, the noise data is fully utilized and the number of training samples is expanded. Fig.12 The step 1300 of correcting the noisy labels shown in FIG. 1 may correspond to Fig.10 Step S1006 and step S1008 shown in FIG.

[0080] Fig.13 Shows the correction Fig. 9 An image example of the process of determining the noise labels of the noisy image samples. Fig.13 Further explanation using classification training as an example Fig.12 The process shown in Fig.13 As shown in FIG, the weakly enhanced image sample set of the noise image sample is predicted by the pre-trained classification model to obtain the pseudo label of the noise image sample, and the label of the image sample is corrected. Specifically, the image (such as Fig.13 In addition, the noisy image samples (such as Fig.13The middle image of the image is shown in Figure 2) and has been effectively corrected and labeled with the correct label, so it can be used to train the classification model. In addition, the uncertain samples (such as Fig.13 As shown in the right image of ), it is not used to train the classification model due to its low prediction probability. Fig.12 Similar to the description, uncertain samples are not used to train the classification model only in the current training process. In the subsequent training process, if the probability that the sample belongs to a certain category is greater than or equal to the preset probability threshold, the sample and the predicted category can be used to train the classification model, thereby further expanding the training sample set. It can be seen that the method for image classification training disclosed in the present invention excludes unavailable noise data on the one hand, while making full use of noise data, expanding the number of training samples, and through iterative training and prediction, more and more noise data can be used for training, further expanding the number of training samples.

[0081] The disclosed method for image recognition training first removes out-of-domain samples to exclude noise samples that are significantly different from the labeled labels for subsequent processing, then preliminarily divides clean samples and noise samples, and then uses a robust semi-supervised learning method to correct the noise labels, making full use of the noise data and expanding the number of training samples, thereby breaking the denoising method's dependence on data distribution and noise distribution assumptions. Through the disclosed training method, the actual recognition performance has been significantly improved.

[0082] It should be understood that although the above reference Figure 4-5 , Figure 7-9 , Fig.11 and Fig.13 The training method of the present disclosure is described using image classification as an example, but those skilled in the art should understand that the training method of the present disclosure can be applied to target detection, image segmentation, etc. In addition, the training method of the present disclosure has a wide range of application scenarios, such as base station corrosion image recognition, etc.

[0083] Below, refer to Fig.14 A device 1100 for image recognition training according to an embodiment of the present disclosure is described. Fig.14 is a schematic diagram of an image processing device according to an embodiment of the present disclosure. Figure 1-Figure 2 The details of the described methods are the same, so for the sake of simplicity, a detailed description of the same contents is omitted here.

[0084] The device for image recognition training disclosed in the present invention includes a processor 1102; and a memory 1101, in which computer-readable instructions are stored, wherein when the computer-readable instructions are executed by the processor, a method for image recognition training is executed, the method comprising: performing a first data enhancement on image samples without valid identification marks to generate a first enhanced image sample set; using an artificial intelligence recognition model and the first enhanced image sample set to perform recognition prediction on the image samples without valid identification marks; using the prediction results to annotate pseudo-labels for the image samples without valid identification marks; and using the image samples without valid identification marks and the pseudo-labels to perform supervised training on the artificial intelligence recognition model.

[0085] Regarding the technical effects of the device 1100 for image recognition training in different embodiments, reference may be made to the technical effects of the method for image recognition training provided in the embodiments of the present disclosure, which will not be repeated here.

[0086] The apparatus 1100 for image recognition training can be used in various appropriate electronic devices.

[0087] Fig.15 is a schematic diagram of a computer-readable storage medium 1200 according to an embodiment of the present disclosure.

[0088] like Fig.15 As shown, the present disclosure also includes a computer-readable storage medium 1200, on which computer-readable instructions 1201 are stored. When the computer-readable instructions are executed by a computer, the computer executes a method for image recognition training, including: performing a first data enhancement on image samples without valid identification marks to generate a first enhanced image sample set; using an artificial intelligence recognition model and the first enhanced image sample set to perform recognition prediction on the image samples without valid identification marks; using the prediction results to annotate pseudo-labels for the image samples without valid identification marks; and using the image samples without valid identification marks and the pseudo-labels to perform supervised training on the artificial intelligence recognition model.

[0089] The various methods / implementations described in this specification may be used alone or in combination, and may be switched during execution. In addition, the processing steps, sequences, flow charts, etc. of the various methods / implementations described in this specification may be changed in order as long as there is no contradiction. For example, with respect to the methods described in this specification, various step units are given in an exemplary order, and are not limited to the specific order given.

[0090] The phrase "based on" used in this specification does not mean "only based on" unless otherwise specified in other paragraphs. In other words, the phrase "based on" means both "only based on" and "at least based on".

[0091] Any reference to a unit using the names "first", "second", etc. used in this specification does not fully limit the number or order of these units. These names may be used in this specification as a convenient method to distinguish between two or more units. Therefore, a reference to a first unit and a second unit does not mean that only two units can be used or that the first unit must precede the second unit in some form.

[0092] When the terms “include”, “comprising”, and their variations are used in this specification or claims, these terms are open ended like the term “having”. Furthermore, the term “or” used in this specification or claims is not an exclusive OR.

[0093] Those skilled in the art will appreciate that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0094] The present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0095] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0096] The present disclosure is described in detail above, but it is obvious to those skilled in the art that the present disclosure is not limited to the embodiments described in this specification. The present disclosure can be implemented as a modification and alteration without departing from the purpose and scope of the present disclosure as determined by the claims. Therefore, the description in this specification is for the purpose of illustration and does not have any restrictive meaning for the present disclosure.

Claims

1. A method for image recognition training, comprising: Performing first data enhancement on image samples without valid identification marks to generate a first enhanced image sample set; Using an artificial intelligence recognition model and the first enhanced image sample set to perform recognition prediction on the image samples without valid recognition marks; Using the prediction results to label the image samples without valid identification marks with pseudo labels; as well as The artificial intelligence recognition model is supervised and trained using the image samples without valid identification marks and the pseudo labels.

2. The method according to claim 1, wherein: Using the image sample without valid identification mark and the pseudo label to supervise the artificial intelligence recognition model includes: Performing second data enhancement on the image samples without valid identification marks to generate a second enhanced image sample set; and Inputting the second enhanced image sample set and the pseudo label into the artificial intelligence recognition model for supervised training, The second data enhancement is different from the first data enhancement.

3. The method according to claim 1 or 2, wherein: Using the prediction result to label the image sample without a valid identification mark with the pseudo label includes: When the maximum probability in the prediction result is greater than or equal to a preset probability threshold, the identification mark with the maximum probability is marked as a pseudo label of the image sample without a valid identification mark.

4. The method according to claim 1 or 2, wherein: The method comprises: Before using the artificial intelligence recognition model and the first enhanced image sample set to perform recognition prediction on the image samples without valid recognition marks, the artificial intelligence classification model is supervisedly trained using image samples with valid recognition marks and their recognition marks.

5. The method according to claim 4, wherein: The method comprises: The training image samples are divided into noise image samples and clean image samples, wherein the noise image samples are used as the image samples without valid identification marks, and the clean image samples are used as the image samples with valid identification marks.

6. The method according to claim 5, wherein: Dividing the training image samples into noisy image samples and clean image samples includes: Performing third data enhancement on the training image samples to generate a third enhanced image sample set; Using the artificial intelligence recognition model to perform recognition prediction on the third enhanced image sample set to determine whether each enhanced image sample in the third enhanced image sample set corresponds to the recognition mark of the training image sample; Determining the proportion of the number of enhanced image samples corresponding to the target mark in the third enhanced image sample set; and When the proportion is greater than or equal to a preset threshold, the training image sample is identified as the clean image sample; otherwise, the training image sample is identified as a noise image sample.

7. The method according to claim 5, wherein: The method comprises: Extracting text features from target labeled text; Extracting image features from image samples in the input image sample set; and Based on the similarity between the text feature and the image feature, a valid image sample is determined from the input image sample set as the training image sample.

8. The method according to claim 2, wherein: The first enhancement is a spatial change to the image samples and the second enhancement comprises a change to the image pixels themselves of the image samples.

9. A device for image recognition training, comprising: processor; A memory storing one or more computer program modules; The one or more computer program modules are configured to execute the method for image recognition training according to any one of claims 1 to 8 when executed by the processor.

10. A non-transitory computer-readable storage medium having computer instructions stored thereon, wherein: When the computer instructions are executed by a processor, the method for image recognition training according to any one of claims 1 to 8 is performed.

Images