Method, device, and storage medium for identifying image and performing training

Abstract

To provide a method, a device, and a storage medium for identifying an image and performing training to improve robustness of a model against noise label-attached data.SOLUTION: A method for identifying an image and performing training includes a step of generating a first extended image sample set by executing first data extension on image samples with no effective identification mark attached, a step of achieving identification and prediction of an image sample with no effective identification mark attached by means of an artificial intelligence identification model and the first extended image sample set, a step of attaching a pseudo label to an image sample that has no effective identification mark attached as the result of the prediction, and a step of executing supervised learning on the artificial intelligence identification model with use of the image sample with no effective identification mark attached and the pseudo label. Accordingly, clean data and noise data are simultaneously used to eliminate dependence on data distribution and noise distribution assumptions.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present application relates to the technical field of processing images, and in particular to a method, an apparatus and a storage medium for identifying and training images. [Background technology]

[0002] When the training set of image classification contains noise data, the strong adaptation ability of neural networks usually leads to overfitting of noise labels in the model training, which greatly reduces the generalization ability of the model to clean datasets. Therefore, it is particularly important to improve the robustness of the model to noise-labeled data.

[0003] Regarding the problem of classifying noise-labeled images, the current mainstream approach is to model synthetic noise datasets (e.g., symmetric or asymmetric noise) based on assumptions made on noise or data distribution. However, in practice, there are significant differences between real noise distributions and synthetic noise. For example, real-world noise datasets usually contain multiple types of noise simultaneously, which are fine-grained noises, while synthetic noise datasets usually contain only a single type of noise, which is assumed to cause confusion between similar large classes. Therefore, traditional denoising methods that rely on assumptions on data and noise distributions are not robust. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of the above problems, the present disclosure provides a method, an apparatus, and a storage medium for identifying and training images.

[0005] According to one aspect of the present disclosure, performing a first data augmentation on the image samples without a valid identification mark to generate a sample set of first augmented images; Using an artificial intelligence classification model and the first augmented image sample set, predicting the valid identification mark-free image sample; pseudo-labeling image samples that do not have valid identifying marks with the predicted results; and performing supervised learning on the artificial intelligence identification model with the image samples without the valid identification marks and the pseudo labels.

[0006] According to an example of the present disclosure, the step of performing supervised learning on the artificial intelligence identification model using the image samples without the valid identification mark and the pseudo-labels includes: performing a second data augmentation on the image samples without the valid identification mark to generate a sample set of second augmented images; inputting the sample set of the second augmented images and the pseudo-labels into the artificial intelligence discriminative model to perform supervised learning; The second data extension is different from the first data extension.

[0007] According to one example of the present disclosure, the step of assigning the pseudo label to the image sample without the valid identifying mark in the predicted result includes a step of labeling the identifying mark having the maximum probability as a pseudo label of the image sample without the valid identifying mark if the maximum probability in the predicted result is greater than or equal to a predetermined probability threshold.

[0008] According to one example of the present disclosure, the method includes a step of performing supervised learning on the artificial intelligence identification model using image samples with a valid identification mark and the identification mark before identifying and predicting with the artificial intelligence identification model and a sample set of the first augmented images for image samples without the valid identification mark.

[0009] According to one example of the present disclosure, the method includes a step of dividing training image samples into noisy image samples and clean image samples, using the noisy image samples as image samples without the valid identifying mark, and using the clean image samples as image samples with the valid identifying mark.

[0010] According to an example of the present disclosure, the step of separating the training image samples into noisy image samples and clean image samples includes: performing a third data augmentation on the training image samples to generate a sample set of third augmented images; Using the artificial intelligence identification model, identifying and predicting the third augmented image sample set, and determining whether each augmented image sample in the third augmented image sample set corresponds to an identification mark of the training image sample; determining a proportion of a number of dilated image samples in the third dilated image sample set that correspond to the target mark; If the ratio is greater than or equal to a preset threshold, identifying the training image sample as the clean image sample, otherwise identifying the training image sample as a noisy image sample.

[0011] According to one example of the present disclosure, the method comprises: Extracting text features from the target marked text; extracting image features from image samples of the input image sample set; determining valid image samples from the input image sample set as the training image samples based on similarities between the text features and the image features.

[0012] According to one example of the present disclosure, the first data augmentation is a spatial modification to the image samples, and the second data augmentation includes modifications to the image pixels of the image samples themselves.

[0013] According to another aspect of the present disclosure, there is provided an apparatus for identifying and training images, comprising a processor and a memory in which one or more computer program modules are stored, the one or more computer program modules being configured to perform the above-mentioned method for identifying and training images when executed by the processor.

[0014] According to yet another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions which, when executed by a processor, perform the above-mentioned method for identifying and training images.

[0015] According to another aspect of the present disclosure, there is provided a computer program product including a medium for implementing a computer program for input into a computer, and a computer program embodied in the medium, the computer program causing a computer to perform the above-described method for identifying and training images.

[0016] In the above aspects of the present disclosure, clean data and noisy data are simultaneously utilized to train a pre-trained model through semi-supervised learning, breaking away from reliance on assumptions on data and noise distributions.

[0017] The above and other objects, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail with reference to the accompanying drawings. The accompanying drawings are used for a better understanding of the embodiments of the present disclosure, constitute a part of the specification, and are used to explain the present disclosure together with the embodiments of the present disclosure, but are not intended to limit the present disclosure. In the drawings, similar reference numerals generally represent the same elements or steps. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 shows a flowchart of a method for identifying and training an image according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 shows a flowchart of a method for identifying and training an image according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 illustrates a flow chart of a process for selecting training image samples from an input image sample set according to an embodiment of the present disclosure. [Figure 4] FIG. 4 shows a schematic diagram of a process for selecting training image samples from an input image sample set according to an example embodiment of the present disclosure. [Diagram 5] FIG. 5 illustrates example images of the process of selecting training image samples from an input image sample set according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates a flowchart of a process for separating training image samples into noisy image samples and clean image samples according to an embodiment of the present disclosure. [Figure 7] FIG. 7 shows a schematic diagram of a process for separating training image samples into noisy and clean image samples according to an example embodiment of the present disclosure. [Figure 8A] FIG. 8A illustrates an example of the effect of consistency scores on separating noisy and clean image samples, according to an embodiment of the present disclosure. [Figure 8B]FIG. 8B illustrates an example of the effect of consistency scores on separating noisy and clean image samples according to an embodiment of the present disclosure. [Figure 8C] FIG. 8C illustrates an example of the effect of consistency scores on separating noisy and clean image samples according to an embodiment of the present disclosure. [Figure 8D] FIG. 8D illustrates an example of the effect of consistency scores on separating noisy and clean image samples, according to an embodiment of the present disclosure. [Figure 9] FIG. 9 shows example images of the process of separating the training image samples into noisy image samples and clean image samples based on the selected training image samples of FIG. [Figure 10] FIG. 10 illustrates a flowchart of a process for training an image identification model in a semi-supervised learning manner using noisy and clean image samples, according to an embodiment of the present disclosure. [Figure 11] FIG. 11 illustrates a schematic diagram of a process for training an image classification model in a semi-supervised learning manner using noisy and clean image samples according to an example embodiment of the present disclosure. [Figure 12] FIG. 12 illustrates a flow chart of a process for modifying noise labels according to an embodiment of the present disclosure. [Figure 13] FIG. 13 shows an example image of the process of modifying the noise labels of the noisy image samples determined in FIG. [Figure 14] FIG. 14 illustrates a functional block diagram of an image processing device according to an embodiment of the present disclosure. [Figure 15] FIG. 15 illustrates a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The technical solutions in the embodiments of the present disclosure are described below clearly and completely with reference to the accompanying drawings in the embodiments of the present disclosure. It should be noted that the described embodiments are only a part of the embodiments of the present disclosure, and are not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts are included in the scope of protection of the present disclosure.

[0020] In the present application, a flow chart illustrates steps of a method according to an embodiment of the present application. It should be understood that the steps do not necessarily have to be performed in order. Instead, various steps can be processed in reverse order or simultaneously. At the same time, other operations can be added to these processes, or one or more steps can be removed from these processes.

[0021] First, a flowchart of an image identification and training method 100 according to an embodiment of the present disclosure will be described with reference to Fig. 1. As shown in Fig. 1, the image identification and training method 100 includes the following steps S102 to S108.

[0022] At S102, a first data augmentation is performed on the image samples without a valid identification mark to generate a sample set of a first augmented image.

[0023] In S104, using an artificial intelligence discrimination model and the first extended image sample set, discrimination and prediction are performed for image samples without valid discrimination marks.

[0024] In S106, the predicted results pseudo-label image samples that do not have valid identifying marks.

[0025] At S108, supervised learning is performed on the artificial intelligence classification model using image samples without valid identifying marks and pseudo labels.

[0026] With reference to FIG. 11, the method 100 for identifying and training an image is further described by taking training for classifying an image as an example. FIG. 11 shows a schematic diagram of a process for training an image classification model in a semi-supervised learning manner using noisy and clean image samples according to an example of an embodiment of the present disclosure. As shown in FIG. 11, some image samples are called clean image samples because their classes are correctly labeled (i.e., they have valid classification marks), while some image samples are called noisy image samples because their classes are incorrectly labeled, and some image samples themselves may not have a class label (not shown). Such image samples with incorrect class labels and image samples without class labels are called image samples without valid classification marks.

[0027] In the example of FIG. 11 , image samples without valid classification marks (e.g., noisy image samples) may be weakly augmented to generate a weakly augmented dataset of noisy image samples. In some embodiments, the weak augmentation is referred to as first data augmentation. Then, using the artificial intelligence classification model and the weakly augmented dataset of noisy image samples, a classification prediction of the noisy image samples may be performed to obtain the probability that the noisy image samples belong to different classes. According to the probability that the noisy image samples belong to different classes, a pseudo-label of the noisy image sample may be obtained.

number

[0028] 11, the artificial intelligence classification model includes an image encoder 44 and a classifier 46, where the classifier 46 can be initialized by text features and the image encoder 44 can be initialized by Contrastive Language-Image Pre-Training (CLIP) parameters. In some implementations, the image encoder 44 can extract image features from the image samples, and the classifier 46 can determine the probability that the image sample belongs to a class corresponding to the initialized text features based on the correlation between the extracted image features and the initialized text features.

[0029] The method 100 for identifying and training images shown in FIG. 1 pseudo-labels image samples without valid identifying marks, allowing image samples without valid identifying marks to be used for training artificial intelligence identification models, breaking away from the reliance of traditional denoising methods on assumptions about data distribution and noise distribution, and increasing the number of samples to be trained.

[0030] 2 shows a flowchart of a method 200 for identifying and training images according to an embodiment of the present disclosure. As shown in FIG. 2, the method 200 for identifying and training images includes three steps: S202, selecting training image samples from an input image sample set; S204, dividing the training image samples into noisy image samples and clean image samples; S206, training an image identification model using the noisy image samples and the clean image samples through a semi-supervised learning method.

[0031] 3 shows a flow chart of a process 300 for selecting training image samples from an input image sample set according to an embodiment of the present disclosure. The process 300 may correspond to step S202 of the method shown in FIG. 2. In the process 300 shown in FIG. 3, in S302, text features may be extracted from the target marked text, in S304, image features are extracted from each image sample in the input image sample set, and then in S306, valid image samples are determined from the input image sample set as training image samples based on the similarity between the text features and the image features.

[0032] Further, FIG. 4 shows a schematic diagram of a process of screening training images from an input image sample set according to an example of an embodiment of the present disclosure. FIG. 4 further explains the process shown in FIG. 3 by taking classification prediction as an example. As shown in FIG. 4, a text encoder 42 is used to extract text features from the text of a target class, and an image encoder 44 is used to extract image features from image samples of the input image sample set, and the text encoder 42 and the image encoder 44 are associated with each other. Then, the similarity between the text features and the image features can be compared to determine whether the input image sample is an Out-of-Domain (OOD) sample or an In-Domain (ID) sample, and the out-of-domain sample is also called an outlier.

[0033] In some embodiments, a zero-shot classification implemented based on the CLIP model can be used to identify samples outside the domain. Specifically, in some embodiments, for every class in the domain, a guide word prompt and a class name class_name (e.g., class c) can be used to generate a text text_c=prompt+class_name. For example, the guide word may be "This is a picture of..." and the class name may be "Blue-and-white Bunting", so that the generated text is "This is a picture of Blue-and-white Bunting". In some embodiments, the text encoder 42 (Text_encoder) can map the generated text to a text vector T(c)=Text_encoder(text_c), i.e., extract text features, as shown by the triangle in the dotted circle in FIG. 4. The image encoder 44 (Image_encoder) can map all image samples belonging to classes in the domain to an image vector I(x c,i )=Image_encoder(x c,i ), that is, image features can be extracted. Next, the cosine similarity sim(x c,i , c) can be calculated individually according to the following formula (1).

number

[0034] In the formula, sim(x c,i ,c) represents the cosine similarity between the image vector of the i-th image of class c and the text vector of class c. Next, based on the calculated cosine similarity between the image and each class, a two-dimensional mixture Gaussian distribution W c,iBy modeling the above, it is possible to determine whether an image belongs to a specific class in the domain based on a preset threshold th, and if the image sample does not belong to any class in the domain, it is determined that the image sample is an out-of-domain sample and is deleted from the input image sample set, and finally an in-domain sample set is obtained. Specifically, it is possible to determine whether an image sample belongs to a specific class in the domain according to the following formula (2).

number

[0035] where GMM is Gaussian Mixture Model. In some implementations, the samples in the domain may be valid image samples as training image samples.

[0036] FIG. 5 shows example images of the process of selecting training image samples from an input image sample set according to an embodiment of the present disclosure. The example images shown in FIG. 5 are based on the WebVision dataset and can be obtained using the process described with reference to FIG. 3 and FIG. 4. As shown in FIG. 5, the target class is "frilled lizard" and the selected out-of-domain samples OOD and the target class are significantly different, but the in-domain sample IDs are all lizards. By removing the out-of-domain samples, significant noise data was reduced and significant noise overfitting was avoided. Those skilled in the art will recognize that the process of selecting training image samples from an input image sample set described with reference to FIG. 3 to FIG. 5 or the step S202 of selecting training image samples from an input image sample set described with reference to FIG. 2 are examples of the method of identifying and training images, and the method of identifying and training images of the present disclosure can also be implemented without the step S202 of selecting training image samples from an input image sample set.

[0037] FIG. 6 shows a flow chart of a process 600 for dividing training image samples into noisy image samples and clean image samples according to an embodiment of the present disclosure. The process 600 may correspond to step S204 of the method shown in FIG. 2. In the process 600 shown in FIG. 6, in S602, a third data augmentation can be performed on the training image samples to generate a third augmented image sample set; in S604, an artificial intelligence classification model can identify and predict for the third augmented image sample set, and determine whether each augmented image sample in the third augmented image sample set corresponds to an identification mark of the training image sample; and then in S606, determine the proportion of the number of augmented image samples in the third augmented image sample set that corresponds to the target mark. Then, in S608, determine whether the proportion is greater than a preset threshold; in S610, if the proportion is greater than or equal to the preset threshold, identify the training image sample as a clean image sample; otherwise, in S612, identify the training image sample as a noisy image sample. In some embodiments, the third data augmentation for the training image sample may be a weak augmentation for the training image sample. In some other embodiments, the third data augmentation for the training image sample may be a strong augmentation for the training image sample. In some other embodiments, the third data augmentation for the training image sample may be a mixture of weak and strong augmentation for the training image sample. Furthermore, it should be understood that the process 600 shown in FIG. 6 is a process for one training image sample, and the process shown in FIG. 6 can be performed for all training image samples in the image sample set to separate the training image samples into noisy image samples and clean image samples.

[0038] Further, FIG. 7 shows a schematic diagram of a process of dividing training image samples into noisy image samples and clean image samples according to an example of the embodiment of the present disclosure. FIG. 7 further explains the process shown in FIG. 6 by taking classification prediction as an example. As shown in FIG. 7, the CLIP model can predict whether the intra-domain image sample generated with reference to FIG. 4 belongs to the class labeled with the image sample, thereby determining whether the intra-domain sample is a noisy image sample or a clean image sample. In the embodiment of FIG. 7, the CLIP model can include an image encoder 44 and a classifier 46, the classifier 46 can be initialized by text features, and the image encoder 44 can be initialized by CLIP pre-training parameters. In some embodiments, the classifier 46 has a function of calculating the probability that the image sample belongs to each class through cosine similarity as described with reference to FIG. 4, and the image encoder 44 can be the same as the image encoder 44 described with reference to FIG. 4.

[0039] As shown in FIG. 7, first, weak and / or strong dilation may be performed on the in-domain image samples generated with reference to FIG. 4 to generate a weakly dilated image sample set, a strongly dilated image sample set, or a mixed set of weakly and strongly dilated image samples. In some embodiments, the weak dilation is a spatial modification to the image samples, and the strong dilation includes a modification to the image pixels of the image samples themselves. In some embodiments, the spatial modification of the image samples may include at least one of translation, flipping, rotation, cropping, and scaling. In some embodiments, the modification to the image pixels of the samples themselves may include at least one of color gamut modification, sharpening, smoothing, and frequency domain filtering. In some embodiments, the third data augmentation may be a weak dilation, while in other embodiments, the third data augmentation may be a strong dilation, and in some embodiments, the third data augmentation is a combination of weak and strong dilation.

[0040] Now that the augmented image sample set has been generated, image encoder 44 can be used to extract image features from the augmented image samples in the augmented image sample set. Based on the correlation between the extracted image features and the text features that initialized classifier 46, classifier 46 determines the probability that the augmented image sample belongs to a class corresponding to the initialized text feature (which may correspond to a class labeled by the training image sample). For example, for an augmented dataset consisting of K augmented image samples, K probabilities can be generated. A consistency score for the sample, score(x i ) can be calculated according to the following formula (3):

number

[0041] In other words, calculate the proportion of samples of the augmented image in the augmented dataset that belong to the class labeled by the training image sample, and if the consistency score is equal to or greater than a preset threshold, label the image sample as clean, and if the consistency score is less than a preset threshold, label the image sample as noisy. Consistency in this specification means that for clean samples, the predicted results of the model for the same sample under different data augmentation are the same and consistent with the class labeled by the training image sample.

[0042] 8A-8D show examples of the impact of consistency scores on clean label / noise label discrimination according to embodiments of the present disclosure. The examples of FIGS. 8A-8D show an original image, a data augmentation result of the image, and a model prediction. In some embodiments, the original image may be an in-domain image sample. FIG. 8A shows that the original image is a blue-green bunting, and the model's prediction for the 10 data augmentation results shows that all of them are blue-green bunting, so the consistency score of the original image in FIG. 8A is 1. FIG. 8B shows that the model's prediction for the 10 data augmentation results shows that four of them are blue-green bunting, so the consistency score of the original image in FIG. 8B is 0.4. Also, FIG. 8C and FIG. 8D show that the model's prediction for the 10 data augmentation results shows that there is no blue-green bunting, so the consistency score of the original image in FIG. 8C and FIG. 8D is 0.

[0043] FIG. 9 shows an example image of the process of dividing the training image samples into noisy image samples and clean image samples based on the selected training image samples of FIG. 5. As shown in FIG. 9, by identifying the noisy image samples, the samples can be labeled with a clean label or a noise label. However, during the labeling process, some clean image samples may be labeled with a noise label. The training image samples can be divided into noisy image samples and clean image samples to reduce the computational effort of subsequent processing.

[0044] It is understood by those skilled in the art that the process of separating training image samples into noisy and clean image samples described with reference to Figures 6 and 7, or step S204 of separating training image samples into noisy and clean image samples described with reference to Figure 2, are examples of the method of identifying and training images. The method of identifying and training images of the present disclosure can also be implemented without the process of separating training image samples into noisy and clean image samples and step S204.

[0045] FIG. 10 illustrates a flowchart of a process 1000 for training an image identification model in a semi-supervised learning manner using noisy and clean image samples according to an embodiment of the present disclosure. The process 1000 may correspond to step S206 of the method illustrated in FIG. 2. In the process 1000 illustrated in FIG. 10, in step S1002, a first data augmentation and a second data augmentation may be performed on an image sample without a valid identification mark to generate a first augmented noisy image set and a second augmented noisy image set, and in step S1004, an image sample with a valid identification mark and the identification mark may be used to perform supervised learning for an artificial intelligence identification model. In some embodiments, the image sample with a valid identification mark itself may be used to train the artificial intelligence identification model. In some embodiments, data augmentation may be performed on an image sample with a valid identification mark to generate a sample set of augmented images, thereby providing more image samples for training the artificial intelligence identification model. In some embodiments, the data augmentation for the image sample with a valid identification mark may be weak augmentation. However, in some other embodiments, the data augmentation for image samples with valid identification marks may be strong augmentation, and each of a plurality of image samples with valid identification marks is used to train the artificial intelligence identification model over multiple rounds, so that the artificial intelligence identification model can obtain a good initial identification effect for subsequent identification prediction.

[0046] After step S1004, in step S1006, the image samples with the valid identification mark, the trained artificial intelligence identification model, and the first augmented image sample set can be used to identify and predict the image samples without the valid identification mark. In step S1008, the predicted results are used to pseudo-label the samples without the valid identification mark. Finally, in S1010, supervised learning is performed on the artificial intelligence identification model using the image samples without the valid identification mark itself, its weakly augmented image sample set, and / or its strongly augmented image sample set. In some embodiments, supervised learning can be performed on the artificial intelligence identification model using the image samples with the valid identification mark itself, its weakly augmented image sample set, and / or its strongly augmented image sample set. Also, in some embodiments, supervised learning can be performed on the artificial intelligence identification model using the image samples with the valid identification mark itself, its weakly augmented image sample set, and / or its strongly augmented image sample set.

[0047] FIG. 11 shows a schematic diagram of a process of training an image classification model in a semi-supervised learning manner using noisy image samples and clean image samples according to an example of an embodiment of the present disclosure. The process of training an image classification model in a semi-supervised learning manner using noisy image samples has been described above, so it will not be repeated here. In the example of FIG. 11, before performing classification prediction on the noisy image samples, the clean image samples are weakly augmented to generate a weakly augmented image sample set of the clean image samples, and the weakly augmented image sample set of the clean image samples and its classification mark are used to perform supervised learning on the artificial intelligence classification model. Although FIG. 11 shows performing supervised learning on the artificial intelligence classification model using a strongly augmented image sample set of the noisy image samples and the pseudo label y^, those skilled in the art can perform supervised learning on the artificial intelligence classification model using the noisy image samples themselves, the weakly augmented image sample set of the noisy image samples, the clean image samples themselves, or the strongly augmented image sample set of the clean image samples. In some embodiments, supervised learning can be performed on the classification model using a weakly augmented image sample set that is different from the weakly augmented image sample set for predicting the noisy image samples.

[0048] FIG. 12 shows a flow chart of a process 1300 for correcting noise labels according to an embodiment of the present disclosure. In the process 1300 shown in FIG. 12, in step S1302, the probability that an image sample without a valid identification mark corresponds to each identification mark can be predicted. Then, in step S1304, it is determined whether the maximum probability is equal to or greater than a probability threshold. If the maximum probability is equal to or greater than a preset probability threshold, in step S1306, the identification mark with the maximum probability is labeled as a pseudo label of the image sample without a valid identification mark; if the maximum probability is less than the preset probability threshold, in step S1308, the noisy image sample is not used to train the identification model. The maximum probability is greater than a certain threshold, ensuring that the model prediction has high reliability. Through the selection of the threshold, the accuracy of the pseudo label participating in the training is improved, and as the model performance improves, more and more noise samples participate in the training. The process 1300 shown in FIG. 12 shows a one-time training process, and the noisy image samples not used in step S1308 may be used in subsequent training according to the subsequent predicted probability in the subsequent training process, rather than being used only in this training. This robust semi-supervised learning makes full use of the noise data and increases the number of training samples. The step 1300 of modifying the noise labels shown in FIG. 12 may correspond to steps S1006 and S1008 shown in FIG. 10.

[0049] FIG. 13 shows an example image of the process of correcting the noise label of the noisy image sample determined in FIG. 9. FIG. 13 further explains the process shown in FIG. 12 by taking classification training as an example. As shown in FIG. 13, by predicting the weakly augmented image sample set of the noisy image sample through a pre-trained classification model, the pseudo label of the noisy image sample is obtained and the label of the image sample is corrected. Specifically, the image (as shown in the left image of FIG. 13) that is erroneously classified into the noisy image sample subset in the noise label identification stage can be reused. In addition, the noisy image sample (as shown in the center image of FIG. 13) is effectively corrected and correctly labeled, so that it can be used to train the classification model. In addition, the uncertain samples (as shown in the right image of FIG. 13) are not used to train the classification model because of their low prediction probability. It should be understood that, similar to the description with reference to Fig. 12, the uncertain samples are not used to train the classification model only in the current training process, and in the subsequent training process, if the prediction shows that the probability that the sample belongs to a certain class is equal to or greater than a preset probability threshold, the sample and the predicted class can be used to train the classification model, thereby further expanding the training samples. The method for image classification training of the present disclosure excludes unavailable noise data, while making full use of the noise data, increasing the number of training samples, and repeating the training and prediction, making more noise data available for training, and further increasing the number of training samples.

[0050] The disclosed image identification and training method first removes noise samples that are significantly different from the labeled marks for subsequent processing by removing samples outside the domain, then first separates clean samples from noise samples, and then uses a robust semi-supervised learning method to correct the noise labels, making full use of the noise data and increasing the number of trained samples, thereby breaking away from the dependency of the denoising method on the assumptions of data distribution and noise distribution. The disclosed training method has significantly improved the actual identification performance.

[0051] Although the training method of the present disclosure has been described with reference to Figures 4-5, 7-9, 11 and 13 using image classification as an example, it should be understood that a person skilled in the art should understand that the training method of the present disclosure can be applied to detecting targets, segmenting images, etc. In addition, the training method of the present disclosure applies to a wide range of application scenarios, such as identifying base station corrosion images.

[0052] Hereinafter, an image identifying and training device 1100 according to an embodiment of the present disclosure will be described with reference to Fig. 14. Fig. 14 shows a schematic diagram of an image processing device according to an embodiment of the present disclosure. The function of the image processing device of this embodiment is the same as the details of the above method described with reference to Figs. 1 and 2, so for simplicity, a detailed description of the same contents will be omitted here.

[0053] The apparatus for identifying and training images of the present disclosure includes a processor 1102 and a memory 1101 storing computer readable instructions, which, when executed by the processor, performs a method for identifying and training images. The method includes the steps of performing a first data augmentation on image samples without a valid identification mark to generate a first augmented image sample set, performing identification and prediction on the image samples without the valid identification mark with an artificial intelligence identification model and the first augmented image sample set, assigning a pseudo label to the image samples without the valid identification mark with the predicted result, and performing supervised learning on the artificial intelligence identification model with the image samples without the valid identification mark and the pseudo label.

[0054] Regarding the technical effects of the image identifying and training apparatus 1100 in different embodiments, reference may be made to the technical effects of the image identifying and training method provided in the embodiments of the present disclosure, and they will not be repeated here.

[0055] The image identification and training device 1100 can be used in a variety of suitable electronic devices.

[0056] FIG. 15 illustrates a schematic diagram of a computer-readable storage medium 1200 according to an embodiment of the present disclosure.

[0057] As shown in FIG. 15, the present disclosure further includes a computer-readable storage medium 1200 having computer-readable instructions 1201 stored therein, which, when executed by a computer, causes the computer to perform a method for identifying and training images, including the steps of performing a first data augmentation on image samples without a valid identifying mark to generate a first augmented image sample set, identifying and predicting the image samples without a valid identifying mark with an artificial intelligence identification model and the first augmented image sample set, pseudo-labeling the image samples without a valid identifying mark with the predicted results, and performing supervised learning on the artificial intelligence identification model with the image samples without the valid identifying mark and the pseudo-labels.

[0058] Each method / embodiment described herein may be used individually or in combination, or may be switched during execution. Also, the processing steps, sequences, flow charts, etc. of each method / embodiment described herein may be reordered unless inconsistent. For example, with respect to the methods described herein, various step units are given in an exemplary order, and are not limited to the particular order given.

[0059] As used herein, the phrase "based on" does not mean "based only on," unless expressly stated otherwise in another paragraph. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0060] As used herein, references to units using names such as "first," "second," etc., are not intended to be inclusive limitations on the number or order of those units. These names may be used herein as a convenient way to distinguish between two or more units. Thus, references to a first unit and a second unit do not imply that only two units may be used or that the first unit must precede the second unit in some way.

[0061] When the terms "comprise," "contain," and variations thereof are used in the present specification or claims, these terms are open-ended, similar to the term "having." Furthermore, when used in the present specification or claims, the term "or" does not mean an exclusive or.

[0062] Those skilled in the art will appreciate that various aspects of the present application may be described and described in several patentable categories or contexts, including any new and useful process, machine, manufacture, or combination of matter, or any new and useful improvement thereof. Thus, various aspects of the present application may be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. Such hardware or software may be referred to as "data blocks," "modules," "engines," "units," "components," or "systems." Furthermore, various aspects of the present application may be embodied as a computer product including computer readable program code and disposed on one or more computer readable mediums.

[0063] The present application uses certain words to describe embodiments of the present application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the present application. Thus, it should be noted that "one embodiment," "one embodiment," or "an alternative embodiment" referenced two or more times in different places in this specification do not necessarily refer to the same embodiment. Also, certain features, structures, or characteristics in one or more embodiments of the present application may be combined as appropriate.

[0064] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. It should also be understood that terms as defined in ordinary dictionaries should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology, and should not be interpreted in an idealized or highly formalized sense unless expressly stated in this specification.

[0065] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described in the present specification. The present disclosure can be modified and changed without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description in the present specification is for illustrative purposes only and does not limit the present disclosure.

Claims

1. performing a first data augmentation on the image samples without a valid identification mark to generate a sample set of first augmented images; predicting the validity of the image samples without the identification mark by using an artificial intelligence identification model and the first augmented image sample set; pseudo-labeling image samples that do not have valid identifying marks with the predicted results; and performing supervised learning on the artificial intelligence identification model using the image samples without the valid identification marks and the pseudo labels.

2. The step of performing supervised learning on the artificial intelligence identification model using the image samples without the valid identification mark and the pseudo-labels includes: performing a second data augmentation on the image samples without the valid identification mark to generate a second sample set of augmented images; and performing supervised learning on the artificial intelligence discrimination model by inputting the sample set of the second augmented images and the pseudo-labels; The method of claim 1 , wherein the second data augmentation is different from the first data augmentation.

3. The step of pseudo-labeling image samples that do not have the valid identifying mark in the predicted results includes: The method for identifying and training images according to claim 1 or 2, further comprising the step of labeling the identification mark having the maximum probability as a pseudo-label for the image sample without the valid identification mark if the maximum probability in the predicted result is equal to or greater than a preset probability threshold.

4. The method comprises: The method for identifying and training images according to claim 1 or 2, comprising a step of performing supervised learning on the artificial intelligence classification model using image samples with a valid identifying mark and the identifying mark before identifying and predicting for image samples without the valid identifying mark with an artificial intelligence identification model and a sample set of the first augmented images.

5. The method comprises:

5. The method for identifying and training images according to claim 4, comprising the steps of: dividing training image samples into noisy image samples and clean image samples; using the noisy image samples as image samples without the valid identifying mark; and using the clean image samples as image samples with the valid identifying mark.

6. The step of dividing the training image samples into noisy image samples and clean image samples includes: performing a third data augmentation on the training image samples to generate a sample set of third augmented images; identifying and predicting the third augmented image sample set with the artificial intelligence identification model, and determining whether each augmented image sample in the third augmented image sample set corresponds to an identification mark of the training image sample; determining a proportion of a number of dilated image samples in the third dilated image sample set that correspond to a target mark; and identifying the training image sample as the clean image sample if the ratio is greater than or equal to a preset threshold, and otherwise identifying the training image sample as a noisy image sample.

7. The method comprises: Extracting text features from the target marked text; extracting image features from image samples of the input image sample set; and determining valid image samples from the input image sample set as the training image samples based on similarities between the text features and the image features.

8. 3. The method of claim 2, wherein the first data augmentation is a spatial modification to an image sample, and the second data augmentation includes modifications to the image pixels of the image sample themselves.

9. A processor; a memory in which one or more computer program modules are stored; An apparatus for identifying and training images, wherein the one or more computer program modules, when executed by the processor, perform the method for identifying and training images according to any one of claims 1 to 8.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, which, when executed by a processor, results in the method for identifying and training images according to any one of claims 1 to 8.

Images