Training method and device of image classification model and image classification method and device

Abstract

The application discloses a training method and device of an image classification model and an image classification method and device. The training method comprises: generating a transformed image by image transformation on an original image; in a current training round, after completing parameter updating on a current batch, determining whether the original image corresponding to a first transformed image sequence in the current batch is a high-probability noise anchor point of the current training round; after the current training round ends, for each high-probability noise anchor point of the current training round, selecting a high-probability noise image from all original images and transformed images of the same category; and setting the loss weight of the high-probability noise anchor point, the high-probability noise image and the corresponding original image and all transformed images of the current training round to be lower than a set weight threshold in the next training round. By applying the application, the robustness of the image classification model to label noise can be improved, and the accuracy of image classification processing can be improved.

Description

Technical Field

[0001] This application relates to image classification technology, and in particular to methods and apparatus for training image classification models and methods and apparatus for image classification. Background Technology

[0002] With the advancement of neural network technology, image classification systems have been increasingly widely used.

[0003] In image classification applications, some require finer-grained image classification, meaning smaller differences and lower discriminative power between different image categories. For example, in vehicle appearance classification, brand identification is crucial. In this type of fine-grained image classification, the differences between classes are small, making classification difficult and image category labeling challenging. Incorrect labeling can lead to noise, affecting the training of the image classification model and the accuracy of image classification processing based on that model. Therefore, for fine-grained image classification, achieving high accuracy requires improving the model's robustness to label noise during training, thereby enhancing the performance and accuracy of the image classification model. However, while there are many label noise diagnostic algorithms, solutions specifically for fine-grained, high-precision processing are lacking. Summary of the Invention

[0004] This application provides a training method and apparatus for an image classification model, as well as an image classification method and apparatus, which can improve the robustness of the image classification model to label noise and improve the accuracy of image classification processing.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] A method for training an image classification model, comprising:

[0007] The original image is subjected to various types of image transformations to generate corresponding transformed images;

[0008] In each training round, feature extraction and classification prediction are performed on all the original images and the transformed images. The loss function is determined by comparing the predicted category with the labeled category. The parameters of the image classification model and the base library features of each category are updated until the training termination condition is met.

[0009] In any given training epoch, after updating the parameters for the current batch, the method further includes: determining a first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch; comparing the feature extraction result of the first transformed image sequence with the base library features of each category; if the similarity between the feature extraction result and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variances of other transformed image sequences composed of each transformed image corresponding to other original images within the same category in the current batch (excluding the first original image), then the original image corresponding to the first transformed image is determined to be a high-probability noise anchor point in the current training epoch; the first original image is any original image in the current batch.

[0010] After any current training round ends, the method further includes: for each high-probability noise anchor point in the current training round, among all original images and transformed images with the same label category, selecting an image whose feature distance to the high-probability noise anchor point is less than a first distance threshold as the high-probability noise image of the current training round; setting the loss weight of the high-probability noise anchor point and the high-probability noise image of the current training round, as well as their corresponding original images and all transformed images, to be lower than a set weight threshold in the next training round.

[0011] Preferably, the feature extraction result of the first transformed image sequence Features of the base library for each category in the image classification model Comparisons include:

[0012] Calculate the minimum average distance between the feature extraction results of the first transformed image sequence and the base library features of different categories. And determine the category h corresponding to the minimum value;

[0013] Calculate the average distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ;

[0014] Calculate the minimum distance between the feature extraction result of the first transformed image sequence and the base library features of category h. ;

[0015] Calculate the maximum distance between the feature extraction result of the first transformed image sequence and the base database features of the same category. ;

[0016] like and D 3< D4. Then, it is determined that the similarity between the feature extraction result and the features of the same category of the base database is less than the similarity between the feature extraction result and the features of different categories of the base database.

[0017] Where k is the label category of the first original image, q is the category index, i is the index of the first original image, j is the index of the transformed image in the transformed image sequence, and P is the number of transformed images in the first transformed image sequence. The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the degree of difference.

[0018] Preferably, the feature variance of the first transformed image sequence is The mean of the feature variance of the transformed image sequence composed of the transformed images corresponding to the other original images within the same category (excluding the first original image) is _____. ,in, , This represents the average value of the feature extraction results for each of the first-transformed image sequences. P' is the average value of the feature extraction results of each transformed image in the other transformed image sequence, P' is the number of transformed images in the other transformed image sequence, and u is the index of the other original image.

[0019] Preferably, after selecting the high-probability noisy image, the method further includes:

[0020] Among the original images and transformed images with the same label category, the image whose feature distance to the high-probability noise image is less than the first distance threshold and whose feature distance to the high-probability noise anchor point is less than the second distance threshold is selected as the high-probability noise image.

[0021] Preferably, the feature distance is the distance between the feature extraction results obtained by using the image classification model after any current training round to extract features from the original image.

[0022] An image classification method, comprising:

[0023] Input the image to be classified into the image classification model generated by any of the training methods described above;

[0024] The image to be classified is processed using the image classification model to determine the category to which the image belongs.

[0025] A training device for an image classification model includes: an image transformation unit, a single-round image training unit, a high-probability noise anchor point selection unit, a high-probability noise image selection unit, a weight adjustment unit, and an output unit.

[0026] The image transformation unit is used to perform various types of image transformations on the original image to generate corresponding transformed images;

[0027] The single-round image training unit is used to perform feature extraction and classification prediction on all the original images and the transformed images in each training round, determine the loss function by comparing the predicted category with the labeled category, update the parameters of the image classification model and update the base library features of each category; and notify the output unit after the training termination condition is met.

[0028] The high-probability noise anchor selection unit, in any current training round performed by the single-round image training unit, after updating the parameters for the current batch, is used to determine the first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch, compare the feature extraction result of the first transformed image sequence with the base library features of each category, and if the similarity between the feature extraction result and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variances of other transformed image sequences composed of each transformed image corresponding to other original images in the same category other than the first original image in the current batch, then the original image corresponding to the first transformed image is determined to be the high-probability noise anchor point of the current training round; the first original image is any original image in the current batch.

[0029] The high-probability noise image selection unit is configured to, after any current training round in the single-round image training unit, for each high-probability noise anchor point in the current training round, select an image whose feature distance to the high-probability noise anchor point is less than a first distance threshold from all original images and transformed images with the same label category as the high-probability noise anchor point in the current training round; and set the loss weight of the high-probability noise anchor point and the high-probability noise image in the current training round, as well as their corresponding original images and all transformed images, to be lower than a set weight threshold in the next training round.

[0030] The output module is used to output the currently generated image classification model after receiving a notification from the single-round image training module.

[0031] Preferably, in the high-probability noise anchor point selection unit, the feature extraction result of the first transformed image sequence is... Features of the base library for each category in the image classification model Comparisons include:

[0032] Calculate the minimum average distance between the feature extraction results of the first transformed image sequence and the base library features of different categories. And determine the category h corresponding to the minimum value;

[0033] Calculate the average distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ;

[0034] Calculate the minimum distance between the feature extraction result of the first transformed image sequence and the base library features of category h. ;

[0035] Calculate the maximum distance between the feature extraction result of the first transformed image sequence and the base database features of the same category. ;

[0036] like and D 3< D 4. Then, it is determined that the similarity between the feature extraction result and the features of the same category of the base database is less than the similarity between the feature extraction result and the features of different categories of the base database.

[0037] Where k is the label category of the first original image, q is the category index, i is the index of the first original image, j is the index of the transformed image in the transformed image sequence, and P is the number of transformed images in the first transformed image sequence. The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the degree of difference.

[0038] Preferably, in the high-probability noise anchor point selection unit, the feature variance of the first transformed image sequence is: The mean of the feature variance of the transformed image sequence composed of the transformed images corresponding to the other original images within the same category (excluding the first original image) is _____. ,in, , This represents the average value of the feature extraction results for each of the first-transformed image sequences. P' is the average value of the feature extraction results of each transformed image in the other transformed image sequence, P' is the number of transformed images in the other transformed image sequence, and u is the index of the other original image.

[0039] Preferably, the high-probability noise image selection unit is further configured to, after selecting the high-probability noise image, select from the original images and transformed images with the same label category an image whose feature distance to the high-probability noise image is less than the first distance threshold and whose feature distance to the high-probability noise anchor point is less than the second distance threshold, as the high-probability noise image.

[0040] Preferably, the feature distance is the distance between the feature extraction results obtained by using the image classification model after any current training round to extract features from the original image.

[0041] An image classification device includes an input unit and an inference unit;

[0042] The input unit is used to input the image to be classified into the image classification model generated by any of the above-described training devices.

[0043] The reasoning unit is used to perform reasoning processing on the image to be classified using the image classification model to determine the category to which the image to be classified belongs.

[0044] As can be seen from the above technical solution, in this application, the original image undergoes various types of image transformations to generate corresponding transformed images; all original images and transformed images are used to train the image classification model. Specifically, in each current training epoch, after updating the parameters for the current batch, a first transformed image sequence is further determined, consisting of the first transformed images corresponding to the first original image in the current batch. The feature extraction result of this first transformed image sequence is compared with the base library features of each category in the image classification model. If the similarity between the feature extraction result and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories, and the feature variance of this first transformed image sequence is less than the mean of the feature variances of other transformed image sequences consisting of the transformed images corresponding to other original images within the same category (excluding the first original image), then the original image corresponding to the first transformed image is determined as a high-probability noise anchor point for the current epoch. This method selects high-probability noise anchor points that have a very high noise probability. The current batch... Each original image is treated as the first original image and the above processing is performed to identify all high-probability noise anchors in the current training round. Next, after each training round, based on each high-probability noise anchor in that training round, images with a feature distance less than a first distance threshold are found among the original images and transformed images of the same category. These images, which are similar to the high-probability noise anchors, are used as high-probability noise images in the current training round. Since high-probability noise anchors have a high probability of being noise images, the noise probability of these similar original images is also relatively high. For the high-probability noise images and high-probability noise anchors in the current training round, as well as their corresponding original images and all transformed images, their loss weights in the next training round are set to be lower than a set weight threshold, which means that learning of these noise images is suppressed in the next training round. The above method first finds high-probability noise anchors by comparing the features of the transformed image with the features of the base database. Then, based on this, similar high-probability noise images are found, which can greatly improve the noise accuracy. At the same time, the loss weights of the found high-probability noise anchors, high-probability noise images and their corresponding original images are reduced to suppress the learning of these noise images and improve the robustness to label noise.

[0045] Furthermore, the image to be classified is input into the image classification model generated according to the above training method. The image classification model is then used to process the image and determine its category. Because the training process of the image classification model improves the accuracy of noise recognition and robustness to label noise, the trained image classification model is more accurate, and the classification results obtained using this image classification model are also more accurate. Attached Figure Description

[0046] Figure 1 This is a schematic diagram illustrating the basic process of the image classification model training method in this application;

[0047] Figure 2 This is a schematic diagram of the basic process of the image classification method in this application;

[0048] Figure 3 This is a schematic diagram illustrating the specific process of the image classification model training method in a particular embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the training structure of the image classification model in a specific embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the basic structure of the image classification model training device in this application;

[0051] Figure 6 This is a schematic diagram of the basic structure of the image classification device in this application. Detailed Implementation

[0052] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0053] Figure 1 This is a schematic diagram illustrating the basic process of the image classification model training method in this application. Figure 1 As shown, the method includes:

[0054] Step 101: Perform various types of image transformations on the original image to generate corresponding transformed images.

[0055] In the image classification model training of this application, the original image is subjected to various types of image transformations (such as flipping, rotating, etc.) to obtain transformed images. Both the original image and the transformed images are used as training images for model training. Each type of transformation can generate a corresponding transformed image, and multiple types of image transformations generate multiple corresponding transformed images.

[0056] Step 102: In each training round, feature extraction and classification prediction are performed on all original and transformed images. The loss function is determined by comparing the predicted category with the labeled category. The parameters of the image classification model and the base library features of each category are updated until the training termination condition is met.

[0057] Training an image classification model requires several training epochs. In each epoch, feature extraction, classification prediction, loss function calculation, model parameter updates, and base database feature updates are performed on all original and transformed images. The base database features represent the intra-class centers for each category, and classification prediction is based on these features. The operations in these training epochs can adopt the processing methods used in the training of existing image classification models.

[0058] In this application, the following steps 102a and 102b are further added to each of the above training rounds to determine the original image and the transformed image with a higher probability of noise, and to suppress the image classification model from learning these images.

[0059] Step 102a: In any current training round, after updating the parameters for the current group, determine the first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch. Compare the feature extraction results of the first transformed image sequence with the base library features of each category in the image classification model. If the similarity between the feature extraction results and the base library features of the same category is less than the similarity between the feature extraction results and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variance of other transformed image sequences composed of each transformed image corresponding to the original images other than the first original image in the same category, then determine the original image corresponding to the first transformed image as the high-probability noise anchor point of the current training round.

[0060] The first original image is any original image in the current batch.

[0061] In each training epoch, due to resource constraints, not all original and transformed images are typically input into the image classification model at once. Instead, they are input in groups for training. Each group is called a batch, and the images in a batch are input into the image classification model for one iteration of training. The specific grouping method is the same as existing methods, and it is usually necessary to ensure that each group includes original and transformed images of multiple categories.

[0062] This step is used to identify high-probability noise anchors, which are the original images with a high probability of noise, after each batch of iterative training. The processing after each batch of iterative training is the same; the following explanation uses the processing of the current batch as an example.

[0063] Specifically, each original image in the current batch is processed as follows. Taking the processing of one of the original images as an example, let's call it the first original image:

[0064] 1) Determine each transformed image corresponding to the first original image in the current batch (to distinguish it from other transformed images, the transformed image corresponding to the first original image in the current batch will be referred to as the first transformed image below), and form a first transformed image sequence from all the first transformed images;

[0065] For a given original image, all its corresponding transformed images may be distributed across different batches. Therefore, each batch typically does not include all the transformed images corresponding to the original image, but it usually includes multiple transformed images corresponding to the original image.

[0066] 2) Compare the feature extraction results of the first transformed image sequence with the base library features of each category in the image classification model;

[0067] The feature extraction result of the first transformed image sequence is the feature extraction result of each first transformed image in the first transformed image sequence. This feature extraction result can be the feature extraction result obtained in the forward processing of this iteration training. The feature extraction result will be referred to as the feature below.

[0068] In training image classification models, if the original image is noisy, its transformed image is often also noisy. Therefore, this application compares the features of multiple transformed images corresponding to the same original image with features from the same and different categories in the base library to select the original image with a higher noise probability. Since this comparison is performed by integrating the features of multiple transformed images, the selection result has higher confidence. In this application, the categories of both the original image and the transformed image refer to the label category.

[0069] 3) If the similarity between the feature extraction result of the first transformed image sequence and the base database feature of the same category is less than the similarity between the feature extraction result and the base database feature of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variance of other transformed image sequences composed of each transformed image corresponding to other original images in the same category in the current batch (excluding the first original image), then the first original image is determined to be a high-probability noise anchor point.

[0070] Specifically, if the similarity between the feature extraction result of the first transformed image sequence and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories (i.e., condition 1 is satisfied), it means that the first original image is less similar to the same category and more similar to other categories, that is, the inter-class distance is small and the intra-class distance is large. Therefore, the probability that the first original image corresponding to the first transformed sequence is a noisy image is relatively high.

[0071] Simultaneously, the variance of the feature extraction results of all first transformed images in the first transformed image sequence is calculated, and it is called the feature variance of the first transformed image sequence. Other original images in the current batch that are of the same category as the first original image are identified. For each other original image, the variance of the feature extraction results of all transformed images corresponding to that other original image in the current batch is calculated, and it is called the feature variance of the other transformed image sequence corresponding to the other original image. The mean of the feature variance of each other transformed image sequence is also calculated. If the feature variance of the first transformed image sequence is less than the mean of the feature variance of each other transformed image sequence (i.e., condition 2 is satisfied), it indicates that the feature dispersion of the first transformed image sequence is greater than the average dispersion of the same category, that is, the feature stability of the first transformed image sequence is low. Then, the probability that the first original image corresponding to the first transformed image sequence is a noisy image is high.

[0072] When the first transformed image sequence satisfies both conditions 1 and 2, the probability that the first original image is a noisy image is very high. This first original image is then identified as the high-probability noise anchor point for the current training round. As mentioned earlier, since this conclusion is drawn from the comparison of multiple transformed images, the confidence level of the result is very high. Based on this high-probability noise anchor point, other images with high noise probabilities can be selected through subsequent processing.

[0073] Through the processing described in 1) to 3), it can be determined whether the first original image is a high-probability noise anchor point. If each original image in the current batch is processed in the same way, it can be determined whether each original image is a high-probability noise anchor point, that is, all high-probability noise anchor points in the current batch are selected.

[0074] By processing each batch in the current training epoch in the same way, all high-probability noise anchors in the current training epoch can be selected. Next, through the processing in step 102b, based on all high-probability noise anchors in the current training epoch, other original and transformed images with high noise probabilities are further determined.

[0075] Step 102b: After any current training round ends, for each high-probability noise anchor point in the current training round, among all original images and transformed images with the same label category, select the image whose feature distance to the high-probability noise anchor point is less than a first distance threshold as the high-probability noise image of the current training round; set the loss weight of all high-probability noise anchor points and their corresponding transformed images, and the loss weight of high-probability noise images and their corresponding original images and transformed images in the next training round to be lower than the set weight threshold.

[0076] Here, after each training epoch, high-probability noise anchors are used to find other high-probability noise images. The processing is the same for each training epoch; the following explanation uses the current training epoch as an example.

[0077] Through the processing in step 102a above, all high-probability noise anchors in the current training round are determined. The same processing is performed on each high-probability noise anchor. Taking the current high-probability noise anchor as an example, the processing is as follows:

[0078] For the current high-probability noise anchor point, calculate the feature distance between each original image and transformed image of the same category and the current high-probability noise anchor point. That is, the distance between the feature extraction result of the current high-probability noise anchor point and the feature extraction results of the original image and transformed image of the same category. If the feature distance is less than the set first distance threshold, it means that the corresponding original image or transformed image is similar to the high-probability noise anchor point. Then the original image or transformed image is also likely to be a noise image. Such original images and transformed images are used as high-probability noise images in the current training round.

[0079] By using the above method, other high-probability noise images can be found based on high-probability noise anchor points, effectively improving the accuracy of noise recognition. Since the original image and its corresponding transformed image are very similar in noise characteristics, if image A is a noise image, its corresponding original image and all its corresponding transformed images (hereinafter referred to as the original image and all transformed images corresponding to noise image A) are usually also noise images. Based on this, in this application, all high-probability noise anchor points and their corresponding transformed images, and high-probability noise images and their corresponding original images and all transformed images (hereinafter referred to as high-probability noise anchor points and high-probability noise images and their corresponding original images and all transformed images) in the current training round are highly likely to be noise images. Therefore, in the next training round, the loss weights corresponding to these images are set below a set weight threshold. This reduces the participation of noise images in the parameter updates of the image classification model in the next training round. Similarly, the same processing is performed in each training round, thereby suppressing the image classification model's learning of these noise images and improving the robustness of the image classification model to label noise.

[0080] At this point, Figure 1 The training process for the image classification model shown is now complete.

[0081] Figure 2 This is a schematic diagram illustrating the basic flowchart of the image classification method in this application. This image classification method is based on… Figure 1 The image classification model generated by the training method shown is used. Figure 2 As shown, this image classification method includes:

[0082] Step 201: Input the image to be classified into the image classification model.

[0083] The image classification model here is an image classification model generated using the image classification model training method of this application.

[0084] Step 202: Use an image classification model to process the input image to be classified and determine the category to which the image belongs.

[0085] At this point, Figure 2 The image classification method flowchart shown has ended.

[0086] The image classification model training method and image classification method described in this application can be applied to various image classification applications, such as vehicle classification applications. In particular, for fine-grained image classification applications such as vehicle brand recognition, it can effectively improve the robustness of fine-grained image classification models to label noise, and improve the performance and accuracy of image classification models.

[0087] The training method of the image classification model in this application is illustrated below through specific embodiments. Figure 3 This is a schematic diagram illustrating the specific process of the image classification model training method in the embodiments of this application. Figure 4 This is a schematic diagram of the overall structure of the training method, as shown below. Figure 3 and Figure 4 As shown, the method includes:

[0088] Step 301: Perform various types of image transformations on the original image to generate corresponding transformed images.

[0089] In this embodiment, using Let i represent the original image, where i is the index of the original image and N is the total number of original images used in the training process.

[0090] Image transformations can include M methods such as random cropping, flipping, rotation, and brightness augmentation, corresponding to a single original image. M transformed images are obtained through image transformation, and each transformed image utilizes... express, A sequence of M transformed images is formed, where j represents the index of the transformed image in the sequence.

[0091] Step 302, the first training round (i.e. Figure 4 Set the training epoch 1 in the current training round.

[0092] Step 303: In the current training round, the first group batch is taken as the current batch.

[0093] Step 304: Input the original image and transformed image of the current batch into the feature extraction network for feature extraction, and perform classification prediction on the feature extraction results. Determine the loss function by comparing the predicted category with the labeled category, and update the parameters of the image classification model.

[0094] The current batch includes original and transformed images that can be sampled from all original and transformed images. Both the original and transformed images in the current batch are used as training images and input into the image classification model. The image classification model includes a feature extraction network, which utilizes the feature extraction network (… Figure 4 The backbone network in the image is used to perform feature extraction processing to obtain the feature extraction results, also known as image features.

[0095] Image features are used for classification prediction. The loss function is determined by comparing the predicted category with the labeled category, and the parameters of the image classification model are updated.

[0096] The aforementioned processes of feature extraction, classification prediction, loss function determination, and parameter updating can all be implemented using existing methods, and will not be elaborated upon here.

[0097] This completes the training iteration for the current batch.

[0098] Step 305: For all original images included in the current batch, determine whether they are high-probability noise anchors.

[0099] As mentioned earlier, when determining whether a certain original image is a high-probability noise anchor point, on the one hand, the image features of the transformed image sequence corresponding to the original image are compared with the base library features of each category, and on the other hand, the feature variance of the transformed image sequence corresponding to the original image is compared with the mean feature variance of other transformed image sequences of the same category.

[0100] First, this embodiment introduces the specific method for comparing the image features of the transformed image sequence corresponding to the original image with the base library features of each category. Taking the first original image as an example, assuming the labeled category of the first original image is k, the first original image... The corresponding transformed image in the current batch There are P in total, that is The comparison of image features of the first transformed image sequence corresponding to the first original image with the base library features of each category specifically includes:

[0101] 1) Calculate the image features of each image in the first transformed image sequence. Minimum of the average distance between the base features of different categories ;in, Represents the transformed image The image features, which can typically be transformed images from the forward processing of the current batch. The feature extraction results, where q is the category index. The base features of category q are represented by MIN, which means taking the minimum value. The calculation here can determine the category h corresponding to the minimum value of the average distance, which is the other category that is closest to the first original image.

[0102] As can be seen from the above calculation method, D 1 can reflect the inter-class distance between the first transformed image sequence and the base library features of other categories. Since the inter-class distance between the first transformed image sequence and other categories can reflect the inter-class distance between the first original image and other categories, therefore, D 1 can also reflect the inter-class distance between the first original image and the base library features of other categories, and is a physical quantity that reflects the inter-class distance;

[0103] 2) Calculate the image features of each image in the first transformed image sequence. Base library features of the same category k FD k average distance between ;

[0104] As can be seen from the above calculation method, D 2 can reflect the intra-class distance between the first Yuanshi image and the features of the same category's base library, and is a physical quantity that reflects the intra-class distance;

[0105] 3) Calculate the image features of each image in the first transformed image sequence. Base library features with category h FD h minimum distance between ;

[0106] As can be seen from the above calculation method, D 3 represents the inter-class distance between the first original image and the nearest class, which is another physical quantity that reflects the inter-class distance;

[0107] 4) Calculate the image features of each image in the first transformed image sequence. Maximum distance between the base features of the same category ;

[0108] As can be seen from the above calculation method, D 4 represents the maximum intra-class distance between the first original image and the same category, which is another physical quantity that reflects intra-class distance;

[0109] when D 3< D 4 and When the result is greater than the inter-class distance when measured from two different perspectives, it can be determined that the similarity between the image features of the first transformed image sequence and the base library features of the same category is less than the similarity between the image features of the first transformed image sequence and the base library features of different categories. Therefore, from the perspective of inter-class distance, the first original image is very likely a noisy image; where, The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the difference between the two categories; a value typically between 1.5 and 4 is acceptable. This is achieved through... D 1 and D Introduced in the comparison of 2 It can compensate for the impact of differences between two categories when comparing intra-class and inter-class distances.

[0110] The above describes the specific processing for comparing the image features of the first transformed image sequence with the base library features of each category in this embodiment. Of course, in practical applications, the comparison method is not limited to the above. Other comparisons of the same category and different categories can also be used to determine whether the similarity between the image features of the first transformed image sequence and the base library features of the same category is less than the similarity between the image features of the first transformed image sequence and the base library features of different categories, thereby determining whether the first original image is very likely a noisy image.

[0111] The following describes the method for comparing characteristic variances in this embodiment. Specifically, it includes:

[0112] 1) Calculate the feature variance of the first transformed image sequence. ;in, This represents the average value of the image features of each of the first transformed image sequences;

[0113] 2) Identify all other original images in the same category within the current batch, excluding the first original image. Calculate the original image The feature variance of the corresponding other transformed image sequences is Among them, the other transformed image sequences are the original images. The sequence of transformed images formed by the various transformed images in the current batch. P' is the average value of the image features of each transformed image in the other transformed image sequence, P' is the number of transformed images in the other transformed image sequence, and u is the index of the other original image;

[0114] 3) Calculate the mean of the feature variances of each of the other transformed image sequences. .

[0115] when When the stability of the first transformed image sequence is lower than the average stability of the same category, it means that the stability is poor. Correspondingly, the stability of the first original image is also poor. Therefore, from the perspective of stability, the first original image is very likely to be a noisy image.

[0116] Considering both the inter-class spacing and stability perspectives mentioned above, if both perspectives determine that the first original image is highly likely to be a noisy image, then the condition is met. , D 3< D 4 and If so, the first original image is determined to be a high-probability noise anchor point.

[0117] For each original image in the current batch, determine whether it is a high-probability noise anchor point in the manner described above.

[0118] Step 306: Take the next batch as the current batch and return to step 304 until all batches have been processed.

[0119] By repeatedly executing steps 304-305, all batches are processed, and at this point, all high-probability noise anchors for the current training epoch can be obtained. Next, step 307 is executed.

[0120] Step 307: Update the base database features for each category.

[0121] The specific update process can be carried out using existing methods, which will not be elaborated here.

[0122] Step 308: Determine whether the training termination condition is met. If yes, end the process; otherwise, continue to step 309.

[0123] This concludes the basic processing for the current training epoch. The next step is to determine if the training termination conditions are met, such as the loss function reaching a set value or the set number of training epochs being reached. If the training termination conditions are met, the trained image classification model is output; otherwise, step 309 is executed to find other high-probability noisy images, and the loss weights of the corresponding noisy images are reset for use in the next training epoch.

[0124] Step 309: For each high-probability noise anchor point in the current training round, select a high-probability noise image from all original and transformed images with the same label category; set the loss weights of all high-probability noise anchor points and high-probability noise images and their corresponding original and transformed images in the current training round to be lower than the set weight threshold in the next training round.

[0125] Based on the high-probability noise anchor points of the current training epoch, other high-probability noise images are searched. The processing for searching other noise images for all high-probability noise anchor points is the same. Here, we still use a current high-probability noise anchor point as an example to illustrate the specific search method.

[0126] In the simplest way, among all the original and transformed images that are of the same category as the current high-probability noise anchor, the image whose feature distance to the current high-probability noise anchor is less than a first distance threshold can be selected as the high-probability noise image for the current training round.

[0127] Here, feature distance refers to the distance between the image features of a high-probability noise anchor and the image features of other images of the same category. The image features of the high-probability noise anchor and the image features of other images can be obtained by extracting features from the corresponding images using the feature extraction network of the current image classification model. When the feature distance is less than a first distance threshold, it indicates that the corresponding image has a high feature similarity to the current high-probability noise anchor. Therefore, given that the current high-probability noise anchor is a noisy image, the corresponding image is also likely to be a noisy image, and thus it is considered a high-probability noise image for the current training epoch.

[0128] In addition to the methods described above for searching high-probability noisy images, a further feature diffusion search approach can be employed. For the selected high-probability noisy images, among the original and transformed images with the same labeling category, images whose feature distance to the selected high-probability noisy image is less than a first distance threshold and whose feature distance to the current high-probability noisy anchor point is less than a second distance threshold are also selected as high-probability noisy images. This allows for further searching for other high-probability noisy images based on the selected high-probability noisy images. Simultaneously, by controlling the feature distance to the current high-probability noisy anchor point, the diffusion search is prevented from gradually moving away from the current high-probability noisy anchor point, ensuring that the searched high-probability noisy images surround the current high-probability noisy anchor point, thus guaranteeing high confidence of the search results. This allows for the acquisition of all high-probability noisy images for the current training round.

[0129] In practical applications, when performing search processing, the image features of the high-probability noise images found can be added to the "high-probability noise feature library" of the category to which the current high-probability noise anchor point belongs. When the high-probability noise feature library for that category no longer needs to add features, the search for that category stops. This process is repeated for each category to generate its own high-probability noise feature library. If a category does not have a corresponding high-probability noise anchor point, that category is skipped. After searching all categories, the original and transformed images corresponding to the features in all high-probability noise feature libraries constitute all the high-probability noise images for the current training round.

[0130] After identifying all high-probability noise images in the current training round, the high-probability noise anchor points and their corresponding transformed images, as well as the high-probability noise images and their corresponding original images and transformed images (i.e., ...) are then used to identify all high-probability noise anchor points in the current training round and their corresponding transformed images. Figure 4 The loss weight of the suspected noise sample sequence in the next training round is set to be lower than the set weight threshold, for example, it can be set to be lower than 0.5. In this way, the participation of these high probability noise anchors and high probability noise images in the model parameter update in the next training round is reduced, the learning of these images by the image classification model is suppressed, and the robustness of the image classification model to label noise is effectively improved.

[0131] Step 310: Set the next training round as the current training round and return to step 303.

[0132] Continue processing for the next training round until the entire training is completed, ending this process.

[0133] The above describes the specific implementation of the model training method in a specific embodiment of this application. In this model training method, by comparing the features of the transformed image sequence with the base database features of each category, and comparing the feature variance of the transformed image sequence with the average feature variance of the same category, high-probability noise anchors that meet the set noise probability requirements are first identified. Then, starting from these high-probability noise anchors, other high-probability noise images are determined through diffusion search, significantly improving the accuracy of noise recognition, especially suitable for fine-grained image classification model training. Simultaneously, improvements are made in two stages of image classification model training: in the first stage, high-probability noise anchors are determined by evaluating the model's generalization ability; in the second stage, high-probability noise images are determined through contrastive learning and feature similarity diffusion principles. Contrastive loss and classification loss functions are designed, and the loss weights of high-probability noise anchors, high-probability noise images, their corresponding original images, and all transformed images are dynamically adjusted during training to suppress the learning of high-probability noise and improve the model's robustness to label noise. The image classification model trained in the above manner exhibits significantly improved performance, and image classification processing based on this model can also effectively improve accuracy, especially for fine-grained image classification models.

[0134] The above describes the specific implementation of the image classification model training method and the image classification method in this application. This application also provides a training device for the image classification model and an image classification device, which can be used to implement the above training method and image classification method, respectively.

[0135] Figure 5 This is a schematic diagram of the basic structure of the training device for the image classification model in this application. Figure 5As shown, the device includes: an image transformation unit, a single-round image training unit, a high-probability noise anchor point selection unit, a high-probability noise image selection unit, a weight adjustment unit, and an output unit.

[0136] The image transformation unit is used to perform various types of image transformations on the original image to generate the corresponding transformed image.

[0137] The single-round image training unit is used to perform feature extraction and classification prediction on all original and transformed images in each training round. It determines the loss function by comparing the predicted class with the labeled class, updates the parameters of the image classification model and the base library features of each class, and notifies the output unit after the training termination condition is met.

[0138] The high-probability noise anchor selection unit, in any current training round of the single-round image training unit, after updating the parameters for the current batch, is used to determine the first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch. The feature extraction results of the first transformed image sequence are compared with the base library features of each category. If the similarity between the feature extraction results and the base library features of the same category is less than the similarity between the feature extraction results and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variances of other transformed image sequences composed of each transformed image corresponding to other original images within the same category (excluding the first original image) in the current batch, then the original image corresponding to the first transformed image is determined as the high-probability noise anchor point of the current training round; the first original image is any original image in the current batch.

[0139] The high-probability noise image selection unit is used to, after any current training round of the single-round image training unit, for each high-probability noise anchor point in the current training round, select images whose feature distance to the high-probability noise anchor point is less than a first distance threshold from all original images and transformed images with the same label category as the high-probability noise anchor point in the current training round as the high-probability noise image in the current training round; and set the loss weight of the high-probability noise anchor point and high-probability noise image in the current training round and their corresponding original images and all transformed images in the next training round to be lower than a set weight threshold.

[0140] The output module is used to output the currently generated image classification model after receiving a notification from the single-round image training module.

[0141] Optionally, in the high-probability noise anchor point selection unit, the feature extraction results of the first transformed image sequence are used. Baseline features of each category in the image classification model The comparison process can specifically include:

[0142] Calculate the minimum average distance between the feature extraction results of the first transformed image sequence and the base library features of different categories. And determine the category h corresponding to the minimum value;

[0143] Calculate the average distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ;

[0144] Calculate the minimum distance between the feature extraction result of the first transformed image sequence and the base library features of category h. ;

[0145] Calculate the maximum distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ;

[0146] like and D 3< D 4. Then, the similarity between the feature extraction results and the features of the same category of the base database is less than the similarity between the feature extraction results and the features of different categories of the base database.

[0147] Where k is the label category of the first original image, q is the category index, i is the index of the first original image, j is the index of the transformed image in the transformed image sequence, and P is the number of transformed images in the first transformed image sequence. The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the difference.

[0148] Optionally, in the high-probability noise anchor point selection unit, the feature variance of the first transformed image sequence is: The mean of the feature variance of the sequence of transformed images formed by the transformed images corresponding to the other original images within the same category (excluding the first original image) is _____. ,in, , This represents the average value of the feature extraction results for each of the first-transformed image sequences. P' is the average value of the feature extraction results of each transformed image in the other transformed image sequence, P' is the number of transformed images in the other transformed image sequence, and u is the index of the other original image.

[0149] Optionally, the high-probability noise image selection unit can be further configured to, after selecting a high-probability noise image, select, from the original image and the transformed image with the same label category, an image whose feature distance to the high-probability noise image is less than a first distance threshold and whose feature distance to the high-probability noise anchor point is less than a second distance threshold, as a high-probability noise image.

[0150] Optionally, the feature distance is the distance between the feature extraction results obtained by using the image classification model to extract features from the original image after any current training round.

[0151] Figure 6 This is a schematic diagram of the basic structure of the image classification device provided in this application, such as... Figure 6 As shown, the image classification device includes an input unit and an inference unit;

[0152] The input unit is used to input the image to be classified into the above-mentioned unit. Figure 5 The image classification model generated by the training device shown;

[0153] The inference unit is used to perform inference processing on the image to be classified using the image classification model to determine the category to which the image belongs.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for an image classification model, characterized in that, include: The original image is subjected to various types of image transformations to generate corresponding transformed images; In each training round, feature extraction and classification prediction are performed on all the original images and the transformed images. The loss function is determined by comparing the predicted category with the labeled category. The parameters of the image classification model and the base library features of each category are updated until the training termination condition is met. In any given training epoch, after updating the parameters for the current batch, the method further includes: determining a first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch; comparing the feature extraction result of the first transformed image sequence with the base library features of each category; if the similarity between the feature extraction result and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variances of other transformed image sequences composed of each transformed image corresponding to other original images in the same category within the current batch (excluding the first original image), then the original image corresponding to the first transformed image is determined to be a high-probability noise anchor point for the current training epoch; the first original image is any original image in the current batch. After any current training round ends, the method further includes: for each high-probability noise anchor point in the current training round, among all original images and transformed images with the same label category, selecting an image whose feature distance to the high-probability noise anchor point is less than a first distance threshold as the high-probability noise image of the current training round; setting the loss weight of the high-probability noise anchor point and the high-probability noise image of the current training round, as well as their corresponding original images and all transformed images, to be lower than a set weight threshold in the next training round.

2. The method according to claim 1, characterized in that, The feature extraction result of the first transformed image sequence Features of the base library for each category in the image classification model Comparisons include: Calculate the minimum average distance between the feature extraction results of the first transformed image sequence and the base library features of different categories. And determine the category h corresponding to the minimum value; Calculate the average distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ; Calculate the minimum distance between the feature extraction result of the first transformed image sequence and the base library features of category h. ; Calculate the maximum distance between the feature extraction result of the first transformed image sequence and the base database features of the same category. ; If and D3 < D4, it is determined that the similarity degree between the feature extraction result and the feature of the same-category database is less than the similarity degree between the feature extraction result and the feature of the different-category database; Where k is the label category of the first original image, q is the category index, i is the index of the first original image, j is the index of the transformed image in the transformed image sequence, and P is the number of transformed images in the first transformed image sequence. The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the degree of difference.

3. The method according to claim 1 or 2, characterized in that, The feature variance of the first transformed image sequence is The mean of the feature variance of the transformed image sequence composed of the transformed images corresponding to the other original images within the same category (excluding the first original image) is _____. ,in, , The average value of the feature extraction results for each first-transformed image sequence. P is the average value of the feature extraction results of each transformed image in the other transformed image sequence, P' is the number of transformed images in the first transformed image sequence, P' is the number of transformed images in the other transformed image sequences, and u is the index of the other original images.

4. The method according to claim 1, characterized in that, After selecting the high-probability noisy image, the method further includes: Among the original images and transformed images with the same label category, the image whose feature distance to the high-probability noise image is less than the first distance threshold and whose feature distance to the high-probability noise anchor point is less than the second distance threshold is selected as the high-probability noise image.

5. The method according to claim 1 or 4, characterized in that, The feature distance is the distance between the feature extraction results obtained by using the image classification model after any current training round to extract features from the original image.

6. An image classification method, characterized in that, include: The image to be classified is input into the image classification model generated by any of the training methods described in claims 1 to 4; The image to be classified is processed using the image classification model to determine the category to which the image belongs.

7. A training device for an image classification model, characterized in that, include: The system includes an image transformation unit, a single-round image training unit, a high-probability noise anchor point selection unit, a high-probability noise image selection unit, a weight adjustment unit, and an output unit. The image transformation unit is used to perform various types of image transformations on the original image to generate corresponding transformed images; The single-round image training unit is used to perform feature extraction and classification prediction on all the original images and the transformed images in each training round, determine the loss function by comparing the predicted category with the labeled category, update the parameters of the image classification model and update the base library features of each category; and notify the output unit after the training termination condition is met. The high-probability noise anchor selection unit, in any current training round of the single-round image training unit, after updating the parameters for the current batch, is used to determine the first transformed image sequence composed of each first transformed image corresponding to the first original image in the current batch. The feature extraction result of the first transformed image sequence is compared with the base library features of each category. If the similarity between the feature extraction result and the base library features of the same category is less than the similarity between the feature extraction result and the base library features of different categories, and the feature variance of the first transformed image sequence is less than the mean of the feature variances of other transformed image sequences composed of each transformed image corresponding to other original images within the same category (excluding the first original image) in the current batch, then the original image corresponding to the first transformed image is determined to be the high-probability noise anchor point of the current training round; the first original image is any original image in the current batch. The high-probability noise image selection unit is configured to, after any current training round in the single-round image training unit, for each high-probability noise anchor point in the current training round, select an image whose feature distance to the high-probability noise anchor point is less than a first distance threshold from all original images and transformed images with the same label category as the high-probability noise anchor point in the current training round; and set the loss weight of the high-probability noise anchor point and the high-probability noise image in the current training round, as well as their corresponding original images and all transformed images, to be lower than a set weight threshold in the next training round. The output unit is used to output the currently generated image classification model after receiving a notification from the single-round image training module.

8. The training device according to claim 7, characterized in that, In the high-probability noise anchor point selection unit, the feature extraction result of the first transformed image sequence is... Features of the base library for each category in the image classification model Comparisons include: Calculate the minimum average distance between the feature extraction results of the first transformed image sequence and the base library features of different categories. And determine the category h corresponding to the minimum value; Calculate the average distance between the feature extraction results of the first transformed image sequence and the base database features of the same category. ; Calculate the minimum distance between the feature extraction result of the first transformed image sequence and the base library features of category h. ; Calculate the maximum distance between the feature extraction result of the first transformed image sequence and the base database features of the same category. ; If and D3 < D4, it is determined that the similarity degree between the feature extraction result and the feature of the same-category database is less than the similarity degree between the feature extraction result and the feature of the different-category database; Where k is the label category of the first original image, q is the category index, i is the index of the first original image, j is the index of the transformed image in the transformed image sequence, and P is the number of transformed images in the first transformed image sequence. The parameters are the degree of difference between the preset categories k and h. The larger the value, the greater the degree of difference.

9. The training device according to claim 7 or 8, characterized in that, In the high-probability noise anchor point selection unit, the feature variance of the first transformed image sequence is: The mean of the feature variance of the transformed image sequence composed of the transformed images corresponding to the other original images within the same category (excluding the first original image) is _____. ,in, , The average value of the feature extraction results for each first-transformed image sequence. P is the average value of the feature extraction results of each transformed image in the other transformed image sequence, P' is the number of transformed images in the first transformed image sequence, P' is the number of transformed images in the other transformed image sequences, and u is the index of the other original images.

10. The training device according to claim 7, characterized in that, The high-probability noise image selection unit is further configured to, after selecting the high-probability noise image, select, from the original images and transformed images with the same label category, an image whose feature distance to the high-probability noise image is less than the first distance threshold and whose feature distance to the high-probability noise anchor point is less than the second distance threshold, as the high-probability noise image.

11. The training device according to claim 7 or 10, characterized in that, The feature distance is the distance between the feature extraction results obtained by using the image classification model after any current training round to extract features from the original image.

12. An image classification device, characterized in that, Includes input units and inference units; The input unit is used to input the image to be classified into the image classification model generated by the training device according to any one of claims 7 to 11. The reasoning unit is used to perform reasoning processing on the image to be classified using the image classification model to determine the category to which the image to be classified belongs.

Images