A fine-grained recognition method under network supervision

Abstract

The application discloses a fine-grained identification method under network supervision, comprising the following steps: training a double network model by using a data set, and constructing a bidirectional mean-teacher module by using the trained double network model; according to the output result of an image passing through an encoder, distinguishing whether the image is a clean sample of a momentum encoder; inputting the image into the momentum encoder, dividing the image into a clean sample or a noise sample according to the prediction of the image passing through the first momentum encoder, updating the second encoder to obtain a final identification model; and inputting the image into the final identification model for prediction; the final model of the application integrates the parameters of the previous generation model, improves the prediction quality, and is more stable in updating, is less affected by noise, and improves the accuracy of the prediction result.

Description

Technical Field

[0001] This invention relates to the field of fine-grained identification technology of network data, specifically a fine-grained identification method under network supervision. Background Technology

[0002] Constructing fine-grained datasets is a challenging task, requiring domain experts to correctly classify fine-grained subclasses by identifying subtle differences. To reduce reliance on manual annotation, there is an increasing tendency to collect images of relevant categories directly from the internet to build network datasets and use them for training. However, such network datasets often contain significant amounts of data noise, and using them directly in training can lead to overfitting, thus affecting accuracy. Fine-grained network datasets typically contain two types of noise: open-set label noise and closed-set label noise. Open-set label noise usually refers to noise that does not belong to any class in the fine-grained domain, while closed-set noise refers to images with incorrect labels within the fine-grained domain.

[0003] Several methods have been proposed to address common label noise, including sample selection, soft labeling, and related loss functions. While these methods have achieved good classification results, they suffer from the following problems: 1) Existing methods primarily rely on high-quality data-driven deep learning to distinguish subordinate categories, but data collection and labeling are costly, often time-consuming and laborious in fine-grained image recognition, resulting in low accuracy. 2) Furthermore, when dealing with noisy images, it is still impossible to transform closed-set labeled noisy images into accurate training images, meaning these images cannot be effectively utilized. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a fine-grained recognition method under network supervision, which aims to improve the accuracy of fine-grained recognition in network datasets.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a fine-grained identification method under network supervision, comprising the following steps:

[0006] Step S1: Collect images to construct a dataset;

[0007] Step S2: Train the dual-network model using the dataset to obtain the trained dual-network model; the dual-network model consists of a first encoder and a second encoder; the first encoder and the second encoder are two ResNet50 network models with different parameters;

[0008] Step S3: Construct a bidirectional mean-teacher module using the trained dual-network model;

[0009] Step S4: Based on the output of the image after passing through the first momentum encoder, distinguish whether the image is a clean sample of the first momentum encoder. Similarly, based on the output of the image after passing through the second momentum encoder, distinguish whether the image is a clean sample of the second momentum encoder.

[0010] Step S5: Input the images in the dataset into the first momentum encoder and the second momentum encoder. Based on the prediction obtained by the first momentum encoder, the image is divided into clean samples or noisy samples of the first momentum encoder. Based on the prediction obtained by the second momentum encoder, the image is divided into clean samples or noisy samples of the second encoder.

[0011] Step S6: Update the second encoder based on the clean samples divided by the first momentum encoder, and update the first encoder based on the clean samples divided by the second momentum encoder. After the update is completed, a fine-grained image recognition model based on bidirectional mean-teacher under network supervision is obtained.

[0012] Step S7: Input the image into a fine-grained image recognition model based on bidirectional mean-teacher under network supervision for prediction.

[0013] Furthermore, the specific process of training the dual-network model using the dataset is as follows: [Image x...] j Input the data into the dual-network model and update the dual-network model; then input the dataset... Let x be the training data with noisy labels, where x j Represents an image in the dataset, y j ∈{0,1} C Let represent the single-hot label corresponding to the j-th image in class C; N is the number of data sets; the parameters of the dual-network model are updated using the following formula:

[0014]

[0015]

[0016] In the formula, This represents the cross-entropy loss function of the first encoder; y represents the cross-entropy loss function of the second encoder; j c x represents j For the label in category C; f1 c (x j ) indicates that the first encoder for x j Prediction for category C; f2 c (x j ) indicates that the second encoder for xj The prediction for category C.

[0017] Furthermore, the specific process of step S3 is as follows: updating the parameters of the first momentum encoder and the second momentum encoder:

[0018]

[0019] In the formula, ∈1 represents the parameters of the first momentum encoder; ∈′1 represents the parameters of the first momentum encoder in the previous iteration; ∈2 represents the parameters of the second momentum encoder; ∈′2 represents the parameters of the second momentum encoder in the previous iteration; γ∈[0,1] represents the target decay rate; ξ1 represents the parameters of the first encoder; ξ2 represents the parameters of the second encoder.

[0020] The consistency cost J1 of the first encoder is defined as the expected distance between the predictions of the first encoder and the first momentum encoder, and the consistency cost J2 of the second encoder is defined as the expected distance between the predictions of the second encoder and the second momentum encoder.

[0021]

[0022] In the formula, f1 c (x j ) indicates that the first encoder for x j Prediction for category c; f2 c (x j ) indicates that the second encoder for x j Prediction for category c; fm1 c (x j ) indicates that the first momentum encoder for x j Prediction for category c; fm2 c (x j ) indicates that the second momentum encoder for x j Prediction for category c.

[0023] Furthermore, the specific process of step S4 is as follows: [Image x] j The data are fed into the first momentum encoder and the second momentum encoder respectively, and the partitioning loss of the first momentum encoder is calculated. The partitioning loss of the second momentum encoder and

[0024]

[0025] In the formula, The cross-entropy loss of the first momentum encoder;

[0026] This represents the entropy loss of the first momentum encoder.

[0027] The cross-entropy loss of the second encoder;

[0028] The entropy loss of the second encoder;

[0029] The first encoder and the second encoder obtain the image x from equation (5). j The resulting partitioning loss for image x j Categorize:

[0030]

[0031] In the formula, A represents a mini-batch of images; τ represents the low loss rate; A c1 and A c2 Both are composed of τ|A|, which minimizes the loss of the first and second momentum encoders; therefore, A is... c1 As the clean sample selected by the first momentum encoder, A c2 The clean samples selected by the second momentum encoder.

[0032] Furthermore, the specific process of step S6 is as follows: update the second encoder with the clean samples obtained from the first momentum encoder, and update the first encoder with the clean samples obtained from the second momentum encoder; the classification loss of the clean samples of the first encoder and the second encoder is:

[0033]

[0034] In the formula, This represents the loss of the clean samples from the first encoder; This represents the loss of the clean samples from the second encoder.

[0035] Furthermore, the final loss function of the network-supervised fine-grained image recognition model based on bidirectional mean-teacher is:

[0036]

[0037] In the formula, and denoted as the loss of the first encoder and the loss of the second encoder in a fine-grained image recognition model based on bidirectional mean-teacher under network supervision.

[0038] Furthermore, the specific process of collecting images to construct the dataset is as follows: use keywords to perform text-based image retrieval in Google Chrome, and use web crawling technology to crawl the images returned by the keywords to construct the dataset.

[0039] Compared with existing technologies, the present invention has the following advantages: The parameters of the fine-grained image recognition model based on bidirectional mean-teacher provided by the present invention are composed of the parameter set of the previous generation version. This integration improves the prediction quality, allows the model to collect information after each iteration rather than after each cycle, and improves the output of all layers. Therefore, the update of this model is more stable and less affected by noise, thereby improving the accuracy of the prediction results. The clean samples obtained by prediction are more consistent with the real labels, which promotes the training effect of the model. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the fine-grained image recognition model based on bidirectional mean-teacher under network supervision according to the present invention. Detailed Implementation

[0041] like Figure 1 As shown, the present invention provides a technical solution: a fine-grained identification method under network supervision, comprising the following steps:

[0042] Step S1: Use keywords to perform text-based image retrieval in Google Chrome, and use web crawling technology to crawl the images returned by the keywords to build a dataset;

[0043] Step S2: Train the dual-network model using the dataset to obtain the trained dual-network model; the dual-network model consists of a first encoder and a second encoder; the first encoder and the second encoder are two ResNet50 network models with different parameters;

[0044] The specific process of training the dual-network model in step S2 is as follows:

[0045] Image x j Input the data into the dual-network model and update the dual-network model; then input the dataset... Let x be the training data with noisy labels, where x j Represents an image in the dataset, y j ∈{0,1} C Let represent the single-hot label corresponding to the j-th image in class C; N is the number of data sets; the parameters of the dual-network model are updated using the following formula:

[0046]

[0047]

[0048] In the formula, This represents the cross-entropy loss function of the first encoder; y represents the cross-entropy loss function of the second encoder;j c x represents j For the label in category C; f1 c (x j ) indicates that the first encoder for x j Prediction for category C; f2 c (x j ) indicates that the second encoder for x j The prediction for category C.

[0049] Step S3: Construct a bidirectional mean-teacher module using the dual-network model trained in step S2;

[0050] In step S3, the parameters of the first momentum encoder and the second momentum encoder are updated using the following formula:

[0051]

[0052] In the formula, ∈1 represents the parameters of the first momentum encoder; ∈′1 represents the parameters of the first momentum encoder in the previous iteration; ∈2 represents the parameters of the second momentum encoder; ∈′2 represents the parameters of the second momentum encoder in the previous iteration; γ∈[0,1] represents the target decay rate; ξ1 represents the parameters of the first encoder; ξ2 represents the parameters of the second encoder.

[0053] The consistency cost J1 of the first encoder is defined as the expected distance between the predictions of the first encoder and the first momentum encoder, and the consistency cost J2 of the second encoder is defined as the expected distance between the predictions of the second encoder and the second momentum encoder.

[0054]

[0055] In the formula, f1 c (x j ) indicates that the first encoder for x j Prediction for category c; f2 c (x j ) indicates that the second encoder for x j Prediction for category c; fm1 c (x j ) indicates that the first momentum encoder for x j Prediction for category c; fm2 c (x j ) indicates that the second momentum encoder for x j The prediction for category c. By minimizing the consistency costs J1 and J2 during training, the first encoder is forced to generate outputs consistent with the second momentum encoder, resulting in a more stable and reliable feature representation for both the first and second encoders.

[0056] Step S4: Based on the output of the image after passing through the first momentum encoder, distinguish whether the image is a clean sample of the first momentum encoder. Similarly, based on the output of the image after passing through the second momentum encoder, distinguish whether the image is a clean sample of the second momentum encoder.

[0057] When image x j The data are fed into the first momentum encoder and the second momentum encoder, respectively. The partitioning loss of the first momentum encoder is calculated using the following loss function. The partitioning loss of the second momentum encoder and

[0058]

[0059] In the formula, The cross-entropy loss of the first momentum encoder;

[0060] This represents the entropy loss of the first momentum encoder.

[0061] The cross-entropy loss of the second encoder;

[0062] The entropy loss is the second encoder's entropy loss; the cross-entropy loss reveals the degree of agreement between the prediction and the label distribution, while the entropy loss reflects the confidence of the prediction. The method of combining cross-entropy loss and entropy loss to select low-loss samples is not only the easiest for the network to learn, but also the easiest to predict, thus ensuring to the greatest extent that the samples in the low-loss are accurately labeled.

[0063] The first encoder and the second encoder obtain the image x from equation (5). j The obtained segmentation loss is used to analyze the image x using the following formula. j Categorize:

[0064]

[0065] In the formula, A represents a mini-batch of images; τ represents the low loss rate; A c1 and A c2 It consists of τ|A|, which minimizes the loss of the first and second momentum encoders; therefore, A is... c1 and A c2 Clean samples selected for the first and second momentum encoders.

[0066] Step S5: Input the images in the dataset into the first momentum encoder and the second momentum encoder. Based on the prediction obtained by the first momentum encoder, the image is divided into clean samples or noisy samples of the first momentum encoder. Based on the prediction obtained by the second momentum encoder, the image is divided into clean samples or noisy samples of the second encoder.

[0067] Step S6: Update the second encoder based on the clean samples divided by the first momentum encoder, and update the first encoder based on the clean samples divided by the second momentum encoder. After the update is completed, a fine-grained image recognition model based on bidirectional mean-teacher under network supervision is obtained.

[0068] The specific process of step S6 is as follows: The clean samples obtained from the first momentum encoder are used to update the second encoder, and the clean samples obtained from the second momentum encoder are used to update the first encoder, in order to filter out different errors caused by label noise; the classification loss of the clean samples from the first encoder and the second encoder is:

[0069]

[0070] In the formula, This represents the loss of the clean samples from the first encoder; This represents the loss of the clean samples from the second encoder;

[0071] The final loss function of the network-supervised fine-grained image recognition model based on bidirectional mean-teacher is:

[0072]

[0073] In the formula, and denoted as the loss of the first encoder and the loss of the second encoder in a fine-grained image recognition model based on bidirectional mean-teacher under network supervision.

[0074] Step S7: Input the image into a fine-grained image recognition model based on bidirectional mean-teacher under network supervision for prediction.

[0075] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fine-grained recognition method under network supervision, characterized in that... Includes the following steps: Step S1: Collect images to construct a dataset; Step S2: Train the dual-network model using the dataset to obtain the trained dual-network model; the dual-network model consists of a first encoder and a second encoder; the first encoder and the second encoder are two ResNet50 network models with different parameters; Step S3: Construct a bidirectional mean-teacher module using the trained dual-network model; Step S4: Based on the output of the image after passing through the first momentum encoder, distinguish whether the image is a clean sample of the first momentum encoder; based on the output of the image after passing through the second momentum encoder, distinguish whether the image is a clean sample of the second momentum encoder. Step S5: Input the images in the dataset into the first momentum encoder and the second momentum encoder. Based on the prediction obtained by the first momentum encoder, the image is divided into clean samples or noisy samples of the first momentum encoder. Based on the prediction obtained by the second momentum encoder, the image is divided into clean samples or noisy samples of the second encoder. Step S6: Update the second encoder based on the clean samples divided by the first momentum encoder, and update the first encoder based on the clean samples divided by the second momentum encoder. After the update is completed, a fine-grained image recognition model based on bidirectional mean-teacher under network supervision is obtained. Step S7: Input the image into a fine-grained image recognition model based on bidirectional mean-teacher under network supervision for prediction; The specific process of step S4 is as follows: [Image] The data are fed into the first momentum encoder and the second momentum encoder respectively, and the partitioning loss of the first momentum encoder is calculated. The partitioning loss of the second momentum encoder and : ； In the formula, The cross-entropy loss of the first momentum encoder; This represents the entropy loss of the first momentum encoder. The cross-entropy loss of the second encoder; The entropy loss of the second encoder; Representing the The image is in The corresponding single hot tag on the class; Indicates the first momentum encoder for No. Category prediction; Indicates the second momentum encoder for No. Category prediction; The first encoder and the second encoder process the image using a segmentation loss. Categorize: ； In the formula, For a small batch of images; To achieve a low loss rate; and Both are minimized by the first momentum encoder and the second momentum encoder. | |Composition; therefore, As the clean samples selected by the first momentum encoder, As clean samples selected by the second momentum encoder; The final loss function of the network-supervised fine-grained image recognition model based on bidirectional mean-teacher is: ； In the formula, and These represent the losses of the first encoder and the second encoder in a fine-grained image recognition model based on bidirectional mean-teacher under network supervision, respectively. This represents the consistency cost of the first encoder; This represents the consistency cost of the second encoder; This represents the loss of the clean samples from the first encoder; This represents the loss of the clean samples from the second encoder.

2. The fine-grained recognition method under network supervision according to claim 1, characterized in that: The specific process of training the dual-network model using the dataset is as follows: [Image...] Input the data into the dual-network model and update the dual-network model; then input the dataset... This is represented as training data with noise labels, where, This represents an image in the dataset; Let be the number of data sets; update the parameters of the dual-network model using the following formula: ； ； In the formula, This represents the cross-entropy loss function of the first encoder; This represents the cross-entropy loss function of the second encoder; express For the Category labels; Indicates the first encoder for The Category prediction; Indicates the second encoder for The Category prediction.

3. The fine-grained recognition method under network supervision according to claim 2, characterized in that: The specific process of step S3 is as follows: updating the parameters of the first momentum encoder and the second momentum encoder: ； In the formula, Indicates the parameters of the first momentum encoder; This represents the parameters of the first momentum encoder in the previous iteration; These represent the parameters of the second momentum encoder; This represents the parameters of the second momentum encoder in the previous iteration; Indicates the target decay rate; Indicates the parameters of the first encoder; Indicates the parameters of the second encoder; The consistency cost of the first encoder Defined as the expected distance between the predictions of the first encoder and the first momentum encoder, the consistency cost of the second encoder is... Defined as the expected distance between the predictions of the second encoder and the second momentum encoder: ； In the formula, Indicates the first encoder for No. Category prediction; Indicates the second encoder for No. Category prediction.

4. The fine-grained recognition method under network supervision according to claim 3, characterized in that: The specific process of step S6 is as follows: update the second encoder with the clean samples obtained from the first momentum encoder, and update the first encoder with the clean samples obtained from the second momentum encoder; the classification loss of the clean samples of the first encoder and the second encoder is: 。 5. The fine-grained recognition method under network supervision according to claim 1, characterized in that: The specific process of collecting images to build the dataset is as follows: use keywords to perform text-based image retrieval in Google Chrome, and use web crawling technology to crawl the images returned by the keywords to build the dataset.

Images