Image classification method based on noise suppression and multi-scale feature fusion

Abstract

The invention provides an image classification method based on noise suppression and multi-scale feature fusion, and the method comprises the steps: building a data set, building a fine-grained image classification model, building a loss function, and classifying images. And respectively training, verifying and testing the fine-grained image classification model by using the training set, the verification set and the test set to obtain a trained fine-grained image classification model, and carrying out image classification to obtain a classification result. According to the invention, the noise suppression module and the multi-scale feature fusion module are introduced to reduce background noise and obtain the possibility of fine-grained category discrimination feature representation in foreground regions of different scales, so that the fine-grained feature representation gathers shallow features and deep features, and the accuracy of the fine-grained category discrimination feature representation is improved. And the performance of the model for a fine-grained classification task is improved.

Description

Technical Field

[0001] The present invention relates to the general field of image data processing or generation technology, and in particular to an image classification method based on noise suppression and multi-scale feature fusion. Background Art

[0002] Fine-grained image classification aims to accurately identify different subcategories with similar appearance, such as cell categories, birds, car brands, and aircraft types. Therefore, fine-grained image classification has important practical applications in various fields such as medical diagnosis, biodiversity conservation, and intelligent transportation, and has received widespread attention.

[0003] However, the common inter-class similarity and relatively high intra-class variance in fine-grained image classification pose a major challenge to this task. This is because fine-grained feature representation often has the following problems: (1) The background region introduces noise into the representation of fine-grained category discriminative features; (2) The association between global semantic information and local detail information is not effectively established, which hinders the ability to represent fine-grained category discriminative features.

[0004] Therefore, a fine-grained image classification method based on noise suppression and multi-scale feature fusion is needed. Summary of the invention

[0005] The present invention aims to solve the problem of fine-grained image classification and provides an image classification method based on noise suppression and multi-scale feature fusion. It is based on the multi-scale feature representation of feature extraction network and feature pyramid network. By introducing noise suppression module and multi-scale feature fusion module, it aims to reduce background noise and improve the model's ability to represent fine-grained category discrimination features.

[0006] The present invention provides an image classification method based on noise suppression and multi-scale feature fusion, comprising the following steps: S1. Establish a fine-grained image classification model dataset in natural scenes and divide it into training set, validation set and test set; S2. Establish a fine-grained image classification model, which includes a noise suppression module and a multi-scale feature fusion module; The noise suppression module includes a feature pyramid network and a classifier. The feature pyramid network obtains feature maps of different scales according to the input image. ,in, For the Layer feature map; classifier based on confidence threshold right The background noise characteristics are suppressed to obtain the Foreground features of layer feature maps And output to the multi-scale feature fusion module; The multi-scale feature fusion module is based on foreground features of different scales. Improving learnable and learnable The information fusion capability between For classification, Used with Information interaction; S3. Establish loss function : ; in, is the real label information; To predict the class probability distribution, , is the number of categories; S4, using the training set, the validation set and the test set to respectively train, validate and test the fine-grained image classification model to obtain a trained fine-grained image classification model; S5. Use the trained fine-grained image classification model to classify images and obtain classification results. An image classification method based on noise suppression and multi-scale feature fusion is completed.

[0007] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, in step S1, the fine-grained image classification model data set includes medical images.

[0008] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is preferably implemented in step S2, wherein the feature pyramid network includes bottom-up and top-down paths to obtain feature maps of different scales. , The output ends of are connected to a classifier respectively; ; in, For the The number of channels of the layer feature map, For the The height of the layer feature map, For the The width of the layer feature map.

[0009] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is a preferred embodiment. .

[0010] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, in step S2, the classification head of the classifier is: ; in, For the The layer feature map corresponds to the weight of the classifier; For the The layer feature map corresponds to the bias of the classifier; .

[0011] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, Confidence corresponding to the layer feature map for: ; Confidence Threshold Confidence The threshold value.

[0012] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is a preferred embodiment. For the general A vector that fills the background area with 0.

[0013] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, in step S2, and The information fusion between them is: ; in, ; ; ; for The position code, for The position code, It integrates spatial location information and hierarchical location information; for function; for and height.

[0014] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, in step S2, the cross attention mechanism is used to improve and The information fusion capability method is: ; in, is the index of the attention head, M is the total number of attention heads, Indicates that the index is The weight of the attention head, is the index of the sampling point, K is the total number of sampling points, for In the attention head , feature map level The sampling point The attention weight of , for In the attention head , feature map level The sampling point Coordinate compensation; It is a fine-grained feature extraction function based on deformable convolution. By introducing deformable convolution, the and information interaction, thereby improving Fine-grained discriminative feature representation.

[0015] The image classification method based on noise suppression and multi-scale feature fusion described in the present invention is, as a preferred embodiment, in step S3, ; in, is a fully connected layer; for function.

[0016] The present invention has the following advantages: (1) The present invention introduces a noise suppression module and a multi-scale feature fusion module to reduce background noise while obtaining the possibility of fine-grained category discrimination feature representation in foreground areas of different scales, so that the fine-grained feature representation brings together shallow features and deep features, thereby improving the performance of the model for fine-grained classification tasks.

[0017] (2) Compared with previous fine-grained classification models, the fine-grained feature representation of the present invention combines noise suppression and multi-scale feature fusion. In addition, the present invention also provides an end-to-end fine-grained feature representation framework, which reduces the training cost of the model and makes it possible to apply it to real scenarios, thus having certain practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of an image classification method based on noise suppression and multi-scale feature fusion; Figure 2A schematic diagram of the noise suppression module structure of an image classification method based on noise suppression and multi-scale feature fusion; Figure 3 is a flow chart of step S2 of an image classification method based on noise suppression and multi-scale feature fusion; Figure 4a An image classification method based on noise suppression and multi-scale feature fusion is to be classified Figure 1 ; Figure 4b An image classification method based on noise suppression and multi-scale feature fusion is to be classified Figure 2 ; Figure 4c An image classification method based on noise suppression and multi-scale feature fusion is to be classified Figure 3 ; Figure 4d An image classification method based on noise suppression and multi-scale feature fusion is to be classified in Figure 4; Figure 4e Figure 5 is an image classification method based on noise suppression and multi-scale feature fusion. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Example 1

[0020] An image classification method based on noise suppression and multi-scale feature fusion, such as Figure 1 As shown, the specific implementation steps are as follows: S1. Establish a dataset of fine-grained image models in natural scenes and divide it into training set, validation set and test set; S2, build the model; Step S2 is specifically implemented as follows: 1) Noise suppression module. The network structure of the noise suppression module is as follows: Figure 2 As shown, first, the input image passes through the feature extraction network, and the feature pyramid network obtains feature maps of different scales. ,in , Indicates Layer feature map, , and Respectively represent The number of channels, height and width of the layer feature map, the classification head can be expressed as: (1); In the above formula, Indicates The weight of the classifier corresponding to the layer feature map; Indicates The bias of the classifier corresponding to the layer feature map; , Indicates The logic corresponding to the layer feature map, where Indicates the number of categories; The confidence can be expressed as: (2); In the above formula, Indicates The confidence corresponding to the layer feature map; according to The confidence threshold , the background noise feature is suppressed, and the foreground feature is expressed as , In simple terms, The background area is filled with 0 vectors, that is, ; 2) Multi-scale feature fusion module, based on foreground features of different scales The multi-scale feature fusion module is used to improve the model's ability to represent fine-grained category discrimination features. Specifically, shallow features have detailed information about the precise location of the target, including contours, edges, colors, textures, and shape features, and shallow features are crucial for fine-grained recognition. Specifically, the morphological classification of cells often integrates the structural changes of cell membranes, cytoplasm, organelles, and nuclei to complete the fine-grained classification of cells; as the number of network downsampling or convolution increases, the receptive field of deep features gradually increases, resulting in rich semantic information; the multi-scale feature fusion module introduces a learnable , ,in Represents the query used for classification, Representation and The query of information interaction aims to improve and The information fusion capability between and The information fusion between can be expressed as: (3); (4); (5); (6); In the above formula, , Respectively and The position code, It integrates spatial location information and hierarchical location information; express function; express and Height; is a linear function, is the layer normalization function.

[0021] Improved by cross-attention mechanism and The information fusion capability can be expressed as: (7); In the above formula, represents the index of the attention head, Indicates that the index is The weight of the attention head, Represents the index of the sampling point, express In the attention head , feature map level The sampling point The attention weights and satisfy , express In the attention head , feature map level The sampling point The coordinate compensation and cross attention module improve the model's ability to interact with feature maps of different levels; Step 3: Establish a loss function; Step 3 is implemented as follows: (8); (9); In the above formula, represents a fully connected layer; express function; Represents the predicted category probability distribution; Indicates the real label information; S4. Use the training set, validation set and test set to train, validate and test the model respectively to obtain a fine-grained image classification model.

[0022] S5. Use the fine-grained image classification model to classify the image and obtain the classification result. The classification process is as follows: Figure 3 shown.

[0023] The classification results of this embodiment are shown in the following table:

[0024] It can be seen that the fine-grained image classification model uses this method to accurately classify cell images.

[0025] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An image classification method based on noise suppression and multi-scale feature fusion, characterized in that: The following steps are involved: S1. Establish a fine-grained image classification model dataset in natural scenes and divide it into training set, validation set and test set; S2. Establishing a fine-grained image classification model, wherein the fine-grained image classification model includes a noise suppression module and a multi-scale feature fusion module; The noise suppression module includes a feature pyramid network and a classifier. The feature pyramid network obtains feature maps of different scales according to the input image. ,in, For the Layer feature map; the classifier is based on the confidence threshold right The background noise characteristics are suppressed to obtain the Foreground features of layer feature maps And output to the multi-scale feature fusion module; The multi-scale feature fusion module is based on foreground features of different scales. Improving learnable and learnable The information fusion capability between For classification, Used with Information interaction; S3. Establish loss function : ； in, is the real label information; To predict the class probability distribution, , is the number of categories; S4, using the training set, the validation set and the test set to respectively train, validate and test the fine-grained image classification model to obtain a trained fine-grained image classification model; S5. Use the trained fine-grained image classification model to perform image classification to obtain classification results, and an image classification method based on noise suppression and multi-scale feature fusion is completed.

2. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S1, the fine-grained image classification model dataset includes medical images.

3. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S2, the feature pyramid network includes bottom-up and top-down paths to obtain feature maps of different scales. , The output ends of are connected to a classifier respectively; ； in, For the The number of channels of the layer feature map, For the The height of the layer feature map, For the The width of the layer feature map.

4. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: 。 5. The image classification method based on noise suppression and multi-scale feature fusion according to claim 3, characterized in that: In step S2, the classification head of the classifier is: ； in, For the The layer feature map corresponds to the weight of the classifier; For the The layer feature map corresponds to the bias of the classifier; 。 6. The image classification method based on noise suppression and multi-scale feature fusion according to claim 5, characterized in that: No. Confidence corresponding to the layer feature map for: ； The confidence threshold Confidence The threshold value.

7. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: For the general A vector that fills the background area with 0.

8. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S2, and The information fusion between them is: ； ； ； ； for The position code, for The position code, It integrates spatial location information and hierarchical location information; for function; for and Height, is a linear function, is the layer normalization function.

9. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S2, the cross attention mechanism is used to improve and The information fusion capability method is: ； in, is the index of the attention head, M is the total number of attention heads, Indicates that the index is The weight of the attention head, is the index of the sampling point, K is the total number of sampling points, for In the attention head , feature map level Sampling point The attention weight of , for In the attention head , feature map level Sampling point Coordinate compensation; Fine-grained feature extraction function based on deformable convolution is improved by introducing deformable convolution and information interaction, thereby improving Fine-grained discriminative feature representation.

10. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S3, ； in, is a fully connected layer; for function.

Images