An Image Classification Method Based on Noise Suppression and Multi-Scale Feature Fusion

Abstract

The present invention provides an image classification method based on noise suppression and multi-scale feature fusion, which includes establishing a data set, establishing a fine-grained image classification model, and establishing a loss function. The training set, validation set, and test set are used to train, validate, and test the fine-grained image classification model respectively to obtain a trained fine-grained image classification model and perform image classification to obtain a classification result. By introducing a noise suppression module and a multi-scale feature fusion module, the present invention aims to reduce background noise while obtaining possible fine-grained category discriminative feature representations in foreground regions of different scales, so that the fine-grained feature representations converge shallow features and deep features, improving the performance of the model for fine-grained classification tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of general image data processing or generation, and particularly to an image classification method based on noise suppression and multi-scale feature fusion. Background Art

[0002] Fine-grained image classification aims to accurately identify different sub-categories with similar appearances, such as cell categories, birds, car brands, and aircraft types. Therefore, fine-grained image classification has important practical applications in various fields such as medical diagnosis, biodiversity conservation, and intelligent transportation, and has received extensive attention.

[0003] However, the common inter-class similarity and relatively high intra-class differences in fine-grained image classification pose significant challenges to this task. The fine-grained feature representation often has the following problems: (1): The background region introduces noise to the representation of fine-grained category discriminative features; (2): The association between global semantic information and local detail information fails to be effectively established; hindering the ability of fine-grained category discriminative feature representation.

[0004] Therefore, a fine-grained image classification method based on noise suppression and multi-scale feature fusion is needed. Summary of the Invention

[0005] The present invention is to solve the problem of fine-grained image classification, and provides an image classification method based on noise suppression and multi-scale feature fusion. Based on the multi-scale feature representation of a feature extraction network and a feature pyramid network, by introducing a noise suppression module and a multi-scale feature fusion module, it aims to reduce background noise and improve the model's ability to represent fine-grained category discriminative features.

[0006] The present invention provides an image classification method based on noise suppression and multi-scale feature fusion, including the following steps:

[0007] S1. Establish a dataset for a fine-grained image classification model in a natural scene, and divide it into a training set, a validation set, and a test set;

[0008] S2. Establish a fine-grained image classification model, where the fine-grained image classification model includes a noise suppression module and a multi-scale feature fusion module;

[0009] The noise suppression module includes a feature pyramid network and a classifier. The feature pyramid network obtains feature maps of different scales according to the input image , where is the -th layer feature map; the classifier suppresses the background noise features of according to the confidence threshold to obtain the foreground features of the -th layer feature map Output the multi-scale feature fusion module at most;

[0010] The multi-scale feature fusion module is based on foreground features of different scales Improve the learnable and learnable information fusion ability between them, where is used for classification, is used for information interaction with ;

[0011] S3. Establish a loss function :

[0012] ;

[0013] where is the true label information; is the predicted class probability distribution, , is the number of classes;

[0014] S4. Use the training set, validation set, and test set to train, validate, and test the fine-grained image classification model respectively to obtain a trained fine-grained image classification model;

[0015] S5. Use the trained fine-grained image classification model to perform image classification to obtain a classification result, and an image classification method based on noise suppression and multi-scale feature fusion is completed.

[0016] In the image classification method based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred mode, in step S1, the fine-grained image classification model dataset includes medical images.

[0017] In the image classification method based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred mode, in step S2, the feature pyramid network includes bottom-up and top-down paths, and then feature maps of different scales are obtained , The output ends of

[0018] ;

[0019] where is the number of channels of the -th layer feature map, is the height of the -th layer feature map, is the width of the -th layer feature map.

[0020] ​A method for image classification based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred embodiment, 。

[0021] A method for image classification based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred embodiment, in step S2, the classification head of the classifier is:

[0022] ;

[0023] Wherein, is the weight of the classifier corresponding to the -th layer feature map; is the bias of the classifier corresponding to the -th layer feature map;

[0024] 。

[0025] A method for image classification based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred embodiment, the confidence corresponding to the -th layer feature map is:

[0026] ;

[0027] The confidence threshold is the threshold of the confidence 。

[0028] A method for image classification based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred embodiment, is a vector that fills the background area of with 0.

[0029] A method for image classification based on noise suppression and multi-scale feature fusion according to the present invention, as a preferred embodiment, in step S2, and The information fusion between is:

[0030] ;

[0031] Wherein,

[0032] ;

[0033] ;

[0034] 。

[0035] is the positional encoding of , is The position encoding incorporates spatial position information and hierarchical position information; is a function; is and the height.

[0036] In a preferred embodiment of the image classification method based on noise suppression and multi-scale feature fusion according to the present invention, in step S2, the method for improving the and information fusion ability by the cross-attention mechanism is as follows:

[0037] ;

[0038] Among them, is the index of the attention head, M is the total number of attention heads, represents the weight of the attention head with index , is the index of the sampling point, K is the total number of sampling points, is at the sampling point of the attention head , feature map level of attention weight, and , is at the sampling point of the attention head , feature map level of coordinate compensation;

[0039] is a fine-grained feature extraction function based on deformable convolution, and by introducing deformable convolution, the and information interaction is improved, and then the fine-grained discriminative feature representation is improved.

[0040] In a preferred embodiment of the image classification method based on noise suppression and multi-scale feature fusion according to the present invention, in step S3,

[0041] ;

[0042] Among them, is a fully connected layer; is a function.

[0043] The present invention has the following advantages:

[0044] (1) By introducing a noise suppression module and a multi-scale feature fusion module, the present invention aims to reduce background noise while obtaining fine-grained category discriminative feature representations in foreground regions at different scales, so that the fine-grained feature representations converge shallow features and deep features, improving the performance of the model for fine-grained classification tasks.

[0045] (2) Compared with previous fine-grained classification models, the fine-grained feature representation of the present invention combines noise suppression and multi-scale feature fusion. In addition, the present invention also provides a framework for end-to-end fine-grained feature representation, reducing the training cost of the model and enabling it to be applied to real scenarios, having certain practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 is a flowchart of an image classification method based on noise suppression and multi-scale feature fusion;

[0047] Figure 2 is a schematic structural diagram of the noise suppression module of an image classification method based on noise suppression and multi-scale feature fusion;

[0048] Figure 3 is a flowchart of step S2 of an image classification method based on noise suppression and multi-scale feature fusion;

[0049] Figure 4a is an image to be classified in an image classification method based on noise suppression and multi-scale feature fusion Figure 1 ;

[0050] Figure 4b is an image to be classified in an image classification method based on noise suppression and multi-scale feature fusion Figure 2 ;

[0051] Figure 4c is an image to be classified in an image classification method based on noise suppression and multi-scale feature fusion Figure 3 ;

[0052] Figure 4d is an image to be classified, Figure 4, in an image classification method based on noise suppression and multi-scale feature fusion;

[0053] Figure 4e is an image to be classified, Figure 5, in an image classification method based on noise suppression and multi-scale feature fusion. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Embodiment 1

[0055] An image classification method based on noise suppression and multi-scale feature fusion, as Figure 1 shown, is specifically implemented according to the following steps:

[0056] S1. Establish a dataset of fine-grained image models in natural scenes and divide it into a training set, a validation set, and a test set;

[0057] S2. Establish a model;

[0058] Step S2 is specifically implemented as follows:

[0059] 1) Noise suppression module. The network structure of the noise suppression module is as Figure 2 shown. First, the input image passes through the feature extraction network, and the feature pyramid network obtains feature maps of different scales , where , represents the -th layer feature map, , and respectively represent the number of channels, height, and width of the -th layer feature map. The classification head can be expressed as:

[0060] (1)

[0061] In the above formula, represents the weight of the classifier corresponding to the -th layer feature map; represents the bias of the classifier corresponding to the -th layer feature map;

[0062] , represents the logic corresponding to the -th layer feature map, where represents the number of categories;

[0063] The confidence can be expressed as:

[0064] (2)

[0065] In the above formula, represents the confidence corresponding to the -th layer feature map;

[0066] According to the confidence threshold of , the background noise features are suppressed, and the foreground features are expressed as , . Simply put, The background area is filled with a 0 vector, that is ;

[0067] 2) Multi-scale feature fusion module, based on foreground features of different scales , through the multi-scale feature fusion module, improve the ability of the model to represent fine-grained category discrimination features. Specifically, the shallow features have detailed information about the exact position of the target, including contour, edge, color, texture, and shape features, and the shallow features are crucial for fine-grained recognition. Specifically, the morphological classification of cells often synthesizes the structural changes of cell membranes, cytoplasm, organelles, and cell nuclei to complete the fine-grained classification of cells; as the network downsamples or the number of convolutions increases, the receptive field of the deep features gradually increases, resulting in rich semantic information; the multi-scale feature fusion module improves the information fusion ability between 、 , where represents the query for classification, represents the query for information interaction with , aiming to improve the information fusion ability between and through the self-attention mechanism. The information fusion between and can be expressed as:

[0068] (3)

[0069] (4) (5)

[0070] (6)

[0071] In the above formula, 、 respectively represent the position encodings of and , integrates spatial position information and hierarchical position information; represents function;

[0072] represents and height;

[0073] is the linear transformation weight function of Q, is the linear transformation weight function of K, is the linear transformation weight function of V, is the layer normalization function.

[0074] Improve through cross-attention mechanism and the information fusion ability, which can be expressed as:

[0075] (7)

[0076] In the above formula, represents the index of the attention head, represents the weight of the attention head with index , represents the index of the sampling point, represents at the attention head , feature map level the sampling point of the attention weight and satisfies , represents at the attention head , feature map level the sampling point of the coordinate compensation. The cross-attention module improves the information interaction ability of the model between different levels of feature maps;

[0077] Step 3, establish a loss function;

[0078] Step 3 is specifically implemented as follows:

[0079] (8)

[0080] (9)

[0081] In the above formula, represents the fully connected layer; represents function; represents the predicted class probability distribution; represents the true label information;

[0082] S4. Use the training set, validation set, and test set to train, validate, and test the model respectively to obtain a fine-grained image classification model.

[0083] S5. Use the fine-grained image classification model to perform image classification to obtain the classification result. The classification process is as Figure 3 shown.

[0084] The classification results of this embodiment are shown in the following table:

[0085]

[0086] It can be seen that the fine-grained image classification model accurately classifies cell images by applying this method.

[0087] As mentioned above, the above are only the preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered by the protection scope of the present invention.

Claims

1. An image classification method based on noise suppression and multi-scale feature fusion, characterized in that: The following steps are involved: S1. Establish a fine-grained image classification model dataset in natural scenes and divide it into training set, validation set and test set; S2. Establishing a fine-grained image classification model, wherein the fine-grained image classification model includes a noise suppression module and a multi-scale feature fusion module; The noise suppression module includes a feature pyramid network and a classifier. The feature pyramid network obtains feature maps of different scales according to the input image. ,in, For the Layer feature map; The output ends of are respectively connected to a classifier, and the classifier is classified according to the confidence threshold right The background noise characteristics are suppressed and The background area is filled with 0 vectors to obtain the Foreground features of layer feature maps And output to the multi-scale feature fusion module; The multi-scale feature fusion module is based on foreground features of different scales. Improving learnable and learnable The information fusion ability between them is improved through the cross attention mechanism and Information fusion capability, including: For classification, Used with Information interaction; and The information fusion between them is: ； in, ； ； ； for The position code, for The position code, It integrates spatial location information and hierarchical location information; for function; for and Height, is the linear transformation weight function of Q, is the linear transformation weight function of K, is the linear transformation weight function of V, is the layer normalization function; S3. Predict category probability classification and establish loss function : ； ； in, To predict the class probability distribution, , is the number of categories; is a fully connected layer; for function, is the real label information; S4, using the training set, the validation set and the test set to respectively train, validate and test the fine-grained image classification model to obtain a trained fine-grained image classification model; S5. Use the trained fine-grained image classification model to perform image classification to obtain classification results, and an image classification method based on noise suppression and multi-scale feature fusion is completed.

2. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S1, the fine-grained image classification model dataset includes medical images.

3. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S2, the feature pyramid network includes bottom-up and top-down paths to obtain feature maps of different scales. ; ； in, For the The number of channels of the layer feature map, For the The height of the layer feature map, For the The width of the layer feature map.

4. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: 。 5. The image classification method based on noise suppression and multi-scale feature fusion according to claim 3, characterized in that: In step S2, the classification head of the classifier is: ； in, For the The layer feature map corresponds to the weight of the classifier; For the The layer feature map corresponds to the bias of the classifier; 。 6. The image classification method based on noise suppression and multi-scale feature fusion according to claim 5, characterized in that: No. Confidence corresponding to the layer feature map for: ； The confidence threshold Confidence The threshold value.

7. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: For the general A vector that fills the background area with 0.

8. The image classification method based on noise suppression and multi-scale feature fusion according to claim 1, characterized in that: In step S2, the cross attention mechanism is used to improve and The information fusion capability method is: ； in, is the index of the attention head, M is the total number of attention heads, Indicates that the index is The weight of the attention head, is the index of the sampling point, K is the total number of sampling points, for In the attention head , feature map level Sampling point The attention weight of , for In the attention head , feature map level Sampling point Coordinate compensation; It is a fine-grained feature extraction function based on deformable convolution. By introducing deformable convolution, the and information interaction, thereby improving Fine-grained discriminative feature representation.

Images