Fine-grained image classification method based on foreground and background multi-level division and feature fusion

Abstract

The present invention relates to a fine-grained image classification method based on foreground-background multi-level division and feature fusion, the method uses a backbone network to obtain the importance distribution of different areas in the detection image, and segments the image by setting a threshold to obtain a foreground image and a multi-level background image; the segmented image is preprocessed and input into the backbone network to obtain a foreground feature and multiple background features; two sets of weights are obtained based on the similarity between each background feature and the foreground feature, and the distance between the original image of each background feature and the foreground image, and the foreground feature and each background feature are weighted twice, and all weighted features are fused and sent to a classifier to obtain a classification result. The present invention adopts foreground-background multi-level division and feature weighted fusion, so that the classification model can more accurately capture the strong discriminant area in the image, while extracting effective information in the image background and removing noise, effectively improving the classification accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of fine-grained image classification, and in particular to a fine-grained image classification method based on foreground-background multi-level division and feature fusion. Background Art

[0002] Fine-grained image classification (FGVC) is a visual classification task that aims to distinguish different subcategories of the same category, such as distinguishing different types of birds in bird images, or different types of flowers in flower images. Fine-grained image classification has a wide range of research needs and application scenarios in both industry and academia. If low-cost fine-grained image recognition can be achieved with the help of computer vision technology, it will be of great significance to both academia and industry.

[0003] The similarities between classes and the differences within classes pose a huge challenge to achieving accurate classification. To address this challenge, the current main approach is to identify highly discriminative regions in the image and extract features from these regions for more accurate classification. A common view is that the background part of the image does not play a positive role in classification and may even interfere with image classification. Most existing work focuses on improving performance by identifying the most discriminative regions as accurately as possible and extracting the most significant features from them. However, these methods ignore the background regions that occupy a considerable part of the image content, resulting in a large amount of information in the image not being fully utilized. In addition, this approach may require precise manual annotations, which increases costs.

[0004] Therefore, it is necessary to propose a fine-grained image classification method based on foreground-background multi-level division and feature weighted fusion, which can utilize the information in the background to perform more accurate fine-grained classification of images. At the same time, it is also necessary to reduce the impact of noise data in the background. By allowing the model to be trained end-to-end in a weakly supervised learning manner, the complexity and training cost of the model can be reduced, thereby improving the practicality of the fine-grained classification method. Summary of the invention

[0005] In view of this, the present invention provides a fine-grained image classification method based on multi-level division of foreground and background and feature fusion, so as to solve the technical problems in the prior art that insufficient attention is paid to background data, effective information in the background is not extracted and utilized, and model classification cost is high and accuracy is low.

[0006] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a fine-grained image classification method based on multi-level division of foreground and background and feature fusion, comprising:

[0008] Acquire a detection image, input the detection image into a preset backbone network, and obtain attention intensity distribution values ​​of different areas in the detection image based on the attention mechanism of the backbone network;

[0009] Segmenting the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image;

[0010] Preprocessing the foreground image and the multi-level background image, inputting the preprocessed foreground image and the multi-level background image into a preset backbone network to obtain a foreground feature and multiple background features;

[0011] A first set of weights is obtained based on the similarity between each background feature and the foreground feature, and the foreground feature and each background feature are weighted according to the first set of weights to obtain a first set of weighted features;

[0012] A second set of weights is obtained based on the distance between each level of background image and foreground image, and the first set of weighted features is weighted according to the second set of weights to obtain a second set of weighted features;

[0013] The second group of weighted features is subjected to feature fusion, and the classification result of the detection image is obtained according to the fused features.

[0014] Furthermore, the preset backbone network is built based on the Vision Transformer network model and adopts a multi-head self-attention mechanism.

[0015] Furthermore, in the multi-head attention mechanism, the calculation formula for the attention weight of each head is:

[0016]

[0017] In the formula, Q, K, and V are query vectors, key-value vectors, and value vectors respectively. k is the dimension of the key-value vector.

[0018] Furthermore, segmenting the detection image based on the attention intensity distribution value includes:

[0019] A plurality of different attention thresholds are preset. When the attention intensity value of any area in the detection image meets the attention threshold range, the values ​​of the pixels in the area remain unchanged; otherwise, the values ​​of the pixels in the area are set to zero. The formula is:

[0020]

[0021] Among them, Ba(i,j) represents the pixel adjustment coefficient, M S (i, j) represents the attention intensity value in the region, δ1 ,δ 2 Indicates the preset attention threshold;

[0022] The image area is divided using bounding box detection to obtain the minimum rectangle that meets the attention threshold range, and the foreground and multi-level background images are obtained.

[0023] Furthermore, the preprocessing of the foreground image and the multi-level background image includes:

[0024] The foreground image and the multi-level background image are scaled so that the specifications of the foreground image and the multi-level background image are consistent with those of the detection image before being input into the backbone network; wherein, when the multi-level background image is scaled, the area to be cropped is blackened.

[0025] Further, a first set of weights is obtained based on the similarity between each background feature and the foreground feature, and the foreground feature and each background feature are weighted according to the first set of weights to obtain a first set of weighted features, including:

[0026] Similarity Calculated using vector cosine:

[0027]

[0028] Based on similarity Adjust the foreground features:

[0029]

[0030] Based on similarity Adjustments to each level of background feature adjustments:

[0031]

[0032] Where i represents the i-th level background, n represents the total number of background levels, ⊙ represents element-by-element multiplication, and Fg and Bg i Respectively represent the foreground features before adjustment and multiple background features before adjustment, Fg' and Bg' i They represent the adjusted foreground features and multiple adjusted background features respectively.

[0033] Furthermore, a second set of weights is obtained based on the distance between each level of background image and foreground image, and the first set of weighted features is weighted according to the second set of weights to obtain a second set of weighted features, including:

[0034] The distances between the original background feature image and the center of the foreground area are sorted in ascending order, and the weight distribution of the second group of weights satisfies:

[0035]

[0036] Among them, i is the ranking number of each background feature, μ is the mean, σ 2 is the variance, σ is the standard deviation, and experiments show that μ=1,σ 2 When =0.2, the model classification effect is better.

[0037] Furthermore, the classification results of the detected image are obtained according to the fused features, including:

[0038] All features after two rounds of weighting are concatenated and input into the fully connected layer as a classifier to obtain the classification result of the detection image.

[0039] In a second aspect, the present invention also provides a fine-grained image classification system based on foreground-background multi-level division and feature fusion, comprising a backbone network module, a foreground-background multi-level segmentation module, a feature weighted fusion module and a classification module;

[0040] The backbone network module is used to obtain a detection image, obtain attention intensity distribution values ​​of different regions in the detection image based on the attention mechanism of the preset backbone network, and input the preprocessed foreground image and multi-level background image into the preset backbone network to obtain foreground features and multiple background features;

[0041] The foreground-background multi-level segmentation module is used to segment the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image;

[0042] The feature weighted fusion module is used to obtain a first set of weights based on the similarity between each background feature and the foreground feature, and weight the foreground feature and each background feature according to the first set of weights to obtain a first set of weighted features; obtain a second set of weights based on the distance between each level of background image and foreground image, and weight the first set of weighted features according to the second set of weights to obtain a second set of weighted features, and perform feature fusion on the second set of weighted features;

[0043] The classification module is used to obtain the classification result of the detection image according to the fused features.

[0044] In the third aspect, the present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps of the fine-grained image classification method based on multi-level division of foreground and background and feature fusion in any of the above-mentioned implementation methods can be implemented.

[0045] The present invention provides a fine-grained image classification method based on foreground-background multi-level division and feature fusion. First, the attention intensity distribution values ​​of different regions in the detection image are obtained by using the attention mechanism of a preset backbone network; secondly, the detection image is segmented based on the attention intensity distribution value to obtain a foreground image and a multi-level background image; the foreground image and the multi-level background image are pre-processed and input into the preset backbone network to obtain a foreground feature and multiple background features; finally, the foreground feature and the background feature are weighted based on the similarity between each background feature and the foreground feature, and then each background feature is weighted twice based on the distance between its original image area and the foreground, and all the features after the second weighting are fused to obtain the classification result of the detection image. The method of the present invention is different from the mainstream approach of focusing only on the strong discrimination area of ​​the image in the existing fine-grained image classification, but uses foreground-background multi-level segmentation and feature weighted fusion to improve the utilization rate of the model for effective information in the image, and improves the accuracy of classification by extracting effective information in the image background and excluding noise data. In addition, the preset backbone network in this method can be replaced by other classic backbone networks, allowing the model to be trained end-to-end in a weakly supervised learning manner, which greatly reduces the training cost and operation complexity of the model and is suitable for promotion and application in practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the process flow of the fine-grained image classification method based on multi-level division of foreground and background and feature fusion provided by the present invention;

[0047] Figure 2 A schematic diagram of the structure of an embodiment of an integrated unit for obtaining attention weights in a backbone network provided by the present invention;

[0048] Figure 3 A schematic diagram of dividing an image foreground and a multi-level background according to an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of a flow chart of an embodiment of feature weighted fusion provided by the present invention;

[0050] Figure 5 A schematic diagram of an embodiment of a processing framework of a fine-grained image classification method for multi-level division of foreground and background and feature fusion provided by the present invention;

[0051] Figure 6 A structural schematic diagram of an embodiment of a fine-grained image classification system based on multi-level foreground-background division and feature fusion provided by the present invention. DETAILED DESCRIPTION

[0052] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0053] The present invention provides a fine-grained image classification method, system and computer-readable storage medium based on multi-level division of foreground and background and feature fusion, which are described below respectively.

[0054] Combination Figure 1 As shown, a specific embodiment of the present invention discloses a fine-grained image classification method based on multi-level division of foreground and background and feature fusion, including:

[0055] Step S101, obtaining a detection image, inputting the detection image into a preset backbone network, and obtaining attention intensity distribution values ​​of different areas in the detection image based on the attention mechanism of the backbone network;

[0056] Step S102, segmenting the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image;

[0057] Step S103, preprocessing the foreground image and the multi-level background image, inputting the preprocessed foreground image and the multi-level background image into a preset backbone network to obtain a foreground feature and multiple background features;

[0058] Step S104: obtaining a first set of weights based on the similarity between each background feature and the foreground feature, and weighting the foreground feature and each background feature according to the first set of weights to obtain a first set of weighted features;

[0059] Step S105: obtaining a second set of weights based on the distance between each level of background image and foreground image, and weighting the first set of weighted features according to the second set of weights to obtain a second set of weighted features;

[0060] Step S106: perform feature fusion on the second group of weighted features, and obtain a classification result of the detection image according to the fused features.

[0061] Compared with the prior art, the method of this embodiment first uses the attention mechanism of a preset backbone network to obtain the attention intensity distribution values ​​of different areas in the acquired detection image; secondly, the detection image is segmented based on the attention intensity distribution values ​​to obtain a foreground image and a multi-level background image; the foreground image and the multi-level background image are pre-processed and input into the preset backbone network to obtain a foreground feature and multiple background features; finally, the foreground feature and the background feature are weighted respectively based on the similarity between each background feature and the foreground feature; each background feature is then weighted twice based on the distance between its original image area and the foreground area, the twice-weighted features are fused, and the classification result of the detection image is obtained according to the fused features.

[0062] The method of this embodiment is different from the mainstream approach of focusing only on the strong discriminant areas of the image in the existing fine-grained image classification. Instead, it uses foreground-background multi-level segmentation and feature weighted fusion to enable the fine-grained recognition model to more accurately capture the strong discriminant areas in the image while extracting effective information from the image background and removing noise to improve the classification accuracy. This method allows the model to be trained end-to-end in a weakly supervised learning manner, which greatly reduces the training cost and operation complexity of the model, and is suitable for promotion and application in practice.

[0063] It should be noted that the above-mentioned fine-grained image classification method based on multi-level division of foreground and background and feature fusion can be applied to most common backbone networks for image classification, such as convolutional neural networks and their variants (such as resnet50, resnet101), Transformer networks (such as ViT, Swim-Transformer), etc., and therefore has the advantage of strong versatility.

[0064] In some embodiments, the detected image is scaled to a uniform size and then input into the backbone network. The input image needs to be converted into a fixed size first for subsequent processing by the model, for example, uniformly converted into a RGB three-channel color image of size 448*448, etc.

[0065] As a preferred embodiment, the preset backbone network is constructed based on the Vision Transformer network model and adopts a multi-head self-attention mechanism.

[0066] Compared with traditional convolutional neural networks, the Vision Transformer (ViT) network has the characteristics of global feature capture. It divides the image into fixed-size tiles and uses each tile as a sequence input. The self-attention mechanism is used to learn the relationship between tiles, which can help ViT capture the global features of the entire image without being limited by the size and level of the convolution kernel. The self-attention mechanism in ViT allows information interaction between tiles at different positions, so that the network can focus on the connection between different areas in the image, which helps the model better utilize the classification information in the image. In addition, the ViT network can be applied to other computer vision tasks through transfer learning, making it suitable for a variety of application fields.

[0067] In a specific embodiment, the selected preset backbone network can load the pre-trained weights of the network after training on datasets such as ImageNet, and the backbone network can accelerate convergence and obtain better classification results through transfer learning.

[0068] The multi-head self-attention mechanism is one of the key components in the Transformer model, which is used to capture the dependencies between different positions in the input sequence. In each attention head, the attention weight is calculated by learning different weight matrices, thereby strengthening the model's attention to different aspects.

[0069] As a preferred embodiment, the calculation formula of the attention weight of each head is:

[0070]

[0071] In the formula, Q, K, and V are query vectors, key-value vectors, and value vectors respectively. k is the dimension of the key-value vector.

[0072] The attention weight of the detected image is obtained according to the multi-head attention mechanism of the backbone network ViT

[0073] M A ∈R h×(P×P) satisfy:

[0074]

[0075] Among them, M A represents the attention matrix with a size of h×(P×P), h represents the number of self-attention heads in the ViT network, n represents the number of transformer encoder layers, and Π represents the matrix product.

[0076] It should be noted that the backbone network has multiple layers to extract image features. The more layers the image passes through, the more abstract the features extracted by the network. In the process of foreground-background segmentation of an image, the image data only needs to be calculated to the middle layer in the backbone network, and then it can be processed in the module responsible for processing the multi-level foreground-background segmentation, thereby saving computing resources and improving classification speed.

[0077] As a preferred embodiment, the integration unit aggregates and normalizes the image attention weight distribution obtained by the multi-head attention mechanism to obtain the standardized attention strength distribution of the backbone network on the original image. Figure 2 As shown, Figure 2 A schematic diagram showing the integrated unit structure for obtaining attention weights in the backbone network.

[0078] In some embodiments, if the backbone network uses a convolutional neural network (CNN) as the backbone network, other methods can be used to replace the Transformer attention map to reflect the degree of attention paid by the network to different regions of the detected image. For example, the backbone network can be used to obtain the importance of different regions of the original image through the Grad-CAM or Grad-CAM++ method.

[0079] As a preferred embodiment, the segmenting of the detection image based on the attention intensity distribution value includes:

[0080] A plurality of different attention thresholds are preset. When the attention intensity value of any area in the detection image meets the attention threshold range, the values ​​of the pixels in the area remain unchanged; otherwise, the values ​​of the pixels in the area are set to zero. The formula is:

[0081]

[0082] Among them, Ba(i,j) represents the pixel adjustment coefficient, M S (i, j) represents the attention intensity value in the region, δ 1 ,δ 2 Indicates the preset attention threshold;

[0083] The image area is divided using bounding box detection to obtain the minimum rectangle that meets the attention threshold range, and the foreground and multi-level background images are obtained.

[0084] In a specific embodiment, by setting different attention thresholds, bounding box detection is used to divide the image area whose attention strength meets the threshold range, wherein the number and size of the thresholds can be set according to the needs of the usage scenario. Experiments have shown that after the attention value is normalized between [0,1], the foreground-background threshold is about 0.7, and the background is generally divided into 2 to 3 levels for the best effect. The threshold for dividing each level of the background is set at equal intervals between 0 and the foreground-background threshold (0.7). The foreground-background division effect diagram is shown in the figure Figure 3 As shown, Figure 3 In the figure, the left is the original image, and the right is the segmentation diagram; the innermost rectangle in the right figure is the image foreground; the area between the innermost rectangle and the outer rectangle is the primary background; the outermost rectangle is the secondary background.

[0085] As a preferred embodiment, the preprocessing of the foreground image and the multi-level background image includes:

[0086] The foreground image and the multi-level background image are scaled so that the specifications of the foreground image and the multi-level background image are consistent with those of the detection image before being input into the backbone network; wherein, when the multi-level background image is scaled, the area to be cropped is blackened.

[0087] The foreground image and multi-level background image are randomly flipped and random Gaussian filtering is applied to increase the generalization of the model and prevent the model from overfitting.

[0088] As a preferred embodiment, the processed image is input into the backbone network again to obtain foreground features and multiple background features.

[0089] The foreground features and background features are weighted based on the similarity between each background feature and the foreground feature, including:

[0090] Calculate the cosine similarity between the features of each level of background and the foreground features, obtain a first set of weights according to the cosine similarity, and perform a first weighting on each of the background features and foreground features to obtain a first set of weighted features;

[0091] A second set of weights is determined according to the distance between the background image and the foreground at each level, and the first set of weighted features is weighted using the second set of weights to obtain a second set of weighted features.

[0092] Based on the similarity between each level of background features and foreground features, the foreground features Fg and multiple background features Bg are respectively i Weighted.

[0093] Similarity Calculated using vector cosine:

[0094]

[0095] Since the foreground contains the classified target, it is necessary to weaken the network's attention to the foreground area that is similar to the background and adjust the foreground feature to Fg':

[0096]

[0097] For the background, it is necessary to enhance the network's attention to the background area that is similar to the foreground, and adjust each level of background features to Bg' i :

[0098]

[0099] Where i represents the i-th level background, n represents the total number of background levels, ⊙ represents element-by-element multiplication, and Fg and Bg i are the foreground features before adjustment and multiple background features before adjustment, Fg' and Bg' i They are the adjusted foreground features and multiple adjusted background features respectively.

[0100] The foreground features and multiple background features are weighted based on the distance between the background image and the foreground image at each level:

[0101] Fg″=k 0 ·Fg′

[0102] Bg″ 1 =k 1 ·Bg 1 ' ...

[0104] Bg″ i =k i ·Bg′ i

[0105] Among them, Fg′, Bg′ i ,Fg″,Bg i ″ are the foreground features and background features before and after distance weighting, respectively, k i is the weight obtained by network training, and its initial value is negatively correlated with the distance between the center of the background image and the foreground image at each level. Experiments show that k i The value of μ=1,σ 2 =0.2, the model classification accuracy is higher.

[0106] As shown in the figure, Figure 4 This is a schematic diagram of feature weighted fusion. From left to right, the foreground and multi-level background features are extracted from the foreground-background image through the backbone network, and the final features are obtained through similarity weighting, distance weighting and fusion. Among them, feature fusion can be done by splicing or superposition.

[0107] As a preferred embodiment, the classification result of the detection image is obtained according to the fused features, including:

[0108] The fused features are input into the fully connected layer to obtain the classification result of the detection image.

[0109] By dividing the foreground and background and fusing features, we can better capture the subtle differences between target categories and improve the accuracy and robustness of classification.

[0110] In order to illustrate the method of the present invention in more detail, Figure 5 As shown, Figure 5 The main processing framework of the above overall processing process is shown. First, the backbone network is selected and the fine-grained image classification model is trained. Then, the detection image is preprocessed to speed up the classification speed and improve the classification accuracy. The processed detection image goes through two stages. The first stage is the multi-level division of foreground and background, and the second stage is the weighted fusion of features. These two stages are interrelated, which enables the model to more accurately capture the strong discriminant areas in the image while also using the effective information in the background to assist classification and reduce the negative impact of noise data in the background.

[0111] The fine-grained image classification model can reasonably divide, process and fuse foreground and background information through these two stages, thereby improving the classification performance of the model. Finally, the fused features are put into the classification module to obtain the classification result of the detection image.

[0112] This embodiment also provides a fine-grained image classification system based on foreground-background multi-level segmentation and feature weighted fusion, which is mainly divided into a backbone network module, a foreground-background multi-level segmentation module, a feature weighted fusion module and a classification module;

[0113] The backbone network module is used to obtain the attention intensity distribution values ​​of different regions of the detection image; and is also used to input the preprocessed foreground image and the multi-level background image into the preset backbone network to obtain the foreground feature and multiple background features;

[0114] The foreground-background multi-level segmentation module is used to segment the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image;

[0115] The feature weighted fusion module is used to obtain a first set of weights based on the similarity between each background feature and the foreground feature, and weight the foreground feature and each background feature according to the first set of weights to obtain a first set of weighted features; obtain a second set of weights based on the distance between each level of background image and foreground image, and weight the first set of weighted features according to the second set of weights to obtain a second set of weighted features, and perform feature fusion on the second set of weighted features;

[0116] The classification module is used to obtain the classification result of the detection image according to the fused features.

[0117] like Figure 6 As shown, Figure 6 is a framework diagram of the classification system, Figure 6 The upper left corner is the detection image, which is input into the backbone network. In the foreground-background multi-level division module, the attention weight value of the multi-layer network for the image is obtained through the backbone network, and the attention intensity distribution is determined through the integration unit. The foreground and multi-level background are divided by the division unit using a preset threshold. The divided foreground and multi-level background are input into the backbone network again for feature extraction to obtain foreground features and multiple background features. In the feature weighting module, the foreground features and background features are weighted based on the similarity between each background feature and the foreground feature; each background feature is weighted twice based on the distance between its original image area and the foreground, and the twice-weighted features are fused. Finally, the classification result is output through the classification module.

[0118] This embodiment also provides a computer-readable storage medium, on which a fine-grained image classification program based on multi-level division of foreground and background and feature fusion is stored. When the fine-grained image classification program based on multi-level division of foreground and background and feature fusion is executed by a processor, the steps in the above embodiment can be implemented.

[0119] The present invention provides a fine-grained image classification method based on foreground-background multi-level segmentation and feature fusion, which utilizes foreground-background multi-level segmentation and feature weighted fusion to enable the fine-grained recognition model to more accurately capture the strong discriminant area in the image, while extracting effective information in the image background and removing noise to improve the classification accuracy, and the backbone network used in the method can be replaced by other commonly used backbone networks. At the same time, the model is allowed to be trained end-to-end in a weakly supervised learning manner, which greatly reduces the training cost and operation complexity of the model. The method of the present invention overcomes the inertial thinking of the prior art, does not adopt the mainstream approach of only focusing on the strong discriminant area of ​​the image, but innovatively achieves higher classification accuracy by extracting effective information in the background and removing noise, and has achieved good results in practical application. Therefore, the present method has the advantages of good versatility, high feasibility, and innovative and efficient.

[0120] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A fine-grained image classification method based on multi-level foreground-background division and feature fusion, characterized in that: include: Acquire a detection image, input the detection image into a preset backbone network, and obtain attention intensity distribution values ​​of different areas in the detection image based on the attention mechanism of the backbone network; Segmenting the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image; Preprocessing the foreground image and the multi-level background image, inputting the preprocessed foreground image and the multi-level background image into a preset backbone network to obtain a foreground feature and multiple background features; A first set of weights is obtained based on the similarity between each background feature and the foreground feature, and the foreground feature and each background feature are weighted according to the first set of weights to obtain a first set of weighted features; A second set of weights is obtained based on the distance between each level of background image and foreground image, and the first set of weighted features is weighted according to the second set of weights to obtain a second set of weighted features; The second group of weighted features is subjected to feature fusion, and the classification result of the detection image is obtained according to the fused features.

2. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 1 is characterized in that: The preset backbone network is built based on the Vision Transformer network model and adopts a multi-head self-attention mechanism.

3. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 2 is characterized in that: In the multi-head self-attention mechanism, the calculation formula of the attention weight of each head is: In the formula, Q, K, and V are query vectors, key-value vectors, and value vectors respectively. k is the dimension of the key-value vector, and T represents the vector transpose.

4. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 1 is characterized in that: The segmenting of the detection image based on the attention intensity distribution value includes: A plurality of different attention thresholds are preset. When the attention intensity value of any area in the detection image meets the attention threshold range, the values ​​of the pixels in the area remain unchanged; otherwise, the values ​​of the pixels in the area are set to zero. The formula is: Among them, Ba(i,j) represents the pixel adjustment coefficient, M S (i, j) represents the attention intensity value in the area, δ1, δ2 represent the preset attention thresholds; The image area is divided using bounding box detection to obtain the minimum rectangle that meets the attention threshold range, and the foreground and multi-level background images are obtained.

5. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 1 is characterized in that: A first set of weights is obtained based on the similarity between each background feature and the foreground feature, and the foreground feature and each background feature are weighted according to the first set of weights to obtain a first set of weighted features, including: Similarity Calculated using vector cosine: Based on similarity Adjust the foreground features: Based on similarity Adjustments to each level of background feature adjustments: Where i represents the i-th level background, n represents the total number of background levels, □ represents element-by-element multiplication, and Fg and Bg i Respectively represent the foreground features before adjustment and multiple background features before adjustment, Fg' and Bg' i They represent the adjusted foreground features and multiple adjusted background features respectively.

6. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 1 is characterized in that: A second set of weights is obtained based on the distance between each level of background image and foreground image, and the first set of weighted features is weighted according to the second set of weights to obtain a second set of weighted features, including: The distances between the original background feature image and the center of the foreground area are sorted in ascending order, and the weight distribution of the second group of weights satisfies: Among them, i is the ranking number of each background feature, μ is the mean, σ 2 is the variance and σ is the standard deviation.

7. The fine-grained image classification method based on foreground-background multi-level division and feature fusion according to claim 1 is characterized in that: The classification results of the detected image are obtained based on the fused features, including: All features after two rounds of weighting are concatenated and input into the fully connected layer as a classifier to obtain the classification result of the detection image.

8. A fine-grained image classification system based on multi-level foreground and background division and feature fusion, characterized in that: It includes a backbone network module, a foreground-background multi-level segmentation module, a feature weighted fusion module, and a classification module; The backbone network module is used to obtain a detection image, obtain attention intensity distribution values ​​of different regions in the detection image based on the attention mechanism of the preset backbone network, and input the preprocessed foreground image and multi-level background image into the preset backbone network to obtain foreground features and multiple background features; The foreground-background multi-level segmentation module is used to segment the detection image based on the attention intensity distribution value to obtain a foreground image and a multi-level background image; The feature weighted fusion module is used to obtain a first set of weights based on the similarity between each background feature and the foreground feature, and weight the foreground feature and each background feature according to the first set of weights to obtain a first set of weighted features; obtain a second set of weights based on the distance between each level of background image and foreground image, and weight the first set of weighted features according to the second set of weights to obtain a second set of weighted features, and perform feature fusion on the second set of weighted features; The classification module is used to obtain the classification result of the detection image according to the fused features.

9. A computer-readable storage medium, characterized in that: Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the fine-grained image classification method based on multi-level division of foreground and background and feature fusion as described in any one of claims 1 to 7.

Images