An image classification method based on improved attention sketch feature fusion

Abstract

The present invention relates to an image classification method based on improved attention sketch feature fusion, belonging to the field of computer image classification. The method comprises the following steps: S1. Acquiring a dataset, the dataset comprising a training set and a test set; S2. Constructing an image classification model based on improved attention sketch feature fusion, the model comprising a sketch encoder, a sketch decoder, a sketch feature extractor, a first original feature extractor, a first attention module, a second original feature extractor, a second attention module, a third original feature extractor, and a classifier; S3. Training the model using images in the training set and optimizing the model using an Adam optimizer with a loss function to obtain a trained model; S4. Applying the trained model to the images to be classified in the test set to obtain classification results. By combining feature information at different levels, the present invention introduces an improved attention mechanism and feature fusion strategy to effectively improve the accuracy and robustness of image classification.

Description

Technical Field

[0001] The present invention belongs to the field of computer image classification, and in particular relates to an image classification method based on improved attention sketch feature fusion. Background Art

[0002] With the rapid development of computer vision technology, image classification has become a key application scenario, widely used in fields such as medical image analysis, autonomous driving, and security monitoring. Traditional image classification methods primarily rely on manual feature extraction and shallow learning algorithms, which often perform poorly when processing complex images. Therefore, the introduction of deep learning, particularly convolutional neural networks (CNNs), has significantly advanced image classification technology. Despite the remarkable achievements of deep learning, model performance remains limited in some cases, particularly when the sample size is small or the class differences are not obvious. Furthermore, existing image classification methods often ignore the importance and contextual information of different image regions, resulting in reduced classification accuracy. To address these shortcomings, researchers have begun to focus on attention mechanisms, which dynamically adjust the weights of input features to more effectively capture key image regions. Furthermore, sketches, as a simplified visual representation, can focus on the outline and shape of objects, providing a different representation from real images. Fusion of sketch features with traditional image features has the potential to enhance the model's understanding of image content. However, in the field of computer image classification, sketch-based feature fusion methods are still in their infancy, and the relevant technology needs to be improved.

[0003] Therefore, an image classification method based on sketch feature fusion with an improved attention mechanism has emerged. It aims to improve the accuracy and robustness of classification by effectively integrating image and sketch features, solve the limitations of existing technologies in feature extraction and context understanding, and provide new ideas and methods for the development of intelligent vision systems. Summary of the Invention

[0004] In order to improve the accuracy and robustness of image classification, the present invention provides an image classification method based on improved attention sketch feature fusion.

[0005] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0006] The present invention provides an image classification method based on improved attention sketch feature fusion, comprising the following steps:

[0007] S1. Obtain a publicly available Sketchy dataset and preprocess the images in the dataset. The Sketchy dataset includes a training set and a test set.

[0008] S2. Construct an image classification model based on improved attention sketch feature fusion, the model comprising a sketch encoder, a sketch decoder, a sketch feature extractor, a first original feature extractor, a first attention module, a second original feature extractor, a second attention module, a third original feature extractor, and a classifier;

[0009] S3. training the model using the images in the training set and optimizing the model using the Adam optimizer through the loss function to obtain a trained image classification model based on improved attention sketch feature fusion;

[0010] S4. The images to be classified in the test set are classified by a trained image classification model based on improved attention sketch feature fusion to obtain classification results.

[0011] Furthermore, step S1 uses the FFDnet image denoising method to denoise the image data in the Sketchy dataset, and encapsulates it using the DataSet class in pytorch to obtain the Sketchy original image. .

[0012] Furthermore, the sketch encoder in step S2 is composed of a feature extraction layer, a feature encoding layer, and a fully connected layer in sequence, specifically:

[0013] The feature extraction layer includes a first convolutional layer, a ReLU activation function, a first pooling layer, a second convolutional layer, a ReLU activation function, a second pooling layer, a third convolutional layer, a sigmoid activation function, and a third pooling layer; the feature encoding layer includes a depthwise convolutional layer, a point-by-point convolutional layer, a pooling layer, and a channel attention layer; the channel attention calculation formula is as follows:

[0014] ,

[0015] in, Represents the feature map of the input channel attention, ⊙ represents element-by-element multiplication, σ represents the Sigmoid function, Represents the initialization weight matrix of the ReLU function, Represents the initialization weight matrix of the Sigmoid function, express The bias term, express The bias term, Indicates the number of feature map rows, represents the number of feature map columns, represents the activation function, Represents the output of channel attention; the number of units in the fully connected layer is 256, and the activation function is the ReLU activation function;

[0016] The Sketchy original image Input into the sketch encoder to obtain sketch encoding features .

[0017] Furthermore, the sketch decoder in step S2 comprises a fully connected layer, a reshaping layer, and an upsampling layer, specifically:

[0018] The number of units in the fully connected layer is 576, and the activation function is the ReLU activation function; the reshape layer uses the Reshape function; the upsampling layer uses transposed convolution;

[0019] Encode the sketch feature Input into the sketch decoder to get the sketch result .

[0020] Furthermore, the sketch feature extractor in step S2 includes a first convolutional layer, a ReLU activation function, a second convolutional layer, a ReLU activation function, and a pooling layer, specifically:

[0021] The sketch results After inputting into the first convolutional layer and ReLU activation function, the first sketch feature is obtained , the first sketch feature Input into the second convolutional layer and ReLU activation function to obtain the second sketch feature , the first sketch feature and the second sketch feature Add the input to the pooling layer to get the sketch features .

[0022] Furthermore, in step S2, the first original feature extractor and the first attention module are specifically:

[0023] The first original feature extractor includes a convolution layer and a sigmoid activation function, and converts the Sketchy original image After inputting into the convolution layer and sigmoid activation function, the sketch features Add to get the first original feature ;

[0024] The first original feature Input into the first attention module to get the first attention , the formula is as follows:

[0025] ,

[0026] in, Represents the first original feature , Represents the first original feature The number of channels, H is the first original feature The height of W is the first original feature The width of LayerNorm represents the normalization layer. represents the output of the first attention module.

[0027] Furthermore, in step S2, the second original feature extractor and the second attention module are specifically:

[0028] The second original feature extractor includes a first convolution layer, a ReLU activation function, a second convolution layer, a ReLU activation function, a third convolution layer, a pooling layer, a flattening layer and a fully connected layer. and sketch features After addition, it is input into the second original feature extractor to obtain the second original feature ;

[0029] The second attention module uses a multi-head attention mechanism to and the second original feature Input into the second attention module to obtain the second attention , the formula is as follows:

[0030] ,

[0031] in, Represents the second original feature , h represents the number of multi-head attention, 、 、 denote the query matrix, key matrix, and value matrix of the j-th sub-attention learnable of the i-th head, respectively. represents the dimension of the bond matrix, represents the learnable adaptive feature selection matrix, and P represents the second original feature Position encoding, Represents the first original feature , represents the dynamic weight, The calculation formula is as follows:

[0032] ,

[0033] Among them, the matrix Represents a learnable linear transformation matrix, GlobalContext represents global pooling, represents the multiplication operation, express function.

[0034] Furthermore, the third original feature extractor in step S2 includes a convolution layer, a ReLU activation function, and a pooling layer, which extracts the second original feature. and sketch features After addition, it is input into the third original feature extractor to obtain the third original feature .

[0035] Furthermore, the classifier in step S2 is composed of a flattening layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and a max function in sequence. The first attention and second attention With the third original feature Add together to get improved attention features , the improved attention feature Input the classifier to get the classification result .

[0036] Furthermore, the loss function in step S3 is cross entropy loss.

[0037] The advantages of the present invention are:

[0038] The present invention first uses a deep convolutional neural network to extract features from the input image, thereby obtaining a multi-level feature representation. On this basis, an improved attention mechanism is introduced, enabling the model to focus on areas with important features through adaptive weight allocation, thereby enhancing the expressive power of key features. Next, the method fuses features from different levels, effectively combining local features with global features through weighted sum or concatenation operations to form a comprehensive feature representation. This process ensures that the model can take into account both details and overall information, improving classification performance. Finally, the fused feature vector is input into the classifier to make the final classification decision, output the probability distribution of each category, and select the final predicted category. Overall, this method successfully overcomes the limitations of traditional image classification techniques in feature selection and fusion, can more accurately capture key information in images, and is widely applicable to various image classification tasks such as object recognition and scene understanding. By introducing an improved attention mechanism and feature fusion strategy, the present invention provides an efficient and powerful solution, laying the foundation for further research and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0040] Figure 1 It is a model structure diagram of the method of the present invention;

[0041] Figure 2 This is a feature visualization comparison diagram of the method of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments derived by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] Example 1

[0044] In this embodiment, Figure 1 As shown, the present invention provides an image classification method based on improved attention sketch feature fusion, the specific steps include:

[0045] (a) Using the FFDnet image denoising method on the images in the publicly available Sketchy dataset to obtain the Sketchy original image ;

[0046] Specifically, we obtain image data from the Sketchy dataset and perform FFDnet image denoising on it. We then use the DataSet class in pytorch to encapsulate the original Sketchy image. , the image size is 224×224.

[0047] (b) Sketchy original image Input into the sketch encoder to obtain sketch encoding features ;

[0048] Specifically, b-1) the sketch encoder consists of a feature extraction layer, a feature encoding layer, and a fully connected layer. b-2) The feature extraction layer consists of a first convolutional layer, a ReLU activation function, a first pooling layer, a second convolutional layer, a ReLU activation function, a second pooling layer, a third convolutional layer, a sigmoid activation function, and a third pooling layer. The first convolutional layer has a 3*3 kernel size, 32 kernels, a stride of 1, and uses "same" padding. The first pooling layer uses max pooling, a pooling window size of 2*2, and a stride of 2. The second convolutional layer has a 3*3 kernel size, 64 kernels, a stride of 1, and uses "same" padding. The second pooling layer uses max pooling, a pooling window size of 2*2, and a stride of 2. The third convolutional layer has a 5*5 kernel size, 128 kernels, a stride of 1, and uses "same" padding. The first pooling layer uses max pooling with a pooling window size of 4*4 and a stride of 1. b-3) The feature encoding layer consists of a depthwise convolutional layer, a pointwise convolutional layer, a pooling layer, and a channel attention layer. The depthwise convolutional layer has a convolution kernel size of 3*3 and a stride of 1, using 'same' padding. The pointwise convolutional layer has a convolution kernel size of 1*1 and a total of 32 convolution kernels. The pooling layer uses average pooling with a pooling window size of 2*2 and a stride of 2. The channel attention layer is calculated as follows:

[0049] ,

[0050] in, is the feature map of the input channel attention, that is, the output of the pooling layer, ⊙ represents element-by-element multiplication, σ represents the Sigmoid function, and Represents two initialized weight matrices, the first one maps the input channel number C to a smaller dimension, and the second one maps the output of the activation function back to the original channel number C. and Respectively represent and The two bias terms of Represents the number of feature map rows, Represents the number of feature map columns. Represents the activation function. Represents the output of channel attention. b-4) The number of units in the fully connected layer is 256, and the activation function is the ReLU activation function. b-5) The Sketchy original image Input into the sketch encoder to obtain sketch encoding features .

[0051] (c) Encoding features in sketches Input into the sketch decoder to get the sketch result .

[0052] Specifically, c-1) the sketch decoder consists of a fully connected layer, a reshape layer, and an upsampling layer. c-2) the number of units in the fully connected layer is 576, and the activation function is the ReLU activation function. c-3) the reshape layer uses the Reshape() function to convert the output of the fully connected layer into a shape of 24*24 suitable for image processing. c-4) the upsampling layer uses transposed convolution to restore the spatial resolution of the image. The transposed convolution kernel size is 3*3, the number of convolution kernels is 64, the stride is 2, and 'same' padding is used. c-5) the sketch encoding features are converted into Input into the sketch decoder to get the sketch result .

[0053] (d) Sketch the result Input into the sketch feature extractor to obtain the sketch feature .

[0054] Specifically, d-1) the sketch feature extractor includes a first convolutional layer, a ReLU activation function, a second convolutional layer, a ReLU activation function, and a pooling layer. d-2) the convolution kernel size of the first convolutional layer is 3*3, the number of convolution kernels is 32, the stride is 1, and 'same' padding is used. The convolution kernel size of the second convolutional layer is 5*5, the number of convolution kernels is 64, the stride is 1, and 'same' padding is used. The pooling layer uses maximum pooling, the pooling window size is 2*2, and the stride is 2. d-3) the sketch result is After inputting into the first convolutional layer and ReLU activation function, the first sketch feature is obtained . Move the first sketch feature Input into the second convolutional layer and ReLU activation function to obtain the second sketch feature . Move the first sketch feature and the second sketch feature Add the input to the pooling layer to get the sketch features .

[0055] (e) Sketchy original image and sketch features Input into the first original feature extractor to obtain the first original feature .

[0056] Specifically, e-1) the first original feature extractor includes a convolution layer and a sigmoid activation function. The convolution kernel size of the convolution layer is 1*1, the number of convolution kernels is 32, the stride is 1, and 'same' padding is used. e-2) The Sketchy original image is After inputting into the convolution layer and sigmoid activation function, it is combined with the sketch features Add together to get the first original feature .

[0057] (f) The first original feature Input into the first attention module to get the first attention .

[0058] Specifically, the calculation formula of f-1) the first attention module is as follows:

[0059]

[0060] in, is the feature map of the first attention module, that is, the first original feature . is the number of channels of the feature map, H is the height of the feature map, W is the width of the feature map, LayerNorm() represents the normalization layer, represents the output of the first attention module. f-2) The first original feature Input into the first attention module to get the first attention .

[0061] (g) The first original feature and sketch features Input into the second original feature extractor to obtain the second original feature .

[0062] Specifically, g-1) The second original feature extractor consists of a first convolutional layer, a ReLU activation function, a second convolutional layer, a ReLU activation function, a third convolutional layer, a pooling layer, a flattening layer, and a fully connected layer. g-2) The convolution kernel size of the first convolutional layer is 3*3, the number of convolution kernels is 64, the stride is 2, and 'same' is used for padding. The convolution kernel size of the second convolutional layer is 1*1, the number of convolution kernels is 256, the stride is 1, and 'same' is used for padding. The convolution kernel size of the third convolutional layer is 3*3, the number of convolution kernels is 512, the stride is 1, and 'same' is used for padding. The pooling layer adopts maximum pooling, the pooling window size is 2*2, and the stride is 2. The flattening layer flattens the pooled output into a one-dimensional vector. The number of units in the fully connected layer is 256. g-3) The first original feature and sketch features After addition, it is input into the second original feature extractor to obtain the second original feature .

[0063] (h) The first original feature and the second original feature Input into the second attention module to obtain the second attention .

[0064] Specifically, h-1) the second attention module adopts a multi-head attention mechanism, and the calculation formula is as follows:

[0065] ,

[0066] in, The feature map of the second attention module is input, that is, the second original feature , h represents the number of multi-head attention, here h=4, 、 、 Represents the query matrix, key matrix and value matrix of the j-th sub-attention learnable of the i-th head, respectively, through the second original feature The linear transformation yields, Indicates the dimensions of the bond matrix. Represents a learnable adaptive feature selection matrix generated by random initialization. P represents the second original feature The position encoding is generated by the embedding() function, Represents the first original feature . Represents dynamic weight, calculated as:

[0067] Among them, the matrix is a learnable linear transformation matrix used to generate dynamic weights associated with the i-th attention head. GlobalContext() represents global pooling. h-2) The first original feature and the second original feature Input into the second attention module to obtain the second attention .

[0068] (i) The second original feature and sketch features Input into the third original feature extractor to obtain the third original feature .

[0069] Specifically, i-1) the third original feature extractor is composed of a convolution layer, a ReLU activation function, and a pooling layer. i-2) the convolution kernel size of the convolution layer is 3*3, the number of convolution kernels is 256, the stride is 1, and 'same' padding is used. The pooling layer uses maximum pooling, the pooling window size is 2*2, and the stride is 2. and sketch features After addition, it is input into the third original feature extractor to obtain the third original feature .

[0070] (j) Focus your attention first and second attention With the third original feature Add together to get improved attention features .

[0071] (k) Attention features will be improved and sketch features Input into the classifier to get the classification result .

[0072] Specifically, k-1) the classifier is composed of a flattening layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and a max() function. k-2) the flattening layer will improve the attention feature and sketch features After addition, the vector is flattened along the channel direction to form a one-dimensional vector. The number of units in the first fully connected layer is 512. The number of units in the second fully connected layer is 128. The number of units in the third fully connected layer is 8. The max() function adds one to the coordinate corresponding to the maximum value in the output of the third fully connected layer, matches it with the classification table dictionary, and outputs the classification result. .

[0073] (l) Using the Adam optimizer to iterate the parameters of the model in steps b) to j) through the loss function to obtain the trained model ;

[0074] (m) Preprocess the image to be classified and input it into the model trained in step l) to output the classification result .

[0075] Example 2

[0076] In this example, we conducted comparative experiments using the publicly available CIFAR-10 dataset. The CIFAR-10 dataset contains 60,000 32x32 color images divided into 10 categories, with 6,000 images per category. These categories include airplanes, cars, birds, cats, deer, dogs, frogs, horses, boats, and trucks. The experimental environment used Python 3.7 and TensorFlow 2.5, and the GPU used was an NVIDIA A100 40G. The present invention was compared with existing mainstream classification methods and implemented on a unified experimental platform. First, a traditional convolutional neural network (CNN) was used. Due to its powerful feature extraction capabilities, parameter efficiency, and adaptability, CNN has become a mainstream method in the field of computer vision and is widely used in various practical scenarios. Second, the FroFA algorithm was used. FroFA is an emerging feature representation method that aims to enhance model performance in feature extraction and classification tasks by introducing a focal attention mechanism.

[0077] Table 1

[0078]

[0079] In this embodiment, Table 1 is a comparison table of experimental parameters, and the accuracy rate represents the ratio of the number of correctly predicted samples to the total number of samples. The precision rate represents the ratio of samples predicted to be positive examples that are actually positive examples. The recall rate represents the ratio of samples that are actually positive examples that are correctly predicted to be positive examples. F1-score represents the harmonic mean of the precision rate and the recall rate, which measures the balance of the model between the two. The method of the present invention outperforms traditional CNN and FroFA in key performance indicators such as accuracy rate (89.7%), precision rate (88.9%) and F1-score (88.2%), showing that its overall recognition ability and reliability are stronger. Although the recall rate (87.6%) is slightly lower than that of FroFA, this method can still effectively reduce misjudgments and is suitable for scenarios with high precision requirements, proving its superiority in specific tasks.

[0080] In this embodiment, Figure 2 This figure shows a comparison of feature visualizations obtained using the method of the present invention and a CNN during the feature extraction phase. The left side shows a feature map extracted using a traditional CNN method, which exhibits less detailed information and significant background noise. The right side shows a feature map extracted using the method of the present invention, which more clearly captures important features and details in the image, demonstrating a stronger feature representation capability.

[0081] The experimental data and visualization results above demonstrate that our improved attention sketch feature fusion method performs well in image classification tasks, improving not only classification accuracy but also the model's ability to handle complex scenes. This demonstrates the effectiveness and practicality of our method, making it suitable for widespread application in practical image classification tasks.

[0082] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A picture classification method based on improved attention sketch feature fusion, characterized in that: The following steps are involved: S1. Obtain a publicly available Sketchy dataset and preprocess the images in the dataset. The Sketchy dataset includes a training set and a test set. S2. Construct an image classification model based on improved attention sketch feature fusion, the model comprising a sketch encoder, a sketch decoder, a sketch feature extractor, a first original feature extractor, a first attention module, a second original feature extractor, a second attention module, a third original feature extractor, and a classifier; Specifically, the Sketchy original image Input into the sketch encoder to obtain sketch encoding features ; Encode the sketch feature Input into the sketch decoder to get the sketch result ; The sketch result Input into the sketch feature extractor to obtain the sketch feature ; Sketchy original image and sketch features Input into the first original feature extractor to obtain the first original feature ; The first original feature Input into the first attention module to get the first attention ; The first original feature and sketch features Input into the second original feature extractor to obtain the second original feature ; The first original feature and the second original feature Input into the second attention module to obtain the second attention ; The second original feature and sketch features Input into the third original feature extractor to obtain the third original feature ; Focus first and second attention With the third original feature Add together to get improved attention features , will improve the attention feature and sketch features Input into the classifier to get the classification result ; S3. training the model using the images in the training set and optimizing the model using the Adam optimizer through the loss function to obtain a trained image classification model based on improved attention sketch feature fusion; S4. The images to be classified in the test set are classified by a trained image classification model based on improved attention sketch feature fusion to obtain classification results.

2. The image classification method based on improved attention sketch feature fusion according to claim 1, characterized in that: In step S1, the image data in the Sketchy dataset is denoised using the FFDnet image denoising method, and the original Sketchy image is obtained by encapsulating it in the DataSet class in pytorch. .

3. The image classification method based on improved attention sketch feature fusion according to claim 2, characterized in that: The sketch encoder in step S2 is composed of a feature extraction layer, a feature encoding layer, and a fully connected layer, specifically: The feature extraction layer includes a first convolutional layer, a ReLU activation function, a first pooling layer, a second convolutional layer, a ReLU activation function, a second pooling layer, a third convolutional layer, a sigmoid activation function, and a third pooling layer; the feature encoding layer includes a depthwise convolutional layer, a point-by-point convolutional layer, a pooling layer, and a channel attention layer; the channel attention calculation formula is as follows: ， in, Represents the feature map of the input channel attention, ⊙ represents element-by-element multiplication, σ represents the Sigmoid function, Represents the initialization weight matrix of the ReLU function, Represents the initialization weight matrix of the Sigmoid function, express The bias term, express The bias term, Indicates the number of feature map rows, represents the number of feature map columns, represents the activation function, Represents the output of channel attention; the number of units in the fully connected layer is 256, and the activation function is the ReLU activation function; The Sketchy original image Input into the sketch encoder to obtain sketch encoding features .

4. The image classification method based on improved attention sketch feature fusion according to claim 3, characterized in that: The sketch decoder in step S2 is composed of a fully connected layer, a reshaping layer, and an upsampling layer, specifically: The number of units in the fully connected layer is 576, and the activation function is the ReLU activation function; the reshape layer uses the Reshape function; the upsampling layer uses transposed convolution; Encode the sketch feature Input into the sketch decoder to get the sketch result .

5. The image classification method based on improved attention sketch feature fusion according to claim 4, characterized in that: The sketch feature extractor in step S2 includes a first convolutional layer, a ReLU activation function, a second convolutional layer, a ReLU activation function, and a pooling layer, specifically: The sketch results After inputting into the first convolutional layer and ReLU activation function, the first sketch feature is obtained , the first sketch feature Input into the second convolutional layer and ReLU activation function to obtain the second sketch feature , the first sketch feature and the second sketch feature Add the input to the pooling layer to get the sketch features .

6. The image classification method based on improved attention sketch feature fusion according to claim 5, characterized in that: In step S2, the first original feature extractor and the first attention module are specifically: The first original feature extractor includes a convolution layer and a sigmoid activation function, and converts the Sketchy original image After inputting into the convolution layer and sigmoid activation function, the sketch features Add to get the first original feature ; The first original feature Input into the first attention module to get the first attention , the formula is as follows: ， in, Represents the first original feature , Represents the first original feature The number of channels, H is the first original feature The height of W is the first original feature The width of LayerNorm represents the normalization layer. represents the output of the first attention module.

7. The image classification method based on improved attention sketch feature fusion according to claim 6, characterized in that: The second original feature extractor and the second attention module in step S2 are specifically: The second original feature extractor includes a first convolution layer, a ReLU activation function, a second convolution layer, a ReLU activation function, a third convolution layer, a pooling layer, a flattening layer and a fully connected layer. and sketch features After addition, it is input into the second original feature extractor to obtain the second original feature ; The second attention module uses a multi-head attention mechanism to and the second original feature Input into the second attention module to obtain the second attention , the formula is as follows: ， in, Represents the second original feature , h represents the number of multi-head attention, 、 、 denote the query matrix, key matrix, and value matrix learnable by the j-th sub-attention of the i-th head, respectively. represents the dimension of the bond matrix, represents the learnable adaptive feature selection matrix, and P represents the second original feature Positional encoding, Represents the first original feature , represents the dynamic weight, The calculation formula is as follows: ， Among them, the matrix Represents a learnable linear transformation matrix, GlobalContext represents global pooling, represents the multiplication operation, express function.

8. The image classification method based on improved attention sketch feature fusion according to claim 7, characterized in that: The third original feature extractor in step S2 includes a convolution layer, a ReLU activation function, and a pooling layer. and sketch features After addition, it is input into the third original feature extractor to obtain the third original feature .

9. The image classification method based on improved attention sketch feature fusion according to claim 8, characterized in that: The classifier in step S2 is composed of a flattening layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, and a max function in sequence; the first attention and second attention With the third original feature Add together to get improved attention features ; The improved attention feature and sketch features Input the classifier to get the classification result .

10. The image classification method based on improved attention sketch feature fusion according to claim 9, characterized in that: The loss function in step S3 is cross entropy loss.

Images