Gesture Image Classification Method Based on DC-Res2Net and Feature Fusion Attention Module

Abstract

The present invention discloses a gesture image classification method based on a feature fusion attention module and feature selection. The method is to first use gesture images in a known data set as gesture images, obtain the category labels of the gesture images, and perform size normalization processing on the gesture images to obtain a training set; then construct a convolutional neural network, which includes a feature extraction module, a feature fusion attention module and a classifier; then input the training set into the convolutional neural network for training; finally, input the gesture images to be classified into the trained convolutional neural network for gesture image classification to obtain classification results. The present invention uses the DC‑Res2Net module and the SK‑Net module in combination to accurately extract high-order features and low-order features of gesture images, uses the feature fusion attention module to enhance low-order features and fuse them with high-order features, and realizes accurate recognition of gesture images by fusion features.

Description

Technical Field

[0001] The present invention relates to a gesture image classification method in the field of image classification, and in particular to a gesture image classification method based on a feature fusion attention module and feature selection. Background Art

[0002] Gestures are a natural form of human communication, offering the advantages of convenience, speed, rich meaning, and rapid understanding. Gesture recognition can provide users with a more authentic experience in natural human-computer interaction. In recent years, gesture recognition has played a vital role in fields such as machine control, virtual reality, and assisted driving. Gesture recognition methods can be broadly categorized into traditional computer vision-based approaches and deep learning approaches primarily based on convolutional neural networks.

[0003] Traditional computer vision-based gesture recognition methods manually design effective gesture features for specific datasets to perform gesture classification. Simply put, manual feature extraction methods are designed to handle specific tasks, making them inadequate for diverse tasks across multiple datasets and resulting in poor generalization performance. Furthermore, the effectiveness of manual feature extraction methods and their ability to handle diverse tasks across multiple datasets are largely dependent on the manual feature extraction methods used. Therefore, the ability of machine vision-based gesture recognition methods to adapt to a variety of tasks is inherently limited, as gesture images captured in complex environments may exhibit a wide variety of conditions.

[0004] Gesture feature extraction based on machine vision is cumbersome and time-consuming. Errors in feature extraction can lead to low recognition accuracy, and the system is susceptible to complex lighting and background effects. Currently, mainstream gesture recognition methods, many of which are based on deep learning, suffer from limited accuracy. Furthermore, the features extracted through a single convolutional neural network are limited in scale and cannot fully describe gesture images. Summary of the Invention

[0005] In order to extract more discriminative gesture image features and solve the information redundancy problem of feature extraction in existing gesture recognition methods, the present invention proposes a more effective gesture image feature extraction and classification method.

[0006] The technical solution adopted in the present invention is:

[0007] 1. A gesture image classification method based on feature fusion attention module and feature selection, the method comprising the following steps:

[0008] Step 1) using gesture images in a known data set as gesture images, obtaining category labels for the gesture images, performing size normalization on the gesture images, and using the size-normalized gesture images as training sets;

[0009] Step 2) Construct a convolutional neural network, which includes a feature extraction module, a feature fusion attention module, and a Softmax classifier;

[0010] Step 3) Inputting the training set of step 1) into the convolutional neural network for training;

[0011] Step 4) Inputting the gesture image to be classified into the convolutional neural network trained in step 3) to perform gesture image classification and obtain a classification result.

[0012] The step 2) is specifically as follows: the convolutional neural network is mainly composed of a feature extraction module, a feature fusion attention module and a Softmax classifier connected in sequence, the gesture image is first input into the feature extraction module to obtain high-order feature tensors and low-order feature tensors respectively, and then the high-order feature tensors and the low-order feature tensors are simultaneously input into the feature fusion attention module for processing to obtain a feature tensor after feature vector fusion, and then the feature tensor after feature vector fusion is input into the Softmax classifier for classification to obtain the classification result of the gesture image.

[0013] The feature extraction module is mainly composed of a DC-Res2Net29 network and three SK-Net modules. The DC-Res2Net29 network includes a first convolutional layer, a first maximum pooling layer and three residual modules. The first convolutional layer is connected to the input end of the first residual module via the first maximum pooling layer. The three residual modules and three SK-Net modules are alternately connected in sequence. Each residual module is mainly composed of three DC-Res2Net modules connected in sequence. The gesture image is input into the first residual module to obtain a low-order feature tensor, and the high-order feature tensor is obtained through the output of the last SK-Net module.

[0014] The DC-Res2Net module is based on the Res2Net module, so that the output of each group of convolution filters in the Res2Net module is input to the next group of convolution filters, and the single convolution layer in the Res2Net module is replaced by grouped convolution.

[0015] The SK-Net module includes a feature separation module, a multi-channel fusion module and a feature selection module connected in sequence; the feature separation module includes a second convolution layer and a hole convolution layer connected in parallel, and the feature vectors output by the residual module are respectively input into the second convolution layer and the hole convolution layer to obtain the first separation feature tensor and the second separated feature tensor The multi-channel fusion module includes a global average pooling layer and a first fully connected layer connected in sequence; the two separated feature tensors obtained by the feature separation module are added and input into the global average pooling layer, and the first fully connected layer outputs the fusion feature as the output result of the multi-channel fusion module; the feature selection module includes a second fully connected layer and a first activation layer; the fusion feature obtained by the multi-channel fusion module is input into the second fully connected layer to obtain the compact feature z, and then the compact feature z is input into the first activation layer; the first weight matrix a is obtained through the activation function c and the first weight matrix b c , and then the first weight matrix a c and the second weight matrix b c The first separated feature tensor obtained by the feature separation module and the second separated feature tensor After multiplication and addition, the feature tensor V after feature vector selection is obtained as the final output of the SK-Net module.

[0016] The feature fusion attention module includes a boundary information enhancement module, a feature weighted enhancement module and a feature fusion module; the boundary information enhancement module includes two convolution branches and a third convolution layer and a second maximum pooling layer; each convolution branch is composed of two convolution layers connected in sequence, and the two convolution branches are set in parallel. The output ends of the two convolution branches are input into the third convolution layer after the addition layer, and the output end of the third convolution layer is connected and input into the second maximum pooling layer; the low-order feature vectors output by the feature extraction module are respectively input into the two convolution branches, and the feature tensors output by the two convolution branches are added through the addition layer and input into the third convolution layer, and then the feature tensor output by the third convolution layer is input into the second maximum pooling layer. In the method, the feature tensor output by the second maximum pooling layer is used as the low-order feature tensor after boundary information enhancement; the feature weighted enhancement module includes a convolution layer and an average pooling layer connected in sequence, the high-order feature vector output by the feature extraction module is input into the average pooling layer, and the feature tensor output by the average pooling layer is input into the fourth convolution layer, and the low-order feature tensor after boundary information enhancement output by the boundary information enhancement module and the feature tensor output by the fourth convolution layer are multiplied to obtain a feature tensor after feature weighting; the feature fusion module directly fuses the high-order feature vector and the feature weighted feature tensor output by the feature weighted enhancement module in an additive manner to obtain a feature tensor after feature vector fusion.

[0017] The beneficial effects of the present invention are:

[0018] (1) The present invention improves the Res2Net module and proposes a DC-Res2Net module; this module obtains more receptive field size combinations through dense connections, which can be used to perform multi-level feature extraction on images, enhance the network's multi-scale feature extraction capability, and improve the classification accuracy.

[0019] (2) The present invention constructs the DC-Res2Net29 network and combines it with the SK-Net module to extract effective low-order features and high-order features from gesture images.

[0020] (3) This paper proposes a feature fusion attention module that performs boundary information enhancement and feature weighting on the low-order features of gesture images and fuses the high-order features with the enhanced low-order features. This module can effectively fuse the high- and low-order features of gesture images, enabling a more accurate description of gesture images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a technical solution flow chart of the method of the present invention;

[0022] Figure 2 Schematic diagram of part of the data set used in the method of the present invention;

[0023] Figure 3 This is a schematic diagram of some low-level features of gestures extracted by the present invention;

[0024] Figure 4 This is a schematic diagram of some high-level features of gestures extracted by the present invention;

[0025] Figure 5 Schematic diagram of the structures of ResNet, Res2Net and DC-Res2Net in the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of the SK-Net module used in the present invention;

[0027] Figure 7 This is a schematic diagram of the feature extraction module structure used in the present invention;

[0028] Figure 8 Schematic diagram of the structure of the feature fusion attention module used in the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, the gesture image classification method of this embodiment includes the following steps:

[0031] Step 1) Figure 2As shown in the figure, gesture images in a known dataset are used as gesture images, and the category labels of the gesture images are obtained. The gesture images are size-normalized and used as training sets. Specifically, the size-normalized gesture images are all 224×224×3 three-channel RGB images. 224×224×3 is used as the input size of the neural network. The three-channel RGB images are then normalized and mapped from integers between 0 and 255 to floating-point numbers between 0 and 1.

[0032] Step 2) Construct a convolutional neural network, which includes a feature extraction module, a feature fusion attention module and a Softmax classifier; Figure 5 and Figure 6 shown.

[0033] Step 3) Inputting the training set of step 1) into the convolutional neural network for training;

[0034] Step 4) Input the gesture image to be classified into the convolutional neural network trained in Step 3) for gesture image classification, obtaining a classification result. Specifically, the feature tensor after feature selection is input into a Softmax classifier for classification, mapping it to probability values for 10 categories. The category with the highest probability is selected as the classification category for the gesture image, obtaining the classification result for the gesture image.

[0035] Among them, the known dataset is the OUHANDS open source sign language dataset.

[0036] Among them, step 2) is specifically as follows: the convolutional neural network is mainly composed of a feature extraction module, a feature fusion attention module and a Softmax classifier connected in sequence. The gesture image is first input into the feature extraction module to obtain high-order feature tensors and low-order feature tensors, such as Figure 3 and Figure 4 The high-order feature tensor and the low-order feature tensor are then simultaneously input into the feature fusion attention module for processing to obtain a feature tensor after feature vector fusion. The feature tensor after feature vector fusion is then input into the Softmax classifier for classification to obtain the classification result of the gesture image.

[0037] like Figure 7As shown in the figure, the feature extraction module is mainly composed of a DC-Res2Net29 network and three SK-Net modules. The DC-Res2Net29 network includes the first convolutional layer, the first maximum pooling layer and three residual modules. The first convolutional layer is connected to the input of the first residual module through the first maximum pooling layer. The three residual modules and three SK-Net modules are alternately connected in sequence. Each residual module is mainly composed of three DC-Res2Net modules connected in sequence. The gesture image is input into the first residual module to obtain a low-order feature tensor, and the high-order feature tensor is obtained through the output of the last SK-Net module.

[0038] The DC-Res2Net module is based on the Res2Net module, so that the output of each group of convolution filters in the Res2Net module is input to the next group of convolution filters, and the single convolution layer in the Res2Net module is replaced by grouped convolution.

[0039] This embodiment proposes the DC-Res2Net module (Densely ConnectedRes2Net) based on the Res2Net module to construct the DC-Res2Net29 network. The DC-Res2Net29 network is then combined with the SK-Net module to construct a feature extraction module for extracting high-order and low-order features of the input image, thereby obtaining high-order and low-order feature tensors. Using the DC-Res2Net module as a basis, every three DC-Res2Net modules are constructed into a residual module, and the first convolutional layer, the first maximum pooling layer, and the three residual modules form the DC-Res2Net29 network.

[0040] Each group of convolutional filters in the DC-Res2Net module receives the output of each previous group of convolutional filters. At the same time, in order to reduce network parameters and computational complexity, grouped convolution is used to replace the original ordinary convolution.

[0041] Specifically:

[0042] The ResNet module is a classic convolutional network structure that solves the gradient explosion and gradient vanishing problems caused by too deep network layers by constructing an identity mapping. The Res2Net module groups the convolution kernels while keeping the size and total number of convolution kernels in the ResNet module unchanged, forming multiple small convolution kernel branches and connecting different convolution kernel branches in a hierarchical residual-like manner.

[0043] In the structure of DC-Res2Net, the input features are first input into the convolution layer with a convolution kernel size of 1×1. Assume that the output feature tensor of the convolution layer is x∈R w×h×b×C, where w×h is the spatial dimension of the feature map, b is the spectral dimension of the feature map, and C is the number of feature channels. In order to achieve hierarchical operation, a new dimension is introduced: s (scale). The 3×3 filter with a total of C channels is divided into s filters, each with s channels, that is, C = s×w. The input feature map x is also divided into x i (i∈{1,2,...,s}). Output feature y i It can be expressed as

[0044]

[0045] Among them, G(*) is a small 3×3 group convolution, and the number of groups is set to 4.

[0046] The output feature tensor y is input into a convolution layer with a convolution kernel size of 1×1. Finally, the output features of the convolution layer are added to the initial output features of the DC-Res2Net module to obtain the final output feature tensor.

[0047] The DC-Res2Net29 network consists of a first convolutional layer, a first maximum pooling layer, and three residual modules. The input feature is transformed into 112×112×64 after passing through the convolution layer with a convolution kernel of 7×7, and then the input feature is transformed into 56×56×64 after passing through the maximum pooling layer with a stride of 2.

[0048] The first residual module is composed of the DC-Res2Net_1 module, the DC-Res2Net_2 module, and the DC-Res2Net_3 module connected in sequence, and the output feature shape is 56×56×64.

[0049] The second residual module is composed of the DC-Res2Net_4 module, the DC-Res2Net_5 module, and the DC-Res2Net_6 module connected in sequence. After the input feature passes through the DC-Res2Net_4 module, the feature shape becomes 28×28×128, and then it is input into the DC-Res2Net_5 module. Its output is then input into the DC-Res2Net_6 module. The output feature shapes of the DC-Res2Net_5 module and the DC-Res2Net_6 module are all 28×28×128.

[0050] The third residual module is composed of the DC-Res2Net_7 module, the DC-Res2Net_8 module, and the DC-Res2Net_9 module connected in sequence. After the input feature passes through the DC-Res2Net_7 module, the feature shape becomes 14×14×256, and then it is input into the DC-Res2Net_8 block, and its output is then input into the DC-Res2Net_9 module. The output feature shapes of the DC-Res2Net_8 module and the DC-Res2Net_9 module are all 14×14×256.

[0051] This embodiment also adds an SK-Net module after each residual module in the DC-Res2Net29 network to form a complete feature extraction network, uses the features output by the first residual module as the low-order feature tensor, and uses the last feature output in the feature extraction network as the high-order feature tensor.

[0052] Specifically:

[0053] In the DC-Res2Net29 network, the output of the first residual module is connected to the SK-Net_1 module, with an output feature shape of 56×56×64. The output of the second residual module is connected to the SK-Net_2 module, with an output feature shape of 28×28×128. The output of the third residual module is connected to the SK-Net_3 module, with an output feature shape of 14×14×256. The output features of the DC-Res2Net_3 module in the first residual module are used as the low-order feature tensor, and the output features of the SK-Net_3 module are used as the high-order feature tensor.

[0054] The SK-Net module includes a feature separation module, a multi-channel fusion module, and a feature selection module connected in sequence; the feature separation module includes a second convolutional layer and a hole convolutional layer connected in parallel, and the feature vectors after the high- and low-order features output by the residual module are input into the second convolutional layer and the hole convolution layer respectively to obtain the first separation feature tensor and the second separated feature tensor The second convolution layer uses a 3×3 convolution kernel, and the hole convolution layer uses a 3×3 convolution kernel with a dilation coefficient of 2. The first separation feature tensor is and the second separated feature tensor It is set specifically through the following formula:

[0055]

[0056]

[0057] Among them, C is the feature vector after the fusion of high-order and low-order features, is a convolution function with a kernel size of 3×3 and an output channel of d. The convolution kernel size is 3×3, and after the expansion factor is 2, the receptive field becomes 5×5, and the output channel is d. is the separated feature tensor output by the convolutional layer, is the separation feature tensor output by the dilated convolutional layer;

[0058] The multi-channel fusion module includes a global average pooling layer and a first fully connected layer connected in sequence. The two separated feature tensors obtained by the feature separation module are added and input into the global average pooling layer. The first fully connected layer outputs the fused feature as the output result of the multi-channel fusion module.

[0059] Specifically, the two separated feature tensors obtained by the splicing feature separation module are set by the following formula:

[0060]

[0061] The obtained feature tensor U is input into the global average pooling layer, which uses global average pooling to embed global information. In order to achieve accurate and adaptive selection guidance in the information, the output of the global average pooling layer is input into the first fully connected layer, which reduces the dimension and can improve efficiency to a certain extent. The specific setting is through the following formula:

[0062] Fuse(U)=F fc (GlobAvgPool c (U))

[0063] Among them, U is the feature tensor obtained by adding two separate feature tensors, Fuse(*) is the multi-channel fusion function; GlobAvgPool c (*) is the global average pooling function in the channel dimension, F fc (*) is the fully connected function;

[0064] The feature selection module includes a second fully connected layer and a first activation layer; the fusion feature obtained by the multi-channel fusion module is input into the second fully connected layer to obtain the compact feature z, and then the compact feature z is input into the first activation layer; the first weight matrix a is obtained through the activation function c and the first weight matrix b c , and then the first weight matrix a c and the second weight matrix b c The first separated feature tensor obtained by the feature separation module and the second separated feature tensor After multiplication and addition, the feature tensor V after feature vector selection is obtained as the final output of the SK-Net module.

[0065] Among them, the first activation layer is the softmax activation function.

[0066] The above process in the feature selection module is expressed as the following formula:

[0067] z=F fc (f)

[0068]

[0069] a c =softmax(z)

[0070] b c =softmax(z)

[0071] a c +b c =1

[0072] Among them, F fc (*) is the fully connected function, f is the result of the multi-channel feature fusion module, softmax(x) is the activation function, x is the input variable of the activation function, z is the compact feature, A and B are random real number matrices with the same shape as the compact feature z, A c and B c Corresponding to the cth row of A and B respectively, A and B are random, a c and b c are different weight matrices obtained through the same activation function.

[0073] like Figure 8 As shown, the feature fusion attention module includes a boundary information enhancement module, a feature weighted enhancement module and a feature fusion module; the boundary information enhancement module includes two convolution branches, a third convolution layer and a second maximum pooling layer; each convolution branch is composed of two convolution layers connected in sequence, and the two convolution branches are set in parallel. The output ends of the two convolution branches are input into the third convolution layer after the addition layer, and the output end of the third convolution layer is connected and input into the second maximum pooling layer;

[0074] In the boundary information enhancement module, to enhance the image boundary information represented by low-level features, two convolution kernels, 1×5 and 5×1, are used to extract gesture edge information and points of interest. While a single 1×5 convolution layer fully extracts local features in gesture images, it results in dimensional information loss. To compensate for this loss of gesture information, two 5×1 and 1×5 convolution layers are designed as information padding to fuse the two feature vectors. The fused features fully display gesture edge information and points of interest while eliminating redundant information in low-level features. This is specifically set using the following formula:

[0075]

[0076] Among them, S(*) is the boundary feature enhancement function, It is a convolution function with a convolution kernel size of 1× 5 and d output channels, and v is the input feature tensor.

[0077] In order to perform dimensionality matching for subsequent feature weighting, a 3×3 convolutional layer is used to increase the feature mapping channels, and then a maximum pooling layer with a stride of 8 is used to reduce the feature dimensionality. The specific settings are as follows:

[0078]

[0079] Where, E(*) is the dimension matching function; is a convolution function with a kernel size of 3×3 and an output channel of d; q is the original high-order feature tensor, MaxPooling s＝8 (*) is the maximum pooling function with a stride of 8.

[0080] The low-order feature vectors output by the feature extraction module are input into two convolution branches respectively. The two convolution kernels of the first branch are 1×5 and 5×1, and the two convolution kernels of the second branch are 5×1 and 1×5. The feature tensors output by the two convolution branches are added through the addition layer and input into the third convolution layer. The feature tensor output by the third convolution layer is then input into the second maximum pooling layer with a stride of 8. The feature tensor output by the second maximum pooling layer is used as the low-order feature tensor after boundary information enhancement.

[0081] The feature weighted enhancement module includes a convolution layer and an average pooling layer connected in sequence. The high-order feature vector output by the feature extraction module is input into the average pooling layer, and the feature tensor output by the average pooling layer is input into the fourth convolution layer. The low-order feature tensor after boundary information enhancement output by the boundary information enhancement module and the feature tensor output by the fourth convolution layer are multiplied to obtain a feature-weighted feature tensor.

[0082] In the feature weighting enhancement module, to highlight gesture features, high-order features containing more key semantic information are used to guide low-order features and precisely combine the two. The high-order features are reduced and simplified using an average pooling layer. The reduced features are then extracted again using a 1×1 convolutional layer to strengthen the feature information, resulting in enhanced high-order features. This feature is then multiplied by the low-order features for weighting. Compared to feature addition, feature multiplication can suppress redundant information and enhance important features for consistency. This is specifically set using the following formula:

[0083]

[0084] Among them, T(*) is the feature weighting function, is a convolution function with a kernel size of 1×1 and an output channel of d, AvgPooing(*) is an average pooling function, h is a high-order feature tensor, and l is the output feature tensor of the boundary information enhancement module.

[0085] The feature fusion module directly performs feature fusion on the high-order feature vector and the feature weighted feature tensor output by the feature weighted enhancement module in an additive manner to obtain the feature tensor after feature vector fusion.

[0086] Although high-order features contain key semantic information, they are prone to losing more gesture details after extraction by deep networks. In addition, due to factors such as background and lighting, target position information may be confused or lost. Therefore, adding improved low-order features to the original high-order features can expand the receptive field of the features.

[0087] In summary, in the feature fusion attention module of this embodiment, the image boundary features of the low-order feature vectors are enhanced through the boundary information enhancement module, the low-order feature vectors are enhanced by the high-order feature vectors through the feature weighted enhancement module, and the feature tensor after the fusion of the high and low order feature vectors is obtained after processing by the feature fusion module.

[0088] The equipment used in this example is as follows: Ubuntu 16.04 operating system, Python version 3.7.2, TensorFlow 2.3.1 deep learning framework, and an Nvidia GeForce GTX 2070Ti graphics card. Comparative experiments were conducted on the OUHANDS open-source sign language dataset using ResNet50, Res2Net29, and the image classification method proposed in this paper, yielding the results shown in the table below.

[0089] Table 1: Comparison of results of this embodiment and other models

[0090]

Claims

1. A gesture image classification method based on DC-Res2Net and feature fusion attention module, characterized by: The method comprises the following steps: Step 1) Use gesture images from a known dataset as gesture images, obtain the category labels of the gesture images, perform size normalization on the gesture images, and use the size-normalized gesture images as the training set; Step 2) Construct a convolutional neural network, which is mainly composed of a feature extraction module, a feature fusion attention module, and a Softmax classifier connected in sequence. The gesture image is first input into the feature extraction module to obtain high-order feature tensors and low-order feature tensors respectively. The high-order feature tensors and low-order feature tensors are then simultaneously input into the feature fusion attention module for processing to obtain a feature tensor after feature vector fusion. The feature tensor after feature vector fusion is then input into the Softmax classifier for classification to obtain the classification result of the gesture image; The feature extraction module is mainly composed of a DC-Res2Net29 network and three SK-Net modules. The DC-Res2Net29 network includes a first convolutional layer, a first maximum pooling layer, and three residual modules. The first convolutional layer is connected to the input end of the first residual module via the first maximum pooling layer. The three residual modules and three SK-Net modules are alternately connected in sequence. Each residual module is mainly composed of three DC-Res2Net modules connected in sequence. The gesture image is input into the first residual module to obtain a low-order feature tensor, and the high-order feature tensor is obtained by outputting the last SK-Net module. Step 3) Input the training set of step 1) into the convolutional neural network for training; Step 4) Input the gesture image to be classified into the convolutional neural network trained in step 3) to perform gesture image classification and obtain the classification result.

2. The gesture image classification method based on DC-Res2Net and feature fusion attention module according to claim 1, characterized in that: The DC-Res2Net module is based on the Res2Net module, so that the output of each group of convolution filters in the Res2Net module is input to the next group of convolution filters, and the single convolution layer in the Res2Net module is replaced by grouped convolution.

3. The gesture image classification method based on DC-Res2Net and feature fusion attention module according to claim 1, characterized in that: The SK-Net module includes a feature separation module, a multi-channel fusion module and a feature selection module connected in sequence; the feature separation module includes a second convolution layer and a hole convolution layer connected in parallel, and the feature vectors output by the residual module are respectively input into the second convolution layer and the hole convolution layer to obtain the first separation feature tensor and the second separated feature tensor ;The multi-channel fusion module includes a global average pooling layer and a first fully connected layer connected in sequence; The two separated feature tensors obtained by the feature separation module are added and input into the global average pooling layer. The first fully connected layer outputs the fusion feature as the output result of the multi-channel fusion module. The feature selection module includes a second fully connected layer and a first activation layer. The fusion feature obtained by the multi-channel fusion module is input into the second fully connected layer to obtain the compact feature. , and then the compact features Input the first activation layer; get the first weight matrix through the activation function and the first weight matrix , and then the first weight matrix obtained and the second weight matrix The first separated feature tensor obtained by the feature separation module and the second separated feature tensor Multiply and add to get the feature tensor after feature vector selection , as the final output of the SK-Net module.

4. The gesture image classification method based on DC-Res2Net and feature fusion attention module according to claim 1, characterized in that: The feature fusion attention module includes a boundary information enhancement module, a feature weighted enhancement module, and a feature fusion module; the boundary information enhancement module includes two convolution branches, a third convolution layer, and a second maximum pooling layer; each convolution branch is composed of two convolution layers connected in sequence, and the two convolution branches are arranged in parallel. The output ends of the two convolution branches are input into the third convolution layer after the addition layer, and the output end of the third convolution layer is connected and input into the second maximum pooling layer; The low-order feature vectors output by the feature extraction module are input into the two convolution branches respectively. The feature tensors output by the two convolution branches are added through the addition layer and input into the third convolution layer. The feature tensor output by the third convolution layer is then input into the second maximum pooling layer. The feature tensor output by the second maximum pooling layer is used as the low-order feature tensor after boundary information enhancement. The feature weighted enhancement module includes a convolution layer and an average pooling layer connected in sequence. The high-order feature vector output by the feature extraction module is input into the average pooling layer, and the feature tensor output by the average pooling layer is input into the fourth convolution layer. The low-order feature tensor after boundary information enhancement output by the boundary information enhancement module and the feature tensor output by the fourth convolution layer are multiplied to obtain a feature tensor after feature weighting; the feature fusion module directly fuses the high-order feature vector and the feature tensor after feature weighting output by the feature weighted enhancement module in an additive manner to obtain a feature tensor after feature vector fusion.

Images