Improved gesture image feature extraction method based on DenseNet network

Abstract

The present invention discloses a method for extracting gesture images based on an improved DenseNet network. The original gesture image is obtained by collecting gestures; the feature tensors of the shallower layer and the deeper layer are selected through convolution downsampling through a convolution downsampling network structure, and the two feature tensors are respectively input into a DenseNet-B module fused with a Drop-Path module to obtain two feature tensors; the feature tensor fused with multi-scale features after fusion is compressed through a transition layer, and then input into a DenseNet-B module fused with a Drop-Path module to obtain a feature tensor containing multi-scale high dimensions; the classification result is obtained through a global average pooling layer, a fully connected layer, and a softmax classifier. The present invention contains feature tensors of different depths in a downsampling network structure, and can accurately identify large and small target objects. At the same time, the Drop-Path module is integrated in the DenseNet network, which effectively reduces the number of parameters without reducing the accuracy, improves the speed of model training, prevents overfitting, and improves the accuracy of gesture recognition.

Description

Technical Field

[0001] The present invention relates to a method for extracting gesture images, and in particular to a method for extracting gesture image features improved based on a DenseNet network. Background Art

[0002] Gesture recognition is a popular human-computer interaction method in recent years, and is widely used in various fields such as sign language recognition, intelligent monitoring and virtual reality. Initially, gesture recognition mainly used wearable devices to directly detect the angles and spatial positions of the joints of the hands and arms. Although these devices can provide good detection results, they are expensive to apply in common fields. In recent years, gesture recognition technology has shifted from wearable devices to two categories: machine vision gesture recognition methods and deep learning-based gesture recognition methods. Although gesture recognition technology has made great progress, it still faces many challenges in real environments. For example, many factors such as lighting and target size will affect the performance of gesture recognition. The gesture feature fusion method based on machine vision can realize the extraction and fusion of complex gesture features to a certain extent, but the extracted features are easily affected by lighting and background, making it difficult for the fused features to accurately describe the gesture image, and the processing speed is slow. At present, the gesture recognition method based on deep learning has gradually become the mainstream. Generally, a convolutional neural network is used to automatically extract the deep features of the contour, skin color and texture of the gesture through the convolution layer, and then fuse them through the convolution layer. However, it is difficult to extract the multi-scale features of images with different gesture target proportions through a single convolutional neural network, and the accuracy performance is poor. Summary of the invention

[0003] In view of the problem that the features obtained by traditional gesture image feature fusion and recognition methods are difficult to accurately identify images with different gesture target proportions in complex backgrounds, and the information redundancy problem that may be caused by dense connections in DenseNet networks, the purpose of the present invention is to propose an effective feature extraction and fusion method that is suitable for images with different gesture target proportions, which can meet the needs of high-precision recognition of gesture targets with different proportions in complex backgrounds.

[0004] The present invention contains feature tensors of different depths in the downsampling network structure, which can accurately identify both large and small targets. At the same time, the Drop-Path module is integrated into the DenseNet network, which effectively reduces the number of parameters without reducing the accuracy, improves the speed of model training, prevents overfitting, and improves the accuracy of gesture recognition.

[0005] The technical solution of the present invention comprises the following steps:

[0006] 1) Capturing images of various gestures to obtain original images, and normalizing the original images to obtain original gesture images;

[0007] 2) The original gesture image is processed by the convolution downsampling network structure for multiple convolution downsampling, and the shallower feature tensors and the deeper feature tensors in the convolution downsampling network structure are selected;

[0008] 3) The shallower feature tensor and the deeper feature tensor are respectively input into the DenseNet-B module integrated with the Drop-Path module to obtain two feature tensors with amplified channels containing semantic information of different scales;

[0009] 4) Fusing the two obtained feature tensors with amplified channels containing semantic information of different scales to obtain a feature tensor of multi-scale feature fusion; the fusion refers to the superposition of channels.

[0010] 5) The obtained multi-scale feature fusion feature tensor is compressed through the transition layer, and then input into the DenseNet-B module fused with the Drop-Path module again after compression to obtain a multi-scale high-dimensional feature tensor;

[0011] 6) The multi-scale high-dimensional feature tensor is passed through a global average pooling layer, a fully connected layer, and a softmax classifier to obtain the classification result of the gesture image.

[0012] The convolutional downsampling network structure is mainly composed of four consecutive second-order convolution modules, a third-order convolution module and an upsampling layer connected in sequence;

[0013] The second-order convolution module is mainly composed of a first convolution layer with stride=2 for downsampling, a second convolution layer of 1×1, a batch normalization layer and a RELU activation layer connected in sequence; the first convolution layer can reduce the negative gradient effect brought by pooling and replace the pooling layer for downsampling, the second convolution layer can improve the degree of nonlinearity and flexibly change the network structure, and the batch normalization layer and RELU activation layer can accelerate network training and convergence speed.

[0014] The third-order convolution module is composed of a bottleneck layer added to the second-order convolution module. The specific position of the bottleneck layer is after the two convolution layers and before the batch normalization layer in the second-order convolution module. The bottleneck layer can compress the dimension and greatly reduce the amount of calculation.

[0015] The upsampling layer is performed using a nearest neighbor interpolation algorithm;

[0016] The outputs of the third second-order convolution module and the upsampling layer are selected as the feature tensor of the shallower layer and the feature tensor of the deeper layer, respectively.

[0017] The second-order convolution module f2(z) and the third-order convolution module f3(z) are specifically expressed according to the following formulas:

[0018]

[0019]

[0020] Where z represents the input image, represents a convolution function with a kernel size of i×i and an output channel number of n. a, b, and c represent the specific number of channels output by the first convolution layer, the second convolution layer, and the bottleneck layer of different convolution modules in the convolution downsampling network structure. BN(*) represents the batch normalization function. δ(*) represents the RELU activation function.

[0021] After the original gesture image passes through the first second-order convolution module, it outputs a feature tensor with 8 channels, which is equivalent to 1 / 2 the size of the original image. After passing through the second second-order convolution module, it outputs a feature tensor with 16 channels, which is equivalent to 1 / 4 the size of the original image. After passing through the third second-order convolution module, it outputs a feature tensor with 32 channels, which is equivalent to 1 / 8 the size of the original image. After passing through the fourth second-order convolution module, it outputs a feature tensor with 64 channels, which is equivalent to 1 / 16 the size of the original image. After passing through a third-order convolution module, it outputs a feature tensor with 32 channels, which is equivalent to 1 / 32 the size of the original image. After passing through the upsampling layer, it outputs a feature tensor with 32 channels, which is equivalent to 1 / 8 the size of the original image.

[0022] The DenseNet-B module is mainly composed of multiple dense layers, each of which is composed of two consecutive convolution modules BRC. The two convolution modules BRC have the same topological structure but different parameters. The convolution module BRC is composed of a batch normalization layer, a RELU activation layer and a single convolution layer connected in sequence; the difference between the two convolution modules BRC is the size of the convolution kernel in the single convolution layer, the convolution kernel in the single convolution layer of one convolution module BRC is 1×1, and the convolution kernel in the single convolution layer of the other convolution module BRC is 3×3; the convolution module BRC is represented by the following formula:

[0023]

[0024] Where z represents the input image, BN(*) represents the batch normalization function, δ(*) represents the RELU activation function, Represents the convolution function with a convolution kernel size of i×i and an output channel number of n; BRC(*) represents the function of the convolution module BRC.

[0025] In the DenseNet-B module, the input of the first dense layer is the feature tensor selected in the convolutional downsampling network structure, and the input of each of the remaining dense layers includes the feature tensor selected in the convolutional downsampling network structure and the output of all dense layers located in front of it, and the feature tensor selected in the convolutional downsampling network structure and the output of all dense layers are connected together as the output of the DenseNet-B module;

[0026] In the specific implementation, four dense layers are set. The input of the first dense layer is the feature tensor selected in the convolutional downsampling network structure, the input of the second dense layer is the feature tensor selected in the convolutional downsampling network structure and the output of the first dense layer, the input of the third dense layer is the feature tensor selected in the convolutional downsampling network structure, the output of the first dense layer and the output of the second dense layer, the input of the fourth dense layer is the feature tensor selected in the convolutional downsampling network structure, the output of the first dense layer, the output of the second dense layer and the output of the third dense layer, and so on.

[0027] Finally, the feature tensor selected from the convolutional downsampling network structure and the outputs of the first to fourth dense layers are fused in the channel dimension, and the fused feature tensor is used as the output of the DenseNet-B module.

[0028] The DenseNet-B module is represented by the following formula setting:

[0029]

[0030] Among them, X0 represents the input of the first dense layer, X l represents the output of the Lth dense layer, H l Represents the convolution module BRC with two convolution kernels of different sizes in layer L. Represents superposition in the channel dimension.

[0031] The DenseNet-B module ensures maximum information flow between layers through feature reuse.

[0032] The DenseNet-B module fused with the Drop-Path module sets a Drop-Path function before each dense layer on the DenseNet-B module to randomly discard certain input sub-paths, but ensures that at least one sub-path is input; the Drop-Path module sets the discard rate by itself, reduces the model training parameters, provides a good regularization effect, and is an extension of Dropout.

[0033] The DenseNet-B module fused with the Drop-Path module is set up and represented by the following formula:

[0034]

[0035] Among them, Drop-Path n (*) indicates the Drop-Path function, and the subscript n indicates the set drop rate.

[0036] The transition layer is mainly composed of a convolutional pooling module BRCM. The convolutional pooling module BRCM adds a maximum pooling layer at the end on the basis of the convolutional module BRC. The convolutional pooling module BRCM can be expressed by the following formula:

[0037]

[0038] in, Represents a maximum pooling layer with a stride of 2 and a convolution kernel size of 2×2.

[0039] The present invention fuses the feature tensors after the number of channels is amplified at two different scales to obtain a feature tensor of multi-scale feature fusion by superimposing on the channel to form a multi-dimensional feature tensor containing different scales, which is then input into the transition layer for compression. The transition layer can make the channel compact, change the scale of the feature tensor, enhance the edge information of the image, and effectively suppress overfitting.

[0040] The present invention uses a downsampling network similar to the target detection algorithm FPN to perform multiple downsampling on the input original gesture image, selects the shallower feature tensors and the deeper feature tensors in the downsampling structure and inputs them into the DenseNet-B module respectively, integrates the Drop-Path module to reduce information redundancy and improve the regularization effect, outputs a multi-dimensional feature tensor containing semantic information of different scales, splices the feature tensors containing semantic information of different scales in the channel dimension, and then uses a transition layer to spatially compress the fused feature tensor, enhances the image edge information and inputs it again into the DenseNet-B module integrated with the Drop-Path module to extract a higher-order multi-scale feature tensor.

[0041] The present invention contains feature tensors of different depths in the downsampling network structure, which can accurately identify both large and small targets. At the same time, the Drop-Path module is integrated into the DenseNet network, which effectively reduces the number of parameters without reducing the accuracy, improves the speed of model training, prevents overfitting, and improves the accuracy of gesture recognition.

[0042] The method proposed in the present invention completes feature extraction of images with different gesture target proportions and multi-scale feature fusion to achieve accurate gesture recognition.

[0043] Beneficial effects of the present invention:

[0044] (1) The present invention proposes a downsampling network structure similar to the target detection algorithm FPN, which is composed of four second-order convolution modules, one third-order convolution module and one upsampling layer. It processes the input original gesture image and can adapt to the detection of images with different gesture target ratios. It effectively helps the convolution network to quickly learn the features of gestures with different ratios, thereby improving the network training speed.

[0045] (2) The present invention adopts a DenseNet-B network model integrated with a Drop-Path module, and inputs the feature tensor containing semantic information of different scales obtained in the downsampling network structure into the DenseNet-B network model integrated with the Drop-Path module. It reduces the redundancy of network image feature information while maximizing the information flow between levels while reusing features, reduces network model parameters, speeds up network training, effectively suppresses overfitting, improves the regularization effect, and can effectively extract and fuse features of gesture targets of different proportions, generate multi-scale feature representation, and further improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the technical solution of the present invention;

[0047] Figure 2 Partial gesture images of the sign language data set used in the method of the present invention;

[0048] Figure 3 Schematic diagram of a second-order convolution module, a basic unit, used in the present invention;

[0049] Figure 4 Schematic diagram of a basic unit three-order convolution module used in the present invention;

[0050] Figure 5 Schematic diagram of the basic unit convolution module BRC used in the present invention;

[0051] Figure 6 Schematic diagram of the basic unit convolutional pooling module BRCM used in the present invention;

[0052] Figure 7 Schematic diagram of the structure of the DenseNet-B module integrated with the Drop-Path module used in the present invention;

[0053] Figure 8 This is a comparison chart of the average parameter values ​​between network models;

[0054] Fig. 9 This is a comparison chart of the average accuracy of the validation set between network models.

[0055] Fig.10This is a comparison result of the average accuracy of the validation set between the network of the present invention and the commonly used network model. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0057] The technical solution flow chart of the present invention is as follows Figure 1 shown.

[0058] The dataset of this invention adopts the ASL (American sign language) open source sign language dataset, some of which are as follows Figure 2 As shown in the figure, it contains gesture images with different gesture target ratios, different angles, different lighting, and different background environments, including 28 gesture categories and 1 non-gesture category, a total of 29 classification categories.

[0059] 1) The size of the original gesture image is normalized to a 224×224×3 RGB image, and then the original gesture image is standardized, and the original gesture image is mapped from an integer between 0 and 255 to a floating point number between 0 and 1 as the input of the neural network.

[0060] 2) Input the standardized gesture image Figure 1 In the downsampling network shown in the figure, the downsampling network is mainly composed of four second-order convolution modules, one third-order convolution module and one upsampling layer connected in sequence. The structure of the second-order convolution module is as follows: Figure 3 , the third-order convolution module is as follows Figure 4 The second-order convolution module f2(z) and the third-order convolution module f3(z) are set according to the following formula:

[0061]

[0062]

[0063] Among them, z represents the input feature vector, Represents a convolution function with a kernel size of i×i and an output channel number of n. In the above formula, a, b, and c represent the specific number of channels output by the first convolution layer, the second convolution layer, and the bottleneck layer of different convolution modules in the convolution downsampling structure; BN(*) represents the batch normalization function; δ(*) represents the RELU activation function.

[0064] The convolution kernel size of the first convolution layer of the second-order convolution module is 3×3, with a stride of 2, and the convolution kernel size of the second convolution layer is 1×1, with a stride of 1. Figure 1The difference between the multiple second-order convolution modules shown in the figure lies in the number of convolution kernels used in the two convolution layers. The number of convolution kernels of the first second-order convolution module and the two convolution layers is 8, the number of convolution kernels of the second second-order convolution module and the two convolution layers is 16, the number of convolution kernels of the third second-order convolution module and the two convolution layers is 32, and the number of convolution kernels of the fourth second-order convolution module and the two convolution layers is 64; the number of convolution kernels of the first convolution layer of the third-order convolution module is 128, the size is 3×3, and the step size is 2, the number of convolution kernels of the second convolution layer is 128, the size is 1×1, the step size is 1, and the number of convolution kernels of the third convolution layer is 32, the size is 1×1, and the step size is 1.

[0065] After the original gesture image passes through the first second-order convolution module, the output feature tensor has a size of 112×112×8. After the second second-order convolution module, the output feature tensor has a size of 56×56×16. After the third second-order convolution module, the output feature tensor has a size of 28×28×32. After the fourth second-order convolution module, the output feature tensor has a size of 14×14×64. After the third-order convolution module, the output feature tensor has a size of 7×7×32.

[0066] After the original gesture image is input into the downsampling network, the shallower and deeper feature tensors are selected as the input of the DenseNet-B module fused with the Drop-Path module. The size of the shallower feature tensor selected is 28×28×32, and the size of the deeper feature tensor is 7×7×32. The feature tensors of different depths contain different semantic information of the gesture image. The receptive field of the shallow network is relatively small, and the geometric detail information representation ability is strong, but the semantic information representation ability is weak. The receptive field of the deep network is relatively large, and the semantic representation ability is strong, but the spatial geometric feature details are lacking, and the geometric information representation ability is weak. Trying to fuse features of different depths to form multi-scale features can effectively help the network improve the recognition rate of gestures of different proportions. The deep feature tensor is then passed through an upsampling layer, and the feature tensor of size 28×28×32 is output according to the nearest neighbor interpolation algorithm.

[0067] 3) Input the shallower feature tensor and the upsampled deep feature tensor into Figure 7 In the DenseNet-B module fused with the Drop-Path module shown, Figure 7 The lighter colored paths in the figure are randomly discarded sub-paths. Figure 7The neural network shown only draws 4 dense layers, the first layer is the input layer. This process uses the DenseNet-B module that contains 6 dense layers and the fused Drop-Path module. The DenseNet-B module inputs the feature map obtained in the current layer to all subsequent layers. For an L-layer network, the traditional CNN contains L connections between layers, and the DenseNet-B module contains a total of 1+2+…L=L×(L+1)÷2 inter-layer connections, which realizes feature reuse and ensures maximum information flow between layers. However, the information flow between these layers tightly contains the features of the previous layers many times, resulting in information redundancy and increasing the computational complexity of model training. Moreover, each time the previous layer is fused, a new set of memory space must be opened up, resulting in a large amount of memory space required to be consumed. The fusion of the Drop-Path module can reduce the number of parameters and alleviate memory usage. At the same time, it can also enhance the generalization ability of the model and improve the recognition accuracy of the model. Figure 7 The BRC*2 in the network structure means two convolution modules BRC, that is, the dense layer is composed of two convolution modules BRC. The structure of the convolution module BRC is as follows: Figure 5 The first convolution module BRC of the dense layer is used as the bottleneck layer of the dense layer. The convolution kernel size is 1×1. The number of channels of the input multidimensional feature tensor is first compressed to 4×k. In this process, k is set to 32 to reduce the number of channels of the feature tensor and reduce the training time of the network. The original nonlinear link is modified and then input to the second convolution module BRC. The convolution kernel size is 3×3. Nonlinear activation is performed and the feature tensor with the number of channels k is output. The number of channels of the input feature tensor of the nth dense layer of the DenseNet-B module without the Drop-Path module is k0+(n-1)×k, where k0 is the input Den The number of channels of the feature tensor of the seNet-B module, the number of channels of the input feature tensor of the nth dense layer of the DenseNet-B module fused with the Drop-Path module is determined by the drop rate d set by the Drop-Path module. The input subpath of the current layer is randomly discarded, and the number of channels of the input feature tensor of the nth dense layer is approximately equal to d×(k0+(n-1)×k). In this network, d is set to 0.5. The Drop-Path module can prevent the network from being too deep and provide a good regularization effect. Finally, the output of the DenseNet-B module fused with the Drop-Path module is as follows Figure 7 The input feature tensor and the superposition of all sub-layers are used. The high-dimensional feature tensor output by the network in this process has a size of 28×28×224.

[0068] Among them, the convolution module BRC and DenseNet-B module can be set by the following formula:

[0069]

[0070]

[0071] Among them, X0 represents the input feature tensor of the DenseNet-B module, X l represents the output of the Lth dense layer, H l Represents the convolution module BRC with two convolution kernels of different sizes in layer L. Indicates channel overlay operation.

[0072] The DenseNet-B module fused with the Drop-Path module can be set by the following formula:

[0073]

[0074] Among them, Drop-Path n (*) indicates the Drop-Path function, and the subscript n indicates the drop rate.

[0075] The output of the DenseNet-B module containing the fused Drop-Path module with L dense layers can be calculated by the following formula:

[0076]

[0077] 4) The two feature tensors with two channels containing semantic information of different scales output by the DenseNet-B module fused with the Drop-Path module are fused, after the channels are expanded. Figure 1 , the channels are superimposed in the channel fusion layer to form a high-dimensional feature tensor containing semantic information of different scales. The size after fusion is 28×28×448, and then it is input to the transition layer for spatial compression. The function of the transition layer is to compact the channels, and the compactness rate is set to 0.5, that is, the number of channels is compressed to half of the original, and the scale of the feature tensor is changed to enhance the edge information of the image, effectively suppressing overfitting. The transition layer is composed of a convolutional pooling module BRCM. The convolutional pooling module BRCM adds a maximum pooling layer on the basis of the convolutional module BRC. The convolutional pooling module BRCM is as follows Figure 6 The pooling layer of the transition layer of the original DenseNet network is an average pooling layer. This network uses a maximum pooling layer, which is conducive to enhancing edge information. Edge information, as an important feature component of gesture images, can effectively improve the accuracy and speed of gesture recognition. The size of the compressed feature tensor is 28×28×224. After compression and pooling in the transition layer, the size of the multi-scale high-dimensional feature tensor finally output is 14×14×224.

[0078] 5) If Figure 1As shown in the figure, this network uses a total of 3 DenseNet-B modules fused with Drop-Path modules. The output of the transition layer is used as the input of the next DenseNet-B module fused with Drop-Path modules. In this process, the number of dense layers is 6, and the learning rate k is set to 16. Finally, according to the formula The output tensor size of the module is calculated to be 14×14×320.

[0079] 6) Finally, the multi-scale high-dimensional feature tensor passes through a global average pooling layer and then a fully connected layer to map the number of channels to 29, and finally inputs it into the softmax classifier to obtain the classification result of the gesture image.

[0080] like Figure 8 and Fig. 9 As shown, network 1 is a DenseNet-B network, network 2 is a DenseNet-B network fused with a Drop-Path module, and network 3 is a DenseNet-B network fused with a downsampling network and a Drop-Path module. All three networks are trained on a training set containing 11,600 images and verified on a validation set containing 2,900 images. The number of model parameters and the accuracy on the validation set are statistically analyzed after multiple training. The average parameters of network 1 are 1,368,000, and the average accuracy is 98.22%. The average parameters of network 2 are 908,000, and the average accuracy is 98.85%. The average parameters of network 3 are 950,000, and the average accuracy is 99.83%.

[0081] The DenseNet-B network that integrates the downsampling network and the Drop-Path module is trained on the same gesture dataset for 20 iterations with the commonly used gesture recognition networks InceptionV3, MobileNet, and AlexNet. The average training time, average model size, and average recognition rate on the validation set between the network models are compared, as shown in Table 1. The validation set accuracy curve of the network model is plotted as shown in Fig.10 .

[0082] Table 1

[0083]

[0084] As you can see, the network model size is only 11.1MB, and the number of parameters is only 9.44×10 5 , the total running time is 919s, and the recognition rate is only 0.06% lower than that of InceptionV3 and MobileNet, with the best overall performance.

[0085] The data obtained through experiments show that the fused Drop-Path module in the present invention can significantly reduce the parameters of model training without reducing the recognition accuracy, effectively suppress overfitting, reduce the storage occupied by the model, and accelerate the convergence of the model. The DenseNet-B network that integrates the downsampling network structure and the Drop-Path module in the present invention can quickly adapt to the extraction of multi-scale features of images with different gesture target proportions, enhance the generalization of model recognition, and improve the accuracy of network model recognition.

Claims

1. A gesture image feature extraction method based on an improved DenseNet network, characterized in that: The steps include: 1) Capturing images of various gestures to obtain original images, and normalizing the original images to obtain original gesture images; 2) The original gesture image is processed by the convolution downsampling network structure for multiple convolution downsampling, and the shallower feature tensors and the deeper feature tensors in the convolution downsampling network structure are selected; 3) The shallower feature tensor and the deeper feature tensor are respectively input into the DenseNet-B module integrated with the Drop-Path module to obtain two feature tensors with amplified channels containing semantic information of different scales; 4) Fusing the two obtained feature tensors with amplified channels containing semantic information of different scales to obtain a feature tensor of multi-scale feature fusion; 5) The obtained multi-scale feature fusion feature tensor is compressed through the transition layer, and then input into the DenseNet-B module fused with the Drop-Path module again after compression to obtain a multi-scale high-dimensional feature tensor; 6) The multi-scale high-dimensional feature tensor is passed through a global average pooling layer, a fully connected layer, and a softmax classifier to obtain the classification result of the gesture image.

2. According to claim 1, the method for extracting gesture images based on the improved DenseNet network is characterized in that: The convolutional downsampling network structure is mainly composed of four consecutive second-order convolution modules, a third-order convolution module and an upsampling layer connected in sequence; The second-order convolution module is mainly composed of a first convolution layer with stride=2 for downsampling, a second convolution layer of 1×1, a batch normalization layer and a RELU activation layer connected in sequence; The third-order convolution module is formed by adding a bottleneck layer on the basis of the second-order convolution module. The specific position of the bottleneck layer is after the two convolution layers and before the batch normalization layer in the second-order convolution module. The upsampling layer is performed using a nearest neighbor interpolation algorithm; The outputs of the third second-order convolution module and the upsampling layer are selected as the feature tensor of the shallower layer and the feature tensor of the deeper layer, respectively.

3. According to claim 2, the method for extracting gesture images based on the improved DenseNet network is characterized in that: The second-order convolution module f2(z) and the third-order convolution module f3(z) are specifically expressed according to the following formulas: Among them, z represents the input image, represents a convolution function with a kernel size of i×i and an output channel number of n. a, b, and c represent the specific number of channels output by the first convolution layer, the second convolution layer, and the bottleneck layer of different convolution modules in the convolution downsampling network structure. BN(*) represents the batch normalization function. δ(*) represents the RELU activation function.

4. According to claim 1, the method for extracting gesture image features based on the improved DenseNet network is characterized in that: The DenseNet-B module is mainly composed of multiple dense layers, each of which is composed of two consecutive convolution modules BRC. The two convolution modules BRC have the same topological structure but different parameters. The convolution module BRC is composed of a batch normalization layer, a RELU activation layer and a single convolution layer connected in sequence; the difference between the two convolution modules BRC is that the size of the convolution kernel in the single convolution layer is different; the convolution module BRC is represented by the following formula: Where z represents the input image, BN(*) represents the batch normalization function, δ(*) represents the RELU activation function, Represents the convolution function with a convolution kernel size of i×i and an output channel number of n; BRC(*) represents the function of the convolution module BRC.

5. According to claim 4, the method for extracting gesture image features based on the improved DenseNet network is characterized in that: In the DenseNet-B module, the input of the first dense layer is the feature tensor selected from the convolutional downsampling network structure, and the input of each of the remaining dense layers includes the feature tensor selected from the convolutional downsampling network structure and the output of all dense layers located in front of it, and the feature tensor selected from the convolutional downsampling network structure and the output of all dense layers are connected together as the output of the DenseNet-B module; the DenseNet-B module is represented by the following formula: Among them, X0 represents the input of the first dense layer, X l represents the output of the Lth dense layer, H l Represents the convolution module BRC with two convolution kernels of different sizes in layer L. Represents superposition in the channel dimension.

6. The method for extracting gesture image features based on an improved DenseNet network according to claim 1, characterized in that: The DenseNet-B module fused with the Drop-Path module is to set the Drop-Path function before each dense layer on the DenseNet-B module to randomly discard the input sub-path; The DenseNet-B module fused with the Drop-Path module is set up and represented by the following formula: Among them, Drop-Path n (*) indicates the Drop-Path function, and the subscript n indicates the set drop rate.

7. The method for extracting gesture image features based on an improved DenseNet network according to claim 1, characterized in that: The transition layer is mainly composed of a convolutional pooling module BRCM. The convolutional pooling module BRCM adds a maximum pooling layer at the end on the basis of the convolutional module BRC. The convolutional pooling module BRCM can be expressed by the following formula: in, Represents a maximum pooling layer with a stride of 2 and a convolution kernel size of 2×2.

Images