A gesture recognition method based on MPE-YOLOv5

Abstract

A gesture recognition method based on MPE-YOLOv5, belonging to the field of computer vision. This invention solves the problems of low detection accuracy and high classification error rate for small targets in existing lightweight network models. The main technical solutions adopted by the method of the present invention are: Step 1, collecting gesture images and constructing a training dataset based on the collected gesture images; Step 2, building an MPE-YOLOv5 network and training the MPE-YOLOv5 network using the constructed training dataset; Step 3, using the trained MPE-YOLOv5 network to detect a single frame of video image to be detected, and obtaining a gesture target positioning result and classification result in the image to be detected. The method of the present invention can be applied to gesture recognition.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a gesture recognition method based on MPE-YOLOv5. Background Art

[0002] With the widespread use of convolutional neural networks in computer vision, the use of deep learning for gesture recognition has become a hot research trend. Compared with traditional gesture recognition methods, gesture recognition methods based on deep learning offer greater robustness and flexibility in complex scenarios, while ensuring fast inference and classification with high recognition accuracy.

[0003] While the recognition accuracy of convolutional neural network models continues to improve, these high-precision models come at the cost of increasing network depth, a growing number of network parameters, and an increasingly bloated and complex network structure. Direct deployment on edge devices is impossible due to hardware limitations, overly complex models, and overly large inference frameworks. Therefore, how to fully utilize limited computing resources and environments, as well as how to design specialized network models for these devices, remain pressing challenges.

[0004] Lightweight networks significantly alleviate the challenges of limited computing power and memory read / write capabilities on edge devices due to a lack of powerful hardware. Lightweight networks optimize convolutional layers, significantly reducing memory read / write operations, network model parameters and weights, and the output of intermediate layers of the convolutional neural network. However, due to the smaller number of parameters and extracted features, lightweight network models can lead to reduced detection accuracy for small objects and higher classification error rates. Summary of the Invention

[0005] The purpose of this invention is to solve the problems of low detection accuracy and high classification error rate of small targets in existing lightweight network models, and to propose a gesture recognition method based on MPE-YOLOv5.

[0006] The technical solution adopted by the present invention to solve the above technical problems is:

[0007] A gesture recognition method based on MPE-YOLOv5, the method specifically comprising the following steps:

[0008] Step 1: Collect gesture images and build a training dataset based on the collected gesture images;

[0009] Step 2: Build the MPE-YOLOv5 network and train the MPE-YOLOv5 network using the constructed training data set;

[0010] The structure of the MPE-YOLOv5 network includes three parts: Backbone, Neck and Head, where:

[0011] The Backbone part includes CBH module, Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit; Stage2 unit includes a MobileNetv3 module and an ECA attention mechanism module, Stage3 unit includes two MobileNetv3 modules and an ECA attention mechanism module, Stage4 unit includes five MobileNetv3 modules and an ECA attention mechanism module, and Stage5 unit includes three MobileNetv3 modules, an SPPF module and an ECA attention mechanism module;

[0012] The Neck part includes the first C3 module to the sixth C3 module, the first CBS module to the sixth CBS module, and the first upsampling layer to the third upsampling layer;

[0013] The head part includes the first to fourth convolutional layers;

[0014] And the connection relationship between the various parts in the MPE-YOLOv5 network is:

[0015] The input of the MPE-YOLOv5 network is used as the input of the CBH module, and the output of the CBH module passes through the Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit in sequence;

[0016] The output of the ECA attention mechanism module in the Stage5 unit is used as the input of the fifth CBS module, and the output of the fifth CBS module is input to the third upsampling layer. The output of the ECA attention mechanism module in the Stage4 unit is spliced with the output of the third upsampling layer, and the spliced result is used as the input of the fifth C3 module;

[0017] The output of the fifth C3 module passes through the third CBS module and the second upsampling layer in sequence, and then the output of the second upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage3 unit, and the spliced result is used as the input of the third C3 module;

[0018] The output of the third C3 module passes through the second CBS module and the first upsampling layer in sequence, and then the output of the first upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage2 unit, and the spliced result is used as the input of the first C3 module;

[0019] The output of the first C3 module is used as the input of the first convolutional layer of the Head part;

[0020] The output of the first C3 module is used as the input of the first CBS module. The output of the first CBS module is then concatenated with the output of the second CBS module, and the concatenated result is used as the input of the second C3 module. The output of the second C3 module is used as the input of the second convolutional layer of the Head part.

[0021] The output of the second C3 module is used as the input of the fourth CBS module. The output of the fourth CBS module is then concatenated with the output of the third CBS module, and the concatenated result is used as the input of the fourth C3 module. The output of the fourth C3 module is used as the input of the third convolutional layer of the Head part.

[0022] The output of the fourth C3 module is used as the input of the sixth CBS module. The output of the fifth CBS module is then concatenated with the output of the sixth CBS module, and the concatenated result is used as the input of the sixth C3 module. The output of the sixth C3 module is used as the input of the fourth convolutional layer of the Head part.

[0023] The gesture target positioning result and gesture classification result are output through the first to fourth convolutional layers of the Head part;

[0024] Step 3: Use the trained MPE-YOLOv5 network to detect the single-frame video image to be detected, and obtain the gesture target positioning result and classification result in the image to be detected.

[0025] Furthermore, the specific process of step one is:

[0026] Step 11: Use a mobile device to capture a gesture image;

[0027] Step 12: performing data amplification on the gesture image captured in step 11 to obtain a data-amplified gesture image; the data amplification method includes cropping, translation, mirroring, changing brightness, and adding Gaussian noise;

[0028] Step 13: randomly select four gesture images from the gesture images captured in step 11, and then stitch the four selected gesture images to obtain a stitched gesture image;

[0029] Similarly, multiple stitched gesture images are obtained;

[0030] Step 14: Use the gesture images after data amplification and the spliced gesture images to form a training data set, and annotate the images in the training data set with category labels.

[0031] Furthermore, the CBH module consists of a 3×3 convolutional layer, a batch normalization layer, and a nonlinear activation function h-swish layer.

[0032] Furthermore, in the Stage 2 unit, the working principle of the ECA attention mechanism module is as follows:

[0033] The input of the MobileNetv3 module is denoted as X. The feature map output by the MobileNetv3 module passes through the ECA attention mechanism module, and then the output of the ECA attention mechanism module is fused with the output of the MobileNetv3 module. As input to the Stage3 unit.

[0034] Furthermore, the ECA attention mechanism module is composed of a global pooling layer, a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer, and a Sigmoid activation function layer in sequence.

[0035] Furthermore, the batch size in the MPE-YOLOv5 network training process is 16, the training iteration cycle is 200, the initial learning rate is 0.01, the final learning rate is 0.1, and the weight decay rate is 0.0005.

[0036] The beneficial effects of the present invention are:

[0037] This paper proposes an MPE-YOLOv5 network model to address the issues of large parameters, high model and computational complexity, and large memory usage in high-precision network models, thereby accelerating the inference speed of the network model. While accelerating inference speed, the MPE-YOLOv5 network model addresses the issues of reduced model accuracy, ineffective recognition of small targets, and inability to handle large-scale target variance in traditional lightweight network models due to their small number of parameters, thereby improving the detection and classification accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a gesture recognition method based on MPE-YOLOv5 of the present invention;

[0039] The preliminary test results include the detection accuracy and gesture classification results. The final test result recorded is the average value of the detection accuracy.

[0040] Figure 2 This is a sample picture of the gesture taken;

[0041] Figure 3 Yes Figure 2 The gesture image after data enhancement of type A gesture in [1].

[0042] Figure 4 Yes Figure 2 Gesture images after data enhancement of H gestures in

[15] ;

[0043] Figure 5Yes Figure 2 Gesture images after data enhancement of U gestures in [1].

[0044] Figure 6 is the spliced gesture image sample;

[0045] Figure 7 This is an example of the sample annotation of the gesture image after splicing;

[0046] Figure 8 This is the architecture diagram of the MPE-YOLOv5 network;

[0047] Figure 9 This is a schematic diagram of the network structure after adding the detection layer;

[0048] Figure 10 This is a schematic diagram of the structure of the backbone network after integrating the ECA attention mechanism module;

[0049] Figure 11 This is a schematic diagram of the ECA-MBv3Conv module;

[0050] Figure 12 This is a graph showing the relationship between the number of training iterations and accuracy of the gesture dataset for the three network models;

[0051] Figure 13 This is a graph showing the relationship between the number of training iterations and the loss function for the network model MPE-YOLOv5 dataset. DETAILED DESCRIPTION

[0052] Specific implementation method 1. Combination Figure 1 and Figure 8 This embodiment describes a gesture recognition method based on MPE-YOLOv5, which specifically includes the following steps:

[0053] Step 1: Collect gesture images and build a training dataset based on the collected gesture images;

[0054] Step 2: Build the MPE-YOLOv5 network and train the MPE-YOLOv5 network using the constructed training data set;

[0055] The structure of the MPE-YOLOv5 network includes three parts: Backbone, Neck and Head, where:

[0056] The Backbone part includes CBH module, Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit; Stage2 unit includes a MobileNetv3 module and an ECA attention mechanism module, Stage3 unit includes two MobileNetv3 modules and an ECA attention mechanism module, Stage4 unit includes five MobileNetv3 modules and an ECA attention mechanism module, and Stage5 unit includes three MobileNetv3 modules, an SPPF module and an ECA attention mechanism module;

[0057] The MobileNetv3 module with a stride of 1 is denoted as MBv3Conv_1, and the MobileNetv3 module with a stride of 2 is denoted as MBv3Conv_2. The MBv3Conv module first upscales the original feature map using a 1×1 convolutional layer. It then extracts features layer by layer through 3×3 convolutional layers in a depthwise separable convolution. A 1×1 convolutional layer then fuses cross-channel information to form a new feature map. After obtaining the new feature map, the SE attention mechanism (consisting of a pooling layer and two fully connected layers) is used to obtain spatial weights to optimize model performance. When the number of input and output channels is the same, an inverse residual connection is performed.

[0058] The Neck part includes the first C3 module (the same structure as the C3 module in the traditional YOLOv5 network) to the sixth C3 module, the first CBS module (the same structure as the CBS module in the traditional YOLOv5 network) to the sixth CBS module, and the first upsampling layer to the third upsampling layer;

[0059] The head part includes the first to fourth convolutional layers;

[0060] And the connection relationship between the various parts in the MPE-YOLOv5 network is:

[0061] The input of the MPE-YOLOv5 network is used as the input of the CBH module, and the output of the CBH module passes through the Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit in sequence;

[0062] The output of the ECA attention mechanism module in the Stage5 unit is used as the input of the fifth CBS module, and the output of the fifth CBS module is input to the third upsampling layer. The output of the ECA attention mechanism module in the Stage4 unit is spliced with the output of the third upsampling layer, and the spliced result is used as the input of the fifth C3 module;

[0063] The output of the fifth C3 module passes through the third CBS module and the second upsampling layer in sequence, and then the output of the second upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage3 unit, and the spliced result is used as the input of the third C3 module;

[0064] The output of the third C3 module passes through the second CBS module and the first upsampling layer in sequence, and then the output of the first upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage2 unit, and the spliced result is used as the input of the first C3 module;

[0065] The output of the first C3 module is used as the input of the first convolutional layer of the Head part;

[0066] The output of the first C3 module is used as the input of the first CBS module. The output of the first CBS module is then concatenated with the output of the second CBS module, and the concatenated result is used as the input of the second C3 module. The output of the second C3 module is used as the input of the second convolutional layer of the Head part.

[0067] The output of the second C3 module is used as the input of the fourth CBS module. The output of the fourth CBS module is then concatenated with the output of the third CBS module, and the concatenated result is used as the input of the fourth C3 module. The output of the fourth C3 module is used as the input of the third convolutional layer of the Head part.

[0068] The output of the fourth C3 module is used as the input of the sixth CBS module. The output of the fifth CBS module is then concatenated with the output of the sixth CBS module, and the concatenated result is used as the input of the sixth C3 module. The output of the sixth C3 module is used as the input of the fourth convolutional layer of the Head part.

[0069] The gesture target positioning result and gesture classification result are output through the first to fourth convolutional layers of the Head part;

[0070] In the MPE-YOLOv5 network model of the present invention, a detection layer for small targets is added (which is more sensitive to small gesture target detection). That is, there are four detection layers, which detect tiny, small, medium, and large targets respectively. This can alleviate the negative impact of drastic changes in the recognition target ratio and the accuracy degradation caused by the lightweight feature extraction network. The structural diagram is shown in FIG. Figure 9 shown.

[0071] Step 3: Use the trained MPE-YOLOv5 network to detect the single-frame video image to be detected, and obtain the gesture target positioning result and classification result in the image to be detected.

[0072] To address the difficulty in deploying complex convolutional neural network models on edge devices, this paper replaces the backbone network CSPDarknet53 of the classic single-stage target detection model YOLOv5 with a lightweight network MobileNetv3, resulting in a network model YOLOv5-MobileNetv3. Furthermore, the lightweight network model YOLOv5-MobileNetv3 addresses issues such as decreased model accuracy due to the reduction in parameters, inability to effectively identify small targets, and inability to handle large-scale target variance.

[0073] The proposed MPE-YOLOv5 network architecture integrates the ECA attention mechanism module into the operations of the intermediate convolutional layers of MobileNetv3 for adaptive feature refinement. This reduces the loss of contextual information in high-level feature maps due to the reduction of feature channels. Furthermore, a detection layer for small objects is added, making it more sensitive to small gesture detection. The proposed fast gesture recognition method based on the MPE-YOLOv5 network achieves an optimal balance between model accuracy and inference time, making it suitable for gesture recognition on hardware-constrained edge mobile terminals.

[0074] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the specific process of step 1 is as follows:

[0075] Step 11: Use a mobile device to capture a gesture image;

[0076] Step 12: performing data amplification on the gesture image captured in step 11 to obtain a data-amplified gesture image; the data amplification method includes cropping, translation, mirroring, changing brightness, and adding Gaussian noise;

[0077] Step 13: randomly select four gesture images from the gesture images captured in step 11, and then stitch the four selected gesture images to obtain a stitched gesture image;

[0078] Similarly, multiple stitched gesture images are obtained;

[0079] The spliced gesture image samples and the annotation examples of the spliced gesture image samples are as follows: Figure 6 and Figure 7 As shown;

[0080] Step 14: Use the gesture images after data amplification and the spliced gesture images to form a training data set, and annotate the images in the training data set with category labels.

[0081] The captured gesture images include 26 gestures from A to Z. The gesture examples are as follows: Figure 2 As shown. New gesture data is generated by mixing the above data enhancement methods. For example, Figure 3 、 Figure 4 and Figure 5 As shown, Figure 3 For Figure 2 The gesture image after data enhancement of type A gesture in Figure 4 For Figure 2 The gesture image after data enhancement of H gestures in Figure 5 For Figure 2 Gesture images after data enhancement of U gestures in .

[0082] The specific method of image stitching in this embodiment is:

[0083] Randomly select four gesture samples for splicing, and use diagonal splicing as the splicing method; determine the pixels of the spliced image, extract the pixel values of the height and width of the four selected gesture samples, and scale them according to the extracted relative pixels so that high-pixel gesture samples account for a larger proportion in the spliced image, and low-pixel gesture samples account for a smaller proportion in the spliced image; determine the splicing center point, and select a vertex for each of the four gesture samples for splicing, that is, select the lower right corner of the upper left sample, the lower left corner of the upper right sample, the upper right corner of the lower left sample, and the upper left corner of the lower right sample as the splicing vertices; create a new empty large image, fill it with each small sample, and construct the spliced gesture image, that is, the large-scale transformed sample image.

[0084] The present invention uses the lightweight network MobileNetv3 as the feature extraction network of YOLOv5, adds a detection layer for small target detection to deal with the large-scale variance of the target, and integrates the attention mechanism module ECA into the backbone network to connect the Backbone part and the Neck part. This reduces the loss of contextual information in high-level feature maps caused by the reduction of feature channels, helps the network model find the target area in the feature map, and retains other modules and structures in the architecture, thus obtaining a gesture fast recognition network model MPE-YOLOv5.

[0085] Other steps and parameters are the same as those in the first embodiment.

[0086] Specific embodiment three: This embodiment differs from specific embodiment one or two in that: the CBH module consists of a 3×3 convolutional layer, a batch normalization layer, and a nonlinear activation function h-swish layer.

[0087] Other steps and parameters are the same as those in the first or second embodiment.

[0088] Specific implementation method 4: Combination Figure 10 and Figure 11This embodiment differs from any one of the first to third embodiments in that the working principle of the ECA attention mechanism module in the Stage 2 unit is as follows:

[0089] The input of the MobileNetv3 module is denoted as X. The feature map output by the MobileNetv3 module passes through the ECA attention mechanism module, and then the output of the ECA attention mechanism module is fused with the output of the MobileNetv3 module. As input to the Stage3 unit.

[0090] The working principle of the ECA attention mechanism module and the nearest MobileNetv3 module in the Stage3 unit, the working principle of the ECA attention mechanism module and the nearest MobileNetv3 module in the Stage4 unit, and the working principle of the ECA attention mechanism module and the SPPF module in the Stage5 unit are all the same as above.

[0091] The other steps and parameters are the same as those in the first to third embodiments.

[0092] Specific embodiment 5. The difference between this embodiment and any one of specific embodiments 1 to 4 is that the ECA attention mechanism module is composed of a global pooling layer, a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer and a Sigmoid activation function layer in sequence.

[0093] The attention mechanism is widely used to improve the performance of convolutional neural networks. Integrating the attention mechanism into the image feature extraction layer of a convolutional neural network can enhance the network's ability to extract image features. The ECA attention mechanism module is lightweight, ensuring plug-and-play integration into existing network frameworks. For lightweight network models, the attention mechanism can significantly improve overall performance without increasing the computational complexity of the network model.

[0094] like Figure 11 As shown in Figure 2, in order to obtain attention in the channel dimension, after the feature map is input, global average pooling is first performed on the width and height of the feature map to reduce the spatial feature dimension to 1×1, as shown in formula (1):

[0095]

[0096] Then, two 1×1 convolutional layers and a ReLU activation function layer are used to efficiently implement local cross-channel interaction, extract the dependencies between channels, and obtain the weight ω of each channel through the Sigmoid activation function, as shown in formula (2):

[0097] ω=σ(C1Dk (z c )) (2)

[0098] Multiply the weights by the corresponding elements of the original input feature map to obtain the final output feature map.

[0099] The ECA attention mechanism is integrated into the MBv3Conv module with different numbers of input channels and output channels, and the ECA attention mechanism is connected to the neck of the network model to reduce the loss of contextual information in high-level feature maps due to the reduction of feature channels.

[0100] The other steps and parameters are the same as those in the first to fourth embodiments.

[0101] Specific embodiment 6. The difference between this embodiment and any one of specific embodiments 1 to 5 is that the batch size in the MPE-YOLOv5 network training process is 16, the training iteration cycle is 200, the initial learning rate is 0.01, the final learning rate is 0.1, and the weight decay rate is 0.0005.

[0102] The other steps and parameters are the same as those in the first to fifth embodiments.

[0103] Experimental results of training the MPE-YOLOv5 model

[0104] In this paper, after merging the small object image data with the augmented image data, the combined data was divided into training, validation, and test sets in a ratio of 8:1.5:0.5. After merging, there were 20,898 gesture samples in total, with 15,890 in the training set, 3,252 in the validation set, and 1,756 in the test set.

[0105] The batch size in the training process is preset to be 16, the training iteration period is 200, the initial learning rate is 0.01, the final learning rate is 0.1, and the weight decay rate is 0.0005. The relationship between the number of training iterations and the accuracy of the gesture dataset of the three network models is as follows: Figure 12 As shown, the relationship between the number of training iterations and the loss function of the network model MPE-YOLOv5 dataset is as follows Figure 13 shown.

[0106] MPE-YOLOv5s network model detection experimental results and performance comparison

[0107] We conducted comparative experiments on the classic YOLOv5 network model, the lightweight MobileNetv3-YOLOv5 model (abbreviated as M-YOLOv5), and the optimized MEP-YOLOv5 network model (a model based on the lightweight MobileNetv3-YOLOv5 model with an object detection layer and ECA attention mechanism module added) to verify the effectiveness of the optimized MEP-YOLOv5 algorithm. Table 1 shows a performance comparison of the various network models.

[0108] Table 1 Performance comparison of various network models

[0109]

[0110] Table 1 compares the performance of various network models. As can be seen from Table 1, while the classic YOLOv5 network model achieves high detection accuracy, reaching 99.3%, its large number of parameters increases the computational complexity required for training, validation, and testing. This makes it unsuitable for deployment on edge devices with more hardware constraints, both in terms of inference speed and model memory usage. Replacing the backbone of the YOLOv5 network model with the lightweight MobileNetv3 network model significantly improves efficiency in terms of parameter count, model complexity, computational complexity, and inference speed. However, this efficiency improvement comes at the expense of model effectiveness. As shown in Table 1, the lightweight YOLOv5 model exhibits a 9.97% decrease in accuracy. However, since the model size is reduced by 0.22 times, its precision density is significantly improved, reaching 4.15 times that of the YOLOv5 network architecture.

[0111] Although the proposed MEP-YOLOv5 algorithm adds a detection layer for small objects, introducing a large number of parameters to the network model, resulting in higher computational complexity, larger model size, and more model parameters than the original lightweight algorithm, it achieves certain results by sacrificing network model efficiency in exchange for effectiveness. Its accuracy is improved by 7.72% compared to the lightweight model. Compared with the classic network model YOLOv5, its model parameter count, model size, and computational complexity are 21.16%, 25.33%, and 27.33% of the original, respectively, which is a significant improvement. YOLOv5 and the proposed new algorithm MEP-YOLOv5 have similar accuracy, with the new algorithm losing only 3.02% of its accuracy. This also verifies the effectiveness of the proposed MEP-YOLOv5 algorithm.

[0112] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A gesture recognition method based on MPE-YOLOv5, characterized in that: The method specifically comprises the following steps: Step 1: Collect gesture images and build a training dataset based on the collected gesture images; Step 2: Build the MPE-YOLOv5 network and train the MPE-YOLOv5 network using the constructed training data set; The structure of the MPE-YOLOv5 network includes three parts: Backbone, Neck and Head, where: The Backbone part includes CBH module, Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit; Stage2 unit includes a MobileNetv3 module and an ECA attention mechanism module, Stage3 unit includes two MobileNetv3 modules and an ECA attention mechanism module, Stage4 unit includes five MobileNetv3 modules and an ECA attention mechanism module, and Stage5 unit includes three MobileNetv3 modules, an SPPF module and an ECA attention mechanism module; The Neck part includes the first C3 module to the sixth C3 module, the first CBS module to the sixth CBS module, and the first upsampling layer to the third upsampling layer; The head part includes the first to fourth convolutional layers; And the connection relationship between the various parts in the MPE-YOLOv5 network is: The input of the MPE-YOLOv5 network is used as the input of the CBH module, and the output of the CBH module passes through the Stage2 unit, Stage3 unit, Stage4 unit and Stage5 unit in sequence; The output of the ECA attention mechanism module in the Stage5 unit is used as the input of the fifth CBS module, and the output of the fifth CBS module is input to the third upsampling layer. The output of the ECA attention mechanism module in the Stage4 unit is spliced with the output of the third upsampling layer, and the spliced result is used as the input of the fifth C3 module; The output of the fifth C3 module passes through the third CBS module and the second upsampling layer in sequence, and then the output of the second upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage3 unit, and the spliced result is used as the input of the third C3 module; The output of the third C3 module passes through the second CBS module and the first upsampling layer in sequence, and then the output of the first upsampling layer is spliced with the output of the ECA attention mechanism module in the Stage2 unit, and the spliced result is used as the input of the first C3 module; The output of the first C3 module is used as the input of the first convolutional layer of the Head part; The output of the first C3 module is used as the input of the first CBS module. The output of the first CBS module is then concatenated with the output of the second CBS module, and the concatenated result is used as the input of the second C3 module. The output of the second C3 module is used as the input of the second convolutional layer of the Head part. The output of the second C3 module is used as the input of the fourth CBS module. The output of the fourth CBS module is then concatenated with the output of the third CBS module, and the concatenated result is used as the input of the fourth C3 module. The output of the fourth C3 module is used as the input of the third convolutional layer of the Head part. The output of the fourth C3 module is used as the input of the sixth CBS module. The output of the fifth CBS module is then concatenated with the output of the sixth CBS module, and the concatenated result is used as the input of the sixth C3 module. The output of the sixth C3 module is used as the input of the fourth convolutional layer of the Head part. The gesture target positioning result and gesture classification result are output through the first to fourth convolutional layers of the Head part; Step 3: Use the trained MPE-YOLOv5 network to detect the single-frame video image to be detected, and obtain the gesture target positioning result and classification result in the image to be detected.

2. The gesture recognition method based on MPE-YOLOv5 according to claim 1, characterized in that: The specific process of step one is: Step 11: Use a mobile device to capture a gesture image; Step 12: performing data amplification on the gesture image captured in step 11 to obtain a data-amplified gesture image; the data amplification method includes cropping, translation, mirroring, changing brightness, and adding Gaussian noise; Step 13: randomly select four gesture images from the gesture images captured in step 11, and then stitch the four selected gesture images to obtain a stitched gesture image; Similarly, multiple stitched gesture images are obtained; Step 14: Use the gesture images after data amplification and the spliced gesture images to form a training data set, and annotate the images in the training data set with category labels.

3. The gesture recognition method based on MPE-YOLOv5 according to claim 2, characterized in that: The CBH module consists of a 3×3 convolutional layer, a batch normalization layer, and a nonlinear activation function h-swish layer.

4. The gesture recognition method based on MPE-YOLOv5 according to claim 3, characterized in that: In the Stage 2 unit, the working principle of the ECA attention mechanism module is as follows: The input of the MobileNetv3 module is denoted as X. The feature map output by the MobileNetv3 module passes through the ECA attention mechanism module, and then the output of the ECA attention mechanism module is fused with the output of the MobileNetv3 module. As input to the Stage3 unit.

5. The gesture recognition method based on MPE-YOLOv5 according to claim 4, characterized in that: The ECA attention mechanism module consists of a global pooling layer, a 1×1 convolution layer, a ReLU activation function layer, a 1×1 convolution layer, and a Sigmoid activation function layer in sequence.

6. The gesture recognition method based on MPE-YOLOv5 according to claim 5, characterized in that: The MPE-YOLOv5 network was trained with a batch size of 16, a training iteration period of 200, an initial learning rate of 0.01, a final learning rate of 0.1, and a weight decay rate of 0.0005.

Images