Lightweight detection method and system for human body key point detection and storage medium

Abstract

The invention belongs to the technical field of computer vision, and particularly relates to a lightweight detection method and system for human body key point detection and a storage medium. The system comprises: an input module configured to input an image to be detected; the feature extraction module is configured to input the to-be-detected image into a feature extraction network to obtain features; the detection module is configured to input the features obtained by the feature extraction module into a decoder to obtain a human body key point detection result; wherein the feature extraction network comprises a lightweight backbone network, a pyramid pooling network module and a first pulse convolution module. The invention further provides a lightweight detection method for human body key point detection using the system. The method and the system have the advantages of saving computing resources and being high in prediction performance, and have good application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a lightweight detection method, system and storage medium for detecting key points of a human body. Background Art

[0002] Human keypoint detection is a fundamental and critical technology in computer vision. It involves identifying and locating specific anatomical locations in human images, such as joints and limb connections. This technology has broad applications in various fields, including but not limited to action recognition, augmented reality, human-computer interaction, health monitoring, and sports analysis. With the rise of deep learning technology, the research and application of human keypoint detection has made significant progress, especially in terms of accuracy and real-time performance.

[0003] The challenge of detecting keypoints in humans lies in handling diverse poses, occlusions, lighting variations, and complex backgrounds in various scenarios. To overcome these challenges, researchers have developed a variety of algorithms and models, including methods based on convolutional neural networks (CNNs), which can learn rich feature representations from large amounts of annotated data. Furthermore, some research is focused on improving the generalization and robustness of models, enabling them to maintain good performance even on unseen data.

[0004] While human keypoint detection technology has achieved considerable success, many challenges remain, such as improving model accuracy in complex scenarios. However, improving model accuracy often requires increasing model complexity, which in turn increases computing resource consumption. Achieving better performance while maintaining a lightweight model is a key challenge in developing new models in this field. Summary of the Invention

[0005] In response to the problems of the prior art, the present invention provides a lightweight detection method, system and storage medium for detecting key points of the human body.

[0006] A lightweight detection system for detecting key points of a human body, comprising: The input module is configured to input an image to be detected; The feature extraction module is configured to input the image to be detected into a feature extraction network to obtain features; A detection module is configured to input the features obtained by the feature extraction module into a decoder to obtain a human body key point detection result; The feature extraction network includes a lightweight backbone network, a pyramid pooling network module and a first nonlinear pulse convolution module; The lightweight backbone network includes a layer layer and a feature fusion network. The layer layer is used to perform preliminary feature extraction to obtain initial features. The feature fusion network is used to downsample the initial features and fuse them to obtain backbone features. The pyramid pooling network module is used to process the backbone features to obtain features of multi-scale information fusion; The first nonlinear pulse convolution module is used to process the initial features and extract 2D features; The features input to the decoder include the multi-scale information fusion features and the 2D features.

[0007] Preferably, it also includes: The preprocessing module is configured to preprocess the image to be detected before the image to be detected is input into the feature extraction module.

[0008] Preferably, the preprocessing includes normalization, denoising and random image rotation.

[0009] Preferably, the layer converts the dimension of the input image from 3 dimensions to 48 dimensions to obtain the initial features.

[0010] Preferably, the feature fusion network downsamples the initial features to obtain a 96-dimensional feature map; then fuses and downsamples the 48-dimensional initial features with the 96-dimensional feature map to obtain three feature maps of 48 dimensions, 96 dimensions, and 144 dimensions; finally, the three feature maps are fused to obtain the 48-dimensional backbone features.

[0011] Preferably, in the pyramid pooling network module, the backbone features are processed in sequence as follows: After passing through the second nonlinear pulse convolution module with a dilation rate of 1, the third nonlinear pulse convolution module with a dilation rate of 2, and the average pooling layer, the three outputs are spliced; Input the data into a fourth nonlinear pulse convolution module for processing, wherein the input feature of the fourth nonlinear pulse convolution module is 144-dimensional and the output feature is 48-dimensional; The four outputs are concatenated after passing through the first nonlinear pulse convolution module with a dilation rate of 1, the second nonlinear pulse convolution module with a dilation rate of 3, the third nonlinear pulse convolution module with a dilation rate of 6, and the average pooling layer; Input the data into a fifth nonlinear pulse convolution module for processing, wherein the input feature of the fifth pulse convolution module is 192 dimensions and the output feature is 48 dimensions; After passing through the sixth nonlinear pulse convolution module with a dilation rate of 1, the seventh nonlinear pulse convolution module with a dilation rate of 4, the eighth nonlinear pulse convolution module with a dilation rate of 8, the ninth nonlinear pulse convolution module with a dilation rate of 12 and the average pooling layer, the five outputs are spliced; The data is input into a tenth nonlinear pulse convolution module for processing. The input feature of the tenth nonlinear pulse convolution module is 240-dimensional, and the output feature is 48-dimensional.

[0012] Preferably, in the pyramid pooling network module, the outputs of all nonlinear pulse convolution modules are 48-dimensional.

[0013] Preferably, the decoder is selected from a basic convolutional module for classification, and the output dimension corresponds to the key point categories of the selected dataset (17 categories for the COCO dataset and 16 categories for the MPII dataset).

[0014] The present invention also provides a lightweight detection method for detecting key points of a human body, which is implemented by applying the lightweight detection system for detecting key points of a human body, and includes the following steps: Input the image to be detected; Inputting the image to be detected into a feature extraction network to obtain features; The features obtained by the feature extraction module are input into the decoder to obtain the human body key point detection result; The feature extraction network includes a lightweight backbone network, a pyramid pooling network module and a first nonlinear pulse convolution module; The lightweight backbone network includes a layer layer and a feature fusion network. The layer layer is used to perform preliminary feature extraction to obtain initial features. The feature fusion network is used to downsample the initial features and fuse them to obtain backbone features. The pyramid pooling network module is used to process the backbone features to obtain features of multi-scale information fusion; The first nonlinear pulse convolution module is used to process the initial features and extract 2D features; The features input to the decoder include the multi-scale information fusion features and the 2D features.

[0015] The present invention also provides a computer-readable storage medium storing: a computer program for implementing the above-mentioned lightweight detection system for detecting key points of the human body, or a computer program for implementing the above-mentioned lightweight detection method for detecting key points of the human body.

[0016] The present invention constructs a feature extraction network for the task of detecting key points on the human body. This feature extraction network achieves good detection performance by extracting features while ensuring lightweightness. Therefore, the present invention combines the advantages of saving hardware computing resources with high detection performance. Specifically, the feature extraction network of the present invention achieves the above advantages through the following design: 1. The lightweight backbone network structure design and module combination of the present invention can achieve high performance under the condition of low parameter count; 2. The pyramid pooling network module (PASP-SNP) can strengthen global image processing and further enhance performance; 3. The nonlinear spike convolution (ConvSNP) module can optimize performance without increasing the amount of computing resources.

[0017] Obviously, based on the above disclosure, various other modifications, substitutions, or changes may be made in accordance with common technical knowledge and customary means in the art, without departing from the basic technical principles of the present invention. Numerals such as "first," "second," and "third" serve only to distinguish different terms and do not indicate any priority or ranking, nor do they limit the scope of the technical features described.

[0018] The following further describes the above content of the present invention in detail through specific embodiments in the form of examples. However, this should not be construed as limiting the scope of the above subject matter of the present invention to the following examples. All technologies implemented based on the above content of the present invention fall within the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the structure of the feature extraction network in Example 1 of the present invention; Figure 2 Schematic diagram of the structure of the feature fusion network in Example 1 of the present invention; Figure 3 Schematic diagram of the structure of the pyramid pooling network module in Example 1 of the present invention; Figure 4 Schematic diagram of the structure of the pulse convolution module in Example 1 of the present invention. DETAILED DESCRIPTION

[0020] It should be noted that the algorithms for data collection, transmission, storage and processing steps not specifically described in the embodiments, as well as the hardware structures, circuit connections, etc. not specifically described can all be implemented through the disclosed content of the prior art.

[0021] Example 1 Lightweight detection system and method for human body key point detection The system of this embodiment includes: The input module is configured to input an image to be detected; A preprocessing module is configured to preprocess the image to be detected before the image to be detected is input into the feature extraction module; The feature extraction module is configured to input the image to be detected into a feature extraction network to obtain features; The detection module is configured to input the features obtained by the feature extraction module into a decoder to obtain a human body key point detection result.

[0022] The method for detecting key points of a human body using this system includes the following steps: Step 1: Input the image to be detected and perform image preprocessing; Step 2: Input the image to be detected into a feature extraction network to obtain features; Step 3: Input the features obtained by the feature extraction module into the decoder to obtain the human body key point detection results.

[0023] In this embodiment, the feature extraction network structure is as follows: Figures 1 to 4 As shown, it specifically includes: a lightweight backbone network, a pyramid pooling network module and a first pulse convolution module.

[0024] The lightweight backbone network includes a layer layer and a feature fusion network. The layer layer converts the dimension of the input image from 3 dimensions to 48 dimensions to obtain the initial features.

[0025] The feature fusion network is used to downsample the initial features and then fuse them. The feature fusion network downsamples the initial features to obtain a 96-dimensional feature map; then the 48-dimensional initial features are fused and downsampled with the 96-dimensional feature map to obtain three feature maps of 48 dimensions, 96 dimensions, and 144 dimensions. Finally, the three feature maps are fused to obtain the 48-dimensional backbone features.

[0026] The pyramid pooling network (PASP-SNP) module is used to process the backbone features to obtain multi-scale information fusion features. Specifically, the backbone features are processed in the following order: After passing through the second nonlinear pulse convolution module with a dilation rate of 1, the third nonlinear pulse convolution module with a dilation rate of 2, and the average pooling layer, the three outputs are spliced; Input the data into a fourth nonlinear pulse convolution module for processing, wherein the input feature of the fourth nonlinear pulse convolution module is 144-dimensional and the output feature is 48-dimensional; The four outputs are concatenated after passing through the first nonlinear pulse convolution module with a dilation rate of 1, the second nonlinear pulse convolution module with a dilation rate of 3, the third nonlinear pulse convolution module with a dilation rate of 6, and the average pooling layer; Inputting the data into a fifth nonlinear pulse convolution module for processing, wherein the input feature of the fifth nonlinear pulse convolution module is 192 dimensions and the output feature is 48 dimensions; After passing through the sixth nonlinear pulse convolution module with a dilation rate of 1, the seventh nonlinear pulse convolution module with a dilation rate of 4, the eighth nonlinear pulse convolution module with a dilation rate of 8, the ninth nonlinear pulse convolution module with a dilation rate of 12 and the average pooling layer, the five outputs are spliced; The data is input into a tenth nonlinear pulse convolution module for processing. The input feature of the tenth nonlinear pulse convolution module is 240-dimensional, and the output feature is 48-dimensional.

[0027] In PASP-SNP, the output of all nonlinear spike convolution modules is 48-dimensional.

[0028] The first nonlinear pulse convolution module is used to process the initial features and extract two-dimensional features.

[0029] Ultimately, the features input to the decoder include the multi-scale information fusion features obtained through PASP-SNP and the two-dimensional features. Because the model already performs sufficient feature extraction, the decoder uses a basic classification convolutional module, and the output feature dimensions correspond to the keypoint types in the selected dataset (17 for the COCO dataset and 16 for the MPII dataset).

[0030] The technical solution of the present invention is further illustrated by experiments below.

[0031] Experimental Example 1 Performance comparison of different lightweight networks for human key point detection 1. Experimental Methods In order to demonstrate the excellent performance of the lightweight network provided by the present invention for the task of human key point detection, this experimental example uses the COCO verification set and the MPII dataset as input to verify the performance difference between the lightweight feature fusion network of the present application and the model of the prior art.

[0032] 2. Experimental Results The results are shown in Tables 1 and 2.

[0033] Table 1 Experimental comparison between the proposed method and existing methods on the COCO validation set Table 2 Experimental comparison between the proposed method and existing methods on the MPII dataset As can be seen from the above table, Example 1 surpasses other existing lightweight models in all evaluation indicators in both datasets while maintaining a low parameter count. This shows that the lightweight detection system and method provided by Example 1 has significant performance advantages over existing lightweight models and also outperforms some complex network models (models with larger parameter counts).

[0034] It can be seen from the above embodiments and experimental examples that the present invention constructs a method and system for the task of human key point detection that has both computational resource savings and high prediction performance, and has good application prospects.

Claims

1. A lightweight detection system for key point detection of the human body, characterized in that: include: The input module is configured to input an image to be detected; The feature extraction module is configured to input the image to be detected into a feature extraction network to obtain features; A detection module is configured to input the features obtained by the feature extraction module into a decoder to obtain a human body key point detection result; The feature extraction network includes a lightweight backbone network, a pyramid pooling network module and a first nonlinear pulse convolution module; The lightweight backbone network includes a layer layer and a feature fusion network. The layer layer is used to perform preliminary feature extraction to obtain initial features. The feature fusion network is used to downsample the initial features and fuse them to obtain backbone features. The pyramid pooling network module is used to process the backbone features to obtain features of multi-scale information fusion; The first nonlinear pulse convolution module is used to process the initial features and extract 2D features; The features input to the decoder include the multi-scale information fusion features and the 2D features.

2. The lightweight detection system for detecting key points of a human body according to claim 1, characterized in that: Also includes: The preprocessing module is configured to preprocess the image to be detected before the image to be detected is input into the feature extraction module.

3. The lightweight detection system for detecting key points of a human body according to claim 2, characterized in that: The preprocessing includes normalization, denoising and random image rotation.

4. The lightweight detection system for detecting key points of a human body according to claim 1, characterized in that: The layer converts the dimension of the input image from 3 dimensions to 48 dimensions to obtain the initial features.

5. The lightweight detection system for detecting key points of a human body according to claim 1 or 4, characterized in that: The feature fusion network downsamples the initial features to obtain a 96-dimensional feature map; then fuses and downsamples the 48-dimensional initial features with the 96-dimensional feature map to obtain three feature maps of 48 dimensions, 96 dimensions, and 144 dimensions; finally, the three feature maps are fused to obtain the 48-dimensional backbone features.

6. The lightweight detection system for detecting key points of a human body according to claim 1, characterized in that: In the pyramid pooling network module, the backbone features are processed in the following order: After passing through the second nonlinear pulse convolution module with a dilation rate of 1, the third nonlinear pulse convolution module with a dilation rate of 2, and the average pooling layer, the three outputs are spliced; Input the data into a fourth nonlinear pulse convolution module for processing, wherein the input feature of the fourth nonlinear pulse convolution module is 144-dimensional and the output feature is 48-dimensional; The four outputs are concatenated after passing through the first nonlinear pulse convolution module with a dilation rate of 1, the second nonlinear pulse convolution module with a dilation rate of 3, the third nonlinear pulse convolution module with a dilation rate of 6, and the average pooling layer; Input the data into a fifth nonlinear pulse convolution module for processing, wherein the input feature of the fifth pulse convolution module is 192 dimensions and the output feature is 48 dimensions; After passing through the sixth nonlinear pulse convolution module with a dilation rate of 1, the seventh nonlinear pulse convolution module with a dilation rate of 4, the eighth nonlinear pulse convolution module with a dilation rate of 8, the ninth nonlinear pulse convolution module with a dilation rate of 12 and the average pooling layer, the five outputs are spliced; The data is input into a tenth nonlinear pulse convolution module for processing. The input feature of the tenth nonlinear pulse convolution module is 240-dimensional, and the output feature is 48-dimensional.

7. The lightweight detection system for detecting key points of a human body according to claim 6, characterized in that: In the pyramid pooling network module, the outputs of all nonlinear pulse convolution modules are 48-dimensional.

8. The lightweight detection system for detecting key points of a human body according to claim 1, characterized in that: The decoder is selected from the basic convolutional module for classification, and the output dimension corresponds to the key point categories of the selected dataset.

9. A lightweight detection method for key point detection of the human body, characterized in that: The lightweight detection system for detecting key points of a human body according to any one of claims 1 to 8 is implemented, comprising the following steps: Input the image to be detected; Inputting the image to be detected into a feature extraction network to obtain features; The features obtained by the feature extraction module are input into the decoder to obtain the human body key point detection result; The feature extraction network includes a lightweight backbone network, a pyramid pooling network module and a first nonlinear pulse convolution module; The lightweight backbone network includes a layer layer and a feature fusion network. The layer layer is used to perform preliminary feature extraction to obtain initial features. The feature fusion network is used to downsample the initial features and fuse them to obtain backbone features. The pyramid pooling network module is used to process the backbone features to obtain features of multi-scale information fusion; The first nonlinear pulse convolution module is used to process the initial features and extract 2D features; The features input to the decoder include the multi-scale information fusion features and the 2D features.

10. A computer-readable storage medium, characterized in that Stored thereon is: a computer program for implementing the lightweight detection system for human key point detection as described in any one of claims 1 to 8, or a computer program for implementing the lightweight detection method for human key point detection as described in claim 9.

Images