A method and system for detecting small human body posture in 2D images

Abstract

The present invention discloses a method and system for detecting a human posture of a small target in a 2D image, wherein the method comprises: obtaining a 2D image of a human target; performing image preprocessing on the 2D image of the human target; performing feature extraction processing on the preprocessed 2D image of the human target to obtain a target feature map; inputting the target feature map into a human posture detection model, and performing human posture detection processing of a small target based on the human posture detection model, wherein the human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training, and the deep neural network model comprises an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module, and a key point regression module. The present invention can better perform deeper feature fusion, improve the accuracy of small target detection, and better avoid the problems of false detection and missed detection.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and system for detecting small target human body posture in a 2D image. Background Art

[0002] Human pose estimation is a hot topic in basic research of computer vision and has attracted extensive attention from many scholars. There are more and more algorithms and applications for human pose estimation. However, the detection and positioning of small targets has always been a difficult point in human pose estimation methods. Small targets have less feature information, a small receptive field range and high background complexity, which leads to incomplete semantic information of small targets and insufficient feature representation ability. When dealing with complex backgrounds, there are often problems such as false detection or missed detection. In addition, due to the low resolution of small targets in feature maps, they are often prone to losing key semantic information during feature fusion, which leads to obvious deviations in key point positioning problems. At present, methods usually prefer deep and complex network structures for small target detection. However, due to the problems of complex calculation, small amount of semantic information and easy loss of feature extraction, context information will inevitably be lost during upsampling / downsampling. This information will continue to accumulate, which will have a serious impact on the final regression. Therefore, these methods all have low accuracy and high missed detection rates. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides a method and system for detecting small target human posture in a 2D image, which can better perform deeper feature fusion, improve the accuracy of small target detection, and better avoid the problems of false detection and missed detection.

[0004] In order to solve the above technical problems, the present invention provides a method for detecting a human body posture of a small target in a 2D image, the method comprising:

[0005] Acquire a 2D image of a human target;

[0006] Performing image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image;

[0007] Perform feature extraction on the preprocessed 2D image of the human target to obtain a target feature map;

[0008] The target feature map is input into a human posture detection model, and human posture detection processing of a small target is performed based on the human posture detection model. The human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training. The deep neural network model includes a HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module.

[0009] Optionally, performing image preprocessing on the human target 2D image to obtain the preprocessed human target 2D image includes:

[0010] Performing grayscale processing on the human target 2D image to obtain a grayscale human target 2D image;

[0011] Performing image enhancement processing on the grayscale human target 2D image to obtain the grayscale human target 2D image after image enhancement processing;

[0012] The grayscale human target 2D image after image enhancement is denoised to obtain a preprocessed human target 2D image.

[0013] Optionally, the HRNet backbone module is used to perform preliminary extraction on the feature map to obtain four feature maps with different resolutions, which are respectively a first feature map, a second feature map, a third feature map and a fourth feature map, and the four feature maps with different resolutions are input into the dynamic expansion receptive field module, the dynamic expansion receptive field module is used to perform further feature extraction on the feature maps with different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction, the efficient fusion semantic information module is used to perform efficient feature fusion processing on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map, and the key point regression module is used to perform key point regression processing on the enhanced feature map to obtain key points of human body posture.

[0014] Optionally, the dynamically expanded receptive field module includes a cascaded dilated convolution component and a gating mechanism component.

[0015] Optionally, the cascaded dilated convolution component includes a dilated convolution layer and two convolution layers, the dilated convolution layer is a dilated convolution layer with a dilation rate of [1, 2, 3] and a convolution kernel of 3*3, the convolution kernel of the two convolution layers is 3*3, and the gating mechanism module includes a global average pooling layer, two consecutive fully connected layers and a residual structure module.

[0016] Optionally, further extracting features from the four feature maps with different resolutions to obtain a first feature map, a second feature map, a third feature map, and a fourth feature map after further feature extraction includes:

[0017] Performing local feature aggregation processing on the first feature map, the second feature map, the third feature map, and the fourth feature map based on the dilated convolution layer in the cascaded dilated convolution component to obtain the first feature map, the second feature map, the third feature map, and the fourth feature map after the local feature aggregation processing;

[0018] Based on the first convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after local feature aggregation processing are subjected to feature fusion by using the BatchNorm function and the ReLU function to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion;

[0019] Based on the second convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are re-fused to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion;

[0020] The global average pooling layer based on the gating mechanism module acquires global information of the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion to obtain corresponding global information;

[0021] The fully connected layer based on the gating mechanism module extracts features from the first feature map, the second feature map, the third feature map, and the fourth feature map after feature fusion, and calculates the attention weight based on the sigmoid function using the feature extraction results and global information;

[0022] The attention weights are used for calibration based on the residual structure module.

[0023] Optionally, the step of performing efficient feature fusion on the first feature map, the second feature map, the third feature map, and the fourth feature map after further feature extraction to obtain an enhanced feature map includes:

[0024] The first feature map after further feature extraction is transformed using a feature enhancement structure to obtain a transformed first feature map;

[0025] Performing upsampling processing on the second feature map, the third feature map, and the fourth feature map after further feature extraction based on a bilinear interpolation method to obtain the second feature map, the third feature map, and the fourth feature map after upsampling processing;

[0026] Based on the Concat feature fusion method, feature map splicing is performed on the first feature map after transformation processing and the second feature map, the third feature map and the fourth feature map after upsampling processing to obtain a spliced ​​feature map;

[0027] A 3*3 convolution process is performed on the spliced ​​feature map to obtain an enhanced feature map.

[0028] Optionally, the Concat feature fusion method is used to perform feature map splicing processing on the first feature map after transformation processing and the second feature map, the third feature map and the fourth feature map after upsampling processing, and after obtaining the spliced ​​feature map, it also includes:

[0029] Based on an efficient channel attention mechanism, weights of different feature channels are assigned to the first feature map after transformation and the second, third and fourth feature maps after upsampling.

[0030] Optionally, the key point regression module adopts an adaptive convolutional neural network.

[0031] In addition, the present invention also provides a 2D image small target human posture detection system, the system comprising:

[0032] Image acquisition module: used to acquire 2D images of human targets;

[0033] An image preprocessing module is used to perform image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image;

[0034] Feature extraction module: used to perform feature extraction processing on the preprocessed 2D image of the human target to obtain a target feature map;

[0035] Human posture detection module: used to input the target feature map into the human posture detection model, and perform human posture detection processing for small targets based on the human posture detection model. The human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training. The deep neural network model includes an HRNet backbone module, a dynamically expanded receptive field module, an efficient fusion semantic information module and a key point regression module.

[0036] In an embodiment of the present invention, a human posture detection model is constructed by an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module. The HRNet backbone module can output four feature maps of different resolutions in parallel, and each feature map contains different semantic information. The dynamic expansion receptive field module can obtain enough feature information, dynamically expand the receptive field, and enhance the feature representation capability. The efficient fusion semantic information module can fully mine the contextual semantic feature information, better perform deeper feature fusion, improve the accuracy of small target detection, and better avoid the problems of false detection and missed detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 12D image small target human body posture detection method according to an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of the structural composition of a 2D image small target human posture detection system in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Embodiment 1

[0042] See also Figure 1 , Figure 1 It is a flow chart of a method for detecting a small target human body posture in a 2D image in an embodiment of the present invention.

[0043] like Figure 1 As shown, a method for detecting a human body posture of a small target in a 2D image comprises:

[0044] S11: Acquire a 2D image of a human target;

[0045] In the specific implementation process of the present invention, a 2D image of a human target is obtained by shooting with a high-resolution camera.

[0046] S12: performing image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image;

[0047] In the specific implementation process of the present invention, the image preprocessing of the human target 2D image to obtain the preprocessed human target 2D image includes: grayscale processing of the human target 2D image to obtain a grayscale human target 2D image; image enhancement processing of the grayscale human target 2D image to obtain a grayscale human target 2D image after image enhancement processing; denoising of the grayscale human target 2D image after image enhancement processing to obtain a preprocessed human target 2D image.

[0048] Specifically, grayscale processing is performed on the 2D image of the human target, pixel values ​​of the three colors of red, green and blue in the 2D image of the human target are calculated, and a pixel average value is calculated based on the pixel values ​​of the three colors of red, green and blue. According to the sensitivity of the human eye to the three colors of red, green and blue, the pixel average value is used to perform weighted averaging, and the 2D image of the human target is processed by the weighted average value to generate a grayscale 2D image of the human target; image enhancement processing is performed on the grayscale 2D image of the human target, frequency domain modulation is performed on the object field of the grayscale image of the human target, and Fourier transform is performed on the frequency domain to obtain the Fourier transform. The imaginary part of the frequency domain after the conversion is taken in absolute value, the 1 / n power is calculated after summing along the frequency, and the enhancement is performed along the vertical axis and the horizontal axis respectively to obtain a grayscale human target 2D image after image enhancement processing; the grayscale human target 2D image after image enhancement processing is denoised, the value of the filter is set, and the grayscale human target 2D image after image enhancement processing is filtered according to the filter after the value setting, and the average grayscale value of adjacent pixels in the grayscale human target 2D image is calculated, and the average grayscale value is assigned to the current pixel point, and the denoising process is completed to obtain the pre-processed human target 2D image.

[0049] S13: performing feature extraction processing on the preprocessed human target 2D image to obtain a target feature map;

[0050] In the specific implementation process of the present invention, a Haar feature classifier is constructed and trained. For the training process of the Haar feature classifier, the parameters required for training the classifier are defined, including the type and number of Haar features, the number of positive and negative samples, the number of classifier cascade layers and the learning rate. The positive sample is an image area containing a target, and the negative sample is an image area not containing a target. After setting the parameters, the train function is used to train the Haar feature classifier. During the training process, the model will continuously adjust the Haar features and set thresholds to remove unnecessary features, and finally form an adaptable and generalizable Haar feature classifier. The preprocessed human target 2D image is input into the trained Haar feature classifier, and feature extraction processing is performed based on the trained Haar feature classifier to obtain a target feature map.

[0051] S14: Input the target feature map into a human posture detection model, and perform human posture detection processing for small targets based on the human posture detection model. The human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training. The deep neural network model includes a HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module.

[0052] In the specific implementation process of the present invention, the HRNet backbone module is used to perform preliminary extraction on the feature map to obtain four feature maps with different resolutions, and the four feature maps with different resolutions are respectively a first feature map, a second feature map, a third feature map and a fourth feature map, and the four feature maps with different resolutions are input into the dynamic expansion receptive field module, the dynamic expansion receptive field module is used to perform further feature extraction on the feature maps with different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction, the efficient fusion semantic information module is used to perform efficient feature fusion processing on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map, and the key point regression module is used to perform key point regression processing on the enhanced feature map to obtain key points of human body posture.

[0053] Furthermore, the dynamically expanded receptive field module includes a cascaded hole convolution component and a gating mechanism component.

[0054] Furthermore, the cascaded dilated convolution component includes a dilated convolution layer and two convolution layers, the dilated convolution layer is a dilated convolution layer with a dilation rate of [1, 2, 3] and a convolution kernel of 3*3, the convolution kernel of the two convolution layers is 3*3, and the gating mechanism module includes a global average pooling layer, two consecutive fully connected layers and a residual structure module.

[0055] Furthermore, the further feature extraction of the feature maps of four different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction includes: performing local feature aggregation processing on the first feature map, the second feature map, the third feature map and the fourth feature map based on the hole convolution layer in the cascade hole convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing; performing feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing using the BatchNorm function and the ReLU function based on the first convolution layer in the cascade hole convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the feature fusion. feature map; based on the second convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are re-fused to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; based on the global average pooling layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are acquired to obtain corresponding global information; based on the fully connected layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are extracted, and the attention weight is calculated based on the sigmoid function using the feature extraction result and the global information; based on the residual structure module, the attention weight is used for calibration.

[0056] Furthermore, the efficient feature fusion of the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map includes: transforming the first feature map after further feature extraction using a feature enhancement structure to obtain the transformed first feature map; upsampling the second feature map, the third feature map and the fourth feature map after further feature extraction based on a bilinear interpolation method to obtain the upsampled second feature map, the third feature map and the fourth feature map; performing feature map splicing processing on the transformed first feature map and the upsampled second feature map, the third feature map and the fourth feature map based on a Concat feature fusion method to obtain a spliced ​​feature map; performing 3*3 convolution processing on the spliced ​​feature map to obtain an enhanced feature map.

[0057] Furthermore, the Concat feature fusion method is used to perform feature map splicing processing on the first feature map after transformation and the second, third and fourth feature maps after upsampling, and after obtaining the spliced ​​feature map, it also includes: assigning weights of different feature channels to the first feature map after transformation and the second, third and fourth feature maps after upsampling based on an efficient channel attention mechanism.

[0058] Furthermore, the key point regression module adopts an adaptive convolutional neural network.

[0059] Specifically, the deep neural network model includes a high-resolution deep neural network (HRNet) backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module. The HRNet backbone module serves as the backbone network in the deep neural network. The HRNet backbone module preliminarily extracts semantic information of different levels from the feature map, and outputs four feature maps of different resolutions in parallel. Each feature map contains different semantic information. The four feature maps of different resolutions are respectively the first feature map, the second feature map, the third feature map and the fourth feature map, and the four feature maps of different resolutions are input into the dynamic expansion receptive field module. The dynamic expansion receptive field module includes a cascaded hole convolution component and a gating mechanism component. The cascaded hole convolution component captures input images with different receptive fields, and the gating mechanism component selectively highlights the semantic information features by assigning weights. The cascaded hole convolution component includes a hole convolution layer and two convolution layers. The gating mechanism module includes a global average pooling layer, two consecutive layers. The fully connected layer and the residual structure module sequentially pass the first feature map, the second feature map, the third feature map and the fourth feature map through the dilation rate [1,2,3] and the convolution kernel 3*3 hole convolution layer, aggregate the different local features of key points of different scales through multiple receptive field sizes, and then perform feature fusion through the first convolution layer with a convolution kernel of 1*1. After training, the BatchNorm function and the ReLU function are used to prevent overfitting, and then the second convolution layer with a convolution kernel of 1*1 is used to perform feature fusion to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; for the gating mechanism component, the global average pooling layer is used to obtain the corresponding global information, and then two consecutive fully connected layers are used to extract the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion, and the sigmoid function is used to calculate the attention weight. Finally, the residual structure module is used to recalibrate the output obtained by using the attention weight. After passing through the dynamic expansion receptive field module, enough feature information can be obtained;The first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction are input into the efficient fusion semantic information module. The efficient fusion semantic information module is used to obtain the correlation of feature information at different levels, assign different weights and perform efficient feature fusion. The efficient fusion semantic information module obtains the first feature map after transformation processing with unchanged size and number of channels through the feature enhancement structure of the first feature map, and then upsamples the second feature map, the third feature map and the fourth feature map by 2 times, 4 times and 8 times respectively. Here, bilinear interpolation upsampling is used instead of deconvolution upsampling to reduce the amount of calculation of the algorithm, so that the sizes of the second feature map, the third feature map and the fourth feature map after upsampling are unified to 128*1 28. The first feature map after transformation and the second, third and fourth feature maps after upsampling are concatenated by continuous (Concatenate, Concat) feature fusion to obtain a concatenated feature map. The Concat operation is used to connect two or more tensors together in a certain dimension to generate a larger tensor, which can be used to increase the number of channels or feature dimensions in a deep learning model so as to better capture the relationship between different features in subsequent layers. The concatenated feature map obtained by the operation has 480 channels and a size of 128*128. The feature maps are then fused by 3*3 convolution to reduce redundant information in the upsampling process and finally obtain an enhanced feature map.In the process of feature fusion, a batch normalization (BN) layer is added to perform batch normalization processing to speed up model convergence and training speed, prevent gradient disappearance, and improve the generalization ability of the network. The ReLU activation function is added after the BN layer, which is conducive to the neural network learning more nonlinear relationships. Since the Concat operation only splices the feature information in the channel dimension, it is impossible to obtain the correlation between each channel. This feature fusion method cannot reflect the importance between feature channels. Therefore, an efficient channel attention mechanism is added after the Concat fusion operation to weight different feature channels. By combining the efficient channel attention mechanism to fuse the feature information of feature maps at different levels, the semantic information of the shallow feature map can be increased, the contextual semantic feature information can be fully mined, and the shallow and deep layers can be better helped to conduct deeper integration. Feature fusion improves the detection capability of small targets. Finally, the enhanced feature map is input into the key point regression module. The key point regression module is used to perform key point regression processing through the enhanced feature map to obtain the key points of human posture. The key point regression module adopts an adaptive convolutional neural network. By training the deep neural network model including the HRNet backbone module, the dynamic expansion receptive field module, the efficient fusion semantic information module and the key point regression module, an applicable human posture detection model is obtained. The target feature map is input into the human posture detection model. The human posture detection processing of small targets is performed based on the human posture detection model, so that the key points of human posture in small targets can be detected and located, the accuracy of small target detection is improved, and the problems of false detection and missed detection are better avoided. ;

[0060] In an embodiment of the present invention, a human posture detection model is constructed by an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module. The HRNet backbone module can output four feature maps of different resolutions in parallel, and each feature map contains different semantic information. The dynamic expansion receptive field module can obtain enough feature information, dynamically expand the receptive field, and enhance the feature representation capability. The efficient fusion semantic information module can fully mine the contextual semantic feature information, better perform deeper feature fusion, improve the accuracy of small target detection, and better avoid the problems of false detection and missed detection.

[0061] Embodiment 2

[0062] See also Figure 2 , Figure 2 It is a schematic diagram of the structural composition of a 2D image small target human posture detection system in an embodiment of the present invention.

[0063] like Figure 2 As shown, a 2D image small target human posture detection system, the system comprises:

[0064] Image acquisition module 21: used to acquire a 2D image of a human target;

[0065] In the specific implementation process of the present invention, a 2D image of a human target is obtained by shooting with a high-resolution camera.

[0066] The image preprocessing module 22 is used to perform image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image;

[0067] In the specific implementation process of the present invention, the image preprocessing of the human target 2D image to obtain the preprocessed human target 2D image includes: grayscale processing of the human target 2D image to obtain a grayscale human target 2D image; image enhancement processing of the grayscale human target 2D image to obtain a grayscale human target 2D image after image enhancement processing; denoising of the grayscale human target 2D image after image enhancement processing to obtain a preprocessed human target 2D image.

[0068] Specifically, grayscale processing is performed on the 2D image of the human target, pixel values ​​of the three colors of red, green and blue in the 2D image of the human target are calculated, and a pixel average value is calculated based on the pixel values ​​of the three colors of red, green and blue. According to the sensitivity of the human eye to the three colors of red, green and blue, the pixel average value is used to perform weighted averaging, and the 2D image of the human target is processed by the weighted average value to generate a grayscale 2D image of the human target; image enhancement processing is performed on the grayscale 2D image of the human target, frequency domain modulation is performed on the object field of the grayscale image of the human target, and Fourier transform is performed on the frequency domain to obtain the Fourier transform. The imaginary part of the frequency domain after the conversion is taken in absolute value, the 1 / n power is calculated after summing along the frequency, and the enhancement is performed along the vertical axis and the horizontal axis respectively to obtain a grayscale human target 2D image after image enhancement processing; the grayscale human target 2D image after image enhancement processing is denoised, the value of the filter is set, and the grayscale human target 2D image after image enhancement processing is filtered according to the filter after the value setting, and the average grayscale value of adjacent pixels in the grayscale human target 2D image is calculated, and the average grayscale value is assigned to the current pixel point, and the denoising process is completed to obtain the pre-processed human target 2D image.

[0069] Feature extraction module 23: used to perform feature extraction processing on the pre-processed human target 2D image to obtain a target feature map;

[0070] In the specific implementation process of the present invention, a Haar feature classifier is constructed and trained. For the training process of the Haar feature classifier, the parameters required for training the classifier are defined, including the type and number of Haar features, the number of positive and negative samples, the number of classifier cascade layers and the learning rate. The positive sample is an image area containing a target, and the negative sample is an image area not containing a target. After setting the parameters, the train function is used to train the Haar feature classifier. During the training process, the model will continuously adjust the Haar features and set thresholds to remove unnecessary features, and finally form an adaptable and generalizable Haar feature classifier. The preprocessed human target 2D image is input into the trained Haar feature classifier, and feature extraction processing is performed based on the trained Haar feature classifier to obtain a target feature map.

[0071] Human posture detection module 24: used to input the target feature map into the human posture detection model, and perform human posture detection processing for small targets based on the human posture detection model. The human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training. The deep neural network model includes a HRNet backbone module, a dynamically expanded receptive field module, an efficient fusion semantic information module and a key point regression module.

[0072] In the specific implementation process of the present invention, the HRNet backbone module is used to perform preliminary extraction on the feature map to obtain four feature maps with different resolutions, and the four feature maps with different resolutions are respectively a first feature map, a second feature map, a third feature map and a fourth feature map, and the four feature maps with different resolutions are input into the dynamic expansion receptive field module, the dynamic expansion receptive field module is used to perform further feature extraction on the feature maps with different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction, the efficient fusion semantic information module is used to perform efficient feature fusion processing on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map, and the key point regression module is used to perform key point regression processing on the enhanced feature map to obtain key points of human body posture.

[0073] Furthermore, the dynamically expanded receptive field module includes a cascaded hole convolution component and a gating mechanism component.

[0074] Furthermore, the cascaded dilated convolution component includes a dilated convolution layer and two convolution layers, the dilated convolution layer is a dilated convolution layer with a dilation rate of [1, 2, 3] and a convolution kernel of 3*3, the convolution kernel of the two convolution layers is 3*3, and the gating mechanism module includes a global average pooling layer, two consecutive fully connected layers and a residual structure module.

[0075] Furthermore, the further feature extraction of the feature maps of four different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction includes: performing local feature aggregation processing on the first feature map, the second feature map, the third feature map and the fourth feature map based on the hole convolution layer in the cascade hole convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing; performing feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing using the BatchNorm function and the ReLU function based on the first convolution layer in the cascade hole convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the feature fusion. feature map; based on the second convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are re-fused to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; based on the global average pooling layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are acquired to obtain corresponding global information; based on the fully connected layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are extracted, and the attention weight is calculated based on the sigmoid function using the feature extraction result and the global information; based on the residual structure module, the attention weight is used for calibration.

[0076] Furthermore, the efficient feature fusion of the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map includes: transforming the first feature map after further feature extraction using a feature enhancement structure to obtain the transformed first feature map; upsampling the second feature map, the third feature map and the fourth feature map after further feature extraction based on a bilinear interpolation method to obtain the upsampled second feature map, the third feature map and the fourth feature map; performing feature map splicing processing on the transformed first feature map and the upsampled second feature map, the third feature map and the fourth feature map based on a Concat feature fusion method to obtain a spliced ​​feature map; performing 3*3 convolution processing on the spliced ​​feature map to obtain an enhanced feature map.

[0077] Furthermore, the Concat feature fusion method is used to perform feature map splicing processing on the first feature map after transformation and the second, third and fourth feature maps after upsampling, and after obtaining the spliced ​​feature map, it also includes: assigning weights of different feature channels to the first feature map after transformation and the second, third and fourth feature maps after upsampling based on an efficient channel attention mechanism.

[0078] Furthermore, the key point regression module adopts an adaptive convolutional neural network.

[0079] Specifically, the deep neural network model includes a high-resolution deep neural network (HRNet) backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module. The HRNet backbone module serves as the backbone network in the deep neural network. The HRNet backbone module preliminarily extracts semantic information of different levels from the feature map, and outputs four feature maps of different resolutions in parallel. Each feature map contains different semantic information. The four feature maps of different resolutions are respectively the first feature map, the second feature map, the third feature map and the fourth feature map, and the four feature maps of different resolutions are input into the dynamic expansion receptive field module. The dynamic expansion receptive field module includes a cascaded hole convolution component and a gating mechanism component. The cascaded hole convolution component captures input images with different receptive fields, and the gating mechanism component selectively highlights the semantic information features by assigning weights. The cascaded hole convolution component includes a hole convolution layer and two convolution layers. The gating mechanism module includes a global average pooling layer, two consecutive layers. The fully connected layer and the residual structure module sequentially pass the first feature map, the second feature map, the third feature map and the fourth feature map through the dilation rate [1,2,3] and the convolution kernel 3*3 hole convolution layer, aggregate the different local features of key points of different scales through multiple receptive field sizes, and then perform feature fusion through the first convolution layer with a convolution kernel of 1*1. After training, the BatchNorm function and the ReLU function are used to prevent overfitting, and then the second convolution layer with a convolution kernel of 1*1 is used to perform feature fusion to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; for the gating mechanism component, the global average pooling layer is used to obtain the corresponding global information, and then two consecutive fully connected layers are used to extract the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion, and the sigmoid function is used to calculate the attention weight. Finally, the residual structure module is used to recalibrate the output obtained by using the attention weight. After passing through the dynamic expansion receptive field module, enough feature information can be obtained;The first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction are input into the efficient fusion semantic information module. The efficient fusion semantic information module is used to obtain the correlation of feature information at different levels, assign different weights and perform efficient feature fusion. The efficient fusion semantic information module obtains the first feature map after transformation processing with unchanged size and number of channels through the feature enhancement structure of the first feature map, and then upsamples the second feature map, the third feature map and the fourth feature map by 2 times, 4 times and 8 times respectively. Here, bilinear interpolation upsampling is used instead of deconvolution upsampling to reduce the amount of calculation of the algorithm, so that the sizes of the second feature map, the third feature map and the fourth feature map after upsampling are unified to 128*1 28. The first feature map after transformation and the second, third and fourth feature maps after upsampling are concatenated by continuous (Concatenate, Concat) feature fusion to obtain a concatenated feature map. The Concat operation is used to connect two or more tensors together in a certain dimension to generate a larger tensor, which can be used to increase the number of channels or feature dimensions in a deep learning model so as to better capture the relationship between different features in subsequent layers. The concatenated feature map obtained by the operation has 480 channels and a size of 128*128. The feature maps are then fused by 3*3 convolution to reduce redundant information in the upsampling process and finally obtain an enhanced feature map.In the process of feature fusion, a batch normalization (BN) layer is added to perform batch normalization processing to speed up model convergence and training speed, prevent gradient disappearance, and improve the generalization ability of the network. The ReLU activation function is added after the BN layer, which is conducive to the neural network learning more nonlinear relationships. Since the Concat operation only splices the feature information in the channel dimension, it is impossible to obtain the correlation between each channel. This feature fusion method cannot reflect the importance between feature channels. Therefore, an efficient channel attention mechanism is added after the Concat fusion operation to weight different feature channels. By combining the efficient channel attention mechanism to fuse the feature information of feature maps at different levels, the semantic information of the shallow feature map can be increased, the contextual semantic feature information can be fully mined, and the shallow and deep layers can be better helped to conduct deeper integration. Feature fusion improves the detection capability of small targets. Finally, the enhanced feature map is input into the key point regression module. The key point regression module is used to perform key point regression processing through the enhanced feature map to obtain the key points of human posture. The key point regression module adopts an adaptive convolutional neural network. By training the deep neural network model including the HRNet backbone module, the dynamic expansion receptive field module, the efficient fusion semantic information module and the key point regression module, an applicable human posture detection model is obtained. The target feature map is input into the human posture detection model. The human posture detection processing of small targets is performed based on the human posture detection model, so that the key points of human posture in small targets can be detected and located, the accuracy of small target detection is improved, and the problems of false detection and missed detection are better avoided. ;

[0080] In an embodiment of the present invention, a human posture detection model is constructed by an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module. The HRNet backbone module can output four feature maps of different resolutions in parallel, and each feature map contains different semantic information. The dynamic expansion receptive field module can obtain enough feature information, dynamically expand the receptive field, and enhance the feature representation capability. The efficient fusion semantic information module can fully mine the contextual semantic feature information, better perform deeper feature fusion, improve the accuracy of small target detection, and better avoid the problems of false detection and missed detection.

[0081] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0082] In addition, the above is a detailed introduction to a 2D image small target human posture detection method and system provided by an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for detecting small human body posture in 2D images, characterized in that: The method comprises: Acquire a 2D image of a human target; Performing image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image; Perform feature extraction on the preprocessed 2D image of the human target to obtain a target feature map; Input the target feature map into a human posture detection model, and perform human posture detection processing on a small target based on the human posture detection model, wherein the human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training, and the deep neural network model includes an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module, and a key point regression module; Wherein, the HRNet backbone module is used to perform preliminary extraction on the feature map to obtain four feature maps with different resolutions, the four feature maps with different resolutions are respectively the first feature map, the second feature map, the third feature map and the fourth feature map, and the four feature maps with different resolutions are input into the dynamic expansion receptive field module, the dynamic expansion receptive field module is used to perform further feature extraction on the feature maps with different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction, the efficient fusion semantic information module is used to perform efficient feature fusion processing on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map, and the key point regression module is used to perform key point regression processing on the enhanced feature map to obtain key points of human posture; The dynamic expansion receptive field module includes a cascaded hole convolution component and a gating mechanism module; The cascaded dilated convolution component includes a dilated convolution layer and two convolution layers, wherein the dilated convolution layer is a dilated convolution layer with a dilation rate of [1, 2, 3] and a convolution kernel of 3*3, and the convolution kernel of the two convolution layers is 3*3, and the gating mechanism module includes a global average pooling layer, two consecutive fully connected layers and a residual structure module; The further feature extraction of the feature maps of four different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction includes: performing local feature aggregation processing on the first feature map, the second feature map, the third feature map and the fourth feature map based on the dilated convolution layer in the cascaded dilated convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing; performing feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing using the BatchNorm function and the ReLU function based on the first convolution layer in the cascaded dilated convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the feature fusion; Based on the second convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are re-fused to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; based on the global average pooling layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are acquired for global information to obtain corresponding global information; based on the fully connected layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are extracted, and the attention weight is calculated based on the sigmoid function using the feature extraction result and the global information; based on the residual structure module, the attention weight is used for calibration; The method of performing efficient feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map includes: performing transformation processing on the first feature map after further feature extraction using a feature enhancement structure to obtain the first feature map after transformation processing; performing upsampling processing on the second feature map, the third feature map and the fourth feature map after further feature extraction based on a bilinear interpolation method to obtain the second feature map, the third feature map and the fourth feature map after upsampling processing; performing feature map splicing processing on the first feature map after transformation processing and the second feature map, the third feature map and the fourth feature map after upsampling processing based on a Concat feature fusion method to obtain a spliced ​​feature map; performing 3*3 convolution processing on the spliced ​​feature map to obtain an enhanced feature map; The method further comprises: assigning weights of different feature channels to the first feature map after the transformation and the second feature map after the upsampling, the third feature map and the fourth feature map after the upsampling based on the Concat feature fusion method, after obtaining the spliced ​​feature map; The key point regression module adopts an adaptive convolutional neural network.

2. The 2D image small target human posture detection method according to claim 1, characterized in that: The performing image preprocessing on the human target 2D image to obtain the preprocessed human target 2D image includes: Performing grayscale processing on the human target 2D image to obtain a grayscale human target 2D image; Performing image enhancement processing on the grayscale human target 2D image to obtain the grayscale human target 2D image after image enhancement processing; The grayscale human target 2D image after image enhancement is denoised to obtain a preprocessed human target 2D image.

3. A 2D image small target human posture detection system, characterized in that: The system comprises: Image acquisition module: used to acquire 2D images of human targets; An image preprocessing module is used to perform image preprocessing on the human target 2D image to obtain a preprocessed human target 2D image; Feature extraction module: used to perform feature extraction processing on the preprocessed 2D image of the human target to obtain a target feature map; Human posture detection module: used to input the target feature map into the human posture detection model, and perform human posture detection processing of small targets based on the human posture detection model. The human posture detection model is a convergence model obtained by inputting the feature map into a deep neural network model for training. The deep neural network model includes an HRNet backbone module, a dynamic expansion receptive field module, an efficient fusion semantic information module and a key point regression module; Wherein, the HRNet backbone module is used to perform preliminary extraction on the feature map to obtain four feature maps with different resolutions, the four feature maps with different resolutions are respectively the first feature map, the second feature map, the third feature map and the fourth feature map, and the four feature maps with different resolutions are input into the dynamic expansion receptive field module, the dynamic expansion receptive field module is used to perform further feature extraction on the feature maps with different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction, the efficient fusion semantic information module is used to perform efficient feature fusion processing on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map, and the key point regression module is used to perform key point regression processing on the enhanced feature map to obtain key points of human posture; The dynamic expansion receptive field module includes a cascaded hole convolution component and a gating mechanism module; The cascaded dilated convolution component includes a dilated convolution layer and two convolution layers, wherein the dilated convolution layer is a dilated convolution layer with a dilation rate of [1, 2, 3] and a convolution kernel of 3*3, and the convolution kernel of the two convolution layers is 3*3, and the gating mechanism module includes a global average pooling layer, two consecutive fully connected layers and a residual structure module; The further feature extraction of the feature maps of four different resolutions to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction includes: performing local feature aggregation processing on the first feature map, the second feature map, the third feature map and the fourth feature map based on the dilated convolution layer in the cascaded dilated convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing; performing feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after the local feature aggregation processing using the BatchNorm function and the ReLU function based on the first convolution layer in the cascaded dilated convolution component to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after the feature fusion; Based on the second convolution layer in the cascaded hole convolution component, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are re-fused to obtain the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion; based on the global average pooling layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are acquired for global information to obtain corresponding global information; based on the fully connected layer of the gating mechanism module, the first feature map, the second feature map, the third feature map and the fourth feature map after feature fusion are extracted, and the attention weight is calculated based on the sigmoid function using the feature extraction result and the global information; based on the residual structure module, the attention weight is used for calibration; The method of performing efficient feature fusion on the first feature map, the second feature map, the third feature map and the fourth feature map after further feature extraction to obtain an enhanced feature map includes: performing transformation processing on the first feature map after further feature extraction using a feature enhancement structure to obtain the first feature map after transformation processing; performing upsampling processing on the second feature map, the third feature map and the fourth feature map after further feature extraction based on a bilinear interpolation method to obtain the second feature map, the third feature map and the fourth feature map after upsampling processing; performing feature map splicing processing on the first feature map after transformation processing and the second feature map, the third feature map and the fourth feature map after upsampling processing based on a Concat feature fusion method to obtain a spliced ​​feature map; performing 3*3 convolution processing on the spliced ​​feature map to obtain an enhanced feature map; The method further comprises: assigning weights of different feature channels to the first feature map after the transformation and the second feature map after the upsampling, the third feature map and the fourth feature map after the upsampling based on the Concat feature fusion method, after obtaining the spliced ​​feature map; The key point regression module adopts an adaptive convolutional neural network.

Images