Human body posture estimation method, system, equipment and medium

Abstract

The invention discloses a human body posture estimation method, system and device and a medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a to-be-estimated picture containing a human body; the method comprises the following steps of: performing feature extraction and down-sampling on a to-be-estimated picture, performing dimension raising, depth separable convolution operation, channel aggregation operation and dimension reduction on a feature map with down-sampling resolution in an output branch after down-sampling to obtain a local feature map, performing up-sampling reconstruction on the local feature map by adopting dynamic weight interpolation, and fusing an output branch which is not down-sampled to obtain a local feature map; obtaining a first fusion feature; taking the first fusion feature as an initial feature, repeating the step of obtaining the first fusion feature, obtaining a second fusion feature, carrying out fusion to obtain a dual-scale fusion feature, extracting a depth perception feature in the dual-scale fusion feature, carrying out human body posture estimation through the depth perception feature, and obtaining a human body key point heat map. According to the invention, multi-scale and global information is obtained through a lightweight structure, and accurate key point positioning is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a human pose estimation method, system, device and medium. BACKGROUND

[0002] Human pose estimation technology aims to analyze the key points of human body in images or videos, detect and locate them, and obtain the motion information of each part of the human body in real time and reconstruct the human skeleton.

[0003] At present, human pose estimation has been widely used in human-computer interaction, behavior recognition, motion analysis, intelligent driving and security monitoring. However, the current large model has high computational complexity and parameter quantity, and it is difficult to deploy it to the existing devices with limited resources due to the quality of feature extraction. SUMMARY

[0004] The present application aims to solve the problems in the prior art by providing a human pose estimation method, system, device and medium.

[0005] The present application specifically provides the following technical solutions: Obtain a to-be-estimated picture containing a human body; Perform feature extraction on the to-be-estimated picture, and downsample the extracted initial features to obtain output branches with different resolutions, and repeatedly downsample the downsampled output branches by a preset number of times; In the downsampled output branches, the feature maps with reduced resolution are upsampled, the upsampled feature maps are subjected to depth separable convolution operation, channel aggregation operation and dimension reduction, local feature maps are obtained, the local feature maps are subjected to dynamic weight interpolation upsampling reconstruction, and the output branches that are not downsampled are fused, and all downsampled feature maps with reduced resolution and the upsampling reconstruction results are fused to obtain first fusion features; wherein the dynamic weight interpolation is provided by a point sampling set; Take the first fusion features as initial features, repeat the steps of obtaining the first fusion features to obtain second fusion features, and fuse the first fusion features and the second fusion features as double-scale fusion features; Extract depth perception features in the double-scale fusion features, perform human pose estimation through the depth perception features, and obtain human key point heat maps; The extraction of the depth perception features in the double-scale fusion features includes: Predict the probability of each pixel in the double-scale fusion features belonging to a discrete depth interval to obtain a depth map and a continuous depth value, and use linear growth discretization to divide the continuous depth value into different discrete depth intervals to obtain initial depth perception features; The predicted depth map is group-convoluted to merge adjacent depth intervals, collect features of all pixels, and use probabilities belonging to depth categories as weights to perform weighted summation to obtain a depth prototype; The depth prototype is used to represent reconstructed new depth perception features, the initial depth perception features and the new depth perception features are spliced to obtain final depth perception features, and the final depth perception features are processed by a convolutional layer to obtain final depth perception features containing reconstructed depth information.

[0006] Preferably, a pre-trained lightweight human pose estimation network model is used to estimate the human pose of the to-be-estimated picture to obtain a human key point heat map, wherein the lightweight human pose estimation network model specifically comprises: The lightweight human pose estimation network model comprises a backbone network and a DFE depth perception feature enhancement module connected in sequence, wherein the backbone network is connected in sequence by two sub-networks with the same structure, each sub-network comprises four parallel sub-networks with different resolutions, and each parallel sub-network with different resolutions is composed of a lightweight feature extraction module and a dynamic sampling-based up-sampling module stacked in series; the lightweight feature extraction module comprises a convolutional layer, a separable convolutional layer and a channel aggregation module.

[0007] Preferably, the dimension-increased feature map is subjected to a depth separable convolution operation, a channel aggregation operation and dimension reduction to obtain a local feature map, specifically as follows: An application of 1x1 convolution reduces the number of channels of the dimension-increased feature map to 1, and aggregates information across channels in the dimension-increased feature map to generate a channel-level representation as an aggregated feature; The weight of the result of subtracting the aggregated feature from the dimension-increased feature map is adjusted by a scaling factor, and the weight of each channel is dynamically adjusted by the scaling factor to obtain an aggregated channel feature after weight adjustment; The aggregated channel feature after weight adjustment is added to the dimension-increased feature map by element-wise addition to generate an output result; the output result contains information enhanced by channel aggregation; The output result is dimension-reduced to obtain a local feature map.

[0008] Preferably, the local feature map is subjected to dynamic weight interpolation for up-sampling reconstruction, and the output branch without down-sampling is fused, specifically as follows: A point sampling set of the local feature map is obtained; The dynamic sampling-based up-sampling module uses the point sampling set to provide dynamic weight interpolation to reconstruct the local feature map to obtain a reconstructed local feature map after up-sampling, which is fused with the output branch without down-sampling.

[0009] Preferably, dynamic weight interpolation is provided using a point sampling set to reconstruct the local feature map, resulting in an upsampled reconstructed local feature map. Specifically: Given a local feature map And the point sampling set is obtained through point sampling calculation. The coordinates of each point in the array; Using point sampling sets The coordinate position in the original feature tensor Bilinear interpolation is performed to map each element to the input feature tensor, and resampling is then performed to obtain a new upsampled feature tensor. , which serves as the reconstructed local feature map after upsampling.

[0010] Preferably, the point sampling set is obtained by giving a shape as Local feature map At the same time, an upsampling scaling factor is given. Then, a size of is generated. The offset is determined, and the local feature map is rearranged using pixel rearrangement. The reshaping yields a size of High-resolution features Through the original sampling network Elements and high-resolution features The point sample set is obtained by adding the elements together. ;in The sampling point number coefficient is the original sampling network. The elements are the initial local feature maps. The elements in.

[0011] This invention provides a human pose estimation system, comprising: The data acquisition module is used to acquire images containing human figures that are to be estimated. The feature extraction module is used to extract features from the image to be estimated, and to downsample the extracted initial features to obtain output branches with different resolutions, and to repeat the downsampling of the downsampled output branches a preset number of times. The first fusion module is used to perform dimensionality upscaling on the downsampled resolution feature map in the downsampled output branch, perform depthwise separable convolution operation, channel aggregation operation and dimensionality reduction on the upsampled feature map to obtain local feature maps, and perform upsampling reconstruction on the local feature maps using dynamic weight interpolation, integrate them into the undownsampled output branch, and fuse all downsampled resolution feature maps with the upsampled reconstruction results to obtain the first fused feature. a second fusion module configured to take the first fusion feature as an initial feature, repeat the step of obtaining the first fusion feature, obtain a second fusion feature, and fuse the first fusion feature and the second fusion feature as a double-scale fusion feature; a pose estimation module configured to extract a depth perception feature in the double-scale fusion feature, and perform human pose estimation through the depth perception feature to obtain a human key point heat map.

[0012] The present application provides a computer device, comprising a memory and a processor, the memory stores a program, and the program is executed by the processor to make the processor execute the steps of the human pose estimation method.

[0013] The present application provides a storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the human pose estimation method.

[0014] Compared with the prior art, the present application has the following advantages: The present application extracts features from the to-be-estimated picture through two parallel subnetworks with the same structure, extracts features and downsamples the to-be-estimated picture in each subnetwork, sets the number of downsampling, upsamples the feature map with the downsampled resolution in the output branch after downsampling, performs depth separable convolution operation, channel aggregation operation and dimension reduction on the upsampled feature map, obtains a local feature map, can highlight the features of small-scale and occluded human key points in the image, improves the detection capability of all key points in the input image, simultaneously uses dynamic weight interpolation to upsample and reconstruct the local feature map, and integrates the output branch without downsampling, thereby constructing and fusing the first fusion feature and the second fusion feature, extracting the depth perception feature in the double-scale fusion feature, and performing human pose estimation through the depth perception feature, which can obtain multi-scale and global information with light weight, is less affected by the environment, can quickly obtain accurate key point positioning, significantly improves the detection accuracy of human pose estimation, and has comparable or better performance compared with the traditional large model deployed when the device is not limited in resources. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a model construction, training and test diagram for human pose estimation according to an embodiment of the present application; Figure 2 is a multi-stage gating human pose estimation network model diagram according to an embodiment of the present application; Figure 3 is a DySample dynamic upsampling module point sampling generator network model diagram according to an embodiment of the present application; Figure 4 is a DySample dynamic upsampling module network model diagram according to an embodiment of the present application; Figure 5 is a network model diagram of the MCAM lightweight feature extraction module proposed in the embodiment of the present application; Figure 6 is a network model diagram of the DFE depth perception feature enhancement module proposed in the embodiment of the present application; Figure 7 is a comparison result diagram of the high-precision lightweight human pose estimation method proposed in the embodiment of the present application and the HRNet method on the COCO dataset, wherein, Figure 7 (a) and Figure 7 (b) of are respectively a first group of comparison result diagrams of the HRNet method on the COCO dataset, Figure 7 (c) and Figure 7 (d) of are respectively a second group of comparison result diagrams of the HRNet method on the COCO dataset, Figure 7 (e) and Figure 7 (f) of are respectively a third group of comparison result diagrams of the HRNet method on the COCO dataset, Figure 7 (g) and Figure 7 (h) of are respectively a fourth group of comparison result diagrams of the HRNet method on the COCO dataset. Figure 8 is a flowchart of a human pose estimation method provided by the present application. DETAILED DESCRIPTION

[0016] The technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should fall within the scope of protection of the present application.

[0017] As shown in Figure 1 and Figure 8 , a human pose estimation method in the embodiment includes the following steps: Step S1: obtaining a picture containing a human body to be estimated.

[0018] The human pose estimation model of the present application is trained based on the public dataset COCO dataset and MPII dataset, as shown in Figure 7 , the class information for labeling human key points in the COCO dataset has 17 categories, and the MPII dataset is a dataset focusing on human pose estimation. Compared with the COCO dataset, MPII focuses more on the labeling of human actions and postures, and is suitable for human behavior analysis and pose estimation model training and evaluation.

[0019] Step S2: The image to be estimated is subjected to feature extraction in parallel two stages, and the extracted initial features are downsampled to obtain output branches with different resolutions. The downsampled output branches are then downsampled a preset number of times.

[0020] A pre-trained lightweight human pose estimation network model is used to estimate the human pose of the image to be estimated, resulting in a human keypoint heatmap, such as... Figure 2 As shown, the lightweight human pose estimation network model is specifically as follows: A multi-level gated lightweight human pose estimation network model is constructed. This model includes a backbone network and a depth-sensing feature enhancement module. The backbone network is formed by sequentially connecting two structurally identical subnetworks. Each subnetwork contains four parallel subnetworks of different resolutions. Each parallel subnetwork of different resolutions is composed of a series of lightweight feature extraction modules and a dynamically sampled upsampling module, which are repeatedly stacked. The dynamically sampled upsampling module is an upsampling function that dynamically calculates weights. The stacking process is as follows: Figure 2 As shown, the feature map upsampling process based on the dynamic sampling upsampling module is as follows: Figure 3 and Figure 4 As shown, the lightweight feature extraction module includes convolutional layers, separable convolutional layers, and a channel aggregation module. The lightweight human pose estimation network model is trained using a human pose estimation dataset to obtain a well-trained lightweight human pose estimation network model.

[0021] First, based on Figure 2 The STEM module in the middle performs downsampling on the input image IMAGE using a conventional 3×3 convolution to extract initial features, and then performs two 3×3 convolutions, such as... Figure 2 The neck feature transfer in the image is performed with step sizes of 1 and 2, resulting in two output branches with feature map resolutions of 1 / 4 and 1 / 8, respectively. Then, the MCAM lightweight feature extraction module is used to highlight features of smaller scale and occluded human body key points in the image, thereby improving the network's ability to detect all key points in the input image.

[0022] Step S3: In the downsampled output branch, the downsampled resolution feature map is upsampled, and the upsampled feature map is subjected to depthwise separable convolution, channel aggregation, and dimensionality reduction to obtain local feature maps. The local feature maps are then upsampled and reconstructed using dynamic weight interpolation and integrated into the undownsampled output branch. All downsampled resolution feature maps are then fused with the upsampled reconstruction results to obtain the first fused feature. Dynamic weight interpolation is provided using a point sampling set.

[0023] Among them, such as Figure 5As shown, the MCAM lightweight feature extraction module first uses 1x1 convolution to increase the dimension of the input feature map, then performs deep separable convolution on the dimension-increased feature map, and then connects a channel aggregation module. Finally, the feature map of the extracted local feature is reduced in dimension by 1x1 convolution for subsequent feature extraction.

[0024] The dimension-increased feature map is subjected to deep separable convolution operation, channel aggregation operation and dimension reduction to obtain a local feature map, specifically: The channel aggregation module is part of the lightweight feature extraction module. When performing the channel aggregation module, 1x1 convolution is applied to reduce the number of channels of the dimension-increased feature map to 1, and the information across the channels of the dimension-increased feature map is aggregated to generate a channel-level representation. The extracted features from the input are used as aggregated features. A scaling factor is used to adjust the weight of the subtraction result. The network can dynamically adjust the weight of each channel by learning this factor, i.e., adjusting the weight of the subtraction result of the aggregated feature and the dimension-increased feature map through the scaling factor, and dynamically adjusting the weight of each channel through the scaling factor to obtain the aggregated channel feature after weight adjustment. Finally, the scaled subtraction result (aggregated channel feature after weight adjustment) and the original input (dimension-increased feature map) are added through element-wise addition to generate an output. The output contains information enhanced by channel aggregation. By strengthening the features of small-scale and occluded human key points in the image, the detection capability of the network for all key points in the input image can be improved, thereby better ensuring the overall performance of the network model. The output is reduced in dimension to obtain a local feature map.

[0025] The two branches of different scales after feature extraction are fused to obtain two branches of 1 / 4 and 1 / 8 feature map resolution after fusion.

[0026] When upsampling the 1 / 8 feature map to 1 / 4, the Sample dynamic sampling-based upsampling module is used, as shown in Figure 3 and Figure 4 As shown, 1. First, bilinear interpolation is used to interpolate the input local feature map to a continuous space to obtain an interpolated feature map. 2. The sampling point generator is responsible for generating a content-aware sampling point set for resampling the interpolated feature map. 3. A linear layer or a linear layer + Plxel Shuffle is used to generate a high-resolution feature O. 4. The high-resolution feature O is added to the offset G of the original sampling grid to obtain the final sampling point set . 5. Finally, the grid_sample function is used to resample the interpolated feature map according to the generated sampling point set to obtain the reconstructed local feature map after upsampling . The process is described in detail as follows:

[0027] As Figure 4 shown, the point sampling set is used to provide dynamic weight interpolation, and the local feature map is reconstructed to obtain the reconstructed local feature map after upsampling, specifically: The point sampling set is used to provide dynamic weight interpolation by giving a feature tensor (local feature map) (size ), wherein is the number of input channels, is the height of the input feature tensor, is the width of the input feature tensor, and the point sampling set is calculated by point sampling to obtain the coordinate position of each point in the point sampling set , that is, the first dimension in the point sampling set calculated by point sampling represents and coordinates, The function uses the coordinate position in the point sampling set to perform bilinear interpolation on the original feature tensor , maps the elements at each position to the input feature tensor, and thus resamples a new upsampled feature tensor

[0028] as the reconstructed local feature map after upsampling. Figure 3 As shown, the point sampling set is generated by giving a feature tensor (local feature map) with a shape of , and giving an upsample ratio factor , using a linear layer with an input channel of and an output channel of to generate an offset with a size of , and reshaping the local feature map to obtain a high-resolution feature with a size of by pixel rearrangement, and obtaining the point sampling set by adding the elements of the original sampling network and the elements of the high-resolution feature ; wherein is a sampling point number coefficient, and the elements of the original sampling network are the elements in the initial local feature map

[0029] . Figure 2 In , the subsequent branch is processed in the same way as the first two branches (1 / 4, 1 / 8), and through the fusion between different scale branches, the resolution of the fused feature map is 1 / 4, 1 / 8, 1 / 16, 1 / 32.

[0030] Step S4: taking the first fusion feature as the initial feature, repeating the step of obtaining the first fusion feature to obtain a second fusion feature, and fusing the first fusion feature and the second fusion feature as a double-scale fusion feature.

[0031] The second parallel subnetwork is consistent with the step in step S3.

[0032] Step S5: extracting a depth perception feature in the double-scale fusion feature, performing human pose estimation through the depth perception feature, and obtaining a human key point heat map.

[0033] Finally, the double-scale feature map is further enhanced by the DFE feature enhancement module to enhance the feature extraction capability, ensure the acquisition of high-precision information, and generate an accurate prediction heat map to provide strong support for subsequent heat map decoding.

[0034] The depth perception feature in the double-scale fusion feature is extracted, specifically: As shown in Figure 6 , the depth perception feature enhancement module DFE regards the depth estimation task as a sequence classification problem. First, the input feature from the backbone network is denoted as , the input feature is processed through two convolutional layers to predict the probability that each pixel in the double-scale feature map belongs to a discrete depth interval, denoted as , to obtain a depth map and a continuous depth value. Then, using linear growth discretization, the continuous depth value is divided into different discrete depth intervals to obtain the initial depth perception feature containing the depth information of each pixel.

[0035] Then, the predicted depth map is combined by group convolution to merge adjacent depth intervals and reduce the number of categories from to , and then the features of all pixels are collected and weighted summed using their probability of belonging to the depth category as weights to obtain the depth prototype .

[0036] Subsequently, the depth prototype is used to represent the reconstructed new depth perception feature , the initial depth perception feature and the reconstructed feature are spliced to obtain the final enhanced depth feature, and then processed through a convolutional layer to obtain the final depth perception feature containing the reconstructed depth information. Wherein , The formula is as follows.

[0037] ; ; wherein, represents a feature map of a pixel in the feature map, is a set of pixels in the feature map, is a probability of each pixel belonging to the i-th depth category after normalization. The pictures in the training set are preprocessed, the human body bounding box is extracted using a pre-trained detector, and the key point coordinates are converted into normalized coordinates relative to the cropped area, then data augmentation is performed by rotation, scaling and color jittering, and then input into the human body estimation model of the present application, the human body pose estimation model is trained using the MSE loss function, and the OKS method is used for verification on the COCO dataset, the formula is as follows:

[0038] The pictures in the training set are preprocessed, the human body bounding box is extracted using a pre-trained detector, and the key point coordinates are converted into normalized coordinates relative to the cropped area, then data augmentation is performed by rotation, scaling and color jittering, and then input into the human body estimation model of the present application, the human body pose estimation model is trained using the MSE loss function, and the OKS method is used for verification on the COCO dataset, the formula is as follows: ; wherein, is the Euclidean distance between the detected human key point coordinates and the true value, represents whether the i-th key point is occluded, that is, if represents that the key point is not occluded, and vice versa, represents the scale factor of the target, represents the decay coefficient of each key point.

[0039] Meanwhile, the PCK evaluation index is selected on the MPII dataset, and the formula is as follows: ; ; wherein, represents the Euclidean distance between the i-th person's key point prediction value and the manually labeled value. represents the scale factor of the i-th person. represents a manually set threshold, , , respectively represents the PCK index of the algorithm under the threshold , .

[0040] ​​​​The test results of the method of the present application and other existing methods on the COCO verification set are shown in Table 1. In Table 1, MGNet represents the model proposed in the present application. Params represents the parameter amount of operation, FLOPs represents the floating point operation amount, AP represents the average precision, AP50, AP75, APM and APL represent the average precision at an intersection over union threshold of 0.50, the average precision at an intersection over union threshold of 0.75, the average precision at all intersection over union thresholds and the average precision at a lower intersection over union threshold, respectively. As can be seen from Table 1, MGNet can achieve the optimal AP precision index with a parameter amount similar to that of a small network, and the precision is similar to that of a large model network. The experimental comparison results on the COCO dataset are shown in Figure 7

[0041] Table 1 Experimental results of each model on the COCO dataset The test results of the method of the present application and other existing methods on the MPII verification set are shown in Table 2. Compared with MobileNetV2, ShuffleNetV2, LEViTPose-T, EfficientViT-M0 and HRNet-W32 algorithms, the MGNet algorithm proposed in the present application achieves good pose estimation results with a lighter model size. The table includes the prediction results of the joint points of the head, shoulder, elbow, wrist, hip, knee and ankle of the human body and the average prediction accuracy. In the evaluation of different parts of the human body, the algorithm of the present application performs well and is at a higher level in most cases.

[0042] Table 2 Experimental results of each model on the MPII verification set The trained network model is used to process input image data containing a human body to be estimated, perform human pose estimation, obtain a human key point heat map, and finally output a human key point visualized image.

[0043] Based on this, the present application proposes a human pose estimation system, which comprises a data acquisition module, a feature extraction module, a first fusion module, a second fusion module and a pose estimation module.

[0044] ​The feature extraction module is configured to perform feature extraction on the to-be-estimated picture, and perform down-sampling on the extracted initial features to obtain output branches with different resolutions, and repeatedly perform down-sampling on the down-sampled output branches for a preset number of times; the first fusion module is configured to, in the down-sampled output branches, perform dimension increasing on the feature maps with down-sampled resolutions, perform depth separable convolution operation, channel aggregation operation and dimension reduction on the dimension-increased feature maps to obtain local feature maps, perform up-sampling reconstruction on the local feature maps by using dynamic weight interpolation, and fuse the up-sampling reconstruction results into the output branches that are not down-sampled, and fuse all the feature maps with down-sampled resolutions after down-sampling and the up-sampling reconstruction results to obtain first fusion features; the second fusion module is configured to take the first fusion features as initial features, repeat the steps of obtaining the first fusion features to obtain second fusion features, and fuse the first fusion features and the second fusion features as double-scale fusion features; and the pose estimation module is configured to extract depth perception features from the double-scale fusion features, and perform human pose estimation by using the depth perception features to obtain human key point heat maps.

[0045] The present application also provides a computer device comprising a memory and a processor, the memory storing a program, the program being executed by the processor to cause the processor to perform the steps of the human pose estimation method.

[0046] According to the disclosed embodiments, the computer device can communicate with one or more external devices (such as a keyboard, a pointing device, a Bluetooth communication, etc.), or with any device (such as a router, a demodulator, etc.) that enables the computer device to communicate with one or more other computer devices.

[0047] The present application also provides a storage medium storing a computer program, the computer program being executed by the processor to implement the steps of the human pose estimation method.

[0048] According to the disclosed embodiments, the storage medium can be a non-volatile computer readable storage medium, which can include but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0049] The above is a further detailed description of the present application in combination with specific preferred embodiments. For those skilled in the art, without departing from the concept of the present application, a number of simple deductions or substitutions can also be made, which should all be considered within the protection scope of the present application.

Claims

1. A human pose estimation method, characterized by, The method comprises the following steps: acquiring a picture to be estimated containing a human body; performing feature extraction on the picture to be estimated, and performing down-sampling on the extracted initial features to obtain output branches with different resolutions, and repeatedly performing down-sampling on the down-sampled output branches for a preset number of times; in the down-sampled output branches, performing dimension increasing on the feature maps with reduced resolution, performing depth separable convolution operation, channel aggregation operation and dimension reduction on the dimension-increased feature maps to obtain local feature maps, performing up-sampling reconstruction on the local feature maps by using dynamic weight interpolation, and fusing the up-sampling reconstruction results into the output branches that have not been down-sampled, and fusing all the down-sampled feature maps with reduced resolution and the up-sampling reconstruction results to obtain first fusion features; wherein the dynamic weight interpolation is provided by a point sampling set; taking the first fusion features as initial features, repeating the step of obtaining the first fusion features to obtain second fusion features, and fusing the first fusion features and the second fusion features as double-scale fusion features; extracting depth perception features in the double-scale fusion features, and performing human pose estimation by using the depth perception features to obtain a human key point heat map; the step of extracting the depth perception features in the double-scale fusion features comprises: predicting the probability of each pixel in the double-scale fusion features belonging to a discrete depth interval to obtain a depth map and a continuous depth value, and using linear growth discretization to divide the continuous depth value into different discrete depth intervals to obtain initial depth perception features; performing group convolution to merge adjacent depth intervals for the predicted depth map, collecting the features of all pixels, and using the probability of belonging to a depth category as a weight to perform weighted summation to obtain a depth prototype; The new depth perception feature is reconstructed by using a deep prototype representation, the initial depth perception feature and the new depth perception feature are spliced to obtain a final depth perception feature, and the final depth perception feature is processed through a convolutional layer to obtain a final depth perception feature containing reconstructed depth information.

2. The human pose estimation method of claim 1, wherein, performing human pose estimation on the picture to be estimated by using a pre-trained lightweight human pose estimation network model to obtain a human key point heat map, wherein the lightweight human pose estimation network model specifically comprises: the lightweight human pose estimation network model comprises a backbone network and a DFE depth perception feature enhancement module connected in sequence, wherein the backbone network is connected in sequence by two sub-networks with the same structure, each sub-network comprises four parallel sub-networks with different resolutions, and each parallel sub-network with different resolutions is composed of a lightweight feature extraction module and a dynamic sampling-based up-sampling module connected in series and repeatedly stacked; the lightweight feature extraction module comprises a convolution layer, a separable convolution layer and a channel aggregation module.

3. The human pose estimation method of claim 2, wherein, the step of performing depth separable convolution operation, channel aggregation operation and dimension reduction on the dimension-increased feature maps to obtain local feature maps specifically comprises: applying a 1x1 convolution to reduce the number of channels of the dimension-increased feature maps to 1, and collecting the information across the channels in the dimension-increased feature maps to generate a channel-level representation as an aggregated feature; adjusting the weight of the difference between the aggregated feature and the dimension-increased feature map by a scaling factor, and dynamically adjusting the weight of each channel by the scaling factor to obtain an aggregated channel feature after weight adjustment; generating an output result by element-wise addition of the aggregated channel feature after weight adjustment and the dimension-increased feature map; the output result contains information enhanced by channel aggregation; performing dimension reduction on the output result to obtain a local feature map.

4. The human pose estimation method of claim 2, wherein, The local feature map is up-sampled and reconstructed by using dynamic weight interpolation, and the output branch without down-sampling is fused, specifically as follows: Obtain a point sampling set of the local feature map; Based on the up-sampling module of dynamic sampling, the point sampling set is used to provide dynamic weight interpolation to perform feature reconstruction on the local feature map, and a reconstructed local feature map after up-sampling is obtained, which is fused with the output branch without down-sampling.

5. The human pose estimation method of claim 4, wherein, The point sampling set is used to provide dynamic weight interpolation to perform feature reconstruction on the local feature map, and a reconstructed local feature map after up-sampling is obtained, specifically as follows: Given a local feature map And the point sampling set is calculated by point sampling The coordinate position of each point in the middle Using point sampling sets The coordinate position in the original feature tensor Bilinear interpolation is performed to map each element to the input feature tensor, and resampling is then performed to obtain a new upsampled feature tensor. , which serves as the reconstructed local feature map after upsampling.

6. The human pose estimation method of claim 5, wherein, The point sampling set is obtained by giving a shape as... Local feature map At the same time, an upsampling scaling factor is given. Then, a size of is generated. The offset is determined, and the local feature map is rearranged using pixel rearrangement. The reshaping yields a size of High-resolution features Through the original sampling network Elements and high-resolution features The point sample set is obtained by adding the elements together. ;in The sampling point number coefficient is the original sampling network. The elements are the initial local feature maps. The elements in.

7. A system for human pose estimation method according to any one of claims 1-6, characterized in that, It comprises: A data acquisition module is configured to acquire a picture to be estimated containing a human body; A feature extraction module is configured to perform feature extraction on the picture to be estimated, down-sample the extracted initial features, obtain output branches with different resolutions, and repeatedly down-sample the output branches after down-sampling for a preset number of times; A first fusion module is configured to, in the output branches after down-sampling, increase the dimension of the feature map with a down-sampling resolution, perform a depth separable convolution operation, a channel aggregation operation and a dimension reduction on the feature map after dimension increasing, obtain a local feature map, perform up-sampling and reconstruction on the local feature map by using dynamic weight interpolation, fuse the output branch without down-sampling, and fuse all the feature maps with a down-sampling resolution after down-sampling and the up-sampling and reconstruction result to obtain a first fusion feature; A second fusion module is configured to take the first fusion feature as an initial feature, repeat the steps of obtaining the first fusion feature to obtain a second fusion feature, and fuse the first fusion feature and the second fusion feature as a double-scale fusion feature; A pose estimation module is configured to extract a depth perception feature from the double-scale fusion feature, perform human pose estimation through the depth perception feature, and obtain a human key point heat map.

8. A computer device, comprising: The computer program is executed by the processor to implement the steps of the human pose estimation method according to any one of claims 1 to 6.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the human pose estimation method according to any one of claims 1 to 6.

Images