Construction method and application of air-to-ground-oriented lightweight human body target attitude estimation model
By introducing a deep convolution transformation module and a multi-scale feature fusion module into the human posture estimation model, and using knowledge distillation technology, the problem of insufficient human posture estimation accuracy on the drone platform is solved, achieving higher detection accuracy and adaptation sex.
Patent Information
- Application Number
- CN202411303531.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-19
- Publication Date
- 2025-01-03
- Estimated Expiration
- 2044-09-19
AI Technical Summary
The existing human posture estimation model cannot effectively detect long-term targets and small targets captured by the drone 5-10m away from the ground on platforms with limited computing power, and the accuracy of the lightweight method is insufficient.
A lightweight human target pose estimation model for air-to-ground lightweight human body is designed, using a deep convolution transformation module based on the HRNet network structure and a multi-scale feature fusion module. Combined with knowledge distillation technology, the training process is optimized to improve the model accuracy.
While keeping the model lightweight, the accuracy of posture estimation is significantly improved, which can better adapt to the air-to-ground human posture estimation, especially in the field of vision and small object detection.
Smart Images

Figure CN119251905A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of human body posture estimation, and in particular to a construction method and application of an air-to-ground lightweight human body target posture estimation model. Background Art
[0002] As an upstream task of behavior recognition, human posture estimation aims to locate the coordinates of key parts such as the head, shoulders, and hips of ground targets in images, thereby constructing a skeleton sequence to provide input data sources for behavior recognition. Posture estimation can be used for violation detection in various dynamic scenes, such as potential threat behavior detection in smart security for key areas. Through specialized training for distant and small targets, human posture estimation can be applied to aerial devices such as drones, effectively estimating human posture from an aerial perspective.
[0003] At present, neural networks used for lightweight human posture estimation, such as the patent "A method and system for multi-person human posture estimation based on knowledge distillation" with patent number "CN 115187660 A", disclose the "GhostPoseNet" network, which uses knowledge distillation technology to generate joint point biases using the student heat map and the target joint point heat map of the data label, and dynamically adjusts the knowledge transfer from the teacher network to the student network to build a lightweight network. Although this method greatly reduces the number of network parameters, the network accuracy also shows a serious decline; in the document with the document number "10.48550 / arXiv.2104.06403", the "Lite-HRNet" network is disclosed, which uses the channel shuffle module Shuffle Block to replace the 3×3 convolution with high computational cost in the network. Although it significantly reduces the complexity of the model, the loss in model accuracy is relatively serious; in the document with the document number "10.48550 / arXiv.2204.10762", the "Dite-HRNet" network is disclosed, which uses dynamic segmentation convolution and adaptive context modeling to design two lightweight modules, dynamic multi-scale context and dynamic global context. Compared with previous methods, the accuracy is improved, but it is still not ideal.
[0004] The common problems in the above-mentioned existing technologies are: the number of high-precision model parameters is too large to be deployed on platforms with limited computing power such as drones, while the accuracy of lightweight methods is often insufficient. At the same time, it is impossible to effectively estimate the human posture of distant targets and small targets photographed by drones 5-10m above the ground. Summary of the invention
[0005] In view of the defects of the existing technologies analyzed above, the present invention proposes a method for constructing and applying a lightweight air-to-ground human target pose estimation model, designs a lightweight human pose estimation network and introduces knowledge distillation technology to solve the problem that the existing human pose estimation algorithm model cannot effectively detect distant targets and small targets photographed by drones 5-10m above the ground on aerial equipment platforms.
[0006] To achieve the above object, the technical solution of the present invention is as follows:
[0007] On the one hand, the present invention discloses a method for constructing an air-to-ground lightweight human target posture estimation model, comprising the following steps:
[0008] S1: Build a dataset for human pose estimation;
[0009] S2: Based on the HRNet network structure, a deep convolution transformation module is designed to replace the residual convolution module in the high-resolution network to obtain a multi-branch feature fusion module;
[0010] S3: Design a multi-scale feature fusion module, and combine it with the multi-branch feature fusion module in step S2 to obtain a lightweight human posture estimation model;
[0011] S4: Design a lightweight model optimization algorithm and use it to optimize the training process of the lightweight human posture estimation model in step S3, and input the data in the data set of step 1 into the optimized lightweight human posture estimation model for retraining.
[0012] Furthermore, the original data set in the above step S1 is selected to be the distant view and small target data taken by the drone 5-10m above the ground, so as to better adapt to the air-to-ground mission, and then the obtained original data set is preprocessed to obtain the input sample.
[0013] Furthermore, the deep convolution transformation module designed in the above step S2 includes a construction module feature extraction layer and a construction module channel mapping layer.
[0014] Based on the above scheme, the specific steps of constructing the module feature extraction layer are as follows:
[0015] Layer normalization is used to eliminate the differences in input samples, 3×3 depth-separable convolution is used to extract the contextual information of the local pixel area of the input sample, and cascade channel attention is introduced to learn the channel features of different input samples;
[0016] The calculation expression of the feature extraction layer is as follows:
[0017] F'=CCA(DWConv 3×3 (LayerNorm(x)))+x
[0018] Among them: x represents the input feature, LayerNorm represents the layer normalization operation, DWConv 3×3 represents a 3×3 depthwise separable convolution, CCA represents cascaded channel attention, and F' represents the output features after the feature extraction layer.
[0019] Based on the above scheme, the channel mapping layer of the construction module further includes layer normalization and two 1×1 convolution operations, and its construction steps are:
[0020] The output features of the feature extraction layer are used as new input features for layer normalization. The first 1×1 convolution is used to expand the feature channels by 2 times to fully learn the information in the new input features. The second 1×1 convolution is used to compress the number of feature channels to the original number of channels to retain important feature information.
[0021] The calculation expression of the channel mapping layer is as follows:
[0022]
[0023] in: They represent the first and second 1×1 convolutions respectively, and F” represents the final output feature of the deep convolution transformation module.
[0024] Furthermore, in the above step S3, designing a multi-scale feature fusion module includes:
[0025] (1) Constructing a feature size redefinition layer;
[0026] (2) Construct the feature information fusion layer.
[0027] Based on the above solution, the feature information fusion layer is further constructed, which consists of three key components:
[0028] The first layer of the feature information fusion layer operates on the superimposed features through 1×1 convolution to compress the number of channels;
[0029] The second layer uses 3×3 depth-wise separable convolution to fuse the spatial information of each branch feature;
[0030] Finally, cascade channel attention is used to further improve the accuracy of fusion. The calculation formula is as follows:
[0031] F b =CCA(DWConv 3×3 (Conv 1×1 (x b )))
[0032] Among them: F b Represents the output features after fusing feature information of different scales on branch b.
[0033] In the last step of feature information fusion, it is necessary to build a cascade channel attention mechanism. The specific steps are:
[0034] First, construct a high-resolution feature channel attention and input feature It will go through two 1×1 convolutions with different numbers of output channels and Get the feature matrix containing multi-spatial information and
[0035] Secondly, construct ECA channel attention and The sizes of are defined as HW×1×1 and HW×C / 4 respectively. The SoftMax function is used to perform nonlinear activation on the feature Q, and then Q is transposed to make its size 1×1×HW, and then matrix product operation is performed with V to compress the spatial information to the weight parameter
[0036] Finally, a 1×1 convolution is applied to MC Restore the number of channels and weight it with the input feature x to get the output feature The calculation formula is as follows:
[0037]
[0038] Among them, F SG and F SM Respectively represent the Sigmoid function and the SoftMax function, * represents the convolution operation, represents matrix product, and ⊙ represents the bit-by-bit multiplication operation.
[0039] Furthermore, in step S4, when constructing a lightweight model optimization algorithm, label knowledge distillation and feature knowledge distillation are included. The network model constructed in steps S1 and S2 is used as a student network, and the trained teacher network is used to supervise the student network. Specifically, the output features of each branch of the teacher model and the student model in the third stage are selected for supervised learning. In the fourth stage, the output features are only convolved by a single layer of 1×1 convolution to obtain the final key point heat map. The attention activation heat map of the bth branch is obtained by compressing the features in each channel dimension, and its expression is as follows:
[0040]
[0041] Among them, b is the index of branches with different resolutions, Before mapping The element value corresponding to the position (h, w) in the i-th channel feature map, To obtain the averaged attention activation heatmap;
[0042] After calculating the attention activation heatmaps of each branch of the teacher model and the student model, the loss function of feature knowledge distillation can be defined as the formula:
[0043]
[0044] Where N represents the number of branches in the third stage, and Represents the attention activation heatmaps of the student model and the teacher model on the b-th branch respectively.
[0045] On the other hand, the present invention also discloses a method for estimating the posture of a lightweight human target from air to ground, comprising the following steps:
[0046] Get the image containing the person's body;
[0047] The acquired image is input into the optimized lightweight human posture estimation model constructed by any of the construction methods described above to obtain an output result of the human posture estimation.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] (1) A lightweight human pose estimation module based on cascaded channel attention and a deep convolutional transformation module are constructed, which can improve the accuracy of pose estimation while keeping the model lightweight.
[0050] (2) A multi-scale feature fusion module is designed to enhance the semantic fusion effect between features of different scales, thereby enhancing the feature richness of the model, which helps the model better understand and process inputs with features of different scales and improves the model's adaptability to diverse scenarios.
[0051] (3) Design a knowledge distillation technology to retrain and optimize the network model, that is, use a more complex but more accurate teacher model to guide the lightweight student model to learn, so that the student model can improve its own detection accuracy without increasing the computational cost.
[0052] (4) Compared with similar methods, the designed network is more accurate in estimating the human pose of distant and small targets. It can effectively distinguish the estimation errors caused by the proximity of key points of small targets and is more suitable for air-to-ground human pose estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 Design a flow chart for lightweight human posture estimation based on knowledge distillation of the present invention;
[0054] Figure 2 It is a deep convolution transformation module diagram;
[0055] Figure 3 It is a multi-scale feature fusion module diagram;
[0056] Figure 4 is the cascade channel attention map;
[0057] Figure 5 is the feature knowledge distillation graph;
[0058] Figure 6 This is the result diagram of human posture visualization. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0060] The present invention provides a method and application for constructing an air-to-ground lightweight human target posture estimation model, such as Figure 1 As shown, first, the present invention first constructs a data set specifically for training air-to-ground tasks, then designs a deep convolution transformation module, and designs a multi-scale feature fusion module to produce a lightweight posture estimation network. Finally, the lightweight human posture estimation optimization based on knowledge distillation is used to complete the design of an efficient and compact posture estimation network model. The specific implementation steps are as follows:
[0061] Step 1, obtain the data set for human posture estimation and preprocess it;
[0062] 1.1 Obtain the original dataset for human posture estimation.
[0063] We use the COCO2017 public dataset and combine it with images of distant and small human objects taken by drones 5-10m above the ground to build an air-to-ground human pose estimation task dataset.
[0064] 1.2 Preprocess the original data set.
[0065] The height and width of the air-to-ground human pose estimation task dataset image are expanded to a fixed aspect ratio of height: width = 4:3, and the adjusted box is cropped from the image and resized to a fixed size of 256×192mm. The present invention uses a random scale factor ([0.75, 1.25]), random rotation ([-30°, 30°]), horizontal flipping and additional half-body enhancement to randomly enhance each person's sample, which is then processed as an input sample.
[0066] Step 2: Based on the HRNet network structure, a deep convolution transformation module is designed to replace the residual convolution module in the high-resolution network to obtain a multi-branch feature fusion module.
[0067] like Figure 2 As shown in the figure, the module uses a combination of 3×3 depthwise separable convolution and cascaded channel attention to construct a feature extraction layer, so as to capture local detail information more specifically while ensuring the lightweight of the model. The specific steps of the construction are as follows:
[0068] 2.1 Building module feature extraction layer;
[0069] Layer normalization is used to eliminate the differences between input samples. Layer normalization helps to ensure that the distribution of input data between different samples is relatively consistent, providing more stable input for subsequent processing. At the same time, a depth-separable convolution with a step size of 3×3 is used to extract the contextual information of the local area of the pixel point, and then the cascade channel attention is introduced to learn the importance of each channel feature of different inputs, so that the model can pay more attention to different channels of the input, thereby improving the perception of multi-channel features. Finally, a residual structure is added to alleviate the gradient disappearance and explosion problems during training. The calculation expression of the feature extraction layer is as follows:
[0070] F'=CCA(DWConv 3×3 (LayerNorm(x)))+x
[0071] Among them, x represents the input feature, LayerNorm represents the layer normalization operation, and DWConv 3×3 represents a 3×3 depthwise separable convolution, CCA represents cascaded channel attention, and F' represents the output features after the feature extraction layer.
[0072] 2.2 Building module channel mapping layer;
[0073] The channel mapping layer structure includes normalization and two 1×1 convolution operations, aiming to improve the nonlinear expression ability of the model. The first 1×1 convolution is used to expand the number of feature channels by 2 times. Through this operation, the model can learn the information in the input features more fully and increase the understanding of rich feature representations. Next, the second 1×1 convolution compresses the number of feature channels to the original number of channels. The purpose is to retain important feature information through nonlinear transformation while reducing the computational burden of the model. When outputting features, the channel mapping layer also uses a residual structure to reduce the risk of gradient explosion during the training process. The calculation formula of the channel mapping layer is as follows:
[0074]
[0075] in, They represent the first and second 1×1 convolutions respectively, and F” represents the final output feature of the deep convolution transformation module.
[0076] Step 3: Design a multi-scale feature fusion module and combine it with the multi-branch feature fusion module in step 2 to obtain a lightweight human posture estimation model.
[0077] Construct a multi-scale feature fusion module, which mainly includes a feature size redefinition layer and a feature information fusion layer. The former redefines the size of multi-scale features from different branches while keeping the number of channels unchanged, and superimposes the features with unified size on the channel dimension as the input of the feature information fusion layer; the latter is responsible for fusing the feature information output from the previous step, thereby improving the fusion accuracy and ensuring that features from different scales can be better fused. The module structure is as follows: Figure 3 As shown, the specific steps of the construction are as follows:
[0078] 3.1 Construct feature size redefinition layer;
[0079] The feature size redefinition layer is composed of three components: AdaptiveAvgPool, Identity, and Upsampling. When fusing the resolution features of other branches, this layer uses adaptive average pooling, identity mapping, and linear upsampling on the input high-resolution (HR), medium-resolution (MR), and low-resolution (LR) features to unify their feature sizes; then, by stacking these features of equal size in the channel dimension, features of the same dimension are provided for each branch. The calculation formula is as follows:
[0080]
[0081] x b =Concat(x bi )(i∈1,...,s)
[0082] Among them, x i represents the characteristics of the i-th branch, x bi Indicates that AdaptiveAvgPool, Identity, or Upsamling is used to pool x i The size of is defined as the feature after the feature size of the bth branch is the same, s represents the total number of branches in the current stage, x b Indicates that the feature x bi Features after superposition in the channel dimension.
[0083] 3.2 Construct feature information fusion layer;
[0084] The feature information fusion layer consists of three key components: 1×1 convolution, 3×3 depthwise separable convolution, and cascaded channel attention. This layer is designed to fuse feature information from different branches through these three operations to improve the accuracy of fusion.
[0085] The first layer of the feature information fusion layer operates on the superimposed features through 1×1 convolution, compressing its number of channels to make it equal to the number of feature channels of the current branch. This operation helps to fuse the information between feature channels and ensure that each channel can work better together; the second layer fuses the spatial information of each branch feature through 3×3 depthwise separable convolution. While maintaining computational efficiency, the depthwise separable convolution effectively captures the relationship between features in the spatial dimension and enhances the expressiveness of features; finally, cascaded channel attention is used to further improve the accuracy of fusion. The cascaded channel attention mechanism can learn the importance of each channel feature of different inputs, making the fusion more targeted; its calculation formula is as follows:
[0086] F b =CCA(DWConv 3×3 (Conv 1×1 (x b )))
[0087] Among them, F b Represents the output features after fusing feature information of different scales on branch b.
[0088] The third layer of the feature information fusion layer is the cascade channel attention mechanism, and its structure is as follows Figure 4 As shown in Figure 2, it consists of two parts: high-resolution feature channel attention and ECA (Efficient Channel Attention) channel attention.
[0089] In the design of high-resolution feature channel attention, the input feature First, it will go through two 1×1 convolutions with different numbers of output channels. and To obtain a feature matrix containing more spatial information and
[0090] Secondly, and The sizes of are defined as HW×1×1 and HW×C / 4 respectively, and the SoftMax function is used to perform nonlinear activation on the feature Q, and then Q is transposed to make its size 1×1×HW, and then matrix product operation is performed with V, so that the spatial information is compressed to the weight parameter
[0091] Finally, a 1×1 convolution is used for MC Restore the number of channels and weight it with the input feature x to get the output feature The calculation formula is as follows:
[0092]
[0093] Among them, F SG and F SM Respectively represent the Sigmoid function and the SoftMax function, * represents the convolution operation, represents matrix product, and ⊙ represents the bit-by-bit multiplication operation.
[0094] In the design of ECA channel attention, after global average pooling (GAP) of the input features, one-dimensional convolution Conv1d is used to realize cross-channel information interaction. The calculation formula is as follows:
[0095]
[0096] Step 4: Design a lightweight model optimization algorithm, use the lightweight model optimization algorithm to optimize the training process of the lightweight human posture estimation model obtained in step 3, and input the data in the data set of step 1 into the optimized lightweight human posture estimation model for retraining.
[0097] The lightweight human posture estimation model is retrained and optimized by offline distillation. At the same time, a feature distillation method based on attention activation is constructed to make the intermediate feature dimensions of the teacher model and the student model the same, so that the loss function can be directly calculated. The feature knowledge distillation is as follows: Figure 5 shown.
[0098] In order to realize feature knowledge distillation, the present invention selects the output features of each branch of the third stage of the teacher model and the student model for supervised learning. In the third stage, both the teacher model and the student model have three feature branches of different scales, so that the model can perform multi-scale feature representation. In the student model, the number of feature channels on each branch is only half of that of the teacher model. Because feature distillation in the early stage of the model will force the student model to fit the output of the teacher model, resulting in excessive learning difficulty and poor training effect, only the features in the third stage are selected for knowledge transfer. In the final fourth stage of the model, the output features are only subjected to a single-layer 1×1 convolution to obtain the final key point heat map, and the feature distribution is slightly different from the predicted heat map, so label knowledge distillation can be directly used.
[0099] The feature distillation process based on attention activation is as follows: The feature map size on the bth branch can be expressed as C b ×H b×W b , then the attention activation heat map of the b-th branch can be obtained by compressing the features in each channel dimension, as shown in the following formula:
[0100]
[0101] Among them, b is the index of the branch with different resolutions, Before mapping The element value corresponding to the position (h, w) in the i-th channel feature map, To obtain the averaged attention activation heat map; after calculating the attention activation heat map of each branch of the teacher model and the student model, the loss function of feature knowledge distillation can be defined as the formula:
[0102]
[0103] N represents the number of branches in the third stage, and Represent the attention activation heatmaps of the student model and the teacher model on the bth branch. Through the above loss function, the student model can learn the degree of attention and importance distribution of the model on key points from the teacher model, making up for the limited transfer of high-level semantic feature knowledge by label knowledge distillation.
[0104] The present invention adopts Object Keypoint Similarity (OKS) as the evaluation criterion, and the calculation is shown in the following formula:
[0105]
[0106] The indicators of model accuracy include average precision AP, AP 50 ,AP 75 ,AP M ,AP L Series value and average recall rate AR. As shown in Table 1, the present invention is compared with multiple traditional lightweight models in experiments.
[0107] Table 1 Comparative experimental results of the present invention and the lightweight model
[0108]
[0109] As can be seen from Table 1, the present invention has achieved significant improvements in various indicators after teacher model knowledge supervision training. When compared with traditional models such as Dite-HRNet-30, Lite-HRnet-30 and GhostPoseNet that use high-resolution backbone feature extraction networks, the performance indicators AP and AR of the lightweight human posture estimation model proposed in the present invention are improved by 1.8 to 14.9 percentage points and 1.6 to 13.7 percentage points respectively, indicating that it can still maintain better performance under a relatively lightweight setting, while at the same time achieving higher precision indicators AP 50 A higher accuracy is obtained, which proves that the lightweight human posture estimation model of the present invention is more effective in recognizing small targets.
[0110] like Figure 6 As shown, the present invention performs human posture estimation on human body pictures taken from air to ground. It can be obviously observed that the lightweight human posture estimation model of the present invention has better robustness for scenes such as incompleteness and occlusion, and has better recognition effect for joint points in image details. These observation results emphasize that the present invention is more effective in air-to-ground scenes, and has more accurate positioning performance in the case of distant scenes, small targets and joint points that are too close when the drone is 5-10m above the ground, proving that the lightweight human posture estimation model of the present invention has better effect for air-to-ground human posture recognition.
[0111] The above is an explanation of the specific implementation of the present invention, rather than a limitation of the present invention. Those skilled in the relevant technical field can also make various equivalent technical solutions without departing from the scope of the present invention, so all equivalent technical solutions should be included in the patent protection scope of the present invention.
Claims
1. A method for constructing an air-to-ground lightweight human target posture estimation model, characterized in that: The following steps are involved: S1: Build a dataset for human pose estimation; S2: Based on the HRNet network structure, a deep convolution transformation module is designed to replace the residual convolution module in the high-resolution network to obtain a multi-branch feature fusion module; S3: Design a multi-scale feature fusion module, and combine it with the multi-branch feature fusion module in step S2 to obtain a lightweight human posture estimation model; S4: Design a lightweight model optimization algorithm and use it to optimize the training process of the lightweight human pose estimation model in step S3, and input the data set of step S1 into the optimized lightweight human pose estimation model for retraining.
2. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 1, characterized in that: In step S1, the data set construction includes: (1) Obtain the original data set for human posture estimation; (2) Preprocess the acquired original data set to obtain input samples.
3. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 2, characterized in that: In step S2, the design of the deep convolution transformation module includes: (1) Building module feature extraction layer; (2) Construct the module channel mapping layer.
4. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 3, characterized in that: The specific steps to construct the feature extraction layer are: Layer normalization is used to eliminate the differences in input samples, 3×3 depth-separable convolution is used to extract the contextual information of the local pixel area of the input sample, and cascade channel attention is introduced to learn the channel features of different input samples; The calculation expression of the feature extraction layer is as follows: F'=CCA(DWConv 3×3 (LayerNorm(x)))+x Among them: x represents the input feature, LayerNorm represents the layer normalization operation, DWConv 3×3 represents a 3×3 depthwise separable convolution, CCA represents cascaded channel attention, and F' represents the output features after the feature extraction layer.
5. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 4, characterized in that: Constructing the channel mapping layer includes layer normalization and two 1×1 convolution operations, and its construction steps are: The output features of the feature extraction layer are used as new input features for layer normalization. The first 1×1 convolution is used to expand the feature channels by 2 times to fully learn the information in the new input features. The second 1×1 convolution is used to compress the number of feature channels to the original number of channels to retain important feature information. The calculation expression of the channel mapping layer is as follows: in: They represent the first and second 1×1 convolutions respectively, and F” represents the final output feature of the deep convolution transformation module.
6. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 1, characterized in that: In step S3, the design of the multi-scale feature fusion module includes: (1) Constructing a feature size redefinition layer; (2) Construct the feature information fusion layer.
7. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 6, characterized in that: Constructing the feature information fusion layer consists of three key components: 1×1 convolution, 3×3 depth-wise separable convolution, and cascaded channel attention.
8. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 7, characterized in that: The specific steps to build the cascade channel attention module are: First, construct a high-resolution feature channel attention and input feature It will go through two 1×1 convolutions with different numbers of output channels and Get the feature matrix containing multi-spatial information and Secondly, construct ECA channel attention and The sizes of are defined as HW×1×1 and HW×C / 4 respectively. The SoftMax function is used to perform nonlinear activation on the feature Q, and then Q is transposed to make its size 1×1×HW, and then matrix product operation is performed with V to compress the spatial information to the weight parameter Finally, a 1×1 convolution is applied to MC Restore the number of channels and weight it with the input feature x to get the output feature The calculation formula is as follows: Among them, F SG and F SM Respectively represent the Sigmoid function and the SoftMax function, * represents the convolution operation, represents matrix product, and ⊙ represents the bit-by-bit multiplication operation.
9. The method for constructing an air-to-ground lightweight human target posture estimation model according to claim 1, characterized in that: In step S4, when constructing a lightweight model optimization algorithm, the output features of each branch of the teacher model and the student model in the third stage are selected for supervised learning. In the fourth stage, the output features are only convolved through a single layer of 1×1 convolution to obtain the final key point heat map. The attention activation heat map of the b-th branch is obtained by compressing the features in each channel dimension, and its expression is as follows: Among them, b is the index of branches with different resolutions, Before mapping The element value corresponding to the position (h, w) in the i-th channel feature map, To obtain the averaged attention activation heatmap; After calculating the attention activation heatmaps of each branch of the teacher model and the student model, the loss function of feature knowledge distillation can be defined as the formula: Where N represents the number of branches in the third stage, and Represents the attention activation heatmaps of the student model and the teacher model on the b-th branch respectively.
10. A method for estimating the pose of a lightweight human target from air to ground, characterized in that: The following steps are involved: Get the image containing the person's body; The acquired image is input into the optimized lightweight human posture estimation model constructed by the construction method described in any one of claims 1 to 9 to obtain an output result of the human posture estimation.
Citation Information
Patent Citations
Knowledge distillation-based human body posture estimation method and system
CN117636403A
Lightweight high-resolution network human body posture estimation method
CN117671779A
Compression method and platform of pre-training language model based on knowledge distillation
US20220067274A1
Cited By
Personnel behavior identification method and equipment used in low-illumination scene, and medium
CN121505697A
