Human body key point detection method and device, electronic equipment and storage medium
By introducing self-attention and cross-attention mechanisms into the human keypoint detection model, and combining underlying semantic features and positional encoding features, the robustness and speed issues of existing methods are solved, and more efficient keypoint localization is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2026-03-13
AI Technical Summary
Existing methods for human keypoint detection have shortcomings in terms of robustness and accuracy. In particular, bottom-up methods are not robust to human bodies of different sizes, while top-down methods, such as regression-based methods, have high computational costs and low detection speeds, and heatmap-based methods have high computational costs and slow speeds.
A keypoint detection model is adopted, including a feature extraction network and a keypoint decoder. It utilizes self-attention and cross-attention mechanisms to extract keypoint features of different granularities through multiple keypoint decoding layers. Combined with low-level semantic features and positional encoding features, the attention mechanism is applied to detect human keypoints.
It improves the accuracy and precision of human body key point localization, reduces computational costs, increases detection speed, and accelerates the convergence speed of model training.
Smart Images
Figure CN121661668A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting key points on the human body. Background Technology
[0002] The goal of human keypoint detection is to detect the two-dimensional spatial location of the human body and its corresponding keypoints from an input image. Therefore, it is also known as human pose estimation and is a prerequisite task for human action recognition, behavior analysis, human-computer interaction, etc.
[0003] Human keypoint detection methods mainly fall into two categories: bottom-up and top-down. Bottom-up methods first detect all keypoints in an image, then cluster them into targets based on their spatial relationships. These methods are less robust to human figures of varying sizes. Top-down methods first detect the person in the image, then determine the location of keypoints. These methods primarily include regression and heatmap methods. Heatmap-based methods are similar to instance segmentation tasks, predicting a score for each pixel in the image to represent the confidence that the location belongs to a keypoint; the pixel with the highest score is then identified as the keypoint. Regression-based methods utilize convolutional neural networks (CNNs) to extract features, then use fully connected layers to directly predict the normalized coordinates of the human body and keypoints.
[0004] However, heatmap-based methods introduce spatial location information for predicting dense points, which increases computational costs and reduces detection speed. Furthermore, the human body, as a non-rigid structure, exhibits considerable freedom in posture, while CNNs excel at predicting relative positional relationships. For example, some methods alleviate regression difficulty by predicting the offset of keypoints relative to the anchor center. This results in points far from the anchor center receiving weaker supervision signals, ultimately leading to poor regression for these points and inaccurate keypoint localization results. Summary of the Invention
[0005] This invention provides a method, device, electronic device, and storage medium for detecting key points on the human body, in order to overcome the deficiencies existing in related technologies.
[0006] This invention provides a method for detecting key points on the human body, comprising: Acquire the image of the human body to be detected; Based on the key point detection model, the key point information of the human body in the human body image to be detected is determined; The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
[0007] According to the present invention, a method for detecting key points of the human body is provided, wherein each key point decoding layer in the plurality of key point decoding layers includes: a self-attention layer and a cross-attention layer; The self-attention layer is used to generate self-attention weights based on the input keypoint features and the corresponding keypoint query vectors, and to query the input keypoint features based on the self-attention weights to obtain self-attention features; the keypoint features input to the self-attention layer of the first keypoint decoding layer in the multiple keypoint decoding layers are the initial keypoint features, and the keypoint features input to the self-attention layer of the non-first keypoint decoding layers are the keypoint features extracted by the previous keypoint decoding layer. The cross-attention layer is used to determine contextual attention weights and spatial attention weights based on the input keypoint features, the corresponding keypoint query vector, the underlying semantic features, the positional encoding features, and the self-attention features, and to query the underlying semantic features based on the contextual attention weights and the spatial attention weights to obtain keypoint features.
[0008] According to the present invention, a method for detecting key points of the human body, wherein the self-attention layer is specifically used for: Based on the key point features of the input, a first query vector and a first key vector are generated, and based on the key point query vector, a second query vector and a second key vector are generated. Based on the first query vector and the second query vector, a self-attention query vector is generated; based on the first key vector and the second key vector, a self-attention key vector is generated; and based on the self-attention query vector and the self-attention key vector, the self-attention weight is generated.
[0009] According to the present invention, a method for detecting key points in the human body, wherein the cross-attention layer is specifically used for: Based on the key point query vector, a two-dimensional coordinate feature is generated, the two-dimensional coordinate feature is position-encoded to obtain the position query feature, and the spatial feature is determined based on the input key point feature; Based on the location query features and the spatial features, a third query vector is determined, and based on the location encoding features, a third key vector is determined. Based on the third query vector and the third key vector, the spatial attention weight is determined. Based on the self-attention features, a fourth query vector is determined, and based on the underlying semantic features, a fourth key vector is determined. Based on the fourth query vector and the fourth key vector, the context attention weight is determined.
[0010] According to the present invention, a method for detecting human key points is provided, wherein the human key point information includes the location and confidence level of the human key points; The end keypoint prediction layer includes a linear layer and a feedforward neural network layer, which are respectively connected to the end keypoint decoding layer. The linear layer is used to output the confidence level; The feedforward neural network layer is used to output the locations of the key points of the human body.
[0011] According to the present invention, a method for detecting human key points is provided, wherein the key point detection model further includes: an encoder and a bounding box decoder connected in sequence; The output of the feature extraction network is connected to the input of the encoder; The encoder is used to encode the underlying semantic features to obtain encoded features; The bounding box decoder is used to predict human bounding box information in the human image to be detected based on the encoded features.
[0012] According to the present invention, a method for detecting human key points is provided, wherein the key point decoder further includes a key point prediction layer connected to each non-tail key point decoding layer in the plurality of key point decoding layers; Accordingly, the keypoint detection model is trained based on the following steps: Human image samples are input into the keypoint prediction results of each initial keypoint prediction layer in the initial keypoint decoder of the initial detection model, as well as the bounding box prediction results output by the initial bounding box decoder. Based on the keypoint prediction results and the human keypoint labels carried by the human image samples, calculate the keypoint prediction loss, and based on the bounding box prediction results and the human bounding box labels carried by the human image samples, calculate the bounding box prediction loss. Based on the keypoint prediction loss and the bounding box prediction loss, the model loss is calculated, and based on the model loss, the model parameters of the initial detection model are iteratively updated to obtain the keypoint detection model.
[0013] According to the present invention, a method for detecting key points of the human body is provided, wherein the feature extraction network includes multiple feature extraction layers of different granularities; The multiple feature extraction layers are used to extract initial semantic features of different granularities from the human image to be detected, map the initial semantic features to the same channel dimension and spatial dimension to obtain the mapping result, and concatenate the mapping result to obtain the underlying semantic features.
[0014] The present invention also provides a human body key point detection device, comprising: Image acquisition module, used to acquire images of the human body to be detected; The key point detection module is used to determine the key point information of the human body in the human body image to be detected based on the key point detection model. The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the human key point detection method as described above.
[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the human key point detection method as described above.
[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the human key point detection method as described above.
[0018] The human keypoint detection method, apparatus, electronic device, and storage medium provided by this invention extract low-level semantic features from the human image to be detected through a feature extraction network, providing basic data for subsequent human keypoint detection. Multiple keypoint decoding layers in the keypoint decoder extract keypoint features of different granularities from the human image to be detected, optimizing the performance and speed of human keypoint regression from coarse to fine, thus improving the accuracy of human keypoint localization. Furthermore, each keypoint decoding layer applies low-level semantic features and their positional encoding features, providing a wealth of location guidance information for human keypoint localization, thereby reducing the difficulty of human keypoint regression. Simultaneously, the application of an attention mechanism allows for explicit focus on the location region of the human keypoint during subsequent human keypoint queries, improving the localization accuracy. The keypoint detection model introduced by this method not only improves the accuracy and precision of human keypoint localization but also avoids the high computational cost and low detection speed problems caused by heatmap methods, reducing computational costs and increasing detection speed compared to heatmap methods. Furthermore, by using the positional encoding features of the underlying semantic features as the input to the model, the model can quickly focus on the key point locations during training, thereby improving the convergence speed of model training. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a flowchart illustrating the human body key point detection method provided by the present invention.
[0021] Figure 2 This is a partial structural diagram of the key point decoder in the human body key point detection method provided by the present invention.
[0022] Figure 3 This is a schematic diagram of the key point detection model in the human body key point detection method provided by the present invention.
[0023] Figure 4 This is a schematic diagram of the human body key point detection device provided by the present invention.
[0024] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0026] Existing bottom-up human keypoint detection methods are not robust to human bodies of different sizes, while top-down human keypoint detection methods based on regression are limited by the CNN structure, resulting in inaccurate keypoint localization results. Methods based on heatmaps have high computational costs and low detection speed.
[0027] Furthermore, while existing regression-based methods introduce Transformer networks on top of CNNs, they only utilize the Transformer network's self-attention mechanism to learn the spatial location information of human key points. This Transformer network only serves to enhance features and cannot enhance the learning of human key points, thus leading to inaccurate key point localization results.
[0028] Based on this, this embodiment of the invention provides a method for detecting key points in the human body.
[0029] Figure 1 This is a flowchart illustrating a method for detecting key human points provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: S1, acquire the image of the human body to be detected; S2, Based on the key point detection model, determine the key point information of the human body in the human body image to be detected; The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
[0030] Specifically, the human key point detection method provided in this embodiment of the invention is executed by a human key point detection device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.
[0031] First, step S1 is executed to obtain the human body image to be detected. This human body image contains a human body and is the image from which the human body keypoint information needs to be detected. Here, the human body keypoint information may include information such as the location and confidence level of the human body keypoints in the human body image to be detected. The confidence level represents a measure of the accuracy of predicting that a pixel in the human body image to be detected belongs to a human body keypoint.
[0032] The human body image to be detected can be any video frame captured by an image acquisition device, and the human body image to be detected can be an RGB color image. The image acquisition device can be a camera, webcam, or other device with image acquisition capabilities, and the image acquisition device can be connected to a human key point detection device to send the human body image to be detected to the human key point detection device.
[0033] Then, step S2 is executed, which inputs the human image to be detected into the key point detection model, and outputs the human key point information in the human image to be detected through the key point detection model.
[0034] The keypoint detection model can be a neural network model, which may include a feature extraction network and a keypoint decoder connected in sequence.
[0035] This feature extraction network can be a backbone network, which may include a CNN. The CNN architecture can be LeNet-5, AlexNet, VGGNet, the Inception series, or ResNet, and can contain a series of convolutional and pooling layers for different downsampling operations. The input to this feature extraction network is the human image to be detected, used to extract the low-level semantic features (backbone content embeddings) of the human image. The output of this feature extraction network is the low-level semantic features of the human image to be detected.
[0036] The keypoint decoder can be built based on a Transformer network and can be named a landmark Transformer network. This keypoint decoder can be used to extract the relative positional relationships between keypoints of the human body in an image to be detected. The input to the keypoint decoder is the low-level semantic features of the human image to be detected and the positional embeddings of the low-level semantic features. These positional embeddings can be obtained by performing sine, cosine, or other transformations on the low-level semantic features; no specific limitations are specified here.
[0037] The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The number of keypoint decoding layers can be set as needed, for example, up to 6. Each keypoint decoding layer is connected sequentially, and the final keypoint decoding layer is connected to the final keypoint prediction layer.
[0038] Each keypoint decoding layer is used to extract keypoint features of different granularities from the human body keypoints in the human body image to be detected by applying an attention mechanism, utilizing the low-level semantic features output by the feature extraction network and the positional encoding features of the low-level semantic features.
[0039] The input to the first and second keypoint decoding layers includes the low-level semantic features output by the feature extraction network and the positional encoding features of these low-level semantic features. In addition, it includes keypoint query vectors (landmark quires) built into the model corresponding to the first and second keypoint decoding layers. These keypoint query vectors can be randomly initialized learnable variables, continuously optimized during the training of the keypoint detection model. During the training of the keypoint detection model, the shallow keypoint decoding layer predicts the offset of a human keypoint relative to a corresponding reference point. Then, the deep keypoint decoding layer further refines the prediction of human keypoints based on this. In this way, the reference points are optimized towards human keypoints, which provides a large amount of prior positional guidance for the regression of reference points, thereby improving the accuracy of keypoint regression.
[0040] Here, each human body key point corresponds to a key point query vector, which is used to represent the location information of the corresponding human body key point.
[0041] The output of the first and second keypoint decoding layer can include coarse-grained keypoint features extracted by applying an attention mechanism through the input of the first and second keypoint decoding layer.
[0042] The input to any keypoint decoding layer, except for the first keypoint decoding layer, can include keypoint features (decoder embeddings) output from the previous keypoint decoding layer, low-level semantic features, and positional encoding features of the low-level semantic features. In addition, it also includes the keypoint query vector built into the model. The output of any keypoint decoding layer can include fine-grained keypoint features extracted using an attention mechanism from the input of any keypoint decoding layer.
[0043] It is understandable that, since the first keypoint decoding layer does not have the previous keypoint decoding layer, the keypoint features output by the previous keypoint decoding layer are empty, that is, the input of the first keypoint decoding layer does not contain the keypoint features output by the previous keypoint decoding layer.
[0044] The attention mechanisms applied in each keypoint decoding layer include self-attention and cross-attention mechanisms. The self-attention mechanism can remove duplicate detections of human keypoints and perform feature interaction between the keypoint query vector and the output of the previous keypoint decoding layer to determine the self-attention features used for the prediction of subsequent human keypoint information. The cross-attention mechanism can aggregate self-attention features, positional encoding features of low-level semantic features, and low-level semantic features to refine the input of the keypoint decoding layer, thereby improving the prediction of human keypoint information in subsequent keypoint decoder layers.
[0045] From the first keypoint decoding layer to the last keypoint decoding layer, the extracted keypoint features can be keypoint features of varying granularity, from coarse to fine. Here, all keypoint features can be contextual features of human body keypoints.
[0046] The input to the end-keypoint prediction layer is the keypoint features extracted by the end-keypoint decoding layer. The end-keypoint prediction layer can predict and output human keypoint information based on these features. This keypoint information may include keypoint location and confidence level. The end-keypoint prediction layer may include a linear layer and a feedforward network (FFN) layer. The linear layer outputs the confidence level, and the feedforward network layer outputs the keypoint location. Here, the end-keypoint prediction layer can be a landmark head. The FFN layer may include two linear transformation layers and a non-linear activation function.
[0047] The human keypoint detection method provided in this embodiment of the invention first acquires an image of the human body to be detected; then, based on a keypoint detection model, it determines the human keypoint information in the image. The keypoint detection model used in this method includes a feature extraction network, a keypoint decoder, and a prediction network connected in sequence. The feature extraction network extracts the low-level semantic features of the human body image to be detected, providing basic data for subsequent human keypoint detection. The keypoint decoder extracts keypoint features of different granularities from the human body keypoints in the image through multiple keypoint decoding layers, optimizing the performance and speed of human keypoint regression from coarse to fine, thus improving the accuracy of human keypoint localization. Furthermore, each keypoint decoding layer applies low-level semantic features and positional encoding features of those features, providing a large amount of location guidance information for human keypoint localization, thereby reducing the difficulty of human keypoint regression. Simultaneously, the application of an attention mechanism allows for explicit focus on the location region of the human keypoint during subsequent human keypoint queries, improving the localization accuracy of the human keypoints. The keypoint detection model introduced by this method not only improves the accuracy and precision of human body keypoint localization but also avoids the problems of high computational cost and low detection speed caused by heatmap methods. Compared with heatmap methods, it can reduce the computational cost of human body keypoints and improve the detection speed. In addition, since the positional encoding features of the underlying semantic features are used as the input of the model, the model can quickly focus on the keypoint locations during training, improving the convergence speed of model training.
[0048] Based on the above embodiments, each of the multiple keypoint decoding layers includes: a self-attention module and a cross-attention module; The self-attention layer is used to generate self-attention weights based on the input keypoint features and the corresponding keypoint query vectors, and to query the input keypoint features based on the self-attention weights to obtain self-attention features; the keypoint features input to the self-attention layer of the first keypoint decoding layer in the multiple keypoint decoding layers are the initial keypoint features, and the keypoint features input to the self-attention layer of the non-first keypoint decoding layers are the keypoint features extracted by the previous keypoint decoding layer. The cross-attention layer is used to determine contextual attention weights and spatial attention weights based on the input keypoint features, the corresponding keypoint query vector, the underlying semantic features, the positional encoding features, and the self-attention features, and to query the underlying semantic features based on the contextual attention weights and the spatial attention weights to obtain keypoint features.
[0049] Specifically, each keypoint decoding layer may include a self-attention layer and a cross-attention layer. The self-attention layer implements the self-attention mechanism, and the cross-attention layer implements the cross-attention mechanism. Here, the keypoint decoding layers are connected sequentially, that is, the output of the first keypoint decoding layer is connected to the input of the next keypoint decoding layer.
[0050] The self-attention layer can generate self-attention weights using the input keypoint features and keypoint query vectors. For each keypoint decoding layer, the internal operations of its self-attention layer are identical; the only difference lies in the input. The input to the self-attention layer of the first keypoint decoding layer is the initial keypoint features, which are obtained through random initialization. The input to the self-attention layer of subsequent keypoint decoding layers is the output of the previous keypoint decoding layer.
[0051] The self-attention layer utilizes the input keypoint features and the corresponding keypoint query vector to generate a self-attention query vector Q1, a self-attention key vector K1, and a self-attention value vector V1. The self-attention weights are obtained by performing a matrix multiplication operation between the self-attention query vector Q1 and the self-attention key vector K1. .
[0052] Subsequently, self-attention weights can be used to query the keypoint features of the input to obtain self-attention features. Self-attention features can be represented as: ; in, This is a self-attention feature. Let K1 be the dimension of each column of features in the target key vector K1. It is a normalized exponential function.
[0053] The cross-attention layer can utilize the input keypoint features, low-level semantic features, and self-attention features to determine the contextual attention weight c, i.e., the contextual attention weight. Through the keypoint query vector and positional encoding features, it can generate the spatial attention weight p. The contextual attention weight c and the spatial attention weight p can constitute the cross-attention weight, i.e.: ; in, These are the cross-attention weights.
[0054] Subsequently, cross-attention weights can be used to query the underlying semantic features to obtain the key point features extracted by the decoding layer of any key point.
[0055] Based on this, in each keypoint decoding layer, the outputs of the self-attention layer and the cross-attention layer can be connected to residual connections and normalization layers (add & normal). The inputs of the residual connections and normalization layers connected to the self-attention layer include the output of the self-attention layer and the self-attention value vector V1. The inputs of the residual connections and normalization layers connected to the cross-attention layer include the output of the cross-attention layer and the cross-attention value vector V2.
[0056] The output of the residual connection and normalization layer connected by the cross-attention layer can also be connected to an FFN layer and a residual connection and normalization layer in sequence. The input of the residual connection and normalization layer includes the output of the residual connection and normalization layer connected by the cross-attention layer and the output of the FFN layer.
[0057] In this embodiment of the invention, the self-attention layer can remove repeated detection of human key points, perform feature interaction between key point query vectors, and enable key point query vectors to clearly understand the features predicted by other key point query vectors, thereby reducing repeated predictions and alleviating the dependence on non-maximum suppression (NMS), which improves the deployment speed of the model. The entire human key point detection process can be optimized end-to-end, providing more convenience for real-time detection.
[0058] By using a cross-attention layer, the keypoint decoder can quickly focus on the location of human keypoints, thus improving the performance of keypoint regression. Furthermore, by decoupling the cross-attention weights into contextual attention weights c and spatial attention weights p, contextual queries can focus on contextual features, and location queries can focus on spatial location encoding features. This accelerates the convergence speed of the keypoint detection model, improves the efficiency of feature representation learning, reduces the difficulty of subsequent keypoint regression, and enhances the detection performance of the keypoint detection model.
[0059] Based on the above embodiments, the self-attention layer is specifically used for: Based on the key point features of the input, a first query vector and a first key vector are generated, and based on the key point query vector, a second query vector and a second key vector are generated. Based on the first query vector and the second query vector, a self-attention query vector is generated; based on the first key vector and the second key vector, a self-attention key vector is generated; and based on the self-attention query vector and the self-attention key vector, the self-attention weight is generated.
[0060] Specifically, in the self-attention layer, the input keypoint features are processed through three linear layers to generate the first query vector Q. content First key vector K contentThe self-attention value vector V1, with the subscript "content" indicating the presence of contextual information, is used. The keypoint query vector is processed through two linear layers to generate the second query vector Q. pos Second bond vector K pos The subscript pos indicates that it contains spatial location information.
[0061] By using the first query vector Q content Second query vector Q pos Adding them together generates a self-attention query vector Q1, which is obtained by adding the first key vector K. content Second bond vector K pos Adding them together generates a self-attention key vector K1. The self-attention weights are then obtained by performing a matrix multiplication operation between the self-attention query vector Q1 and the self-attention key vector K1. .
[0062] Furthermore, by combining the self-attention value vector V1, it is possible to query the key point features of the input and obtain the self-attention features. .
[0063] In this embodiment of the invention, the self-attention weight is determined by the query vector and key vector generated by the input key point features and key point query vector respectively. The self-attention weight can be determined by comprehensively considering the input key point features and key point query vector, making the obtained self-attention weight more accurate.
[0064] Based on the above embodiments, the cross-attention layer is specifically used for: Based on the key point query vector, a two-dimensional coordinate feature is generated, the two-dimensional coordinate feature is position-encoded to obtain the position query feature, and the spatial feature is determined based on the input key point feature; Based on the location query features and the spatial features, a third query vector is determined, and based on the location encoding features, a third key vector is determined. Based on the third query vector and the third key vector, the spatial attention weight is determined. Based on the self-attention features, a fourth query vector is determined, and based on the underlying semantic features, a fourth key vector is determined. Based on the fourth query vector and the fourth key vector, the context attention weight is determined.
[0065] Specifically, in the cross-attention layer, the keypoint query vector can first pass through a feedforward neural network to generate two-dimensional coordinate features (2D coord embedding). These 2D coordinate features can then be normalized using a normalization function to obtain normalized 2D coordinate features. Here, both the 2D coordinate features and the normalized 2D coordinate features are used to represent the reference points of the keypoints. The normalization function can be the sigmoid function.
[0066] Two-dimensional coordinate features can be represented as: ; in, It is a two-dimensional coordinate feature. This is the query vector for key points.
[0067] Subsequently, the normalized two-dimensional coordinate features can be encoded using a sinusoidal positional encoding function to obtain positional query features (query embedding), which can be keypoint query vectors with positional information.
[0068] Subsequently, the input keypoint features can be processed by a feedforward neural network to determine spatial embedding.
[0069] Subsequently, the location query features and spatial features can be multiplied to obtain the third query vector. That is: ; Where f represents the input keypoint features. This is a sinusoidal position encoding function.
[0070] Third query vector This will provide location information for contextual queries of key points on the human body.
[0071] By passing the location features through a linear layer, the third key vector can be obtained. Furthermore, the third query vector and the third key vector Matrix multiplication can be used to determine the spatial attention weights. .
[0072] By passing the self-attention features through a linear layer, a fourth query vector can be obtained. Furthermore, by passing the underlying semantic features through two linear layers, we can obtain the cross-attention value vector V2 and the fourth key vector. Furthermore, the fourth query vector and the fourth bond vector Matrix multiplication can be used to determine the context attention weights. .
[0073] Therefore, the cross-attention weights can be expressed as: .
[0074] Understandably, in the cross-attention layer, the third query vector is used... With the fourth query vector By concatenating the vectors, we can obtain the cross-attention query vector Q2, which is obtained by using the third key vector. With the fourth key vector By concatenating the vectors, we can obtain the cross-attention key vector K2.
[0075] In this embodiment of the invention, spatial attention weights and contextual attention weights are generated separately, which decouples contextual queries from location queries, thereby accelerating the convergence speed of the keypoint detection model and improving its detection performance. Furthermore, the two-dimensional coordinate features generated by the keypoint query vector represent the reference points of the keypoints. The final prediction of human keypoints is achieved by predicting the offset of the human keypoints relative to the reference points, providing a location prior for human keypoint prediction, which is more effective for detecting non-rigid structures like the human body.
[0076] Based on the above embodiments, the human body key point information includes the location and confidence level of the human body key points; The end keypoint prediction layer includes a linear layer and a feedforward neural network layer, which are respectively connected to the end keypoint decoding layer. The linear layer is used to output the confidence level; The feedforward neural network layer is used to output the locations of the key points of the human body.
[0077] Specifically, the processing steps of the final keypoint decoding layer and the keypoint prediction layer are as follows: Figure 2 As shown, the input to the final keypoint decoding layer includes the keypoint features output from the previous keypoint decoding layer and the keypoint query vector. By combining the keypoint features output from the previous keypoint decoding layer with the keypoint query vector, the self-attention value vector V1, the self-attention query vector Q1, and the self-attention key vector K1 can be obtained respectively. After passing through the self-attention layer, the self-attention features are obtained. Self-attention characteristics The self-attention value vector V1 is then processed through residual connections and a normalization layer to update the self-attention features, resulting in the updated self-attention features.
[0078] The key point query vector is processed by a feedforward neural network and a sigmoid function to obtain normalized two-dimensional coordinate features, which are then encoded using sinusoidal position coding to obtain position query features.
[0079] The keypoint features output from the previous keypoint decoding layer are processed by a feedforward neural network to obtain spatial features. By multiplying the location query features with the spatial features, the third query vector can be obtained. .
[0080] By passing the updated self-attention features through a linear layer, we can obtain the fourth query vector. .
[0081] By passing the location features through a linear layer, the third key vector can be obtained. .
[0082] By passing the underlying semantic features through two linear layers, we can obtain the cross-attention value vector V2 and the fourth key vector. .
[0083] The third query vector With the fourth query vector By concatenating the vectors, we can obtain the cross-attention query vector Q2, which is obtained by using the third key vector. With the fourth key vector By concatenating the vectors, we can obtain the cross-attention key vector K2.
[0084] The cross-attention value vector V2, cross-attention query vector Q2, and cross-attention key vector K2 are passed through the cross-attention layer to obtain cross-attention features. The cross-attention features and the updated self-attention features are then passed through residual connections and normalization layers to obtain the key point features output by the final key point decoding layer.
[0085] Subsequently, the keypoint features output by the end keypoint decoding layer are input to the FFN layer, which outputs the human body keypoint positions. The keypoint features output by the end keypoint decoding layer are input to the linear layer, which outputs the confidence level.
[0086] The locations of key points on the human body can be represented as: ; in, These are key locations on the human body.
[0087] Confidence level can be expressed as: ; in, , where is the confidence level.
[0088] Based on the above embodiments, the key point detection model further includes: an encoder and a bounding box decoder connected in sequence; The output of the feature extraction network is connected to the input of the encoder; The encoder is used to encode the underlying semantic features to obtain encoded features; The bounding box decoder is used to predict human bounding box information in the human image to be detected based on the encoded features.
[0089] Specifically, such as Figure 3As shown, the keypoint detection model further includes an encoder and a bounding box decoder connected sequentially. The encoder may include multiple encoding layers of different granularities, and the bounding box decoder may include multiple bounding box decoding layers of different granularities. The number of encoding layers, the number of bounding box decoding layers, and the number of keypoint decoding layers can all be equal, for example, all can be N. In this embodiment of the invention, N can be 6.
[0090] The output of the feature extraction network can be connected to the input of the encoder and the input of the keypoint decoder, respectively, to input initial semantic features into the encoder and the keypoint decoder.
[0091] Meanwhile, the input to the keypoint decoder also includes the positional encoding of the human image to be detected. The encoder, bounding box decoder, and keypoint decoder can together form a keypoint Transformer network.
[0092] The encoder can encode the underlying semantic features at different granularities to obtain encoded features.
[0093] The bounding box decoder extracts the spatial relationships between pixels and uses encoded features to predict the bounding box information of the human body in the image to be detected. This bounding box information can include the target category and target location of the human bounding box. The target location can include the center point position and the size of the human bounding box. The prediction principle of the center point position is the same as that of the keypoint prediction principle. The size of the human bounding box can include width and height, which can be a set of normalized values predicted by the bounding box decoder.
[0094] Understandably, a bounding box decoder can include linear layers and a feedforward neural network. The linear layer outputs the target category of the human bounding box, while the feedforward neural network outputs the target location of the human bounding box. After determining the target location of the human bounding box, it can be annotated on the input human image to be detected. Similarly, after determining the location of human keypoints, the keypoint decoder can annotate the human keypoints on the input human image to be detected.
[0095] In this embodiment of the invention, by configuring an encoder and a bounding box decoder connected in sequence in the key point detection model, the key point detection model can simultaneously have the dual functions of human bounding box detection and human key point detection, thereby improving the application scope of the key point detection model.
[0096] Based on the above embodiments, the feature extraction network includes multiple feature extraction layers with different granularities; The multiple feature extraction layers are used to extract initial semantic features of different granularities from the human image to be detected, map the initial semantic features to the same channel dimension and spatial dimension to obtain the mapping result, and concatenate the mapping result to obtain the underlying semantic features.
[0097] Specifically, a feature extraction network can include multiple feature extraction layers of different granularities, each of which can extract initial semantic features at one granularity. For example... Figure 3 As shown, the feature extraction network can include four feature extraction layers, namely P1, P2, P3, and P4, with the granularity decreasing from coarse to fine.
[0098] Subsequently, the initial semantic features can be mapped to the same channel dimension and spatial dimension to obtain the mapping results. These mapping results are then concatenated to obtain the underlying semantic features. The same channel dimension means the same number of channels, and the same spatial dimension means the width and height of the initial semantic features are the same. Therefore, the underlying semantic features obtained by concatenation can be represented as a long cube obtained by concatenating four congruent cubes.
[0099] In this embodiment of the invention, by extracting initial semantic features of different granularities from the human body image to be detected, semantic information of different granularities in the human body image to be detected can be obtained, which facilitates the provision of richer information for subsequent human body keypoint detection and human body bounding box detection. Moreover, by mapping each initial semantic feature to the same channel dimension and spatial dimension, and concatenating the mapping results, the input of the subsequent model structure can be simplified, the computational load of the model can be reduced, and the detection efficiency can be improved.
[0100] Based on the above embodiments, the key point detection model provided in this embodiment of the invention further includes a key point prediction layer connected to each non-end key point decoding layer in the plurality of key point decoding layers. Accordingly, the keypoint detection model is trained based on the following steps: Human image samples are input into the keypoint prediction results of each initial keypoint prediction layer in the initial keypoint decoder of the initial detection model, as well as the bounding box prediction results output by the initial bounding box decoder. Based on the keypoint prediction results and the human keypoint labels carried by the human image samples, calculate the keypoint prediction loss, and based on the bounding box prediction results and the human bounding box labels carried by the human image samples, calculate the bounding box prediction loss. Based on the keypoint prediction loss and the bounding box prediction loss, the model loss is calculated, and based on the model loss, the model parameters of the initial detection model are iteratively updated to obtain the keypoint detection model.
[0101] Specifically, the keypoint decoder may also include a keypoint prediction layer connected to each non-tail keypoint decoding layer in a plurality of keypoint decoding layers, that is, the output of each keypoint decoding layer is connected to a keypoint prediction layer.
[0102] Furthermore, the keypoint detection model can be trained from the initial detection model. The initial detection model can have the same structure as the keypoint detection model, the difference being the model parameters. The parameters of the initial detection model are initialized, while the parameters of the keypoint detection model are trained. In other words, the initial detection model includes a feature extraction network and a keypoint Transformer network. The feature extraction network consists of a CNN for low-level semantic feature extraction; the keypoint Transformer network can include an encoder, a bounding box decoder, and a keypoint decoder.
[0103] During training, human image samples are first input into the keypoint prediction results output by each initial keypoint prediction layer in the initial keypoint decoder of the initial detection model, as well as the bounding box prediction results output by the initial bounding box decoder. Human image samples can carry human keypoint labels and human bounding box labels. The human keypoint labels include the location of the human keypoints in the human image sample, and the human bounding box labels include the category and location of the human bounding boxes in the human image sample.
[0104] By using the keypoint prediction results output by each initial keypoint prediction layer and the human keypoint labels, the prediction loss of each initial keypoint prediction layer can be calculated, and the sum of the prediction losses of all initial keypoint prediction layers is the keypoint prediction loss.
[0105] The bounding box prediction loss can be calculated using the bounding box prediction results and the human bounding box labels carried by the human image samples.
[0106] The model loss is obtained by weighted summing the keypoint prediction loss and the bounding box prediction loss. This model loss can then be used to iteratively update the parameters of the initial detection model until the loss value converges or a preset number of iterations is reached, thus obtaining the keypoint detection model. Here, the preset number of iterations can be set as needed, and this embodiment of the invention does not impose a specific limitation on it.
[0107] In summary, the keypoint detection model used in this embodiment of the invention can be deployed in an end-to-end manner without requiring post-processing such as non-maximum suppression. Furthermore, it can... Figure 2 The keypoint regression learning process shown is integrated into the decoder structure of the Transformer network. Local and global spatial location information helps in the learning of keypoint features, further improving the regression performance of keypoints.
[0108] like Figure 4 As shown, based on the above embodiments, this embodiment of the invention provides a human key point detection device, including: Image acquisition module 41 is used to acquire images of the human body to be detected; The key point detection module 42 is used to determine the key point information of the human body in the human body image to be detected based on the key point detection model. The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
[0109] Based on the above embodiments, the human key point detection device provided in the embodiments of the present invention includes, in the plurality of key point decoding layers, each key point decoding layer includes: a self-attention layer and a cross-attention layer; The self-attention layer is used to generate self-attention weights based on the input keypoint features and the corresponding keypoint query vectors, and to query the input keypoint features based on the self-attention weights to obtain self-attention features; the keypoint features input to the self-attention layer of the first keypoint decoding layer in the multiple keypoint decoding layers are the initial keypoint features, and the keypoint features input to the self-attention layer of the non-first keypoint decoding layers are the keypoint features extracted by the previous keypoint decoding layer. The cross-attention layer is used to determine contextual attention weights and spatial attention weights based on the input keypoint features, the corresponding keypoint query vector, the underlying semantic features, the positional encoding features, and the self-attention features, and to query the underlying semantic features based on the contextual attention weights and the spatial attention weights to obtain keypoint features.
[0110] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention, wherein the self-attention layer is specifically used for: Based on the key point features of the input, a first query vector and a first key vector are generated, and based on the key point query vector, a second query vector and a second key vector are generated. Based on the first query vector and the second query vector, a self-attention query vector is generated; based on the first key vector and the second key vector, a self-attention key vector is generated; and based on the self-attention query vector and the self-attention key vector, the self-attention weight is generated.
[0111] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention, wherein the cross-attention layer is specifically used for: Based on the key point query vector, a two-dimensional coordinate feature is generated, the two-dimensional coordinate feature is position-encoded to obtain the position query feature, and the spatial feature is determined based on the input key point feature; Based on the location query features and the spatial features, a third query vector is determined, and based on the location encoding features, a third key vector is determined. Based on the third query vector and the third key vector, the spatial attention weight is determined. Based on the self-attention features, a fourth query vector is determined, and based on the underlying semantic features, a fourth key vector is determined. Based on the fourth query vector and the fourth key vector, the context attention weight is determined.
[0112] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention includes human key point information including human key point location and confidence level; The end keypoint prediction layer includes a linear layer and a feedforward neural network layer, which are respectively connected to the end keypoint decoding layer. The linear layer is used to output the confidence level; The feedforward neural network layer is used to output the locations of the key points of the human body.
[0113] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention further includes an encoder and a bounding box decoder connected in sequence. The output of the feature extraction network is connected to the input of the encoder; The encoder is used to encode the underlying semantic features to obtain encoded features; The bounding box decoder is used to predict human bounding box information in the human image to be detected based on the encoded features.
[0114] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention includes a feature extraction network comprising multiple feature extraction layers of different granularities. The multiple feature extraction layers are used to extract initial semantic features of different granularities from the human image to be detected, map the initial semantic features to the same channel dimension and spatial dimension to obtain the mapping result, and concatenate the mapping result to obtain the underlying semantic features.
[0115] Based on the above embodiments, the human key point detection device provided in this embodiment of the invention further includes a key point prediction layer connected to each non-tail key point decoding layer in the plurality of key point decoding layers. Accordingly, the keypoint detection model is trained based on the following steps: Human image samples are input into the keypoint prediction results of each initial keypoint prediction layer in the initial keypoint decoder of the initial detection model, as well as the bounding box prediction results output by the initial bounding box decoder. Based on the keypoint prediction results and the human keypoint labels carried by the human image samples, calculate the keypoint prediction loss, and based on the bounding box prediction results and the human bounding box labels carried by the human image samples, calculate the bounding box prediction loss. Based on the keypoint prediction loss and the bounding box prediction loss, the model loss is calculated, and based on the model loss, the model parameters of the initial detection model are iteratively updated to obtain the keypoint detection model.
[0116] Specifically, the functions of each module in the human key point detection device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.
[0117] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute the human key point detection method provided in the above embodiments.
[0118] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0119] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the human key point detection method provided in the above embodiments.
[0120] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the human key point detection method provided in the above embodiments.
[0121] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting key points on the human body, characterized in that, include: Acquire the image of the human body to be detected; Based on the key point detection model, the key point information of the human body in the human body image to be detected is determined; The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
2. The method for detecting key human body points according to claim 1, characterized in that, Each of the multiple keypoint decoding layers includes: a self-attention layer and a cross-attention layer; The self-attention layer is used to generate self-attention weights based on the input keypoint features and the corresponding keypoint query vectors, and to query the input keypoint features based on the self-attention weights to obtain self-attention features; the keypoint features input to the self-attention layer of the first keypoint decoding layer in the multiple keypoint decoding layers are the initial keypoint features, and the keypoint features input to the self-attention layer of the non-first keypoint decoding layers are the keypoint features extracted by the previous keypoint decoding layer. The cross-attention layer is used to determine contextual attention weights and spatial attention weights based on the input keypoint features, the corresponding keypoint query vector, the underlying semantic features, the positional encoding features, and the self-attention features, and to query the underlying semantic features based on the contextual attention weights and the spatial attention weights to obtain keypoint features.
3. The method for detecting key human body points according to claim 2, characterized in that, The self-attention layer is specifically used for: Based on the key point features of the input, a first query vector and a first key vector are generated, and based on the key point query vector, a second query vector and a second key vector are generated. Based on the first query vector and the second query vector, a self-attention query vector is generated; based on the first key vector and the second key vector, a self-attention key vector is generated; and based on the self-attention query vector and the self-attention key vector, the self-attention weight is generated.
4. The method for detecting key human body points according to claim 2, characterized in that, The cross-attention layer is specifically used for: Based on the key point query vector, a two-dimensional coordinate feature is generated, the two-dimensional coordinate feature is position-encoded to obtain the position query feature, and the spatial feature is determined based on the input key point feature; Based on the location query features and the spatial features, a third query vector is determined, and based on the location encoding features, a third key vector is determined. Based on the third query vector and the third key vector, the spatial attention weight is determined. Based on the self-attention features, a fourth query vector is determined, and based on the underlying semantic features, a fourth key vector is determined. Based on the fourth query vector and the fourth key vector, the context attention weight is determined.
5. The method for detecting key human body points according to claim 2, characterized in that, The human body key point information includes the location and confidence level of the human body key points; The end keypoint prediction layer includes a linear layer and a feedforward neural network layer, which are respectively connected to the end keypoint decoding layer. The linear layer is used to output the confidence level; The feedforward neural network layer is used to output the locations of the key points of the human body.
6. The method for detecting key human body points according to any one of claims 1-5, characterized in that, The key point detection model also includes: an encoder and a bounding box decoder connected in sequence; The output of the feature extraction network is connected to the input of the encoder; The encoder is used to encode the underlying semantic features to obtain encoded features; The bounding box decoder is used to predict human bounding box information in the human image to be detected based on the encoded features.
7. The method for detecting key human body points according to claim 6, characterized in that, The keypoint decoder also includes a keypoint prediction layer connected to each of the plurality of keypoint decoding layers that is not the last keypoint decoding layer. Accordingly, the keypoint detection model is trained based on the following steps: Human image samples are input into the keypoint prediction results of each initial keypoint prediction layer in the initial keypoint decoder of the initial detection model, as well as the bounding box prediction results output by the initial bounding box decoder. Based on the keypoint prediction results and the human keypoint labels carried by the human image samples, calculate the keypoint prediction loss, and based on the bounding box prediction results and the human bounding box labels carried by the human image samples, calculate the bounding box prediction loss. Based on the keypoint prediction loss and the bounding box prediction loss, the model loss is calculated, and based on the model loss, the model parameters of the initial detection model are iteratively updated to obtain the keypoint detection model.
8. The method for detecting key human body points according to any one of claims 1-5, characterized in that, The feature extraction network includes multiple feature extraction layers of different granularities; The multiple feature extraction layers are used to extract initial semantic features of different granularities from the human image to be detected, map the initial semantic features to the same channel dimension and spatial dimension to obtain the mapping result, and concatenate the mapping result to obtain the underlying semantic features.
9. A human body key point detection device, characterized in that, include: Image acquisition module, used to acquire images of the human body to be detected; The key point detection module is used to determine the key point information of the human body in the human body image to be detected based on the key point detection model. The key point detection model includes a feature extraction network and a key point decoder connected in sequence. The feature extraction network is used to extract the low-level semantic features of the human image to be detected; The keypoint decoder includes multiple cascaded keypoint decoding layers and a final keypoint prediction layer connected to the final keypoint decoding layer. The multiple keypoint decoding layers are used to extract keypoint features of different granularities in the human body keypoints in the image to be detected by applying an attention mechanism based on the underlying semantic features and the positional encoding features of the underlying semantic features. The final keypoint prediction layer is used to predict the human body keypoint information based on the keypoint features extracted by the final keypoint decoding layer.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the human key point detection method as described in any one of claims 1-8.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the human key point detection method as described in any one of claims 1-8.