High-resolution human body key point detection method and system

Abstract

The invention discloses a high-resolution human body key point detection method and system. The method comprises the following steps: S1, selecting a data set; s2, preprocessing the data set; s3, constructing a pyramid-type high-resolution network, wherein the pyramid-type high-resolution network comprises a shuffling module and a feature fusion module; wherein the shuffling module is composed of a feature reservation branch and a feature extraction branch, and the feature extraction branch comprises a PSA module and a convolutional layer; the feature fusion module fuses from a high-resolution branch to a low-resolution branch, then fuses from the low-resolution branch to a high-resolution branch, and finally adds the two fusion results as output; s4, training the human body key point network constructed in the step S3 by adopting the data set preprocessed in the step S2; and S5, selecting a network with optimal performance in training to carry out human body key point prediction, and introducing a super-resolution head heat map post-processing method to obtain key point positions. According to the method, rich multi-scale space information can be extracted, and cross-dimension important features are captured in the channel attention vector.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a high-resolution human key point detection method and system. Background Art

[0002] Human key point detection technology is widely used in sports, virtual reality, medical rehabilitation, and sports posture correction. In recent years, heatmap-based methods have achieved remarkable results in human key point detection. Specifically, this method generates a heatmap for each keypoint, each of which contains a two-dimensional Gaussian distribution centered on the true value joint position, which can suppress false positives and smooth the training process. Therefore, heatmap-based methods have stronger generalization ability and are easier to optimize. However, since continuous coordinate values ​​need to be discretized into small-scale heatmaps and the resolution of the heatmap limits the coordinate accuracy, there is a non-negligible quantization error. Due to the existence of quantization error, many heatmap-based methods perform poorly in direct decoding. Most heatmap-based methods rely heavily on further post-processing to mitigate quantization errors, but this post-processing is poorly optimized.

[0003] Studies have shown that high-resolution networks have great advantages in detecting key points of the human body. They connect high-resolution to low-resolution convolutional flows in parallel; repeatedly exchange information between layers, and the features obtained are semantically richer and spatially more precise. At the same time, the attention mechanism can capture the key point information of the human body very well. It is widely used in computer vision fields such as image classification, target detection, instance segmentation, semantic segmentation, scene parsing, and action localization. There are currently two attention mechanisms: channel attention and spatial attention. Studies have shown that by using channel attention, spatial attention, or both, performance can be significantly improved. The most commonly used channel attention method is the Squeeze-and-Excitation Networks (SENet), which can significantly improve performance at a fairly low cost, but the disadvantage of SENet is that it ignores the importance of spatial information.

[0004] Therefore, some researchers proposed the Bottleneck Attention Module (BAM) and the Convolutional Block Attention Module (CBAM) to effectively combine spatial attention and channel attention to enrich the attention map. However, there are still two important and challenging problems. One is how to effectively capture and utilize the spatial information of feature maps of different scales to enrich the feature space. Second, channel or spatial attention can only effectively capture local information, but cannot establish long-term channel dependencies.

[0005] Correspondingly, other methods have been proposed in the industry to solve the above two problems, such as a method based on multi-scale feature representation and cross-channel information interaction (PyConv, Res2Net, HS-ResNet). However, the above methods all bring high network complexity and heavy computational burden to the network. Summary of the invention

[0006] Aiming at the deficiencies of existing human key point detection methods in processing spatial information and establishing long-distance channel dependencies, as well as the quantization error problem in heat map post-processing, the present invention proposes a high-resolution human key point detection method and system based on the Pyramid Squeeze Attention (PSA) module. The present invention can extract rich multi-scale spatial information and capture important cross-dimensional features in the channel attention vector.

[0007] The technical solution of the present invention is as follows:

[0008] A high-resolution human key point detection method specifically comprises the following steps:

[0009] S1. Select a data set, which contains multiple human body images in natural scenes and records the coordinates of key points of the human body in each image.

[0010] S2. Before feature extraction, preprocess the data set of step S1.

[0011] S3. Based on the HRNet network, the Pyramid Squeeze Attention-High Resolution Network (PSA-HRNet) is constructed in combination with the Shuffle-PSA module: The Pyramid Squeeze Attention-High Resolution Network contains two main modules: the shuffle module and the feature fusion module. The shuffle module consists of a feature retention branch and a feature extraction branch. The feature retention branch is a residual connection that retains the original features. The feature extraction branch includes the PSA module and the standard convolutional layer. This shuffle module enables the network to learn a variety of feature representations. The fusion process of the feature fusion module is to first fuse from the high-resolution branch to the low-resolution branch, then fuse the high-resolution branch from the low-resolution branch, and finally add the results of the two fusions as the output of the feature fusion module.

[0012] S4, using the human key point data set preprocessed in step S2 to train the human key point network constructed in step S3.

[0013] S5. After the training in step S4 is completed, the network with the best performance in the training is selected to predict the key points of the human body. During the prediction, a super-resolution head (SR Head) heat map post-processing method is introduced to obtain the key point positions with low errors.

[0014] As a preferred solution, in step S1, each human body image is a three-channel color image, and the total number of key points of the human body is 17.

[0015] As a preferred solution, in step S1, there is only one human object in each human image.

[0016] As a preferred embodiment, in step S1, the 17 key points in the human body are left eye, right eye, nose, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle and right ankle.

[0017] As a preferred solution, in step S2, the specific method of data preprocessing is: scaling the width and height of the pedestrian detection frame in the training data to the corresponding scale while maintaining the original ratio of the target, and then performing corresponding padding, cropping the pedestrians in the image of the training data using a fixed aspect ratio of 4:3, and scaling to a fixed size of 256x192 after cropping.

[0018] As a preferred solution, all the cropped images obtained above are randomly flipped horizontally with a probability of 0.5, randomly scaled at a scaling ratio between 0.65 and 1.35, and randomly rotated within a rotation angle range between -45° and +45°.

[0019] As a preferred solution, in step S3, the feature extraction branch includes a 1x1 convolution, a PSA module, a 1x1 convolution, and a channel shuffle connected in sequence.

[0020] As a preferred solution, in step S3, batch normalization (BN) and ReLU activation function operations are added after each convolution; and batch normalization BN operations are added after the PSA module.

[0021] As a preferred solution, in step S3, the specific formula for the feature fusion module to fuse from the high-resolution branch to the low-resolution branch is:

[0022]

[0023] Where n is the total number of branches; A i is the output result of the i-th branch before feature fusion; H iIt is the fusion result of the i-th branch when high resolution is fused to low resolution; the F(a,b,H) function represents the resolution conversion of the output result H of branch a to achieve fusion with branch b, where a is the high-resolution branch adjacent to b.

[0024] As a preferred solution, in step S3, the specific formula for the lightweight feature fusion module to fuse from the low-resolution branch to the high-resolution branch is:

[0025]

[0026] Where n is the total number of branches; A i is the output result of the i-th branch before feature fusion; L i is the fusion result of the i-th branch when low resolution is fused to high resolution; the F(a′,b′,L) function represents the resolution conversion of the output result L of the a′ branch to achieve fusion with the b′ branch, where a′ is the low-resolution branch adjacent to b′.

[0027] As a preferred solution, in step S3, the number of basic channels of the Stage2, Stage3, and Stage4 branches of lightweight human key point detection is set in an incremental manner, and the number of channels of the same branch is different in different stages.

[0028] Let the network learn different human key point information at different scales to improve the generalization ability of the network, and design a multi-scale convolutional neural network structure to enable the network to learn different human key point information at different scales.

[0029] As a preferred solution, in step S4, the loss function used to calculate the key points is as follows:

[0030]

[0031] Among them, H j (p) is the prediction result, which indicates the probability of key point j at p. It is a real generated heat map. Whether the key point j is visible is indicated by v j Indicates that when key point j is invisible, v j = 0, when key point j is visible v j = 1. The convolutional neural network is trained using the above loss function.

[0032] As a preferred solution, in step S5, a lightweight heat map post-processing method SR Head is introduced. The key points are encoded as key point embeddings through the encoder, followed by decoding and splicing operations, which effectively reduces the quantization error. After using PSA-HRNet to predict human key points, a more accurate key point heat map can be obtained through heat map decoding processing of SR Head. The SRPSA-HRNet network constructed by combining PSA-HRNet and SR Head not only significantly enhances the generalization ability of the network, but also reduces the quantization error.

[0033] In the present invention, the PSA module improves network performance by enhancing multi-scale characterization capabilities and long-distance channel dependencies. At the same time, the present invention introduces a lightweight heat map post-processing method Super-Resolution Head, which first encodes various key points into key point embeddings through a key point encoder, then decodes them through a separate large-core convolution, calculates multiple heat maps about the data, and finally splices all the key point heat maps together through pixel shuffle to obtain a high-resolution heat map of the data, thereby effectively reducing the quantization error.

[0034] The present invention introduces the PSA module into the shuffle block to replace the traditional 3x3 convolution block, thereby constructing a pyramid shuffle (Shuffle-Pyramid Squeeze Attention, Shuffle-PSA) module, so that the network can learn diverse feature representations. On this basis, the Shuffle-PSA module is integrated into the HRNet high-resolution network to replace the redundant Basic Block module, thereby constructing a more efficient human key point detection network - the Pyramid Squeeze Attention-High Resolution Network (PSA-HRNet).

[0035] After the heat map decoding process of SR Head, a more accurate key point heat map can be obtained. Combining PSA-HRNet with the Super Resolution Pyramid SqueezeAttention-High Resolution Network (SRPSA-HRNet) constructed by SR Head not only significantly enhances the generalization ability of the network and reduces the quantization error, but also shows flexible adaptability in diverse application scenarios, thus promoting the continuous advancement of key point detection technology.

[0036] The present invention also discloses a high-resolution human key point detection system, which is used to execute the above method and includes the following modules:

[0037] Data set selection module: selects a data set, the data set includes human body images, and records the coordinates of key points of the human body in the images;

[0038] Preprocessing module: preprocess the selected data set;

[0039] Network construction module: Construct a pyramid-shaped high-resolution network, which includes two modules: a shuffle module and a feature fusion module. The shuffle module consists of a feature preservation branch and a feature extraction branch. The feature extraction branch includes a PSA module and a convolutional layer. The feature fusion module first fuses from the high-resolution branch to the low-resolution branch, and then fuses the high-resolution branch from the low-resolution branch. Finally, the results of the two fusions are added together as the output.

[0040] Network training module: uses the preprocessed data set to train the human key point network constructed by the network construction module;

[0041] Detection module: Select the best performing network in training to predict key points of the human body, and introduce the super-resolution head heat map post-processing method to obtain the key point position.

[0042] In summary, compared with the prior art, the present invention has the following characteristics:

[0043] The present invention collects data containing human body images and performs preprocessing operations to enhance the generalization ability of the network and improve the accuracy of the network.

[0044] The present invention proposes a PSA-HRNet network, which can extract rich multi-scale spatial information and capture important cross-dimensional features in channel attention, enhance the multi-scale representation capability and long-distance channel dependency, enable the network to learn diversified feature representations, effectively capture the frequently changing information relationships in the input feature map, and improve network performance.

[0045] The present invention introduces a lightweight heat map post-processing method, SR Head, which encodes various key points into key point embeddings through a key point encoder, decodes them through a separate large-kernel convolution, calculates multiple heat maps about the data, and finally splices all the key point heat maps together through pixel shuffle to obtain the HR heat map of the data, effectively reducing the quantization error.

[0046] The present invention allows the same branch to have different numbers of channels at different stages in a human key point detection network, so that the deeper the network is, the more and more valuable features can be extracted, and the feature expression capability is stronger. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 The figure is a flow chart of a method for detecting key points of a human body according to a preferred embodiment of the present invention.

[0049] Figure 2 Schematic diagram of predefined human body key point template.

[0050] Figure 3 This is a network structure diagram of the human body key point detection method according to the preferred embodiment of the present invention.

[0051] Figure 4 This is the PSA structure diagram of the pyramid attention module.

[0052] Figure 5 This is the structure diagram of the pyramid shuffle module Shuffle-PSA.

[0053] Figure 6 This is the SR Head network diagram of the heat map post-processing method.

[0054] Figure 7 The figure is a block diagram of a human key point detection system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be described clearly and completely in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0056] like Figure 1 As described above, this embodiment provides a high-resolution human key point detection method based on the PSA module, and builds a PSA-HRNet network, which can extract rich multi-scale spatial information without significantly increasing the complexity of the algorithm, and capture important cross-dimensional features in channel attention. By enhancing the multi-scale representation capability and long-distance channel dependency, the network performance is improved. Specifically, the following steps are included:

[0057] S1, obtain the human key point detection data set;

[0058] This embodiment uses the COCO dataset, which contains more than 200,000 pictures and 250,000 human bodies marked with 17 key points. The COCOtrain2017 training set contains 57K pictures and 150K human images, and the COCOva12017 validation set contains 5,000 pictures. The training set is used to train the network, and the validation set is used to evaluate the performance of the trained network.

[0059] In this embodiment, the number of key points of the human body is 17. Figure 2 As shown in Table 1, a schematic diagram of the structure of 17 key points of the human body is shown.

[0060] Table 1 Human body key point structure

[0061] serial number meaning serial number meaning serial number meaning 1 nose 2 Left Eye 3 Right eye 4 left ear 5 Right ear 6 Left shoulder 7 Right shoulder 8 left elbow 9 right elbow 10 left wrist 11 right wrist 12 Left hip 13 Right hip 14 Left knee 15 Right knee 16 Left ankle 17 Right ankle

[0062] S2, data preprocessing;

[0063] The acquired raw data is preprocessed. First, for the COCO2017 dataset, the image resolution used in network training is 256x192, so the image needs to be processed accordingly. This process is not simply to stretch the image to 256x192, but to keep the original ratio of the target, scale it to the appropriate size, and then perform necessary padding. The pedestrians in the dataset images are cropped with a fixed aspect ratio of 4:3, and then scaled to a fixed size of 256x192 after cropping. In addition, some enhancement measures are implemented on the data, including random rotation, random scaling, random horizontal flipping, and Half Body (there is a certain probability of cropping the target, retaining only the key points of the upper or lower body). For example, the cropped image is randomly flipped horizontally with a probability of 0.5, randomly scaled at a scaling ratio between 0.65 and 1.35, and randomly rotated within the rotation angle range of -45° to +45°.

[0064] S3, network construction;

[0065] HRNet has powerful functions in semantic segmentation, human key point detection, object detection, image classification, etc. The present invention aims to tap its potential in the field of human key point detection. In this embodiment, the high-resolution network based on the PSA module is improved. Based on the HRNet network, this embodiment constructs the Shuffle-PSA module to build a more powerful human key point detection network PSA-HRNet, such as Figure 3As shown in the figure. The network first uses convolution operations to downsample, then passes through Layer 1, and then passes through three transition layers (Transition) and three feature extraction stages (Stage). After each transition layer, a new scale branch is added to build a multi-branch network structure. Each feature extraction stage continuously extracts feature information with the help of this multi-branch parallel structure, and fuses different resolution features from different branches to form a feature fusion module to obtain richer feature representation. Finally, the network outputs the extracted features of the highest resolution branch.

[0066] The first layer (Layer 1) is four stacked Shuffle-PSA modules. The Shuffle-PSA module is a fusion of the shuffle module and the PSA module. The structure of each module is as follows: the input information first passes through the Channel Split module, which evenly divides the channels of the input image into two branches, namely the feature retention branch and the feature extraction branch. The feature retention branch is a residual connection that retains the original features. The feature extraction branch first passes through a convolution kernel of size 1x1 and stride 1, a BN (Batch Normalization) layer and a ReLU activation function, then passes through a PSA module, then connects to a BN (Batch Normalization) layer, and finally passes through a convolution kernel of size 1x1 and stride 1, a BN layer, and a ReLU activation function. The PSA module can effectively capture the frequently changing information relationship in the input feature map and prevent the subsequent fixed shuffle rules from causing a lack of diversity in information transmission. The PSA module consists of SEWeight, Softmax, SPC and other parts. Among them, the SEWeight module is a variant of the SE module, which is used to extract channel attention weights. It obtains the channel information of the input feature map through global average pooling, and then generates the attention weight of each channel through a fully connected layer and activation functions (such as ReLU and Sigmoid). The Softmax function is used to convert the original attention weights output by the SEWeight module into normalized weights that can be weighted and summed. Softmax ensures that the attention weights of all channels add up to 1, so that the contributions of different channels can be weighted. The SPC module is the core part of the PSA module and is responsible for implementing multi-scale feature extraction. The input feature map is first divided into S groups in the channel dimension, each group has C / S channels. Then, each group of feature maps is processed by convolution kernels of different scales to capture spatial information of different scales. The processed multi-scale feature maps are spliced ​​again in the channel dimension to form a feature map that integrates multi-scale information. Its detailed structure is shown in the attached. Figure 4 .

[0067] Finally, the residual connection and feature extraction parts are spliced ​​together, and then the channels are shuffled through the Channel Shuffle module to form the Shuffle-PSA module to achieve feature fusion between groups. The detailed structure is shown in the attached Figure 5 .

[0068] Extraction stages 2, 3, and 4 are operations on multiple branches after transition layer 1 (Transition1), transition layer 2 (Transition2), and transition layer 3 (Transition3). Each branch must pass through four superimposed Shuffle-PSA modules and then fuse information of different scales. The output of each scale branch is fused by the output of all branches.

[0069] S4. Use the loss function to train the convolutional neural network. The loss function used to calculate the key points is as follows:

[0070]

[0071] Among them, J is the key point of the human body, H j (p) is the prediction result, which indicates the probability of key point j at p. It is a real generated heat map. Whether the key point j is visible is indicated by v j Indicates that when key point j is invisible, v j = 0, when key point j is visible v j =1.

[0072] S5. Select the best performing network in training to predict human key points. When predicting, introduce a lightweight heat map post-processing method SR Head. Use the key point encoder to encode various key points into key point embeddings, decode them through a separate large kernel convolution, calculate multiple heat maps about the data, and finally splice all the key point heat maps together through pixel shuffle to obtain the HR heat map of the data, which effectively reduces the quantization error and alleviates the problem of low-resolution features being difficult to supervise. Its detailed structure is shown in the attached Figure 6 In this embodiment, the SRPSA-HRNet network is constructed by combining PSA-HRNet and SR Head, which significantly enhances the generalization ability of the network and reduces the quantization error.

[0073] In summary, this embodiment proposes a high-performance human key point detection method. Based on the HRNet network, a human key point detection model SPSA-HRNet that can effectively establish long-distance channel dependency is built; a new module Shuffle-PSA is added to the HRNet extraction module, which can maintain a high accuracy without significantly increasing the number of parameters. The PSA-HRNet human key point detection model proposed in the present invention enhances the multi-scale characterization capability and long-distance channel dependency, allowing the network to learn diverse features. At the same time, a lightweight heat map prediction method SR Head is introduced to effectively reduce the quantization error. The SPSA-HRNet model constructed by combining PSA-HRNet and SR Head not only significantly enhances the generalization ability of the network and reduces the quantization error, but also shows flexible adaptability in diverse scenarios, promoting the continuous progress of key point detection.

[0074] like Figure 7 As shown, this embodiment discloses a high-resolution human key point detection system, which is used to execute the above method, and includes the following modules:

[0075] Data set selection module: selects a data set, the data set includes human body images, and records the coordinates of key points of the human body in the images;

[0076] Preprocessing module: preprocess the selected data set;

[0077] Network construction module: Construct a pyramid-shaped high-resolution network, which includes two modules: a shuffle module and a feature fusion module. The shuffle module consists of a feature preservation branch and a feature extraction branch. The feature extraction branch includes a PSA module and two standard convolutional layers. The feature fusion module first fuses from the high-resolution branch to the low-resolution branch, and then fuses the high-resolution branch from the low-resolution branch. Finally, the results of the two fusions are added together as the output.

[0078] Network training module: uses the preprocessed data set to train the human key point network constructed by the network construction module;

[0079] Detection module: Select the best performing network in training to predict key points of the human body, and introduce the super-resolution head heat map post-processing method to obtain the key point position.

[0080] For other contents of this embodiment, please refer to the above method embodiment.

[0081] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A high-resolution human key point detection method, characterized by The steps include: S1. Select a data set, the data set includes human body images, and records the coordinates of key points of the human body in the images; S2, preprocessing the data set of step S1; S3. Construct a pyramid-shaped high-resolution network, which includes two modules: a shuffle module and a feature fusion module. The shuffle module consists of a feature preservation branch and a feature extraction branch. The feature extraction branch includes a PSA module and a convolutional layer. The feature fusion module first fuses from the high-resolution branch to the low-resolution branch, then fuses the high-resolution branch, and finally adds the results of the two fusions as the output. S4, using the data set pre-processed in step S2 to train the human body key point network constructed in step S3; S5. Select the network with the best performance in training to predict the key points of the human body, and introduce the super-resolution head heat map post-processing method to obtain the key point position.

2. A high-resolution human key point detection method as claimed in claim 1, characterized in that: Step S2 The preprocessing is as follows: the width and height of the pedestrian detection box in the dataset are scaled to the set scale while maintaining the original ratio of the target, and then the corresponding padding is performed. The pedestrians in the dataset image are cropped with a fixed aspect ratio of 4:3, and then scaled to a fixed size of 256x192 after cropping.

3. A high-resolution human key point detection method as claimed in claim 2, characterized in that: The cropped image is randomly flipped horizontally with a probability of 0.5, randomly scaled with a scaling ratio between 0.65 and 1.35, and randomly rotated within a rotation angle range between -45° and +45°.

4. A high-resolution human key point detection method as claimed in claim 1, characterized in that: In step S3, the feature extraction branch includes a 1x1 convolution, a PSA module, a 1x1 convolution, and a channel shuffle connected in sequence.

5. A high-resolution human key point detection method as claimed in claim 4, characterized in that: In step S3, a BN layer and a ReLU activation function are added after the 1x1 convolution; a BN layer is added after the PSA module.

6. A high-resolution human key point detection method as claimed in any one of claims 1 to 5, characterized in that: In step S3, the specific formula for the feature fusion module to fuse from the high-resolution branch to the low-resolution branch is: Where n is the total number of branches; A i is the output result of the i-th branch before feature fusion; H i It is the fusion result of the i-th branch when high resolution is fused to low resolution; the F(a,b,H) function represents the resolution conversion of the output result H of branch a to achieve fusion with branch b, where a is the high-resolution branch adjacent to b.

7. A high-resolution human key point detection method as claimed in claim 6, characterized in that: In step S3, the specific formula for the feature fusion module to fuse from the low-resolution branch to the high-resolution branch is: Where, L i is the fusion result of the i-th branch when low resolution is fused to high resolution; the F(a,b,L) function represents the resolution conversion of the output result L of the a′ branch to achieve fusion with the b′ branch, where a′ is the low-resolution branch adjacent to b′.

8. A high-resolution human key point detection method as claimed in any one of claims 1 to 5, characterized in that: In step S5, the human body key point network is trained using a loss function, and the loss function is as follows: Among them, J is the key point of the human body, H j (p) is the prediction result, which indicates the probability of key point j at p. is the real generated heat map, v j Indicates whether key point j is visible.

9. A high-resolution human key point detection system, used to execute the method according to any one of claims 1 to 8, characterized in that: Includes the following modules: Data set selection module: selects a data set, the data set includes human body images, and records the coordinates of key points of the human body in the images; Preprocessing module: preprocess the selected data set; Network construction module: Construct a pyramid-shaped high-resolution network, which includes two modules: a shuffle module and a feature fusion module. The shuffle module consists of a feature preservation branch and a feature extraction branch. The feature extraction branch includes a PSA module and a convolutional layer. The feature fusion module first fuses from the high-resolution branch to the low-resolution branch, and then fuses the high-resolution branch from the low-resolution branch. Finally, the results of the two fusions are added together as the output. Network training module: uses the preprocessed data set to train the human key point network constructed by the network construction module; Detection module: Select the best performing network in training to predict key points of the human body, and introduce the super-resolution head heat map post-processing method to obtain the key point position.

Images