Multi-person key point recognition method and device based on deep learning, and storage medium

Abstract

The application discloses a multi-person key point recognition method and device based on deep learning and a storage medium. The method comprises the following steps: constructing a multi-person key point recognition model based on deep learning, constructing a loss function, optimizing parameters of the multi-person key point recognition model based on deep learning based on the loss function, and recognizing multi-person key points of a to-be-detected image by using the optimized multi-person key point recognition model. The application can guarantee that global and local features in an image are fully utilized while maintaining more fine constraint capabilities among a human body, a posture and key points, so that end-to-end human posture estimation is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human pose estimation, and in particular to a multi-person key point recognition method based on deep learning, a device and a storage medium. BACKGROUND

[0002] The purpose of multi-person pose estimation is to accurately locate the key points of all human body targets in a picture in order to better assist computers in understanding human behavior. Nowadays, it plays an important role in computer vision tasks such as human-computer interaction, activity recognition, motion capture, motion tracking, pedestrian re-identification, augmented reality and virtual reality.

[0003] There are currently some excellent human pose estimation algorithms, and the overall framework structure can be divided into single-stage and two-stage, and the two-stage structure can be further divided into top-down and bottom-up structures. The top-down algorithm first relies on a human detector to detect all people from the image, and then uses a single-person pose estimation method to estimate the pose of all people. The disadvantage is that the algorithm running efficiency decreases with the increase of the number of people, and part of the occluded people cannot be detected, and the accuracy is not high. The bottom-up algorithm first detects all the key points of the people, then connects the key points to form a graph, and finally removes the wrong connections through graph optimization method to realize multi-person pose estimation. The advantage of the bottom-up algorithm is that the running time does not increase linearly with the increase of the number of people, which is more conducive to real-time multi-person pose estimation. However, this method involves subsequent grouping processing of key points, which usually requires manual operation, and the grouping algorithm involves tedious parameter adjustment skills. The two-stage method is too complex, so people are actively exploring a direct end-to-end multi-person pose estimation method.

[0004] “End-to-End Trainable Multi-Instance Pose Estimation with Transformers” (In Proceedings of the arXiv preprint arXiv:2103.12115, 2021) POET is the first to introduce an end-to-end target detection algorithm based on Transformer into multi-person pose estimation, and regards multi-instance image pose estimation as a direct set prediction problem. However, since it is directly improved based on the DETR structure, it only uses local dependent keys for key point regression, which has the defects of slow convergence speed and low performance.

[0005] Poseur: Direct human pose regression with transformers” (In Proceedings of the arXiv preprint arXiv:2201.07412, 2022) Poseur attempts to directly apply DETR to a single-stage framework and finds that its performance drops significantly. This can be an optimization conflict between global and local dependency learning using a shared encoder.

[0006] “End-to-End Multi-person Pose Estimation with Transformers” (In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 11069-11078) discloses a multi-person pose estimation structure based on Deformable DETR hierarchical set prediction, which effectively correlates the relationships between human key points while improving the convergence speed of the model. However, this method only uses the dependency relationship of local features to regress key points and does not fully utilize the information between contexts, making it difficult to align features.

[0007] “Explicit Box Detection Unifies End-to-end Multi-Person Pose Estimation.” (In Proceedings of the International Conference on Learning Representations, 2023) adopts an explicit bounding box detection structure, which considers target detection and pose estimation as two explicit bounding box detection processes with unified representation and regression supervision, and extracts global features from the encoding token through a human decoder as input to the key point decoder, thereby more effectively improving the convergence speed of the model. However, this method unnecessarily complicates the regression of human key points as bounding boxes and ignores the kinematic relationships between human bounding boxes and key points.

[0008] The patent specification with publication number CN116246338A provides a behavior recognition method based on a graph convolution and a Transformer composite neural network. The method includes processing human body part key node coordinate data into a graph data structure, and using a graph convolution model to model features in the spatial dimension, while using a Transformer model to model features in the time dimension. The method combines the advantages of both graph convolution neural network and Transformer neural network structures, solving the problem of limited receptive field of convolution limited by convolution kernel size. The method uses human skeleton sequences as input to ensure lightweight and anti-interference capability of input data; uses a graph convolution neural network structure to model spatial dimension information to overcome the weakness of weak spatial modeling capability of Transformer network structure; uses a Transformer network structure to model time dimension information to overcome the defect of graph convolution limited by convolution kernel size that cannot effectively model video segments with long duration. The invention provides a behavior recognition method that meets the requirements of lightweight input data, strong anti-interference capability, and effective modeling of long duration videos.

[0009] The patent specification with publication number CN114550305A discloses a human pose estimation method and system based on Transformer to solve the problems existing in the prior art. The method includes a feature extraction module, a feature interaction module, and a feature recovery module. The feature extraction module uses residual networks, compressed convolution, and pose attention mechanism units to realize local feature extraction and feature channel attention. The feature interaction module realizes global feature interaction on the input image. The feature recovery module recovers the feature of the interactive image to obtain the predicted feature image. The training of the feature extraction module, the feature interaction module, and the feature recovery module is completed by calculating the loss value of the predicted feature image and the heat map of the true value key node. The invention uses a lightweight network based on Transformer, which not only improves the effect, but also realizes the fast running of the network, improves the overall operation efficiency and effect.

[0010] Therefore, it is a very challenging problem to ensure that the human key points can be accurately identified while maintaining the real-time analysis of the human pose estimation system. Human bodies in crowded situations may encounter various complex poses, blurring, occlusion, and small targets, etc. This requires the model to have more fine feature extraction capability, especially for human recognition, pose query, and pose decoder to identify key points through a better initialization method to ensure that global information and local features in the image are fully utilized while improving the convergence speed of model training. SUMMARY

[0011] The application provides a multi-person key point recognition method based on deep learning, which can maintain more fine constraints among human bodies, postures and key points while fully utilizing global and local features in images, so as to realize end-to-end human posture estimation.

[0012] A multi-person key point recognition method based on deep learning comprises the following steps.

[0013] (1) Constructing a multi-person key point recognition model based on deep learning comprises the following steps.

[0014] S1, for an input image, features at three different scale positions are extracted by using a feature extraction network, then the features are respectively flattened and spliced after channel number alignment by a 1x1 convolution layer, to obtain image feature labels L; position encoding features corresponding to the image feature labels L are generated according to a position encoding function; the image feature labels L and the corresponding position encoding features are superimposed to obtain joint features as the input of a Transformer encoder;

[0015] S2, the Transformer encoder is a 6-layer deformable DETR encoder sequentially cascaded, each layer of the deformable DETR encoder comprises a multi-scale deformable attention module and a feedforward neural network, and the joint features are refined and merged by the Transformer encoder to obtain corresponding encoding label features and corresponding entity scores wherein L is the length of the joint features , D is an encoding dimension, and can be 256;

[0016] S3, a 2-layer cascaded human target detection decoder is used; the encoding label features F are sorted from high to low according to the entity scores S, and the first T label features are selected as the input of the human target detection decoder wherein T is the number of selected label features, representing the number of target human body boxes in the rough screening candidate boxes; the human target detection decoder outputs the first M image label features arranged from high to low according to the entity scores and corresponding position encoding features thereof M is less than T;

[0017] S4, a 2-layer cascaded human-posture decoder is used; a set of learnable key point position matrices is initialized wherein K is the number of key points, Y m is expanded to M dimensions and added to to obtain image label features and as input of the human-pose decoder; the human-pose decoder outputs image label features, position encoding features of M human targets and key points and corresponding entity scores, wherein represents position coordinates of K+2 human targets and key points of the i-th person;

[0018] S5, a two-level cascaded human-key point decoder is adopted; image label features, position encoding features of M human targets and key points are arranged in descending order of entity scores and the first N are selected as input of the human-key point decoder, and a final position sequence of human targets and key points is outputted;

[0019] (2) a loss function is constructed, including human classification loss L cls , key point regression loss L oks , human ground truth box regression loss L hum , human ground truth box L1 loss L h_reg and key point L1 loss L k_reg ;

[0020] (3) based on the loss function, parameters of the deep learning-based multi-person key point recognition model are optimized;

[0021] (4) the optimized multi-person key point recognition model is used to recognize multi-person key points of a to-be-detected image.

[0022] Step S1 can specifically include the following steps:

[0023] S11, multi-scale features C3, C4, C5 of the input image are extracted by a feature extraction network Resnet-50, and corresponding strides are 8, 16 and 32 respectively; H and E correspond to height and width of the input image I respectively;

[0024] S12, a spatial fully connected layer is adopted to project the multi-scale features onto a feature map with 256 channels through 1×1 convolution, and then the feature map is flattened into feature labels C′3, C′4, C′5:

[0025] C′ i =H / 2 i ×W / 2 i ×256

[0026] wherein, i is 3, 4 or 5;

[0027] S13, the feature labels obtained in step S12 are spliced to obtain image feature labels L=C′3+C′4+C′5;

[0028] S14, each element in the position encoding features corresponding to the image feature labels L is calculated according to the following formula

[0029]

[0030] wherein L(t) represents the value at the t-th position in the image feature marker L, i is the index of the t-th position in the image feature marker L, k is π, d represents the encoding dimension, which is 256;

[0031] S15, the joint feature is calculated according to the following formula

[0032]

[0033] wherein, represents the element-level addition.

[0034] In step S3, the position encoding feature H output by the d-th human body target detection decoder can be calculated according to the following formula d is:

[0035] H d = σ(σ -1 (H d-1 + △H d ))

[0036] wherein d is 1 or 2, △H d is the offset of the d-th layer prediction, σ and σ -1 represent the Sigmoid function and the inverse Sigmoid function respectively, and H0 is a randomly initialized matrix.

[0037] In step S5, the position encoding feature HJ of the human body target and the key point output by the d-th human body-key point decoder can be calculated according to the following formula d is:

[0038] HJ d = σ(σ -1 (HJ d-1 + △HJ d ))

[0039] wherein d is 1 or 2, HJ0 is a randomly initialized matrix, and △HJ d is the offset of the d-th layer prediction, which is obtained by a joint regression head and a multi-layer perception manner;

[0040] The sequence decoded layer by layer in step S5 is output through a feedforward neural network to obtain the final position sequence of the human body target and the key point.

[0041] The loss function constructed in the multi-person key point recognition method based on deep learning can be:

[0042] L all= λ1L cls + λ2L oks + λ3L hum + λ3L h_reg + λ5L k_reg

[0043] wherein λ1, λ2, λ3, λ4 and λ5 are weights corresponding to loss terms respectively.

[0044] Human classification loss L cls can be calculated as follows:

[0045] L cls = -a (1 - p n ) γ log*(p n ), where p n = p if y = 1, p n = 1 - p if y ≠ 1

[0046] Key point regression loss L oks can be calculated as follows:

[0047]

[0048] Human ground truth box regression loss L hum can be calculated as follows:

[0049] L hum = 1 - GIOU

[0050] Human ground truth box L1 loss L h_reg can be calculated as follows:

[0051] L h_reg = ‖H - H * ‖

[0052] Key point L1 loss L k_reg can be calculated as follows:

[0053] L k_reg = ‖P - P * ‖

[0054] wherein: p n is the probability of the nth predicted human target in the first T human targets output by the human target detection decoder in step S3; a, γ represent parameters; p is the probability value of predicting the presence of a human output by the deep learning-based multi-human key point recognition model; y represents the class of the true value, y = 1 indicates that the class is a human target, otherwise it is not a human target; K is the number of predicted human key points; v i represents whether the key points of the human can be observed, v i = 0 indicates that the key points cannot be observed, vi 1 means that the key point is labeled but occluded, v i 2 means that the key point is labeled and visible; s is the square root of the area occupied by the human target; k i is the decay coefficient of each key point; δ(v i >0) represents the normalized factor of the i-th key point; GIOU is the generalized rectangle intersection over union, whose value is a floating point number between 0 and 1; ‖H-H * ‖ represents the Euclidean distance between the detected human target frame coordinates H and the human target frame real coordinates H * ; ‖P-P * ‖ represents the Euclidean distance between the detected human key point coordinates P and the human key point real coordinates P * ; represents the Euclidean distance between the detected human key point i coordinates P i and the human key point i real coordinates .

[0055] The application also provides a computer device comprising a memory and a processor, the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, and the computer program runs to make the processor execute the multi-person key point recognition method based on deep learning.

[0056] The application also provides a computer readable storage medium, the computer readable storage medium stores a program or instructions, and the program or instructions are executed by a computer device to make the computer device execute the multi-person key point recognition method based on deep learning.

[0057] Compared with the prior art, the application has the following beneficial effects:

[0058] 1. The application is a simple, fast, efficient and end-to-end multi-person target key point recognition method, which can accurately recognize the positions of target human bodies and key points in an image, and simultaneously and in parallel output the postures of all target human bodies in the image, and can be applied to various human behavior analysis and virtual reality technology.

[0059] 2. The application provides a structural target character selection method, which can directly analyze the posture of a target human body at a certain moment in real time and retain the spatial position information. DETAILED DESCRIPTION

[0060] Figure 1 is a flowchart of training human targets and human key points in the embodiment of the application.

[0061] Figure 2The specific implementation flowchart of the human target detection decoder (human decoder), human-pose decoder and human-key point decoder involved in the embodiment of the present application.

[0062] Figure 3 The flowchart of the multi-person key point recognition method based on deep learning in the embodiment of the present application.

[0063] Figure 4 The target image in which the human pose needs to be recognized.

[0064] Figure 5 The Figure 4 picture in which each human target and pose position is recognized. DETAILED DESCRIPTION

[0065] The present application will be further described below in conjunction with the drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0066] Referring to Figures 1-3 , a multi-person key point recognition method based on deep learning, comprising:

[0067] (1) constructing a multi-person key point recognition model based on deep learning, comprising steps of:

[0068] S1, for an input image, using a feature extraction network to extract features at three different scale positions, then performing channel number alignment through a 1x1 convolution layer, and then respectively flattening and splicing the features to obtain an image feature label L; generating a position encoding feature corresponding to the image feature label L according to a position encoding function; superimposing the image feature label L and the corresponding position encoding feature to obtain a joint feature as the input of the Transformer encoder. Specifically comprising steps of:

[0069] S11, extracting multi-scale features C3, C4, C5 of the input image through a feature extraction network Resnet-50, the corresponding strides of which are 8, 16 and 32 respectively; H and E correspond to the height and width of the input image I respectively;

[0070] S12, using a spatial fully connected layer to project the multi-scale features onto a feature map with 256 channels through a 1x1 convolution, and then flattening it into feature labels C'3, C'4, C'5:

[0071] C' i = H / 2 i x W / 2 i x 256

[0072] wherein i is 3, 4 or 5;

[0073] S13, the feature markers obtained in step S12 are concatenated to obtain image feature markers L=C′3+C′4+C′5;

[0074] S14, calculate the positional coding feature corresponding to the image feature label L using the following formula. The elements in

[0075]

[0076] Where L(t) represents the value at position t in the image feature label L, i is the index of position t in the image feature label L, k is π, and d represents the encoding dimension, which is 256;

[0077] S15, calculate the joint features according to the following formula.

[0078]

[0079] in, This indicates element-level addition.

[0080] S2, the Transformer encoder consists of six layers of deformable DETR encoders cascaded sequentially. Each deformable DETR encoder layer contains a multi-scale deformable attention module and a feedforward neural network, jointly featuring... After refinement and merging by the Transformer encoder, the corresponding encoded tag features are obtained. and corresponding entity scores Where L is the joint feature The length of the encoding dimension is D, which is 256. The deformable DETR algorithm can adopt existing techniques, such as the method published in "Deformable detr: Deformable transformers for end-to-end object detection" (International Conference on Learning Representations, 2021).

[0081] S3 employs a two-layer cascaded human target detection decoder; the encoded marker features F are sorted from high to low according to the entity score S, and the top T marker features (T is 1000 in this embodiment) are selected as the input of the human target detection decoder. Where T is the number of selected marker features, representing the number of target human bounding boxes in the coarsely filtered candidate boxes; Q PH1The classification head is input into the human target detection decoder multi-task prediction head, and predicts the confidence score (entity score) S' of each human target detection box through a linear projection layer; the human target detection decoder outputs the top M (M is 500 in this embodiment) image labeling features arranged in descending order of entity score S' and the corresponding position encoding features 500 human target detection boxes represent the position coordinates of the jth human target detection box of the ith person, j = 1 represents the upper left corner coordinates of the human target detection box, j = 2 represents the lower right corner coordinates of the human target detection box, and D represents the encoding dimension 256.

[0082] In step S3, the position encoding feature H output by the dth human target detection decoder is calculated according to the following formula d

[0083] H d = σ(σ -1 (H d-1 +△H d ))

[0084] where d is 1 or 2, △H d is the offset predicted by the dth layer, σ and σ -1 represent the Sigmoid function and the inverse Sigmoid function respectively, and H0 is a randomly initialized matrix that is updated jointly with the model parameters during the training process. This cross-attention structure can better query the visual features more relevant to the human target. The human target coordinates H1 are directly predicted by the first human target detection decoder layer and used as the input of the second human target detection decoder to gradually refine the human target.

[0085] In step S4, a two-layer cascaded human-pose decoder is used; a set of learnable key point position matrices is initialized where K is the number of key points, and D is the encoding dimension. After Y m is expanded to M dimensions (M is 500), it is added to respectively to obtain the image labeling features and as the input of the human-pose decoder; the human-pose decoder outputs the image labeling features, position encoding features of the M (M is 500) human targets and key points, and the corresponding entity scores, where represents the position coordinates of the K+2 human targets and key points of the ith person;

[0086] ​S5, adopt 2 layers of cascaded human-body-keypoint decoders; image marking features and position coding features of M (M is 500) human-body targets and key points are arranged in descending order according to entity scores, and the first N (N is 100 in the embodiment) are selected as inputs of the human-body-keypoint decoders, and finally, the position sequence of the human-body targets and key points is output.

[0087] In step S5, the position coding features HJ of the human-body targets and key points output by the dth human-body-keypoint decoder are calculated according to the following formula: d

[0088] HJ d = σ (σ -1 (HJ d-1 +△HJ d ))

[0089] wherein d is 1 or 2, HJ0 is a randomly initialized matrix, and △HJ d is an offset predicted by the dth layer, which is predicted by a joint regression head and a multi-layer perception method.

[0090] The sequence decoded layer by layer in step S5 is output as the final position sequence of the human-body targets and key points by a feedforward neural network.

[0091] In the method of the application, each layer decoder except the first layer receives the output of the previous layer decoder and the output of the corresponding encoder layer as the input of the current layer decoder.

[0092] (2) Construct a loss function, including human-body classification loss L cls , key point regression loss L oks , human-body ground truth box regression loss L hum , human-body ground truth box L1 loss L h_reg and key point L1 loss L k_reg .

[0093] The constructed loss function is:

[0094] L all = λ1L cls + λ2L oks + λ3L hum + λ3L h_reg + λ5L k_reg

[0095] wherein λ1, λ2, λ3, λ4 and λ5 are weights of the corresponding loss terms.

[0096] The human-body classification loss L cls can be calculated according to the following formula:

[0097] L cls ​= -a(1 - p n ) γ log*(p n ), where p n = p if y = 1, p n = 1 - p if y ≠ 1

[0098] The key point regression loss L oks can be calculated as follows:

[0099]

[0100] The human ground truth box regression loss L hum can be calculated as follows:

[0101] L hum = 1 - GIOU

[0102] The human ground truth box L1 loss L h_reg can be calculated as follows:

[0103] L h_reg = ‖H - H * ‖

[0104] The key point L1 loss L k_reg can be calculated as follows:

[0105] L k_reg = ‖P - P * ‖

[0106] where p n is the probability of the nth predicted human target in the first T human targets output by the human target detection decoder in step S3; a and y represent parameters, in the present embodiment, a is 0.25 and y is 2; p is the probability value of predicting the presence of a human output by the deep learning-based multi-human key point recognition model; y represents the class of the ground truth, y = 1 indicates that the class is a human target, otherwise (i.e., y ≠ 1) it is not a human target; K is the number of predicted human key points; v i represents whether the key points of the human can be observed, v i = 0 indicates that the key points cannot be observed, v i = 1 indicates that the key points are labeled but are occluded, and v i = 2 indicates that the key points are labeled and visible; s is the square root of the area occupied by the human target; k i is the attenuation coefficient of each key point; and d(v i> 0) represents the normalization factor of the i-th key point, which in this embodiment is obtained by calculating the standard deviation of all true values in the existing human key point dataset coco2017 dataset, reflecting the influence degree of the current key point on the whole, the greater the value, the worse the labeling effect of this key point in the whole dataset, the smaller the value, the better the labeling effect of this key point in the whole dataset; GIOU is the generalized rectangle intersection over union, whose value is a floating point number between 0 and 1; ‖H-H * ‖ represents the Euclidean distance between the detected human target box coordinates H and the human target box true coordinates H * * ‖ represents the Euclidean distance between the detected human key point coordinates P and the human key point true coordinates P * represents the Euclidean distance between the detected human key point i coordinates P i and the human key point i true coordinates

[0107] (3) Parameter optimization of the multi-person key point recognition model based on deep learning based on the loss function;

[0108] (4) Multi-person key point recognition of the to-be-detected image by using the optimized multi-person key point recognition model.

[0109] Figure 4 is a target image in which human posture needs to be recognized. After the image feature markers and position coding features are extracted according to the above method, the final human target and key point position are obtained by using the above optimized multi-person key point recognition model based on deep learning, as shown in Figure 5

[0110] The application further provides a computer device comprising a memory and a processor, wherein the memory is used for storing a computer program, and the processor is used for executing the computer program stored in the memory, and the computer program runs to make the processor execute the above multi-person key point recognition method based on deep learning.

[0111] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a program or instructions, and the program or instructions make the computer device execute the above multi-person key point recognition method based on deep learning when the program or instructions are executed by the computer device.

[0112] In addition, it should be understood that, after reading the above description of the application, those skilled in the art can make various modifications or modifications to the application, and these equivalent forms also fall within the scope defined by the claims attached to the present application.​​​​

Claims

1. A multi-person key point recognition method based on deep learning, characterized in that, Comprise: (1) Construct a multi-person key point recognition model based on deep learning, comprising steps: S1, for the input image, features at three different scale positions are extracted by a feature extraction network, then the features are respectively flattened and spliced after channel number alignment by a 1x1 convolutional layer, to obtain an image feature label L; a position encoding feature corresponding to the image feature label L is generated according to a position encoding function; the image feature label L and the corresponding position encoding feature are superimposed to obtain a joint feature as the input of the Transformer encoder; S2, the Transformer encoder is sequentially cascaded with 6 deformable DETR encoders, each of which contains a multi-scale deformable attention module and a feed-forward neural network, and the joint features After refinement and merging by the Transformer encoder, the corresponding encoding mark features are obtained and the corresponding entity scores where L is the length of the joint feature and D is the encoding dimension S3, using a two-layer cascaded human target detection decoder; according to the entity score S, the encoded label features F are sorted from high to low, and the first T label features are selected as the input of the human target detection decoder wherein T is the number of selected label features, and D represents the number of target human body boxes in the rough screening candidate box. The human object detection decoder outputs the top M image label features ranked by entity score from high to low and their corresponding location encoding features M is less than T; S4, adopt 2 layers of human-pose decoder in cascade; initialize a set of learnable keypoint position matrix where K is the number of key points, Y m After expanding to M dimensions, respectively Add to obtain image label features and As the input of human-pose decoder; The human-pose decoder outputs image label features, position encoding features of M human targets and key points and the corresponding entity scores, wherein represents the position coordinates of K+2 human targets and key points of the ith person S5, adopt a 2-layer cascaded human-key point decoder; The image label features, position coding features of M human body targets and key points are arranged in descending order according to entity scores, and the first N are selected as the input of the human-key point decoder, and the final position sequence of the human body target and the key point is output; (2) constructing a loss function, including a human body classification loss L cls , a key point regression loss L oks , a human body true value box regression loss L hum , a human body true value box L1 loss L h_reg , and a key point L1 loss L k_reg ; (3) Parameter optimization of the multi-person key point recognition model based on deep learning based on the loss function; (4) Use the optimized multi-person key point recognition model to recognize the multi-person key points of the image to be detected. 2.The multi-person key point recognition method based on deep learning according to claim 1, wherein, Step S1 specifically comprises steps: S11, extracting a multi-scale feature C3, C4, C5 of the input image I through a feature extraction network Resnet-50, wherein strides corresponding to the multi-scale feature C3, C4, C5 are 8, 16 and 32 respectively; H and W respectively correspond to height and width of the input image I; S11, extracting a multi-scale feature C3, C4, C5 of the input image I through a feature extraction network Resnet-50, wherein strides corresponding to the multi-scale feature C3, C4, C5 are 8, 16 and 32 S12, adopt a spatial fully connected layer to project multi-scale features onto a 256-channel feature map through 1x1 convolution, and then flatten it into feature labels C'3, C'4, C'5: C′ i = H / 2 i x W / 2 i x 256 Wherein, i is 3, 4 or 5; S13, the feature labels obtained in step S12 are spliced to obtain image feature labels L=C'3+C'4+C'5; S14, the position encoding feature corresponding to the image feature label L is calculated according to the following formula each element in Wherein, L(t) represents the value at position t in the image feature label L, i is the index of position t in the image feature label L, k is π, and d represents the coding dimension, which is 256; S15, the joint feature is calculated according to the following formula wherein represents an element-wise addition. 3.The deep learning-based multi-person key point recognition method of claim 1, wherein, In step S3, the position encoding feature H output by the dth human target detection decoder is calculated as follows d is given by: H d = σ(σ -1 (H d-1 +△H d )) where d is 1 or 2, AH d is the offset of the dth layer prediction, σ and σ -1 denote the Sigmoid function and the inverse Sigmoid function, respectively, and H0is a randomly initialized matrix. 4.The deep learning-based multi-person key point recognition method of claim 1, wherein, In step S5, the position encoding features HJ of the human target and the key points output by the dth human key point decoder are calculated according to the following formula d are: HJ d = σ(σ -1 (HJ d-1 + ΔHJ d )) where d is 1 or 2, HJ0is a randomly initialized matrix, AHJ d is the offset of the dth layer prediction, which is predicted by a joint regression head and using a multi-layer perception approach. The sequence decoded layer by layer in step S5 is output through a feedforward neural network to obtain the final position sequence of the human body target and the key point. 5.The deep learning-based multi-person key point recognition method of claim 1, wherein, The constructed loss function is: L all = λ1L cls + λ2L oks + λ3L hum + λ3L h_reg + λ5L k_reg Wherein, λ1, λ2, λ3, λ4 and λ5 are the weights of the corresponding loss terms. 6.The multi-person key point recognition method based on deep learning according to claim 1 or 5, characterized in that, human classification loss L cls is calculated as follows: L cls = -a(1 - p n ) γ log*(p n ), where p n = p if y = 1, p n = 1 - p if y ≠ 1 Keypoint regression loss L oks is calculated as follows: Human ground truth box regression loss L hum is calculated as follows: L hum =1-GIOU Human ground truth box L1 loss L h_reg is calculated as follows: L h_reg =‖H-H * ‖ Key point L1 loss L k_reg is calculated as follows: L k_reg =‖P-P * ‖ wherein: p n is the probability of the n-th human target in the first T human targets output by the human target detection decoder in step S3 being predicted as a human target; a, g represent parameters; p is the probability value of predicting the existence of a human target output by the deep learning-based multi-human key point recognition model; y represents the true value of the category, y = 1 indicates that the category is a human target, otherwise it is not a human target; K is the number of predicted human key points; v i represents whether the key points of the human can be observed, v i = 0 indicates that the key points cannot be observed, v i = 1 indicates that the key points are labeled but are occluded, v i = 2 indicates that the key points are labeled and visible; s is the square root of the area occupied by the human target; k i is the attenuation coefficient of each key point; d(v i > 0) represents the normalization factor of the i-th key point; GIOU is the generalized rectangle intersection over union, whose value is a floating point number between 0 and 1; ‖H-H * ‖ represents the Euclidean distance between the detected human target frame coordinates H and the human target frame true coordinates H * ; ‖P-P * ‖ represents the Euclidean distance between the detected human key point coordinates P and the human key point true coordinates P * ; represents the Euclidean distance between the detected human key point i coordinates P i and the human key point i true coordinates .

7. A computer device comprising a memory for storing a computer program and a processor for executing the computer program stored in the memory, characterized in that, The computer program runs to make the processor execute the multi-person key point recognition method based on deep learning in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores programs or instructions, and when the programs or instructions are executed by the computer device, the computer device executes the multi-person key point recognition method based on deep learning in any one of claims 1-6.

Images