An illumination adaptive single-stage 2D multi-person pose estimation method

Abstract

The application discloses a kind of illumination adaptive single-stage 2D multi-person pose estimation methods.The method comprises the following steps: constructing decomposition network, obtaining reflectivity image and illumination image;Channel attention mechanism is constructed, and illumination adaptive weight factor is generated;Denoising recovery network is constructed, and the optimized reflectivity image is obtained;The backbone network is constructed, and the joint backbone feature of the optimized reflectivity image and input image is extracted;Feature enhancement and fusion module are constructed, illumination adaptive feature enhancement is carried out based on joint backbone feature and illumination adaptive weight factor, and enhanced global feature is output;Key point detection head is constructed, and human key point is positioned based on enhanced global feature;Single-stage 2D multi-person pose estimation model is constructed and trained, and single-stage 2D multi-person pose estimation is carried out on image.The application can significantly improve the multi-person pose estimation detection precision in low-light scene, and does not significantly increase the calculation overhead, improves inference speed.

Description

TECHNICAL FIELD

[0001] The present application relates to a 2D human pose estimation method, more particularly, to a single-stage 2D multi-person pose estimation method with adaptive illumination. BACKGROUND

[0002] The human pose estimation method for dense crowds is widely used in the fields of automatic driving and intelligent security. In addition to the challenges brought by occlusion, compactness, and singular poses, these fields often face uneven lighting environments such as night, weather, and strong light, which reduces the image quality captured by the camera, lacks image information, and has noise, resulting in a significant decline in the performance of existing multi-person pose estimation methods. Current human pose estimation methods mainly focus on solving problems such as dense occlusion and singular poses in complex scenes, with little research on coping with changes in lighting environments. We have counted the performance of the latest methods under the ExLPose low-light dataset, and the best single-stage multi-person pose estimation method CID has an average detection accuracy of less than 1% in the image set with an average pixel intensity of 1.4 (Wang D, Zhang S. Contextual Instance Decoupling for Robust Multi-Person Pose Estimation [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2022: 11060-11068.).

[0003] Existing 2D human pose estimation methods for handling lighting scenarios focus on night-time human pose estimation, which can be generally divided into three categories: 1) multi-modal fusion: infrared cameras are used to capture infrared images, and RGB images are integrated for night-time human pose estimation. 2) image enhancement: image enhancement techniques are used to improve the brightness and denoise of the input image, and then downstream pose estimation is performed. 3) domain transfer: combined with knowledge distillation and other means, the model learns the knowledge and features of the normal light source domain to obtain better performance in the low light target domain (Zhang Y, Zhang J, Guo X. Kindling the darkness: A practical low-light image enhancer [C] / / Proceedings of the 27th ACM international conference on multimedia. 2019: 1632-1640.). These human pose estimation methods that deal with changes in lighting environments face the following challenges. 1) Difficulty in obtaining data sets: due to the low popularity and high cost of infrared or temperature devices, it is difficult to collect multi-modal data. In addition, it is challenging to obtain aligned pairs of low-light-normal light images in actual scenarios, and pose estimation data sets also involve more complex key point labeling processes. 2) Model complexity makes it difficult to implement: for example, the current low-light pose estimation method based on domain transfer and some similar work mentioned in it, need to train the model in stages with paired images, and the models used for training and inference are different. This not only makes the model have high computational and parameter quantities, and cannot be trained and learned end-to-end, but also increases the difficulty of model implementation. In addition, another method that directly adds an image enhancement network to the input image for global brightening and denoising not only produces redundant computational costs for global enhancement, but also ignores the special needs of downstream tasks, which may introduce some additional background noise (Sun K, Xiao B, Liu D, et al. Deep high-resolution representation learning for human pose estimation [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2019: 5693-5703.). SUMMARY

[0004] The present application aims to improve the low-light human detection accuracy by modeling the light adaptive characteristics in low-light scenes. In order to achieve the above purpose, according to the light adaptive single-stage 2D multi-person pose estimation method provided by the present application, the original input image is effectively decomposed into an illumination map and a reflectance map, the features of the optimized illumination-independent reflectance image are extracted to compensate for the low-light human information, and the features of the illumination map are extracted and the channel attention mechanism is designed to adaptively weight the subsequent features, thereby achieving the purpose of dynamically enhancing human features according to the original image illumination, and improving the detection accuracy of low-light human pose estimation.

[0005] The purpose of the present application is achieved by at least one of the following technical solutions.

[0006] A light adaptive single-stage 2D multi-person pose estimation method, comprising the following steps:

[0007] Step one, build a decomposition network, input a single image into the decomposition network for decomposition to obtain a reflectance image and an illumination image;

[0008] Step two, build a channel attention mechanism, generate a light adaptive weight factor based on the illumination image;

[0009] Step three, build a denoising recovery network, optimize and improve the reflectance image to obtain an optimized reflectance image;

[0010] Step four, build a backbone network, extract the joint backbone features of the optimized reflectance image and the input image;

[0011] Step five, build a feature enhancement and fusion module, perform light adaptive feature enhancement based on the joint backbone features and the light adaptive weight factor, and output enhanced global features;

[0012] Step six, build a key point detection head, locate the human key points based on the enhanced global features;

[0013] Step seven, sequentially connect the decomposition network, the channel attention mechanism, the denoising recovery network, the backbone network, the feature enhancement and fusion module, and the key point detection head to obtain a single-stage 2D multi-person pose estimation model, train the single-stage 2D multi-person pose estimation model during the training process, and use the trained single-stage 2D multi-person pose estimation model to perform single-stage 2D multi-person pose estimation on the image during the inference process.

[0014] Further, in step one, the decomposition network is a two-branch network structure based on the Retinex image enhancement theory, including two branches of illumination map and reflectance map;

[0015] The reflectance map branch comprises 5 layers of U-shaped fully convolutional networks connected in sequence, a convolutional layer and a Sigmoid activation layer, the original input image is input into the reflectance map branch, and the Sigmoid activation layer outputs a reflectance image;

[0016] The illumination map branch comprises a spatial pooling operation, two convolutional layers, two ReLU activation layers and a convolutional layer for connecting the reflectance map branch connected in sequence, the original input image is input into the illumination map branch, and the convolutional layer for connecting the reflectance map branch outputs an illumination image;

[0017] The sizes of the finally obtained reflectance image and illumination image are consistent with the size of the original image.

[0018] Further, one layer of U-shaped fully convolutional network comprises a 3X3 convolutional layer, a ReLU activation layer, a first basic block and a second basic block connected in sequence, and the output of the second basic block and the output of the 3X3 convolutional layer are spliced by channels to obtain the output of one layer of U-shaped fully convolutional network;

[0019] The first basic block comprises a 3X3 convolutional layer, a ReLU activation layer, a 3X3 convolutional layer, a ReLU activation layer and a 2X2 spatial maximum pooling layer connected in sequence;

[0020] The second basic block comprises a 3X3 convolutional layer, a ReLU activation layer and an up-sampling layer connected in sequence.

[0021] Further, in step two, after the illumination image enters the channel attention mechanism, the illumination image is first subjected to a spatial pooling operation to down-sample the resolution of the illumination image to 1 / 4 of the original size, then three 3x3 convolutional layers are used to further extract features of the down-sampled illumination image to obtain a 960-channel feature map, and then global spatial pooling is performed on the 960-channel feature map to generate a one-dimensional 960-channel weight, i.e., an illumination adaptive weight factor;

[0022] The nonlinear activation layer comprises a first 960 1 fully connected layer, a ReLU activation layer, a second 960 1 fully connected layer and a Sigmoid activation layer connected in sequence.

[0023] Further, in step three, the denoising and restoration network comprises 5 layers of U-shaped fully convolutional networks and a Sigmoid activation layer connected in sequence.

[0024] Further, in step four, the backbone network comprises a high-resolution backbone network;

[0025] The high-resolution backbone network comprises three branches, each branch has different down-sampling resolution, wherein the first branch firstly performs convolution and down-sampling to obtain an output of 1 / 4 image resolution, and maintains the resolution during the convolution process, the second branch and the third branch are further down-sampled based on the resolution of the previous branch to obtain and maintain 1 / 8, 1 / 16 original image resolution outputs during the convolution process, the outputs of each resolution branch are fused during the convolution process to obtain multi-scale information, the second branch and the third branch are respectively outputted to be up-sampled by 2 times or 4 times to maintain the same resolution as the first branch, and finally the outputs of the three branches are added and fused to obtain the output of the high-resolution backbone network.

[0026] The high-resolution backbone network respectively performs feature extraction on the input image in step one and the optimized reflectance image in step three, and then splices the channels to obtain the joint backbone features of the output of the backbone network.

[0027] Further, in step five, the feature enhancement and fusion module comprises enhancement in the channel dimension and the spatial dimension composed of channel pooling and spatial pooling, a feature fusion process comprising a per-channel product, a 1x1 convolution layer, a BN regularization layer and a nonlinear activation layer, and the feature fusion process is as follows:

[0028]

[0029] wherein, is the fusion feature, is the input joint backbone feature, represents a 1x1 convolution layer, represents a BN regularization layer, represents a nonlinear activation layer; the fusion feature is multiplied by the illumination adaptive weight factor in step two to output the enhanced joint backbone feature.

[0030] Further, in step six, the key point detection head is an existing single-stage multi-person pose estimation detector, comprising a human body detection branch and a key point detection branch, the human body detection branch outputs the number of human bodies and the human body center point heat map, and the key point detection branch outputs the instance-level key point heat map.

[0031] The final key point position coordinates are calculated as follows: obtaining the pixel value coordinate position (x, y) of the highest response of the instance-level key point heat map and performing 4 times scaling to adapt to the size of the input image in step one, and additionally adding a human-set 0.5 bias parameter, the heat map is converted into the final key point position formula as follows:

[0032]

[0033] wherein, represents the two-dimensional coordinate position of the final key point in the input image, represents the heat map output corresponding to the jth key point of the ith person.

[0034] Further, in step seven, the single-stage end-to-end training is performed on the single-stage 2D multi-person pose estimation model, and the training loss only uses the multi-person pose estimation heat map loss, and the real heat map generation formula is as follows:

[0035]

[0036] wherein, represents the real heat map of the jth key point of the ith person, represents the human body size adaptive standard deviation. , represents the labeled key point two-dimensional space position coordinate of the jth key point of the ith person;

[0037] The heat map loss calculation formula is as follows:

[0038]

[0039] wherein, represents the predicted heat map of the jth key point of the ith person, and FL represents the Focal loss, L represents the heat map loss.

[0040] Further, in the training process, the key point detection head only outputs the heat map to calculate the heat map loss, and the trainable weights of the decomposition network, the channel attention mechanism, the denoising recovery network, the backbone network, the feature enhancement and fusion module and the key point detection head are updated through back propagation; in the inference process, the maximum response value two-dimensional coordinates in the output heat map are scaled and biased to solve the quantization error.

[0041] Compared with the prior art, the advantages of the present application are:

[0042] The present application proposes a single-stage 2D multi-person pose estimation network adaptive to illumination, which effectively extracts human features independent of illumination based on original image decomposition and subsequent feature enhancement means to adapt to multi-person pose estimation under different illumination scenarios. The present application can significantly improve the 2D multi-person pose estimation detection accuracy in low light scenes, without significantly increasing the computational overhead, and realizes fast inference speed. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a flowchart of the illumination adaptive single-stage 2D multi-person pose estimation method in the present embodiment;

[0044] Figure 2 is a structure diagram of the channel attention mechanism design in the present embodiment;

[0045] Figure 3 The configuration diagram of the feature enhancement and fusion module in this embodiment is shown in the following figure:

[0046] Figure 4 The structure diagram of the denoising recovery network in this embodiment is shown in the following figure:

[0047] Figure 5 The construction flowchart of the joint backbone feature in this embodiment is shown in the following figure. DETAILED DESCRIPTION

[0048] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be taken as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the various embodiments described herein can be made without departing from the scope and spirit of the disclosure. In addition, descriptions of well-known functions and constructions can be omitted for clarity and conciseness.

[0049] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but are merely used by the inventor to enable a clear and consistent understanding of the present disclosure. Accordingly, it should be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustration purpose only and not for limiting the present disclosure as defined by the appended claims and their equivalents.

[0050] The present disclosure will be described in detail with reference to the accompanying drawings and specific embodiments.

[0051] Embodiment:

[0052] A single-stage 2D multi-person pose estimation method adaptive to illumination, as shown in Figure 1 includes the following steps:

[0053] Step one, construct a decomposition network, input a single image to the decomposition network for decomposition to obtain a reflectance image and an illumination image;

[0054] The decomposition network is a two-branch network structure based on the Retinex image enhancement theory, including two branches of illumination map and reflectance map;

[0055] The reflectance map branch includes a 5-layer U-shaped fully convolutional network, a convolutional layer and a Sigmoid activation layer connected in sequence, the original input image is input to the reflectance map branch, and the Sigmoid activation layer outputs the reflectance image;

[0056] The illumination map branch comprises, in sequence, one spatial pooling operation, two convolutional layers, two ReLU activation layers, and one convolutional layer for connecting the reflectance map branch, the original input image is input into the illumination map branch, and the convolutional layer connected with the reflectance map branch outputs an illumination image;

[0057] One U-shaped full convolutional network comprises, in sequence, one 3X3 convolutional layer, one ReLU activation layer, a first basic block, and a second basic block, and the output of the second basic block and the output of the 3X3 convolutional layer are spliced by channels to obtain the output of the one U-shaped full convolutional network;

[0058] The first basic block comprises, in sequence, one 3X3 convolutional layer, one ReLU activation layer, one 3X3 convolutional layer, one ReLU activation layer, and one 2X2 spatial maximum pooling layer;

[0059] The second basic block comprises, in sequence, one 3X3 convolutional layer, one ReLU activation layer, and an up-sampling layer;

[0060] The sizes of the finally obtained reflectance image and the illumination image are consistent with the size of the input image, and the relationship between the input image and the reflectance image and the illumination image can be represented by the following formula:

[0061]

[0062] Wherein, I represents a single RGB input image, R represents a reflectance image (the reflectance image contains structural, material, color, and other information irrelevant to illumination), and L represents an illumination image.

[0063] In this embodiment, the decomposition network weight pre-trained based on the dataset LOL is used.

[0064] Step two, construct a channel attention mechanism to generate an illumination adaptive weight factor based on the illumination image;

[0065] As shown in Figure 2 , after the illumination image enters the channel attention mechanism, it is first subjected to a spatial pooling operation to down-sample the resolution of the illumination image to 1 / 4 of the original size, then three 3x3 convolutional layers are used to further extract features from the down-sampled illumination image to obtain a 960-channel feature map, and then global spatial pooling and a nonlinear activation layer are used to generate a one-dimensional 960-channel weight, i.e., an illumination adaptive weight factor;

[0066] The nonlinear activation layer comprises, in sequence, a first 960 1 fully connected layer, a ReLU activation layer, a second 960 1 fully connected layer, and a Sigmoid activation layer.

[0067] Step three, construct a denoising recovery network to optimize and improve the reflectance image, and obtain an optimized reflectance image;

[0068] As shown in Figure 4 , the denoising recovery network includes a 5-layer U-shaped full convolutional network connected in sequence and a Sigmoid activation layer.

[0069] Step four, construct a backbone network to extract joint backbone features of the optimized reflectance image and the input image;

[0070] As shown in Figure 5 , the high-resolution backbone network includes three branches, each branch has different down-sampling resolution, wherein the first branch first convolves and down-samples the image to obtain a 1 / 4 image resolution output, and maintains the resolution during the convolution process, the second branch and the third branch are further down-sampled based on the resolution of the previous branch, and 1 / 8, 1 / 16 original image resolution outputs are obtained and maintained during the convolution process, the outputs of each resolution branch are fused during the convolution process to obtain multi-scale information, the second branch and the third branch are respectively outputted and are respectively up-sampled by 2 times or 4 times to maintain the same resolution as the first branch, and the outputs of the last three branches are added and fused to obtain the output of the high-resolution backbone network.

[0071] The high-resolution backbone network respectively extracts features of the input image in step one and the optimized reflectance image in step three, and then splices them by channel to obtain joint backbone features output by the backbone network.

[0072] Step five, construct a feature enhancement and fusion module, perform illumination adaptive feature enhancement based on the joint backbone features and the illumination adaptive weight factor, and output enhanced global features;

[0073] As shown in Figure 3 , the feature enhancement and fusion module includes channel dimension and spatial dimension enhancement composed of channel pooling and spatial pooling, includes feature fusion process including channel-by-channel product, 1x1 convolution layer, BN regularization layer and nonlinear activation layer, and the feature fusion process is as follows:

[0074]

[0075] Among them, is the fusion feature, is the input joint backbone feature, represents a 1x1 convolution layer, represents a BN regularization layer, represents a nonlinear activation layer; the fusion feature is multiplied by the illumination adaptive weight factor in step two, and the enhanced joint backbone feature is output.

[0076] Step six, construct the key point detection head, based on the enhanced global features, locate the human key points;

[0077] The key point detection head is an existing single-stage multi-person pose estimation detector, including a human detection branch and a key point detection branch. The human detection branch outputs the number of human bodies and the human center point heat map, and the key point detection branch outputs the instance-level key point heat map.

[0078] The final key point position coordinates are calculated as follows: obtaining the pixel value coordinate position (x, y) of the highest response of the instance-level key point heat map and scaling it by 4 times to adapt to the size of the input image in step one, and additionally adding a human-set 0.5 bias parameter. The heat map is converted into the final key point position formula as follows:

[0079]

[0080] wherein, represents the two-dimensional coordinate position of the final key point in the input image, represents the heat map output corresponding to the jth key point of the ith human body.

[0081] Step seven, sequentially connect the decomposition network, the channel attention mechanism, the denoising recovery network, the backbone network, the feature enhancement and fusion module, and the key point detection head to obtain a single-stage 2D multi-person pose estimation model. In the training process, the single-stage 2D multi-person pose estimation model is trained. In the inference process, the trained single-stage 2D multi-person pose estimation model is used to perform single-stage 2D multi-person pose estimation on the image.

[0082] In one embodiment, both use the weights pre-trained on the paired normal-light-low-light dataset LOL.

[0083] The single-stage 2D multi-person pose estimation model is trained in a single-stage end-to-end manner. The training loss only uses the multi-person pose estimation heat map loss. The real heat map generation formula is as follows:

[0084]

[0085] wherein, represents the real heat map of the jth key point of the ith human body, represents the human size adaptive standard deviation. , represents the two-dimensional spatial position coordinates of the labeled key point of the jth key point of the ith human body.

[0086] The heat map loss calculation formula is as follows:

[0087]

[0088] wherein, the predicted heat map of the jth key point of the ith person, FL represents the Focal loss, L represents the heat map loss.

[0089] Further, in the training process, the key point detection head only outputs the heat map to calculate the heat map loss, and the trainable weights of the decomposition network, the channel attention mechanism, the denoising restoration network, the backbone network, the feature enhancement and fusion module and the key point detection head are updated by back propagation; in the inference process, the two-dimensional coordinates of the maximum response value in the output heat map are scaled and a bias is added to solve the quantization error.

[0090] In one embodiment, training and testing are performed based on the ExLPose dataset, the training period is 80 epochs, 4 NVIDIA GeForce GTX 2080ti graphics cards are used in the training process, the evaluation index is the average precision (AP%), the inference speed is frames per second (FPS), and the parameter amount Param (M). The results show that, compared with the recent excellent method, the present application not only achieves the best performance in accuracy, but also does not significantly increase the parameter amount, and achieves a fast inference speed.

[0091] Table 1: Comparison of experimental results in low-light complex scenes

[0092]

[0093] Note: LL and WL represent the low-light and normal-light training data sets in ExLPose. All the methods mentioned above, except for the baseline and the method in this section, use a two-stage architecture. " indicates that the evaluation does not include the overhead of the target detection model.

[0094] Compared with the prior art, the present application can significantly improve the 2D multi-person pose estimation detection accuracy in low-light scenes, and does not significantly increase the calculation overhead, and realizes a fast inference speed.

[0095] The above embodiments of the present application are only examples for clearly illustrating the present application, and are not a limitation on the embodiments of the present application. Based on the above description, other different forms of changes or variations can be made by those of ordinary skill in the art. Here, it is not necessary or possible to exhaust all embodiments. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A single-stage 2D multi-person pose estimation method adaptive to illumination, characterized in that: The method comprises the following steps: Step one, constructing a decomposition network, inputting a single image into the decomposition network for decomposition to obtain a reflectance image and an illumination image; Step two, constructing a channel attention mechanism, generating an illumination adaptive weight factor based on the illumination image; Step three, constructing a denoising recovery network, optimizing and improving the reflectance image to obtain an optimized reflectance image; Step four, constructing a backbone network, extracting joint backbone features of the optimized reflectance image and the input image; Step five, constructing a feature enhancement and fusion module, performing illumination adaptive feature enhancement based on the joint backbone features and the illumination adaptive weight factor to output enhanced global features; Step six, constructing a key point detection head, locating human key points based on the enhanced global features; Step seven, sequentially connecting the decomposition network, the channel attention mechanism, the denoising recovery network, the backbone network, the feature enhancement and fusion module, and the key point detection head to obtain a single-stage 2D multi-person pose estimation model, training the single-stage 2D multi-person pose estimation model during the training process, and using the trained single-stage 2D multi-person pose estimation model to perform single-stage 2D multi-person pose estimation on the image during the inference process.

2. The single-stage 2D multi-person pose estimation method with illumination adaptation according to claim 1, wherein, In step one, the decomposition network is a two-branch network structure based on the Retinex image enhancement theory, comprising an illumination map branch and a reflectance map branch; The reflectance map branch comprises five sequentially connected U-shaped full convolution networks, a convolution layer, and a Sigmoid activation layer, the original input image is input into the reflectance map branch, and the Sigmoid activation layer outputs a reflectance image; The illumination map branch comprises a spatial pooling operation, two convolution layers, two ReLU activation layers, and a convolution layer for connecting the reflectance map branch, the original input image is input into the illumination map branch, and the convolution layer for connecting the reflectance map branch outputs an illumination image; The final obtained reflectance image and illumination image have the same size as the original image.

3. The single-stage 2D multi-person pose estimation method of claim 2, wherein, A U-shaped full convolution network comprises a 3x3 convolution layer, a ReLU activation layer, a first basic block, and a second basic block connected in sequence, and the output of the second basic block and the output of the 3x3 convolution layer are spliced by channels to obtain the output of the U-shaped full convolution network; The first basic block comprises a 3x3 convolution layer, a ReLU activation layer, a 3x3 convolution layer, a ReLU activation layer, and a 2x2 spatial maximum pooling layer connected in sequence; The second basic block comprises a 3x3 convolution layer, a ReLU activation layer, and an up-sampling layer connected in sequence.

4. The single-stage 2D multi-person pose estimation method with illumination adaptation according to claim 1, wherein, In step two, after the illumination image enters the channel attention mechanism, it is first subjected to a spatial pooling operation to down-sample the resolution of the illumination image to 1 / 4 of the original size, then three 3x3 convolution layers are used to further extract features from the down-sampled illumination image to obtain a 960-channel feature map, and finally a global spatial pooling and a non-linear activation layer are used to generate a one-dimensional 960-channel weight, i.e., an illumination adaptive weight factor; The nonlinear activation layer includes a first 960 1 fully connected layer, ReLU activation layer, second 960 1 fully connected layer and Sigmoid activation layer.

5. The single-stage 2D multi-person pose estimation method of claim 3, wherein, In step three, the denoising recovery network comprises five U-shaped full convolution networks and a Sigmoid activation layer connected in sequence.

6. The single-stage 2D multi-person pose estimation method of claim 1, wherein, In step four, the backbone network comprises a high-resolution backbone network; The high-resolution backbone network comprises three branches, each branch has different down-sampling resolution, wherein the first branch firstly performs convolution and down-sampling to obtain an output of 1 / 4 image resolution, and maintains the resolution during the convolution process, the second branch and the third branch are further down-sampled based on the resolution of the previous branch to obtain and maintain 1 / 8, 1 / 16 original image resolution outputs during the convolution process, and the outputs of each resolution branch are fused during the convolution process to obtain multi-scale information, the outputs of the second branch and the third branch are respectively up-sampled by 2 or 4 times to maintain the same resolution as the first branch, and the outputs of the last three branches are added and fused to obtain the output of the high-resolution backbone network; The high-resolution backbone network respectively extracts features from the input image in step one and the optimized reflectance image in step three, and then splices them by channel to obtain the joint backbone features of the backbone network output.

7. The single-stage 2D multi-person pose estimation method of claim 1, wherein, In step five, the feature enhancement and fusion module comprises channel dimension and spatial dimension enhancement composed of channel pooling and spatial pooling, feature fusion process comprising channel-by-channel product, 1x1 convolution layer, BN regularization layer and nonlinear activation layer, and the feature fusion process is as follows: wherein, is a fused feature, is an input joint backbone feature, denotes a 1x1 convolutional layer, denotes a BN regularization layer, denotes a nonlinear activation layer; the fused feature is multiplied by the illumination adaptive weight factor in step two, and an enhanced joint backbone feature is output.

8. The single-stage 2D multi-person pose estimation method of claim 1, wherein, In step six, the key point detection head is an existing single-stage multi-person pose estimation detector, which comprises a human body detection branch and a key point detection branch, the human body detection branch outputs the number of human bodies and the human body center point heat map, and the key point detection branch outputs the instance-level key point heat map; The final key point position coordinate calculation is to obtain the highest response pixel value coordinate position (x, y) of the instance-level key point heat map and scale it by 4 times to adapt to the size of the input image in step one, and additionally add a human-set 0.5 bias parameter, and the heat map is converted into the final key point position formula as follows: wherein, denotes the two-dimensional coordinate position of the final key point in the input image, denotes the heat map output corresponding to the jth key point of the ith person.

9. The single-stage 2D multi-person pose estimation method of claim 1, wherein, In step seven, the single-stage 2D multi-person pose estimation model is trained in a single-stage end-to-end manner, and the training loss only uses the multi-person pose estimation heat map loss, and the real heat map generation formula is as follows: wherein, represents a real heat map of the jth key point of the ith person body, represents a body size adaptive standard deviation; , ) represents the labeled key point two-dimensional space position coordinates of the jth key point of the ith person body. The heat map loss calculation formula is as follows: wherein, represents the predicted heat map of the jth key point of the ith person body, FL represents the Focal loss, L represents the heat map loss.

10. The single-stage 2D multi-person pose estimation method of claim 9, wherein, During the training process, the key point detection head only outputs the heat map to calculate the heat map loss, and the trainable weights of the decomposition network, the channel attention mechanism, the denoising restoration network, the backbone network, the feature enhancement and fusion module, and the key point detection head are updated by back propagation; In the inference process, the maximum response value two-dimensional coordinate in the output heat map is scaled and biased to solve the quantization error.

Images