Dark light multi-person posture estimation method based on cascade channel attention

Abstract

The invention discloses a dark light multi-person posture estimation method based on cascade channel attention. The method comprises the following steps: firstly, acquiring an image construction data set, and then constructing a network model: based on a high-resolution network, in a coding stage, adopting a human body posture estimation method introducing a cascade channel attention mechanism; in a decoding stage, a traditional heat map method is improved, each pixel is uniformly divided into a plurality of intervals by adopting a classification thought, and a final attitude estimation result is generated. And finally, training the network model, and carrying out attitude estimation in a dark light multi-person scene. According to the method, while the internal high-resolution features are kept, the model can learn the importance of each channel of the input features, so that the characterization capability is enhanced, and information loss is effectively reduced. According to the method, the accuracy is improved, accurate human body posture estimation can be realized in a dark light scene, the adaptability requirement in practical application is met, the generalization ability is higher, and the method can adapt to wider application scenes.

Description

Technical Field

[0001] This invention relates to the field of pose estimation in deep learning, specifically to a method for multi-person pose estimation in low light based on cascaded channel attention. Background Technology

[0002] With the rapid development of computer vision technology, pose estimation has become increasingly important in many fields. However, traditional pose estimation methods often rely on good lighting conditions, and their performance deteriorates significantly in low-light scenes. Low-light environments can lead to increased image noise and loss of detail, thus challenging the accuracy and robustness of pose recognition. In daily life, many important scenarios, such as nighttime activities, indoor gatherings, and movement in low-light environments, face the problem of low illumination. Especially in the fields of public safety and surveillance, the ability to accurately identify the actions and poses of multiple people under low-light conditions is crucial. Therefore, developing a method for effectively estimating the poses of multiple people in low-light scenes not only has academic value but also broad practical application potential.

[0003] In traditional pose estimation tasks, channel attention mechanisms are often employed, along with pooling operations to compress the resolution of input features. While this approach enhances feature representation to some extent, it significantly sacrifices spatial information. The integrity of spatial information is crucial in pose estimation, as accurate keypoint localization and relative relationships require nuanced spatial context. This decrease in accuracy is particularly pronounced in low-light environments. Furthermore, traditional methods typically use heatmaps as labels to supervise model output; however, these heatmaps are usually smaller than the original image, inevitably leading to quantization errors when the coordinates are magnified back to the original image. Therefore, traditional designs limit the effectiveness and reliability of pose estimation tasks in practical applications. Summary of the Invention

[0004] Purpose of the invention: In order to solve the problems of the prior art, this invention proposes a method for low-light multi-person pose estimation based on cascaded channel attention.

[0005] The technical solution adopted in this invention is a method for estimating the pose of multiple people in low light based on cascaded channel attention, comprising the following steps:

[0006] S1: Acquire multi-person pose image data and convert it into a unified standard format;

[0007] S2: Construct an improved network model and use it for feature extraction to generate representations for the corresponding key points;

[0008] S3: Next, train the improved network model;

[0009] S4: Use the trained network model to estimate poses in low-light multi-person scenes.

[0010] Furthermore, in step S2, improvements are made based on the HRNet network during the encoding stage; the basic high-resolution network HRNet reduces the resolution of the input image to one-quarter; then, in each stage, the network introduces new low-resolution feature branches, so that the resolution of the features is gradually halved; after four stages, the high-resolution features in the first branch are adjusted in terms of channel number through convolution operations so as to effectively fuse with the features of other branches.

[0011] Based on this, a series of improvements were made to HRNet. First, preliminary features were extracted through convolution, and then four branches were generated sequentially using a parallel structure.

[0012] In the encoding stage, traditional convolutional modules are replaced with depthwise separable convolution and cascaded channel attention modules. The specific process is as follows: First, shallow initial features are extracted in the transition stage. Then, in subsequent stages, a designed feature enhancement module is introduced to improve feature extraction performance. The feature enhancement module first employs layer normalization to eliminate differences between input samples, ensuring feature consistency. Next, traditional convolution is replaced by depthwise separable convolution to more effectively extract contextual information from local pixel regions. Furthermore, cascaded channel attention allows the model to learn the importance of features from different input channels, enhancing its ability to focus on key features. Finally, a residual structure is used to add the extracted features to the original input features X to ensure effective information transfer. The final output is F. out Specifically, this can be expressed as:

[0013] F out =CCA(DWConv(LN(X)))+X (1)

[0014] Where X represents the input feature, LN(·) represents the normalization layer, DWConv(·) is the depthwise separable convolution, and CCA(·) is the cascaded channel attention operation, F out The output features are obtained after cascaded channel attention operations. Finally, through normalization and convolutional layers, the number of feature channels is compressed back to the original number of channels, and a residual structure is introduced when outputting features. This not only preserves important feature information but also effectively improves the nonlinear expressive power of the model, which can be expressed as:

[0015] F′ out =Conv(Conv(LN(F out )))+Fout (2)

[0016] Among them, F′ out These are the final output features of the feature enhancement module; through these improvements, the final network model can process high-resolution features more efficiently in keypoint detection and feature extraction tasks, thereby significantly improving model performance.

[0017] For the CCA(·) operation, given the input features X∈R (H×W×X) Where H, W, and X are the number of elements along the height, width, and channel size X, respectively. First, it undergoes two convolutions W... q ∈R (1，C，1,1) and W v ∈R (X / 4,X,1,1 ), respectively forming the characteristic matrix Q∈R (H×W×1) and V∈R (H×W×C / 4 Next, the dimensions of Q and V are changed back to HW×1 and HW×X / 4, respectively, and a Softmax function is applied to the feature Q for non-linear activation. After this, Q is transposed and multiplied with V to compress spatial information and generate the weight parameter. Finally, the number of channels ∈ R(1) is recovered through convolution. ×1×X The input feature is then multiplied by the input feature using a weighted multiplication to generate the output feature x′, which can be expressed as:

[0018]

[0019] Where x′ is the output feature after processing by the channel attention module, σ′ indicates that the activation function used is the Sigmoid function, and σ indicates that the activation function used is the Softmax function. ⊙ represents matrix multiplication, and ⊙ represents digit-by-digit multiplication. To address the information loss caused by feature channel compression while maintaining high resolution, a channel attention module is introduced to enhance the model's expressive power. Next, global average pooling is performed on the input feature x′ to compress the spatial dimension of the feature map into a single value, thus obtaining the average feature representation for each channel. Subsequently, one-dimensional convolution is applied to capture the correlation and information flow between channels; its calculation formula can be expressed as:

[0020] x″=σ′(Conv(GAP(x′)))⊙ x′(4)

[0021] In this formula, GAP(·) represents the global average pooling operation on the input feature x′, x″ is the final output feature result of the cascaded channel attention module, and σ′ is the Sigmoid activation function.

[0022] Furthermore, in step S2, during the decoding stage, in order to achieve the classification task, each continuous coordinate value is uniformly discretized into integers, which serve as the class labels for model training: c x ∈[1, N x ] and c y ∈[1, N y ]; where N x =W·k and N y =H·k represents the intervals of the horizontal and vertical axes, respectively; here, k is a segmentation factor, set to k≥1 to reduce quantization error and achieve sub-pixel level positioning accuracy, and W and H are the width and height; when generating the final prediction, based on the n keypoint representations learned by the backbone network, the horizontal and vertical coordinates are independently classified, that is, given the representation of the m-th keypoint as input, the predicted coordinate result of the point is... and Expressed as:

[0023]

[0024] The above formula means: finding the maximum value o in a one-dimensional vector. x (·) and o y The positions (·) are determined by their indices i and j. These indices are then divided by the scaling factor k to map back to the scale of the original image, thus restoring the corresponding coordinates. and Here, argmax represents the input position where a function achieves its maximum value within a specific input range.

[0025] Furthermore, in step S3, a two-stage augmentation training strategy is adopted, first strong data augmentation followed by weak data augmentation. First, strong data augmentation is used to train for 180 epochs, and then weak data augmentation is used for an additional 30 epochs. During training, the weight decay rate is set to 0.05, the initial learning rate is 0.004, and the batch size is 32. This strategy aims to gradually improve the robustness and performance of the model.

[0026] Beneficial Effects: This invention employs a cascaded channel attention mechanism instead of traditional channel attention, effectively reducing complexity, enhancing feature representation capabilities, and significantly reducing information loss. Furthermore, addressing the quantization error introduced in traditional heatmap-supervised training methods, this invention uniformly divides each pixel into multiple intervals during discretization, effectively reducing quantization error and achieving sub-pixel-level localization accuracy. Finally, by optimizing the training strategy, the robustness and performance of the model are further improved. Experimental results show that the improved network achieves a 1.2 percentage point improvement in AP score for pose estimation compared to the basic high-resolution HRNet network. Attached Figure Description

[0027] Figure 1 This is a flowchart of the present invention.

[0028] Figure 2 This is a schematic diagram of the overall network structure.

[0029] Figure 3 This is a schematic diagram of the cascaded channel attention module.

[0030] Figure 4 This is a structural diagram of the coordinate classification module.

[0031] Figure 5 The figure shows the experimental results of an example of the present invention. Detailed Implementation

[0032] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0033] This invention proposes a method for low-light multi-person pose estimation based on cascaded channel attention, the process of which is as follows: Figure 1 As shown, it specifically includes the following:

[0034] S1: Acquire multi-person pose image data and convert it into a unified standard format;

[0035] S2: Construct an improved network model and use it for feature extraction to generate representations for the corresponding key points.

[0036] The encoding stage is based on the HRNet network and improved upon it. The basic high-resolution HRNet network reduces the resolution of the input image to one-quarter. Then, in each stage, the network introduces new low-resolution feature branches, so that the resolution of the features is gradually halved. After four stages, the high-resolution features in the first branch are adjusted for the number of channels through convolution operations so as to effectively fuse the features with those of other branches.

[0037] Based on this, a series of improvements were made to HRNet, and its overall structure is as follows: Figure 2 As shown: First, preliminary features are extracted through convolution, and then four branches are generated sequentially using a parallel structure.

[0038] In the encoding stage, traditional convolutional modules are replaced with depthwise separable convolution and cascaded channel attention modules. The specific process is as follows: First, shallow initial features are extracted in the transition phase. Then, in subsequent stages, a designed feature enhancement module is introduced to improve feature extraction performance. The feature enhancement module first employs layer normalization to eliminate differences between input samples, ensuring feature consistency. Next, traditional convolution is replaced by depthwise separable convolution to more effectively extract contextual information from local pixel regions. Furthermore, a cascaded channel attention module enables the model to learn the importance of features from different input channels, enhancing its ability to focus on key features. Finally, a residual structure is used to add the extracted features to the original input features X to ensure effective information transfer. The final output is F. out Specifically, this can be expressed as:

[0039] F out =CCA(DWConv(LN(X)))+X (1)

[0040] Where X represents the input feature, LN(·) represents the normalization layer, DWConv(·) is the depthwise separable convolution, and CCA(·) is the cascaded channel attention operation, F out This is the output feature after cascaded channel attention operations. Finally, through normalization layers and convolutional layers, the number of feature channels is compressed back to the original number of channels, and a residual structure is introduced when outputting features. This not only preserves important feature information but also effectively improves the nonlinear expressive power of the model, which can be expressed as:

[0041] F′ out =Conv(Conv(LN(F out )))+F out (2)

[0042] Among them, F′ out This is the final output feature of the feature enhancement module. Through these improvements, the final network model can process high-resolution features more efficiently in keypoint detection and feature extraction tasks, thereby significantly improving model performance.

[0043] like Figure 3 The diagram shown is a structural schematic of the cascaded channel attention module.

[0044] For the CCA(·) operation, given the input features X∈R (H×W× X), where H, W, and X are the number of elements along the height, width, and channel size of X. First, it undergoes two convolutions W... q ∈R (1，C，1，1) and Wv ∈R (X / 4,X,1,1) Each forms a characteristic matrix Q∈R (H×W×1) and V∈R (H×W×C / 4) Next, the dimensions of Q and V are changed back to HW×1 and HW×C / 4, respectively, and a Softmax function is applied to the feature Q for non-linear activation. After this, Q is transposed and multiplied with V to compress spatial information and generate the weight parameter. Finally, the number of channels ∈ R(1) is recovered through convolution. ×1×X The input feature is then multiplied by the input feature using a weighted multiplication to generate the output feature x′, which can be expressed as:

[0045]

[0046] Where x′ is the output feature after processing by the channel attention module, σ′ indicates that the activation function used is the Sigmoid function, and σ indicates that the activation function used is the Softmax function. ⊙ represents matrix multiplication, and ⊙ represents digit-by-digit multiplication. To address the information loss caused by feature channel compression while maintaining high resolution, a channel attention module is introduced to enhance the model's expressive power. Next, global average pooling is performed on the input feature x′, compressing the spatial dimension of the feature map into a single value, thus obtaining the average feature representation for each channel. Subsequently, one-dimensional convolution is applied to capture the correlation and information flow between channels. Its calculation formula can be expressed as:

[0047] x″=σ′(Conv(GAP(x′)))⊙ x′(4)

[0048] In this formula, GAP(·) represents the global average pooling operation on the input feature x′, x″ is the final output feature result of the cascaded channel attention module, and σ′ is the Sigmoid activation function.

[0049] like Figure 4 The diagram shown is a structural schematic of the coordinate classification module. During the decoding phase, to achieve the classification task, each continuous coordinate value is uniformly discretized into integers, which serve as the class labels for model training: c x ∈[1, N x ] and c y ∈[1, N y ]. Wherein, N x =W·k and N y=H·k represents the intervals of the horizontal and vertical axes, respectively. Here, k is a segmentation factor, set to k≥1 to reduce quantization error and achieve sub-pixel-level positioning accuracy. W and H are the width and height, respectively. When generating the final prediction, based on the n keypoint representations learned by the backbone network, horizontal and vertical coordinates are independently classified. That is, given the representation of the m-th keypoint as input, the predicted coordinate result is... and Expressed as:

[0050]

[0051] The above formula means: finding the maximum value o in a one-dimensional vector. x (·) and o y The positions (·) are determined by their indices i and j. These indices are then divided by the scaling factor k to map back to the scale of the original image, thus restoring the corresponding coordinates. and Here, argmax represents the input position where a function achieves its maximum value within a specific input range.

[0052] S3: Next, train the improved network model and optimize the training strategy, employing a two-stage augmentation strategy: strong data augmentation followed by weak data augmentation. First, train with strong data augmentation for 180 epochs, then switch to weak data augmentation for an additional 30 epochs. During training, the weight decay rate is set to 0.05, the initial learning rate to 0.004, and the batch size to 32. This strategy aims to gradually improve the model's robustness and performance. Experimental results show that using the basic high-resolution network requires 2.9 × 10⁻⁶ data. 7 With 73.4% AP achieved by 1 parameter and 7.1 gigabit floating-point operations per second, the improved network can achieve an AP of 74.6%, which can significantly improve accuracy.

[0053] S4: Use the trained network model to estimate poses in low-light multi-person scenes. Figure 5 The figures shown are experimental results for an example of this invention. It can be seen that as the light intensity gradually decreases, the method designed in this invention can effectively cope with and demonstrate good robustness. Overall, the method of this invention can achieve relatively accurate estimation results in low-light scenes.

[0054] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A dark light multi-person pose estimation method based on cascaded channel attention, characterized in that: The following steps are involved: S1: Obtain multi-person posture image data and convert it into a unified standard format; S2: Build an improved network model and use it to extract features to generate representations for corresponding key points; S3: Then train the improved network model; S4: Use the trained network model to estimate the pose of multiple people in dark light scenes.

2. A dark light multi-person posture estimation method based on cascaded channel attention as claimed in claim 1, characterized in that: In step S2, improvements are made based on the HRNet network in the encoding stage; the basic high-resolution network HRNet reduces the resolution of the input image to one-fourth; then, in each stage, the network introduces a new low-resolution feature branch, so that the resolution of the feature is gradually halved; after four stages, the high-resolution feature in the first branch is adjusted through a convolution operation to adjust the number of channels so as to be effectively fused with the features of other branches; On this basis, a series of improvements were made to HRNet. First, the preliminary features were extracted through convolution, and then four branches were generated in sequence using a parallel structure. In the encoding stage, the traditional convolution module is replaced by the depthwise separable convolution and cascaded channel attention modules. The specific process is as follows: first, the initial features of the shallow layer are extracted in the transition stage; then, in the subsequent stage, the designed feature enhancement module is introduced; The feature enhancement module first uses layer normalization technology to eliminate the differences between input samples; then, the traditional convolution is replaced by a depth-separable convolution; in addition, the cascaded channel attention module enables the model to learn the importance of different input channel features; finally, the residual structure is used to add the extracted features to the original input features X; finally, the output F out The specific expression is: F out =CCA(DWConv(LN(X)))+X (1) Where X represents the input features, LN(·) represents the normalization layer, DWConv(·) is the depthwise separable convolution, and CCA(·) is the cascaded channel attention operation. out is the output feature after the cascade channel attention operation; finally, through the normalization layer and the convolution layer, the number of feature channels is compressed back to the original number of channels, and the residual structure is introduced when outputting the features, which is expressed as: F′ out =Conv(Conv(LN(F out )))+F out (2) Among them, F′ out It is the final output feature of the feature enhancement module; For the CCA(·) operation, given the input feature X∈R (H×W×C) , where H, W, C are the number of elements along the height, width, and channel size of X; first, two convolutions W q ∈R (1，C，1，1) and W v ∈R (C / 4,C,1,1 ), respectively forming the feature matrix Q∈R (H×W×1) and V∈R (H×W×C / 4 ); Next, the sizes of Q and V are changed back to HW×1 and HW×C / 4, and the Softmax function is applied to the feature Q for nonlinear activation processing; after that, Q is transposed and matrix-multiplied with V to compress spatial information and generate weight parameters Weight; finally, the number of channels ∈ R is restored through convolution (1×1×C) , and weightedly multiply it with the input feature to finally generate the output feature x′, expressed as: Among them, x′ is the output feature after being processed by the channel attention module, σ′ indicates that the activation function used is the Sigmoid function, and σ indicates that the activation function used is the Softmax function. is matrix multiplication, ⊙ is a bit-by-bit multiplication operation; in order to deal with the problem of information loss caused by feature channel compression while maintaining high resolution, a channel attention module is introduced to enhance the expressiveness of the model; then a global average pooling operation is performed on the input feature x′ to compress the spatial dimension of the feature map into a single value, thereby obtaining the average feature representation of each channel; then, a one-dimensional convolution is applied to capture the association and information flow between channels; its calculation formula is expressed as: x″=σ′(Conv(GAP(x′)))⊙x′ (4) In this formula, GAP(·) represents the global average pooling operation on the input feature x′, x″ is the final output feature result of the cascade channel attention module, and σ′ is the Sigmoid activation function.

3. A dark light multi-person posture estimation method based on cascaded channel attention as claimed in claim 2, characterized in that: In step S2, in the decoding stage, in order to achieve the classification task, each continuous coordinate value is evenly discretized into an integer as a class label for model training: x ∈[1,N x ] and cy∈[1,N y ]; where N x =W·k and N y =H·k represents the interval of the horizontal axis and the vertical axis respectively; here k is the segmentation factor, which is set to k≥1 to reduce the quantization error and achieve sub-pixel positioning accuracy. W and H are width and height. When generating the final prediction, the horizontal and vertical coordinates are classified independently according to the n key point representations learned by the backbone network, that is, given the representation of the mth key point as input, the coordinate result of the predicted point and Expressed as: The above formula means: find the maximum value o in a one-dimensional vector x (·) and o y (·) positions i and j, unindexed, and then this index is divided by the scaling factor k and mapped back to the scale of the original image, thereby restoring the corresponding coordinate position and Here, argmax represents the input position where a function achieves its maximum value within a specific input range.

4. A dark light multi-person posture estimation method based on cascaded channel attention as claimed in claim 3, characterized in that: In step S3, a two-stage enhancement training strategy of first strong and then weak is adopted; first, strong data enhancement is used for training for 180 epochs, and then weak data enhancement is switched to for an additional 30 epochs; during the training process, the weight decay rate is set to 0.05, the initial learning rate is set to 0.004, and the batch size is set to 32.

Images