Two-dimensional human pose estimation method and system based on frequency domain guidance feature fusion
Patent Information
- Application Number
- CN202611221711.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-12
- Publication Date
- 2026-11-03
AI Technical Summary
现有基于视觉Transformer的二维人体姿态估计方法在复杂姿态、人体遮挡及局部结构模糊场景下,难以有效捕获人体边缘、关节轮廓和局部几何变化,且频域特征融合方式不足,导致关键点定位偏移和鲁棒性不足。
By extracting high-frequency features and visual embedding features for spatial-frequency fusion, and combining a hierarchical cross-token fusion mechanism, the frequency domain structural information is enhanced to participate in the layer-by-layer evolution of token features in multiple coding layers of the visual Transformer. Dual-branch decoding is used to improve the accuracy and robustness of key point localization.
It improves the localization accuracy and robustness of 2D human pose estimation, especially the structural perception capability in complex poses and occluded scenes, and enhances the stability and visibility prediction of key points.
Smart Images

Figure CN123033259A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and human pose estimation technology, specifically to a two-dimensional human pose estimation method and system based on frequency domain guided feature fusion. Background Technology
[0002] With the widespread application of human pose estimation technology in action recognition, human-computer interaction, intelligent rehabilitation, security monitoring, virtual reality, and robot perception, 2D human pose estimation has become an important research direction in computer vision. While traditional convolutional neural network-based pose estimation methods can extract local texture, edge, and contour features of the human body, their global structure modeling capabilities remain limited when faced with complex poses, human occlusion, limb overlap, and long-distance keypoint dependencies. To address these issues, the Visual Transformer has been introduced into 2D human pose estimation. Unlike traditional convolutional networks that primarily rely on local convolutional receptive fields for feature extraction, the Visual Transformer models the global relationships between image patches through a self-attention mechanism, thereby better capturing the spatial dependencies between human keypoints. However, existing 2D human pose estimation methods based on visual Transformers still have common shortcomings: First, they mainly rely on spatial domain image patch embedding for representation learning. When human joints are occluded, limb boundaries are unclear, or local structures are similar, they easily ignore fine-grained structural information such as human edges, joint contours, and local geometric changes, leading to keypoint localization offsets. Second, existing frequency domain feature utilization methods are mostly concentrated in image classification, image restoration, or dense prediction tasks, and are insufficiently applied in sparse keypoint localization scenarios in 2D human pose estimation. Third, existing spatial and frequency domain feature fusion methods usually only perform simple splicing or addition in a single stage, making it difficult to continuously provide structural guidance in multiple coding layers of the visual Transformer, resulting in frequency domain structural information not being able to fully participate in the layer-by-layer evolution process of token features. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide a two-dimensional human pose estimation method and system based on frequency domain guided feature fusion, which addresses the above-mentioned problems of the prior art. The present invention aims to enhance the human structure perception capability in complex pose, human body occlusion and local structural ambiguity scenarios, and improve the positioning accuracy and robustness of two-dimensional human pose estimation.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A two-dimensional human pose estimation method based on frequency domain guided feature fusion includes the following steps: S101, The input feature map is extracted from the input image containing the human target through the image encoder; S102, extract high-frequency features and visual embedding features from the input feature map respectively, and concatenate the high-frequency features and visual embedding features to obtain the spatial-frequency fusion representation of the input image; S103, perform hierarchical cross-token fusion on the spatial-frequency fusion representation to obtain a fusion feature map; S104, the fused feature map decoding is performed with dual-branch decoding to obtain a two-dimensional human pose estimation result. The two-dimensional human pose estimation result includes the predicted heatmap and predicted visibility probability of each key point of the human target obtained by the two-branch decoding.
[0005] Optionally, the image encoder in step S101 is a visual Transformer backbone network, which includes an image patch embedding layer and multiple visual Transformer coding layers. The image patch embedding layer is used to divide the input image into... N The non-overlapping image blocks are mapped to image block embeddings. The image block embeddings are then passed through multiple visual Transformer coding layers and rearranged or projected to obtain the input feature map output by the visual Transformer backbone network.
[0006] Optionally, the high-frequency features in step S102 refer to the high-frequency structure diagram. And high-frequency structure diagram The extraction includes: S201, Perform a two-dimensional fast Fourier transform on the input feature map channel by channel along the spatial dimension to obtain the spectral features. ; S202, Construct a high-pass mask according to a preset frequency cutoff ratio. The high-pass mask This is used to suppress low-frequency components in the central region of the spectrum while preserving high-frequency structural components corresponding to human body edges, limb contours, and joint neighborhoods; the high-pass mask is constructed according to a preset frequency cutoff ratio. The function expression is: ; ; in, Qualcomm mask Middle position The mask value at that location, Positions x and y coordinates This represents the frequency cutoff ratio. and These represent the height and width of the human image to be estimated; S203, based on the high-pass mask Spectral characteristics High-pass filtering is performed, and the spectral features obtained after high-pass filtering are mapped back to the spatial domain through a two-dimensional inverse fast Fourier transform to obtain a high-frequency structure diagram. .
[0007] Optionally, in step S203, based on the high-pass mask... Spectral characteristics The function expression for high-pass filtering is: ; ; ; in, For centralized spectral characteristics, This is a spectrum centering function used to shift the zero-frequency component of the spectrum to the center of the spectrum. Centralized spectral characteristics via Qualcomm mask The filtered spectrum centering result In position The high-pass spectrum value at that location, Centralized spectral characteristics Middle position Pixel value at that location, Qualcomm mask Middle position The mask value at that location, This is the inverse spectrum centering function, used to restore the spectrum centering result to the original spectrum arrangement position. The spectral characteristics are obtained after high-pass filtering.
[0008] Optionally, the visual embedding features in step S102 refer to spatial adaptation representations. And space adaptation means The extraction includes: extracting high-frequency structure maps representing high-frequency features. Flatten and perform linear projection to obtain a frequency domain structure representation. Project the input feature map onto the frequency domain structure representation. The same feature space is used to obtain a spatially adapted representation. .
[0009] Optionally, in step S103, the hierarchical cross-token fusion of the spatial-frequency fusion representation to obtain the fused feature map is implemented by a hierarchical cross-token fusion module. This module includes multiple local projection networks, a shared mapping network, and multiple ViT coding layers. The local projection network and the shared mapping network are multilayer perceptrons. The ViT coding layers have the same number of layers as the local projection networks and correspond one-to-one. Each ViT coding layer includes a multi-head self-attention module and a feedforward neural network. The function expression for performing hierarchical cross-token fusion of the spatial-frequency fusion representation to obtain the fused feature map is: ; ; ; ; in, The first output of the shared mapping network An enhanced spatial-frequency fusion representation, To share the mapping network, For the first A local projection network, For the spatial-frequency fusion representation of the input, The number of ViT coding layers; For the first The input tokens of each ViT coding layer represent... For the first Hierarchical spatial-frequency fusion features of each ViT coding layer For layer normalization, For multi-head self-attention modules, For the first The intermediate token representation of each ViT encoding layer after being updated by a multi-head self-attention module. For the first The output token of each ViT encoding layer is represented.
[0010] Optionally, step S102 is implemented through the spatial-frequency feature fusion module, and step S104 is implemented through the dual-branch decoding module. Before step S101, the method further includes end-to-end training of the two-dimensional human pose estimation network composed of the image encoder, spatial-frequency feature fusion module, hierarchical cross-token fusion module, and dual-branch decoding module, and updating the network parameters of the two-dimensional human pose estimation network using the total loss function. The function expression of the total loss function used during training is as follows: ; ; ; in, For the total loss function, and The heatmap regression loss is respectively and visibility prediction loss The weight, K The number of key points for the human body target. and Here are the height and width of the predicted and actual heatmaps for all key points, respectively. and The first The key point lies in the values in the predicted heatmap and the actual heatmap. For the first Predicted visibility probability of key points The binary visibility label is obtained by converting keypoint annotations. The set of key points for participating in the visibility loss calculation. for The size of the set.
[0011] A two-dimensional human pose estimation system based on frequency domain guided feature fusion includes a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion.
[0012] A computer-readable storage medium storing a computer program or instructions programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion via a processor.
[0013] A computer program product includes a computer program or instructions that are programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion via a processor.
[0014] Compared with existing technologies, this invention mainly achieves the following beneficial effects: By performing a two-dimensional fast Fourier transform on the input feature map and using a high-pass mask to extract high-frequency structural components related to human keypoints, this invention can enhance fine-grained geometric information such as human edges, limb contours, and joint neighborhoods; by fusing frequency domain structural representation and spatial adaptation representation into the same latent space, it can simultaneously preserve spatial semantic information and local structural information, improving the consistency of human structure and the accuracy of keypoint localization; by continuously injecting spatial-frequency structural guidance into multiple ViT coding layers through a hierarchical cross-token fusion mechanism, it can enable frequency domain structural cues to participate in the layer-by-layer evolution of token representation, enhancing the model's ability to express complex poses and occluded scenes; by performing dual-branch decoding through heatmap regression and visibility prediction branches, it can improve the stability and robustness of pose estimation under conditions where keypoints are invisible or local structures are blurred. The fusion mechanism of this invention is lightweight and easy to integrate with existing visual Transformer backbone networks, which can improve the engineering deployment value of two-dimensional human pose estimation. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.
[0016] Figure 2 This is a schematic diagram illustrating the principle of the spatial-frequency feature fusion module generating a spatial-frequency fused representation in an embodiment of the present invention.
[0017] Figure 3 This is a schematic diagram illustrating the principle of the hierarchical cross-token fusion module generating fusion feature maps in an embodiment of the present invention. Detailed Implementation
[0018] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.
[0019] like Figure 1 and Figure 2 As shown, the two-dimensional human pose estimation method based on frequency domain guided feature fusion in this embodiment includes the following steps: S101, the input image containing the human target is processed by an image encoder to extract the input feature map. ; S102, for the input feature map High-frequency features and visual embedding features are extracted separately, and the high-frequency features and visual embedding features are concatenated to obtain the spatial-frequency fusion representation of the input image; S103, perform hierarchical cross-token fusion on the spatial-frequency fusion representation to obtain a fusion feature map; S104, the fused feature map decoding is performed with dual-branch decoding to obtain a two-dimensional human pose estimation result. The two-dimensional human pose estimation result includes the predicted heatmap and predicted visibility probability of each key point of the human target obtained by the two-branch decoding.
[0020] In this embodiment, the image encoder in step S101 is a visual Transformer backbone network. The visual Transformer backbone network includes an image block embedding layer and multiple visual Transformer coding layers. The image block embedding layer is used to divide the input image into... N Each non-overlapping image patch is mapped to an image patch embedding. These image patch embeddings are then sequentially passed through multiple visual Transformer coding layers, and subsequently rearranged or projected to obtain the input feature map output by the visual Transformer backbone network. During the training phase, input images containing human targets can be expanded through preprocessing, including one or more of the following: human region cropping, scale scaling, random flipping, rotation perturbation, color enhancement, and pixel normalization.
[0021] In this embodiment, the high-frequency feature in step S102 refers to the high-frequency structure diagram. And high-frequency structure diagram The extraction includes: S201, Perform a two-dimensional fast Fourier transform on the input feature map channel by channel along the spatial dimension to obtain the spectral features. : ; in, Indicates the first c Each channel on the frequency coordinate The complex Fourier coefficients (spectrum) at the corresponding spatial frequency. These are the frequencies in the horizontal and vertical directions, respectively. and These represent the input feature maps respectively. Height and width, c Indicates the channel index. Input feature map In planar coordinates The value at point, where e is the natural constant. The imaginary unit, These are the x and y coordinates in the planar coordinate system, respectively. S202, Construct a high-pass mask according to a preset frequency cutoff ratio. The high-pass mask This is used to suppress low-frequency components in the central region of the spectrum while preserving high-frequency structural components corresponding to human body edges, limb contours, and joint neighborhoods; the high-pass mask is constructed according to a preset frequency cutoff ratio. The function expression is: ; ; in, Qualcomm mask Middle position The mask value at that location, Positions x and y coordinates This represents the frequency cutoff ratio. and These represent the height and width of the human image to be estimated; , The cutoff bandwidth is determined based on the frequency cutoff ratio; when the frequency coordinates When located in the center region of the spectrum, Used to suppress low-frequency components; when the frequency coordinate When located outside the center region of the spectrum, , used to retain high-frequency structural components; S203, based on the high-pass mask Spectral characteristics High-pass filtering is performed, and the spectral features obtained after high-pass filtering are mapped back to the spatial domain through a two-dimensional inverse fast Fourier transform to obtain a high-frequency structure diagram. High-frequency structure diagram Structural components related to human body edges, limb contours, and joint neighborhoods.
[0022] High frequency structure diagram The spectral features can be obtained by mapping back to the spatial domain and projecting. In step S203 of this embodiment, the high-pass mask is used... Spectral characteristics The function expression for high-pass filtering is: ; ; ; in, For centralized spectral characteristics, This is a spectrum centering function used to shift the zero-frequency component of the spectrum to the center of the spectrum. Centralized spectral characteristics via Qualcomm mask The filtered spectrum centering result In position The high-pass spectrum value at that location, Centralized spectral characteristics Middle position Pixel value at that location, Qualcomm mask Middle position The mask value at that location, This is the inverse spectrum centering function, used to restore the spectrum centering result to the original spectrum arrangement position. The spectral characteristics are obtained after high-pass filtering.
[0023] In this embodiment, the visual embedding feature in step S102 refers to spatial adaptation representation. And space adaptation means The extraction includes: extracting high-frequency structure maps representing high-frequency features. Flatten and perform linear projection to obtain a frequency domain structure representation. Project the input feature map onto the frequency domain structure representation. The same feature space is used to obtain a spatially adapted representation. The preprocessed image is divided into... N Non-overlapping image patches, and for the N Visual embedding is performed on non-overlapping image patches to obtain image patch embeddings. ; through learnable adaptation matrices Image patch embedding Projecting yields a spatially adapted representation. , can be represented as: ; in , N The number of image patches, D Embed dimensions for image patches, For learnable adaptation matrix, c Let be the dimension representing the frequency domain structure. Finally, concatenating the high-frequency features and visual embedding features yields the spatial-frequency fusion representation of the input image, which can be expressed as: ; in, The initial spatial-frequency fusion representation contains both spatial semantic information and high-frequency structural information of the human image to be estimated.
[0024] In step S103 of this embodiment, the hierarchical cross-token fusion of the spatial-frequency fusion representation to obtain a fused feature map is achieved through a hierarchical cross-token fusion module. This module performs residual injection of the spatial-frequency fusion representation with the current token representation of the corresponding coding layer to obtain a pose feature sequence, and then rearranges the pose feature sequence into a fused feature map. For example... Figure 3As shown, the hierarchical cross-token fusion module in this embodiment includes multiple local projection networks, one shared mapping network, and multiple ViT coding layers. The local projection networks and the shared mapping network are multilayer perceptrons, and the N local projection networks are respectively represented as multilayer perceptrons MLP1 to MLP2. N The ViT coding layer has the same number of layers as the local projection network and corresponds one-to-one with each layer. Each ViT coding layer has a multi-head self-attention module and a feedforward neural network. The function expression for performing hierarchical cross-token fusion on the spatial-frequency fusion representation to obtain the fused feature map is as follows: ; ; ; ; in, The first output of the shared mapping network An enhanced spatial-frequency fusion representation, To share the mapping network, For the first A local projection network, For the spatial-frequency fusion representation of the input, The number of ViT coding layers; For the first The input tokens of each ViT coding layer represent... For the first Hierarchical spatial-frequency fusion features of each ViT coding layer For layer normalization, For multi-head self-attention modules, For the first The intermediate token representation of each ViT encoding layer after being updated by a multi-head self-attention module. For the first The output token of each ViT encoding layer is represented.
[0025] In this embodiment, step S102 is implemented through a spatial-frequency feature fusion module, and step S104 is implemented through a dual-branch decoding module. The dual-branch decoding module includes a heatmap regression branch and a visibility prediction branch. The heatmap regression branch is used to predict the human body key point heatmap based on the spatial feature map, and the visibility prediction branch is used to predict the visibility probability of human body key points based on the spatial feature map. Based on the human body key point heatmap and the visibility probability of human body key points, the two-dimensional human pose estimation result is output.
[0026] In this embodiment, the dual-branch decoding module uses two loss functions during training: heatmap regression loss and visibility prediction loss. The heatmap regression loss constrains the difference between the predicted keypoint heatmap and the actual keypoint heatmap, and the visibility prediction loss constrains the difference between the predicted keypoint visibility probability and the actual visibility label. The total training loss is obtained by weighted summation. Specifically, before step S101, this embodiment further includes end-to-end training of the two-dimensional human pose estimation network composed of the image encoder, spatial-frequency feature fusion module, hierarchical cross-token fusion module, and dual-branch decoding module, and updating the network parameters of the two-dimensional human pose estimation network using the total loss function. The expression of the total loss function used during training is: ; ; ; in, For the total loss function, and The heatmap regression loss is respectively and visibility prediction loss The weight, K The number of key points for the human body target. and Here are the height and width of the predicted and actual heatmaps for all key points, respectively. and The first The key point lies in the values in the predicted heatmap and the actual heatmap. For the first Predicted visibility probability of key points The binary visibility label is obtained by converting keypoint annotations. The set of key points for participating in the visibility loss calculation. for The size of the set.
[0027] To verify the effectiveness of the frequency-domain guided feature fusion and hierarchical cross-token fusion mechanism in the two-dimensional human pose estimation method based on frequency-domain guided feature fusion in this embodiment, a performance comparison experiment was conducted, and the accuracy of human key point localization was used as the performance index. The final results are shown in Table 1.
[0028] Table 1: Performance comparison results of the method in this embodiment on the COCO dataset
[0029] In Table 1, the input size of each model is 256×192 pixels, the number of parameters is in M (millions of parameters), the computational cost is in GFLOPs, and AP, AP50, AP75, AP(M), AP(L), and AR represent the average precision, average precision at different thresholds, average precision for mesoscale targets, average precision for large-scale targets, and average recall rate in the human keypoint detection task, respectively. The units for AP, AP50, AP75, AP(M), AP(L), and AR are all percentages. The comparison methods in Table 1 include SimpleBaseline-Res50, a classic convolutional baseline method based on ResNet-50; HRNet-W32, a representative pose estimation method that preserves high-resolution feature representation; TokenPose-B, a Transformer pose estimation method that models human keypoints as tokens; ViTPose-B, a visual Transformer pose estimation method with a backbone size similar to the baseline model in this embodiment; and HRPVT-L, a representative method using a pyramid visual Transformer backbone network. The “small model in this embodiment” is a lightweight 2D human pose estimation network using ViT-S / 16 as the image encoder and configured with a spatial-frequency feature fusion module, a hierarchical cross-token fusion module, and a dual-branch decoding module. The “benchmark model in this embodiment” is a benchmark 2D human pose estimation network using ViT-B / 16 as the image encoder and configured with a spatial-frequency feature fusion module, a hierarchical cross-token fusion module, and a dual-branch decoding module. As shown in Table 1, with an input size of 256×192, the benchmark model in this embodiment achieves superior performance on the COCO dataset, with AP, AP50, AP75, and AR reaching 76.8, 90.8, 83.9, and 82.1, respectively. Compared to the existing method ViTPose-B, the benchmark model in this embodiment, with a similar number of parameters, improves AP from 75.8 to 76.8, AP75 from 83.2 to 83.9, and AP(L) from 78.4 to 83.6. Therefore, the two-dimensional human pose estimation method based on frequency domain guided feature fusion in this embodiment can improve the model's ability to represent the local structure of the human body and the positional relationship of key points by introducing frequency domain structure representation, spatial-frequency feature fusion and hierarchical cross-token fusion, thereby improving the performance of two-dimensional human pose estimation.
[0030] Furthermore, this embodiment also provides a two-dimensional human pose estimation system based on frequency domain guided feature fusion, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion via a processor. This embodiment also provides a computer program product including a computer program or instructions programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion via a processor.
[0031] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0032] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A two-dimensional human pose estimation method based on frequency domain guided feature fusion, characterized in that, Includes the following steps: S101, The input feature map is extracted from the input image containing the human target through the image encoder; S102, extract high-frequency features and visual embedding features from the input feature map respectively, and concatenate the high-frequency features and visual embedding features to obtain the spatial-frequency fusion representation of the input image; S103, perform hierarchical cross-token fusion on the spatial-frequency fusion representation to obtain a fusion feature map; S104, the fused feature map decoding is performed with dual-branch decoding to obtain a two-dimensional human pose estimation result. The two-dimensional human pose estimation result includes the predicted heatmap and predicted visibility probability of each key point of the human target obtained by the two-branch decoding.
2. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 1, characterized in that, The image encoder in step S101 is a visual Transformer backbone network, which includes an image patch embedding layer and multiple visual Transformer coding layers. The image patch embedding layer is used to divide the input image into... N The non-overlapping image blocks are mapped to image block embeddings. The image block embeddings are then passed through multiple visual Transformer coding layers and rearranged or projected to obtain the input feature map output by the visual Transformer backbone network.
3. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 1, characterized in that, The high-frequency features in step S102 refer to the high-frequency structure diagram. And high-frequency structure diagram The extraction includes: S201, Perform a two-dimensional fast Fourier transform on the input feature map channel by channel along the spatial dimension to obtain the spectral features. ; S202, Construct a high-pass mask according to a preset frequency cutoff ratio. The high-pass mask This is used to suppress low-frequency components in the central region of the spectrum while preserving high-frequency structural components corresponding to human body edges, limb contours, and joint neighborhoods; the high-pass mask is constructed according to a preset frequency cutoff ratio. The function expression is: ; ; in, Qualcomm mask Middle position The mask value at that location, Positions x and y coordinates This represents the frequency cutoff ratio. and These represent the height and width of the human image to be estimated; S203, based on the high-pass mask Spectral characteristics High-pass filtering is performed, and the spectral features obtained after high-pass filtering are mapped back to the spatial domain through a two-dimensional inverse fast Fourier transform to obtain a high-frequency structure diagram. .
4. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 3, characterized in that, In step S203, based on the high-pass mask Spectral characteristics The function expression for high-pass filtering is: ; ; ; in, For centralized spectral characteristics, This is a spectrum centering function used to shift the zero-frequency component of the spectrum to the center of the spectrum. Centralized spectral characteristics via Qualcomm mask The filtered spectrum centering result In position The high-pass spectrum value at that location, Centralized spectral characteristics Middle position Pixel value at that location, Qualcomm mask Middle position The mask value at that location, This is the inverse spectrum centering function, used to restore the spectrum centering result to the original spectrum arrangement position. The spectral characteristics are obtained after high-pass filtering.
5. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 3, characterized in that, The visual embedding features in step S102 refer to spatial adaptation representations. And space adaptation means The extraction includes: extracting high-frequency structure maps representing high-frequency features. Flatten and perform linear projection to obtain a frequency domain structure representation. Project the input feature map onto the frequency domain structure representation. The same feature space is used to obtain a spatially adapted representation. .
6. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 1, characterized in that, In step S103, the hierarchical cross-token fusion of the spatial-frequency fusion representation to obtain the fused feature map is implemented through a hierarchical cross-token fusion module. This module includes multiple local projection networks, a shared mapping network, and multiple ViT coding layers. The local projection networks and the shared mapping network are multilayer perceptrons. The ViT coding layers have the same number of layers as the local projection networks and correspond one-to-one. Each ViT coding layer includes a multi-head self-attention module and a feedforward neural network. The function expression for performing hierarchical cross-token fusion of the spatial-frequency fusion representation to obtain the fused feature map is: ; ; ; ; in, The first output of the shared mapping network An enhanced spatial-frequency fusion representation, To share the mapping network, For the first A local projection network, For the spatial-frequency fusion representation of the input, The number of ViT coding layers; For the first The input tokens of each ViT coding layer represent... For the first Hierarchical spatial-frequency fusion features of each ViT coding layer For layer normalization, For multi-head self-attention modules, For the first The intermediate token representation of each ViT encoding layer after being updated by a multi-head self-attention module. For the first The output token of each ViT encoding layer is represented.
7. The two-dimensional human pose estimation method based on frequency domain guided feature fusion according to claim 3, characterized in that, Step S102 is implemented through the spatial-frequency feature fusion module, and step S104 is implemented through the dual-branch decoding module. Before step S101, the training also includes end-to-end training of the two-dimensional human pose estimation network composed of the image encoder, spatial-frequency feature fusion module, hierarchical cross-token fusion module, and dual-branch decoding module, and updating the network parameters of the two-dimensional human pose estimation network using the total loss function. The expression of the total loss function used during training is as follows: ; ; ; in, For the total loss function, and The heatmap regression loss is respectively and visibility prediction loss The weight, K The number of key points for the human body target. and Here are the height and width of the predicted and actual heatmaps for all key points, respectively. and The first The key point lies in the values in the predicted heatmap and the actual heatmap. For the first Predicted visibility probability of key points The binary visibility label is obtained by converting keypoint annotations. The set of key points for participating in the visibility loss calculation. for The size of the set.
8. A two-dimensional human pose estimation system based on frequency domain guided feature fusion, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the two-dimensional human pose estimation method based on frequency domain guided feature fusion as described in any one of claims 1 to 7.
9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the two-dimensional human pose estimation method based on frequency domain guided feature fusion as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the two-dimensional human pose estimation method based on frequency domain guided feature fusion as described in any one of claims 1 to 7.
