Lightweight attitude estimation method based on frequency domain attention mechanism

Abstract

The invention belongs to the technical field of human body posture estimation, and discloses a lightweight posture estimation method based on a frequency domain attention mechanism, and the method comprises the steps: compressing input feature information through grouping convolution, and then restoring the compressed feature information through three attention modules. Firstly, the frequency attention module uses the filtering module to extract different frequency components, important frequency components are selected through softmax normalization key values, and then the channel and space attention module better captures global information and accurately positions key features. The calculation complexity of the frequency domain attention mechanism module provided by the invention is linearly related to the image resolution, the calculation complexity is smaller, the problem that the calculation amount is rapidly expanded along with the increase of the resolution can be solved, and meanwhile, the module can reduce information loss caused by packet convolution feature compression, so that the accuracy of the algorithm is improved. Therefore, the human body posture estimation accuracy in the lightweight scene is further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of human body posture estimation, and specifically relates to a lightweight posture estimation method based on frequency domain attention mechanism. Background Art

[0002] Human pose estimation refers to the process of reconstructing human pose by detecting and locating key points on the human body. This task is one of the important challenges in the field of computer vision and plays a key role in many computer vision applications, such as action recognition, intelligent video surveillance, and human-computer interaction. At present, various human pose estimation methods have been proposed under the condition of high-resolution data, and have achieved good results. When the image resolution is reduced, the loss of image information will cause the performance of existing models to drop sharply, which cannot meet the growing production and application needs. The pose estimation algorithm extracts multi-scale features and designs a complex decoder head to establish the relationship between multi-scale features. These improvements come at the cost of huge parameters and high computational costs.

[0003] At present, the top-down method has fully met the requirements of industrial applications in terms of accuracy. However, because the task of human posture estimation itself is a pixel-level intensive prediction task, it is required to maintain high-resolution spatial dimensions when extracting features through neural networks. This leads to huge computing resources required for deep neural networks. The stringent requirements of these large networks on computing resources make it difficult to deploy them on increasingly common mobile devices. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a lightweight posture estimation method based on the frequency domain attention mechanism. The method proposes a feature data grouping compression, and then uses the matching degree of different frequencies to restore the feature information, constructing a lightweight resolution human posture estimation method to solve the technical problem of huge computing resources required by deep neural networks.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention discloses a lightweight posture estimation method based on a frequency domain attention mechanism. The lightweight posture estimation method is based on a lightweight feature extraction network N based on a frequency domain attention mechanism. x Implementation, the lightweight feature extraction network N based on frequency domain attention mechanism x Including convolutional layer, group position encoding, lightweight human posture estimation network module M x and a fully connected layer, the specific lightweight posture estimation method specifically includes the following steps:

[0007] Step 1: Select the MSCOCO human pose estimation dataset as the training dataset, and obtain the input feature map by capturing the image through the human detection frame in the labeled data;

[0008] Step 2: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x , the lightweight human posture estimation network module M x Including frequency domain attention module, channel attention module and spatial attention module,

[0009] The frequency domain attention module includes four dynamic low-pass filter modules, four dynamic high-pass filter modules, and a frequency matching kernel module;

[0010] Each of the dynamic low-pass filter modules includes an average pooling layer, a bilinear interpolation layer, and a ReLU layer, and the pooling sizes of the four dynamic low-pass filter modules are 1×1, 2×2, 3×3, and 6×6, respectively;

[0011] Each of the dynamic high-pass filtering modules includes a grouped deep convolution layer, and the four grouped deep convolution layers are respectively composed of convolution layers with convolution kernels of 1×1×1, 3×3×3, 5×5×5 and 7×7×7;

[0012] The frequency matching kernel module includes a linear layer and a frequency domain partial calculation layer;

[0013] Step 3: Based on the lightweight human posture estimation network module M constructed in step 2 x , build a lightweight feature extraction network N based on frequency domain attention mechanism x , generate visual features by convolutional coding the input feature map intercepted in step 1;

[0014] Step 4: Use the constructed training data set to train the lightweight human posture estimation network module M based on the frequency domain attention mechanism x ;

[0015] Step 5: Use the trained lightweight human pose estimation network based on the frequency domain attention mechanism to perform pose estimation on the human body image to obtain the corresponding human pose estimation result.

[0016] A further improvement of the present invention is that: in step 2, a lightweight human posture estimation network module M based on the frequency domain attention mechanism is constructed x , including frequency domain attention module extraction, channel attention module extraction and spatial attention module extraction, specifically including the following steps:

[0017] Step 2.1, frequency domain attention module extraction: The input feature map obtained in step 1 is passed through the convolution layer of the frequency matching kernel module to obtain the input feature vector x, and the input feature vector x is input into the linear layer of the frequency matching kernel module to obtain the query tensor Q, key tensor K and value tensor V. The attention matrix of the query tensor Q, key tensor K and value tensor V is calculated to obtain the frequency domain attention feature tensor FF. At the same time, the low-frequency feature LF is extracted through the dynamic low-pass filter module, and the high-frequency feature HF is extracted through the dynamic high-pass filter module. The low-frequency feature LF, the high-frequency feature HF and the frequency domain attention feature tensor FF are added together to obtain the frequency domain attention module M. f ;

[0018] Step 2.2: Compress the feature vector in step 2.1 along the channel of the channel attention module, compress the spatial dimension through global average pooling and global maximum pooling, generate global features describing each channel, send the generated global features to a shared multi-layer perceptron (MLP), learn the attention weights of each channel, and output the global features M through the module. c ;

[0019] Step 2.3: The global feature M output from step 2.2 c Perform global average pooling and global maximum pooling to generate two two-dimensional feature maps. These two two-dimensional feature maps are concatenated and processed through a convolution layer to generate spatial attention weights. The module outputs the spatial attention weight feature M. s ;

[0020] Step 2.4: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x :

[0021] M x =M f +M c ×M f +M s ×M f .

[0022] A further improvement of the present invention is that the step 2.1 of extracting the low-frequency feature LF through the dynamic low-pass filtering module specifically comprises the following steps:

[0023] Step 2.1.1, get the height and width of the input feature map;

[0024] Step 2.1.2, divide the input feature map into m parts according to the number of channels, m ≥ 4, input each segmented feature map into the corresponding filter, and use bilinear interpolation to upsample the output feature map to the original size;

[0025] Step 2.1.3, concatenate the m upsampled feature maps in the channel dimension, activate them through the ReLU layer of the dynamic low-pass filter module, and finally output the low-frequency feature LF after low-pass filtering: the expression is as follows:

[0026] LF=bilinear(AvgPooling k×k (V)

[0027] Among them, V is the value tensor, k is the cutoff frequency, and the cutoff frequency of each channel is different.

[0028] A further improvement of the present invention is that in step 2.1, the high frequency feature HF is extracted by a dynamic high pass filtering module, which specifically includes the following steps:

[0029] Step 2.2.1, divide the value tensor V into n groups, and use the grouped deep convolution layer in the dynamic high-pass filter module to simulate the tensor characteristics of the cutoff frequencies in different high-pass filters;

[0030] Step 2.2.2: Concatenate the tensor features obtained in step 2.2.1 to obtain the high-frequency feature HF:

[0031] HF=depthwise_conv k×k (V).

[0032] A further improvement of the present invention is that: in step 2.1: the frequency matching kernel module A i，j It is expressed as:

[0033]

[0034] Among them, k i represents the i-th frequency component of the key tensor K, v j represents the jth frequency component of the value tensor V;

[0035] By frequency matching kernel module A i，j Obtaining the frequency domain attention feature tensor FF specifically includes the following steps:

[0036] Step 2.3.1, divide the input feature vector x into multiple groups, and divide the input feature vector x into i Shoot to query tensor Q i , key tensor K i , value tensor V i ;

[0037] Step 2.3.2: Key tensor K i Perform softmax operation to get Will Sum value tensor V i Multiply them together and you get

[0038] Step 2.3.3: Use the einsum function to query the tensor Q i and Multiply them together to get the final attention matrix A i ;

[0039] Step 2.3.4: Concatenate the attention features obtained from each group together to finally obtain the frequency domain attention feature tensor FF.

[0040] A further improvement of the present invention is that step 4 specifically comprises the following steps:

[0041] Step 4.1: Build a human posture estimator, encode the visual features extracted in step 3, and obtain H heat maps of human joints, where H represents the number of human joint categories pre-defined in the dataset;

[0042] Step 4.2, the H heat maps of human joints obtained in step 4.1 and the Gaussian heat maps constructed according to the annotation information are used to calculate the loss through the mean square error loss function to optimize the network;

[0043] The mean square error loss function is LMSE specifically:

[0044]

[0045] Among them, H represents the human joint point category pre-defined in the dataset, Represents the true value heat map corresponding to the h-th joint point, Represents the predicted heat map corresponding to the h-th joint point.

[0046] The beneficial effects of the present invention are as follows: the present invention explores and constructs the correlation between characteristic human body images of different frequency components, proposes a method for characteristic data grouping compression, and then uses the matching degree of different frequencies to restore characteristic information to construct a method for lightweight resolution human body posture estimation; the method

[0047] The present invention can significantly reduce the amount of network parameters while alleviating the information loss caused by compression features. Compared with traditional lightweight networks such as MobileNet, it has higher network accuracy.

[0048] The present invention introduces a lightweight human posture estimation network module with frequency domain attention, which compresses features in multiple segments through high-pass and low-pass filtering modules with different cutoff frequencies. Compared with the traditional grouping compression method, this module can retain more useful semantics of the corresponding frequencies; the grouped feature attention mechanism can significantly reduce the computational overhead introduced by the TransFormer network due to multi-head attention.

[0049] The present invention solves the problem of information loss caused by feature compression. Compared with methods such as HRFormer, this module only needs a small number of parameters to reconstruct data feature details, while achieving better model performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the process of the present invention.

[0051] Figure 2 It is a flow chart of the frequency domain attention module of the present invention.

[0052] Figure 3 It is a flow chart of the lightweight human posture estimation network module based on the frequency domain attention mechanism of the present invention.

[0053] Figure 4 It is a structural diagram of the lightweight feature extraction network based on the frequency domain attention mechanism of the present invention. DETAILED DESCRIPTION

[0054] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0055] like Figure 1 As shown, the present invention is a lightweight human posture estimation method based on the frequency domain attention mechanism, which improves the basic human posture estimation process, and retains useful information to the greatest extent while lightweighting the original backbone network, so as to perform human posture estimation in scenarios such as mobile terminals. The lightweight posture estimation method is based on a lightweight feature extraction network N based on the frequency domain attention mechanism. x The lightweight feature extraction network Nx based on the frequency domain attention mechanism includes a convolutional layer, a group position encoding, and a lightweight human posture estimation network module M. x and a fully connected layer, the specific lightweight posture estimation method specifically includes the following steps:

[0056] Step 1: Select the MSCOCO human pose estimation dataset as the training dataset, and obtain the input feature map by capturing the image through the human detection frame in the labeled data;

[0057] Step 2: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x , the lightweight human posture estimation network module M x It includes frequency domain attention module, channel attention module and spatial attention module.

[0058] The frequency domain attention module includes four dynamic low-pass filtering modules, four dynamic high-pass filtering modules, and a frequency matching kernel module; each of the dynamic low-pass filtering modules includes an average pooling layer, a bilinear interpolation layer, and a ReLU layer, and the pooling sizes of the four dynamic low-pass filtering modules are 1×1, 2×2, 3×3, and 6×6, respectively; each of the dynamic high-pass filtering modules includes a grouped deep convolutional layer, and the four grouped deep convolutional layers are respectively composed of convolutional layers with convolution kernels of 1×1×1, 3×3×3, 5×5×5, and 7×7×7; the frequency matching kernel module includes a linear layer and a frequency domain partial calculation layer.

[0059] Among them, the dynamic low-pass filter module performs average pooling and bilinear interpolation of different sizes on the m group features to extract low-frequency semantic information. The low-pass filter module obtains m low-frequency components of the feature. The dynamic high-pass filter module divides the key tensor K and the value tensor V into n groups. For each group, a convolution layer with different kernels is used to simulate the cutoff frequency in different high-pass filters. The matrix dot multiplication of the obtained high-frequency features is performed to reduce the high-frequency noise of the human body. The high-pass filter module obtains n high-frequency components of the feature. The frequency domain attention module uses a matching kernel of a fixed size to represent the correspondence between different frequency components. The feature extracted by the convolution layer is grouped, the key tensor K and the value tensor V of the frequency component are calculated, and the softmax operation is used to normalize the key between the frequency components. The important frequency components are selected by querying the frequency matching kernel to reduce the loss of feature information caused by group calculation.

[0060] In step 2, a lightweight human posture estimation network module M based on the frequency domain attention mechanism is constructed. x , including frequency domain attention module extraction, channel attention module extraction and spatial attention module extraction, specifically including the following steps:

[0061] Step 2.1, frequency domain attention module extraction: The input feature map obtained in step 1 is passed through the convolution layer of the frequency matching kernel module to obtain the input feature vector x, and the input feature vector x is input into the linear layer of the frequency matching kernel module to obtain the query tensor Q, key tensor K and value tensor V. The attention matrix of the query tensor Q, key tensor K and value tensor V is calculated to obtain the frequency domain attention feature tensor FF. At the same time, the low-frequency feature LF is extracted through the dynamic low-pass filter module, and the high-frequency feature HF is extracted through the dynamic high-pass filter module. The low-frequency feature LF, the high-frequency feature HF and the frequency domain attention feature tensor FF are added together to obtain the frequency domain attention module M. f .

[0062] Among them, Figure 2 As shown, extracting low-frequency features LF through a dynamic low-pass filtering module specifically includes the following steps:

[0063] Step 2.1.1, get the height and width of the input feature map;

[0064] Step 2.1.2, split the input feature map into 4 parts according to the number of channels, input each split feature map into the corresponding filter, and use bilinear interpolation to upsample the output feature map to the original size;

[0065] Step 2.1.3, concatenate the m upsampled feature maps in the channel dimension, activate them through the ReLU layer of the dynamic low-pass filter module, and finally output the low-frequency feature LF after low-pass filtering: the expression is as follows:

[0066] LF=bilinear(AvgPooling k×k (V)

[0067] Among them, V is the value tensor, k is the cutoff frequency, and the cutoff frequency of each channel is different.

[0068] The high-frequency feature HF is extracted by a dynamic high-pass filtering module, which specifically includes the following steps:

[0069] Step 2.2.1, divide the value tensor V into n groups, and use the grouped deep convolution layer in the dynamic high-pass filter module to simulate the tensor characteristics of the cutoff frequencies in different high-pass filters;

[0070] Step 2.2.2: Concatenate the tensor features obtained in step 2.2.1 to obtain the high-frequency feature HF:

[0071] HF=depthwise_conv k×k (V).

[0072] Among them, the frequency matches the kernel module A i，j It is expressed as:

[0073]

[0074] Among them, k i represents the i-th frequency component of the key tensor K, v j represents the jth frequency component of the value tensor V;

[0075] By frequency matching kernel module A i，j Obtaining the frequency domain attention feature tensor FF specifically includes the following steps:

[0076] Step 2.3.1, divide the input feature vector x into multiple groups, and divide the input feature vector x into i Shoot to query tensor Q i , key tensor K i , value tensor V i ;

[0077] Step 2.3.2: Key tensor K i Perform softmax operation to get Will Sum value tensor V i Multiply them together and you get

[0078] Step 2.3.3: Use the einsum function to query the tensor Q i and Multiply them together to get the final attention matrix A i ;

[0079] Step 2.3.4: Concatenate the attention features obtained from each group together to finally obtain the frequency domain attention feature tensor FF.

[0080] Step 2.2: Compress the feature vector in step 2.1 along the channel of the channel attention module, compress the spatial dimension through global average pooling and global maximum pooling, generate global features describing each channel, send the generated global features to a shared multi-layer perceptron (MLP), learn the attention weights of each channel, and output the global features M through the module. c , enhance the weights of important channel components;

[0081] Step 2.3: The global feature M output from step 2.2 c Perform global average pooling and global maximum pooling to generate two two-dimensional feature maps. These two two-dimensional feature maps are concatenated and processed through a convolution layer to generate spatial attention weights. The module outputs the spatial attention weight feature M. s ;

[0082] Step 2.4: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x :

[0083] M x =M f +M c ×M f +M s ×M f .

[0084] Step 3: Based on the lightweight human posture estimation network module M constructed in step 2 x , build a lightweight feature extraction network N based on frequency domain attention mechanism x , the input feature map intercepted in step 1 is used to generate visual features through convolutional coding, such as Figure 3 shown.

[0085] Step 4: Use the constructed training data set to train the lightweight human posture estimation network module M based on the frequency domain attention mechanism x , specifically including the following steps:

[0086] Step 4.1: Build a human posture estimator, encode the visual features extracted in step 3, and obtain H heat maps of human joints, where H represents the number of human joint categories predefined in the dataset. Figure 3 As shown in the figure, the feature map information of the image is first obtained through the convolution module; then the feature information of the image is grouped and added to the corresponding position code to obtain the keypoint token containing the position information. A lightweight human posture estimation network module M based on the frequency domain attention mechanism is used x , replace the MHSA module of the multi-head attention mechanism in the Transformer network. Finally, through the MLP fully connected network layer, the heat map of human joints required for human posture estimation is obtained.

[0087] Step 4.2, the H heat maps of human joints obtained in step 4.1 and the Gaussian heat maps constructed according to the annotation information are used to calculate the loss through the mean square error loss function to optimize the network;

[0088] The mean square error loss function is LMSE specifically:

[0089]

[0090] Among them, H represents the human joint point category pre-defined in the dataset, Represents the true value heat map corresponding to the h-th joint point, Represents the predicted heat map corresponding to the h-th joint point.

[0091] Step 5: Use the trained lightweight human pose estimation network based on the frequency domain attention mechanism to perform pose estimation on the human body image to obtain the corresponding human pose estimation result.

[0092] In order to verify the present invention, the present invention is compared with the simcc_mobilenetv2 and pose_shufflenetv2 methods, and the results are shown in Table 1 below:

[0093] Table 1

[0094]

[0095] As can be seen from Table 1 above, the network model parameters of the present invention are only 3.118M, which is similar to the parameters of similar lightweight models. Compared with other similar lightweight models, the improved model has better performance in precision and accuracy.

[0096] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A lightweight attitude estimation method based on frequency domain attention mechanism, characterized by: The lightweight pose estimation method is based on a lightweight feature extraction network N based on the frequency domain attention mechanism. x Implementation, the lightweight feature extraction network N based on frequency domain attention mechanism x Including convolutional layer, group position encoding, lightweight human posture estimation network module M x and a fully connected layer, the specific lightweight posture estimation method specifically includes the following steps: Step 1: Select the MSCOCO human pose estimation dataset as the training dataset, and obtain the input feature map by capturing the image through the human detection frame in the labeled data; Step 2: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x , the lightweight human posture estimation network module M x Including frequency domain attention module, channel attention module and spatial attention module, The frequency domain attention module includes four dynamic low-pass filter modules, four dynamic high-pass filter modules, and a frequency matching kernel module; Each of the dynamic low-pass filter modules includes an average pooling layer, a bilinear interpolation layer, and a ReLU layer, and the pooling sizes of the four dynamic low-pass filter modules are 1×1, 2×2, 3×3, and 6×6, respectively; Each of the dynamic high-pass filtering modules includes a grouped deep convolution layer, and the four grouped deep convolution layers are respectively composed of convolution layers with convolution kernels of 1×1×1, 3×3×3, 5×5×5 and 7×7×7; The frequency matching kernel module includes a linear layer and a frequency domain partial calculation layer; Step 3: Based on the lightweight human posture estimation network module M constructed in step 2 x , build a lightweight feature extraction network N based on frequency domain attention mechanism x , generate visual features by convolutional coding the input feature map intercepted in step 1; Step 4: Use the constructed training data set to train the lightweight human posture estimation network module M based on the frequency domain attention mechanism x ; Step 5: Use the trained lightweight human pose estimation network based on the frequency domain attention mechanism to perform pose estimation on the human body image to obtain the corresponding human pose estimation result.

2. According to claim 1, a lightweight posture estimation method based on frequency domain attention mechanism is characterized in that: In step 2, a lightweight human posture estimation network module M based on the frequency domain attention mechanism is constructed. x , including frequency domain attention module extraction, channel attention module extraction and spatial attention module extraction, specifically including the following steps: Step 2.1, frequency domain attention module extraction: The input feature map obtained in step 1 is passed through the convolution layer of the frequency matching kernel module to obtain the input feature vector x, and the input feature vector x is input into the linear layer of the frequency matching kernel module to obtain the query tensor Q, key tensor K and value tensor V. The attention matrix of the query tensor Q, key tensor K and value tensor V is calculated to obtain the frequency domain attention feature tensor FF. At the same time, the low-frequency feature LF is extracted through the dynamic low-pass filter module, and the high-frequency feature HF is extracted through the dynamic high-pass filter module. The low-frequency feature LF, the high-frequency feature HF and the frequency domain attention feature tensor FF are added together to obtain the frequency domain attention module M. f ; Step 2.2: Compress the feature vector in step 2.1 along the channel of the channel attention module, compress the spatial dimension through global average pooling and global maximum pooling, generate global features describing each channel, send the generated global features to a shared multi-layer perceptron (MLP), learn the attention weights of each channel, and output the global features M through the module. c ; Step 2.3: The global feature M output from step 2.2 c Perform global average pooling and global maximum pooling to generate two two-dimensional feature maps. These two two-dimensional feature maps are concatenated and processed through a convolution layer to generate spatial attention weights. The module outputs the spatial attention weight feature M. s ; Step 2.4: Construct a lightweight human posture estimation network module M based on the frequency domain attention mechanism x : M x =M f +M c ×M f +M s ×M f 。 3. The lightweight posture estimation method based on frequency domain attention mechanism according to claim 2, characterized in that: The step 2.1 extracts the low-frequency feature LF through the dynamic low-pass filtering module and specifically includes the following steps: Step 2.1.1, get the height and width of the input feature map; Step 2.1.2, divide the input feature map into m parts according to the number of channels, m ≥ 4, input each segmented feature map into the corresponding filter, and use bilinear interpolation to upsample the output feature map to the original size; Step 2.1.3, concatenate the m upsampled feature maps in the channel dimension, activate them through the ReLU layer of the dynamic low-pass filter module, and finally output the low-frequency feature LF after low-pass filtering: the expression is as follows: LF=bilinear(AvgPooling k×k (V)) Among them, V is the value tensor, k is the cutoff frequency, and the cutoff frequency of each channel is different.

4. The lightweight posture estimation method based on frequency domain attention mechanism according to claim 2, characterized in that: In step 2.1, the high-frequency feature HF is extracted by a dynamic high-pass filtering module, which specifically includes the following steps: Step 2.2.1, divide the value tensor V into n groups, and use the grouped deep convolution layer in the dynamic high-pass filter module to simulate the tensor characteristics of the cutoff frequencies in different high-pass filters; Step 2.2.2: Concatenate the tensor features obtained in step 2.2.1 to obtain the high-frequency feature HF: HF=depthwise_conv k×k (V)。 5. The lightweight posture estimation method based on frequency domain attention mechanism according to claim 2, characterized in that: In step 2.1: frequency matching kernel module A i，j It is expressed as: Among them, k i represents the i-th frequency component of the key tensor K, v j represents the jth frequency component of the value tensor V; By frequency matching kernel module A i，j Obtaining the frequency domain attention feature tensor FF specifically includes the following steps: Step 2.3.1, divide the input feature vector x into multiple groups, and divide the input feature vector x into i Shoot to query tensor Q i , key tensor K i , value tensor V i ; Step 2.3.2: Key tensor K i Perform softmax operation to obtain the intermediate distribution vector Will Sum value tensor V i Multiply them together and you get V i ; Step 2.3.3: Use the einsum function to query the tensor Q i and V i Multiply them together to get the final attention matrix A i ; Step 2.3.4: Concatenate the attention features obtained from each group together to finally obtain the frequency domain attention feature tensor FF.

6. The lightweight posture estimation method based on frequency domain attention mechanism according to claim 1, characterized in that: Step 4 specifically includes the following steps: Step 4.1: Build a human posture estimator, encode the visual features extracted in step 3, and obtain H heat maps of human joints, where H represents the number of human joint categories pre-defined in the dataset; Step 4.2, the H heat maps of human joints obtained in step 4.1 and the Gaussian heat maps constructed according to the annotation information are used to calculate the loss through the mean square error loss function to optimize the network; The mean square error loss function is LMSE specifically: Among them, H represents the human joint point category pre-defined in the dataset, Represents the true value heat map corresponding to the h-th joint point, Represents the predicted heat map corresponding to the h-th joint point.

Images