Low-resolution human pose estimation method and system based on high-frequency information enhancement

Abstract

The method and system disclosed by the application relate to the technical field of human pose estimation and comprise the following steps: constructing a high-frequency information enhanced down-sampling module; constructing a low-resolution human pose estimation model based on high-frequency information enhancement; training the low-resolution human pose estimation model based on high-frequency information enhancement by using a labeled human pose data set to obtain a trained model; and performing human pose estimation prediction by using the trained low-resolution human pose estimation model to obtain a human pose estimation prediction result. The low-resolution human pose estimation method based on high-frequency information enhancement provided by the application effectively improves the accuracy and stability of pose estimation by enhancing high-frequency information in a low-resolution image, and the application achieves better results in terms of accuracy, stability and overcoming high-frequency information loss.

Description

Technical Field

[0001] This invention relates to the field of human pose estimation technology, specifically a low-resolution human pose estimation method based on high-frequency information enhancement. Background Technology

[0002] Human pose estimation is an important research area in computer vision. Its goal is to extract human pose information from images or videos, including the position, orientation, and posture of body joints. It is a crucial prerequisite task for many fields such as human action recognition, behavior analysis, and human-computer interaction.

[0003] Currently, existing pose estimation network models have achieved significant results in high-resolution image environments. However, when the resolution of the input image decreases, due to the limited fine-grained information contained in low-resolution images, existing pose estimation network models often fail to extract sufficient pose features, severely impacting the algorithm's detection performance. To address the lack of fine-grained information in low-resolution images, most existing methods rely on feature pyramids or feature super-resolution techniques. While this approach can improve the prediction accuracy of low-resolution human pose estimation network models, it also incurs substantial computational overhead, significantly reducing the practicality of the network models.

[0004] In addition to the loss of fine-grained information due to the image itself, existing pose estimation network models typically use pooling layers or strided convolutional layers to downsample features. These downsampling methods also lead to the loss of fine-grained information, which severely limits the performance of human pose estimation network models.

[0005] Since fine-grained information in images mostly resides in their high-frequency information, this invention proposes a low-resolution human pose estimation method based on high-frequency information enhancement, taking a frequency domain perspective. This method replaces the pooling layers and stride convolutional layers in current pose detection network models with a high-frequency information enhancement downsampling module. This encodes and enhances the high-frequency information in the original features, thereby preserving task-relevant high-frequency information to the maximum extent while downsampling features. This improvement significantly enhances the network's feature extraction capabilities, substantially reduces the object misalignment prediction rate, and thus achieves more accurate low-resolution human pose estimation. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is that existing human pose estimation methods lack fine-grained information, have high computational overhead, and suffer from the loss of fine-grained information, as well as the optimization problem of how to achieve more accurate low-resolution human pose estimation.

[0009] To address the aforementioned technical problems, this invention provides the following technical solution: a low-resolution human pose estimation method based on high-frequency information enhancement, comprising constructing a downsampling module for high-frequency information enhancement, including a spatial information to channel information unit, a high-frequency enhancement fusion unit, a high-frequency component decoupling unit, and a cross-domain feature fusion unit; constructing a low-resolution human pose estimation model based on high-frequency information enhancement; and training the low-resolution human pose estimation model based on high-frequency information enhancement using an annotated human pose dataset and performing human pose estimation prediction.

[0010] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the high-frequency information enhancement downsampling module includes a spatial information to channel information unit; interval sampling is used to segment the original features into multiple sub-features according to a scaling scale, and the sub-features are concatenated and recombined according to the channel dimension to form a new feature, represented as:

[0011] X std =Concatenate(Split(X, s), axis=channel)

[0012] In this context, Split(X, s) represents dividing X into s×s sub-features according to spatial dimensions, Concatenate represents concatenating and recombining the obtained sub-features into a completely new feature, and axis = channel represents the concatenation operation along the channel dimension.

[0013] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the high-frequency information enhancement downsampling module further includes a high-frequency enhancement fusion unit; the features are subjected to discrete cosine transform to obtain the corresponding multi-frequency coefficients K. std , represented as:

[0014] K std =DCT(X) std )

[0015]

[0016] Wherein, DCT stands for Discrete Cosine Transform; the multi-frequency coefficients are respectively subjected to high-frequency enhancement coding, represented as:

[0017] K h=MLP(GlobalMaxPool(K std )

[0018]

[0019] Among them, K h To enhance high-frequency information features, GlobalMaxPool is a global max pooling operation, and MLP is a multilayer perceptron consisting of two fully connected layers; multi-frequency joint encoding obtains high-frequency enhancement coefficients and multi-frequency enhancement coefficients, expressed as:

[0020] K m =MLP(GlobalAvgPool(K std )

[0021]

[0022] Among them, K m For multi-frequency information enhancement features, GlobalAvgPool is a global average pooling operation; it is used to enhance high-frequency information from multiple frequency bands of the feature. The enhanced multi-frequency coefficients are obtained by adding the high-frequency enhancement coefficients and the multi-frequency enhancement coefficients, expressed as:

[0023] K = K m +K h

[0024]

[0025] Wherein, K represents the enhanced feature after high-frequency enhancement coding; multiplying the enhanced multi-frequency coefficients by the feature yields the enhanced feature of high-frequency information, expressed as:

[0026] X ag =K×X std

[0027] X he =Fusion(X) ag (axis = channel)

[0028]

[0029]

[0030] Among them, X ag For the enhanced features, X he The final output features are defined by Fusion(axis = channel), which fuses features along the channel dimension.

[0031] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the high-frequency information enhancement downsampling module further includes a high-frequency component decoupling unit; the weights of the initial high-frequency filter are expressed as:

[0032]

[0033] Here, LH, HL, and HH are three fixed-weight filters. The LH filter captures high-frequency information that a single high-pass filter cannot obtain; the HL filter extracts information from the high-frequency components; and the HH filter extracts the high-frequency features of the high-frequency components. The initialized high-frequency filters are decoupled alternately along the vertical and horizontal directions of the original features to extract all features containing high-frequency information. All features containing high-frequency information are combined according to the channel dimension to form high-frequency features, represented as:

[0034] X High =Concatenate(HLC(X),LHC(X),HHC(X),axis=channel)

[0035]

[0036] Here, HLC, LHC, and HHC are three filter layers with fixed weights, and Concatenate(axis = channel) is a concatenation operation performed along the channel dimension. By reducing the channel dimension of high-frequency features, noise in the high-frequency features is reduced, and the high-frequency components are obtained, represented as:

[0037] X hf =Fusion(X) High (axis = channel)

[0038]

[0039] Here, Fusion(axis = channel) is used to fuse features according to the channel dimension.

[0040] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the high-frequency information enhancement downsampling module further includes a cross-domain feature fusion unit; the spatial domain features and frequency domain features are linearly encoded and mapped to the same feature domain, and the two features in the same feature domain are added together to obtain the final output feature, expressed as:

[0041] X out =LE(X) he )+LE(X hf )

[0042]

[0043] Where LE stands for linear coding layer.

[0044] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the low-resolution human pose estimation model includes constructing a backbone network model.

[0045] The pooling layers and stride convolutional layers in the backbone network model are replaced with downsampling modules that enhance high-frequency information, forming a backbone network model to extract human pose features from low-resolution images.

[0046] As a preferred embodiment of the low-resolution human pose estimation method based on high-frequency information enhancement described in this invention, the training of the low-resolution human pose estimation model based on high-frequency information enhancement includes: reading input samples from the training set; performing data preprocessing on the input samples to obtain input data and annotations; feeding the input data into the low-resolution human pose estimation model based on high-frequency information enhancement to obtain output data; using a loss function to measure the difference between the prediction result and the annotation, optimizing the model parameters, and obtaining the trained model; reading input samples from the test set, performing data preprocessing to obtain input data, and feeding the input data into the trained low-resolution human pose estimation model to obtain the human pose estimation prediction result.

[0047] Another objective of this invention is to provide a low-resolution human pose estimation system based on high-frequency information enhancement, which solves the problem of high-frequency information loss during feature downsampling by replacing the pooling layer and stride convolution layer in the low-resolution human pose estimation network model.

[0048] As a preferred embodiment of the low-resolution human pose estimation system based on high-frequency information enhancement described in this invention, it includes: a spatial information to channel information module, a high-frequency enhancement and fusion module, a high-frequency component decoupling module, a cross-domain feature fusion module, a human pose estimation model construction module, and a training and prediction module; the spatial information to channel information module uses interval sampling to segment the original features into multiple sub-features, and merges the sub-features into new features according to the channel dimension; the high-frequency enhancement and fusion module fuses the new features with the original features by performing discrete cosine transform, high-frequency enhancement coding, multi-frequency joint coding, and feature enhancement on the features; the high-frequency component decoupling module uses a high-frequency filter to decompose the high-frequency components in the original features, supplementing the high-frequency information lost by interval sampling; the cross-domain feature fusion module combines spatial domain features and frequency domain features. Linear mapping to the same feature domain and linear fusion are performed to obtain the final downsampled features. The human pose estimation model construction module constructs a backbone network model, replacing the pooling layers and stride convolutional layers in the backbone network model with high-frequency information enhancement downsampling modules to form a backbone network model to extract human pose features from low-resolution images. The training and prediction module reads input samples from the training set, performs data preprocessing to obtain input data and labels, feeds the input data into the human pose estimation model to obtain output data, uses a loss function to measure the difference between the prediction result and the label, and obtains the trained model. Input samples are read from the test set, and data preprocessing is performed to obtain input data, which is then fed into the trained low-resolution human pose estimation model to obtain the human pose estimation prediction result.

[0049] A computer device includes a memory and a processor, the memory storing a computer program, characterized in that the processor executes the computer program to implement a low-resolution human pose estimation method based on high-frequency information augmentation.

[0050] A computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of a low-resolution human pose estimation method based on high-frequency information enhancement.

[0051] The beneficial effects of this invention are as follows: The low-resolution human pose estimation method based on high-frequency information enhancement provided by this invention effectively improves the accuracy and stability of pose estimation by enhancing the high-frequency information in low-resolution images. By replacing the pooling layer and stride convolutional layer, and introducing a downsampling module for high-frequency information enhancement, the problem of high-frequency information loss during feature downsampling can be overcome. This invention achieves better results in terms of accuracy, stability, and overcoming high-frequency information loss. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0053] Figure 1 The flowchart shows the overall process of the low-resolution human pose estimation method based on high-frequency information enhancement provided in the first embodiment of the present invention.

[0054] Figure 2 This is a schematic diagram of the low-resolution human pose estimation result of the prior art HRNet network, which is based on the high-frequency information enhancement low-resolution human pose estimation method provided in the second embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the low-resolution human pose estimation results of the HRNet network based on high-frequency information enhancement, which is the method for low-resolution human pose estimation based on high-frequency information enhancement provided in the second embodiment of the present invention.

[0056] Figure 4 The overall flowchart of the low-resolution human pose estimation system based on high-frequency information enhancement provided for the third embodiment of the present invention is shown. Detailed Implementation

[0057] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0060] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0061] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0062] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0063] Example 1

[0064] Reference Figure 1 As an embodiment of the present invention, a low-resolution human pose estimation method based on high-frequency information enhancement is provided, comprising:

[0065] S1: Construct a downsampling module for high-frequency information enhancement, including a spatial information to channel information unit, a high-frequency enhancement fusion unit, a high-frequency component decoupling unit, and a cross-domain feature fusion unit.

[0066] Furthermore, the high-frequency information enhancement downsampling module includes a spatial information to channel information unit, a high-frequency enhancement fusion unit, a high-frequency component decoupling unit, and a cross-domain feature fusion unit.

[0067] It should be noted that the spatial information to channel information unit uses interval sampling to divide the original feature into multiple sub-features according to the scaling scale, and then concatenates and recombines the sub-features according to the channel dimension to form a new feature, represented as:

[0068] X std =Concatenate(Split(X, s), axis=channel)

[0069] In this context, Split(X, s) represents dividing X into s×s sub-features according to spatial dimensions, Concatenate represents concatenating and recombining the obtained sub-features into a completely new feature, and axis = channel represents the concatenation operation along the channel dimension.

[0070] It should also be noted that the high-frequency enhancement fusion unit performs a discrete cosine transform on the features to obtain the corresponding multi-frequency coefficients K. std , is represented as:

[0071] K std =DCT(X) std )

[0072]

[0073] Wherein, DCT stands for Discrete Cosine Transform; the multi-frequency coefficients are respectively subjected to high-frequency enhancement coding, represented as:

[0074] K h =MLP(GlobalMaxPool(K std )

[0075]

[0076] Among them, K h To enhance high-frequency information features, GlobalMaxPool is a global max pooling operation, and MLP is a multilayer perceptron consisting of two fully connected layers; multi-frequency joint encoding obtains high-frequency enhancement coefficients and multi-frequency enhancement coefficients, expressed as:

[0077] K m =MLP(GlobalAvgPool(K std )

[0078]

[0079] Among them, K m For multi-frequency information enhancement features, GlobalAvgPool is a global average pooling operation; it is used to enhance high-frequency information from multiple frequency bands of the feature. The enhanced multi-frequency coefficients are obtained by adding the high-frequency enhancement coefficients and the multi-frequency enhancement coefficients, expressed as:

[0080] K = K m +K h

[0081]

[0082] Wherein, K represents the enhanced feature after high-frequency enhancement coding; multiplying the enhanced multi-frequency coefficients by the feature yields the enhanced feature of high-frequency information, expressed as:

[0083] X ag =K×X std

[0084] X he =Fusion(X) ag (axis = channel)

[0085]

[0086]

[0087] Among them, X ag For the enhanced features, X he The final output features are defined by Fusion(axis = channel), which fuses features along the channel dimension.

[0088] It should also be noted that the weights of the high-frequency filter initialized by the high-frequency component decoupling unit are expressed as follows:

[0089]

[0090] Wherein, LH, HL, and HH are three fixed-weight filters. The LH filter captures high-frequency information that a single high-pass filter cannot obtain; the HL filter extracts information from the high-frequency components; and the HH filter extracts the high-frequency features of the high-frequency components. The initialized high-frequency filters are decoupled alternately along the vertical and horizontal directions of the original features to extract all features containing high-frequency information. All features containing high-frequency information are combined according to the channel dimension to form high-frequency features, represented as:

[0091] X High =Concatenate(HLC(X),LHC(X),HHC(X),axis=channel)

[0092]

[0093] Here, HLC, LHC, and HHC are three filter layers with fixed weights, and Concatenate(axis = channel) is a concatenation operation performed along the channel dimension. By reducing the channel dimension of high-frequency features, noise in the high-frequency features is reduced, and the high-frequency components are obtained, represented as:

[0094] X hf =Fusion(X) High (axis = channel)

[0095]

[0096] Here, Fusion(axis = channel) is used to fuse features according to the channel dimension.

[0097] It should also be noted that the cross-domain feature fusion unit performs linear encoding on the spatial domain features and frequency domain features respectively, mapping them to the same feature domain. Two features in the same feature domain are then added together to obtain the final output feature, represented as:

[0098] X out =LE(X) he )+LE(X hf )

[0099]

[0100] Where LE stands for linear coding layer.

[0101] It should also be noted that the spatial information to channel information unit divides the original features according to the scaling scale through interval sampling, generating multiple sub-features. These sub-features are spliced ​​and recombined according to the channel dimension to form a new feature. This process helps to extract and retain spatial information in the image. The high-frequency enhancement fusion unit performs discrete cosine transform on the above features to obtain the corresponding multi-frequency coefficients. These multi-frequency coefficients are processed by high-frequency enhancement coding and encoded through a fully connected layer composed of global max pooling and multilayer perceptrons. This can enhance the representation ability of high-frequency information and obtain high-frequency enhancement coefficients and multi-frequency enhancement coefficients. The high-frequency component decoupling unit initializes the weights of a set of high-frequency filters. These filters are used to capture high-frequency information that a single high-pass filter cannot obtain, extract information in high-frequency components, and extract high-frequency features of high-frequency components. By applying these initialized high-frequency filters to the vertical and horizontal directions of the original features, features containing high-frequency information can be decoupled. These features are combined according to the channel dimension to form high-frequency features, reducing noise in high-frequency features. The cross-domain feature fusion unit performs linear encoding on spatial domain features and frequency domain features and maps them to the same feature domain. Two features in the same feature domain are added to obtain the final output feature.

[0102] S2: Construct a low-resolution human pose estimation model based on high-frequency information enhancement.

[0103] Furthermore, constructing a low-resolution human pose estimation model based on high-frequency information enhancement includes building a backbone network model and extracting pose features.

[0104] It should be noted that the pooling layers and stride convolutional layers in the backbone network model are replaced with downsampling modules that enhance high-frequency information, forming the backbone network model to extract human pose features from low-resolution images.

[0105] It should also be noted that replacing the pooling layers and strided convolutional layers in the backbone network model with high-frequency information-enhanced downsampling modules is done to enhance the representation of high-frequency information and reduce information loss at low resolutions when extracting pose features from low-resolution images. By using high-frequency information-enhanced downsampling modules as a replacement, more detailed information can be preserved in the backbone network. This is especially important for tasks like human pose estimation, where high-frequency information is crucial for accuracy. Specifically, high-frequency information-enhanced downsampling modules can replace the pooling layers and strided convolutional layers in the backbone network. These alternative downsampling modules are composed of units such as spatial information to channel information units, high-frequency enhancement fusion units, high-frequency component decoupling units, and cross-domain feature fusion units. They extract features by enhancing high-frequency information. This alternative approach introduces a high-frequency information-enhanced downsampling process into the backbone network, thereby better extracting human pose features from low-resolution images. This helps to mitigate information loss caused by downsampling and improves the performance of the pose estimation model.

[0106] S3: Train a low-resolution human pose estimation model based on high-frequency information augmentation using an annotated human pose dataset and perform human pose estimation and prediction.

[0107] Furthermore, training a low-resolution human pose estimation model based on high-frequency information enhancement and performing human pose estimation prediction includes training the model and the prediction results.

[0108] It should be noted that an input sample is randomly read from the training set, and the input sample is preprocessed to obtain the corresponding input data and labels. The input data is then fed into a low-resolution human pose estimation model based on high-frequency information enhancement to obtain the corresponding output data. MSELoss is used to measure the difference between the prediction results and the labels, and Adamw is used to optimize the network model parameters, thereby obtaining the trained model.

[0109] It should also be noted that through repeated training, optimization, and evaluation, the model will gradually learn the features of the input data and pose-related information, thereby improving its pose estimation ability on low-resolution images.

[0110] Example 2

[0111] Reference Figure 2-3 As an embodiment of the present invention, a low-resolution human pose estimation method based on high-frequency information enhancement is provided. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0112] from Figure 2As can be seen, the low-resolution human pose estimation results provided by the existing HRNet network are insufficient in low-resolution image environments due to the limited fine-grained information in the images themselves. This results in the inability of existing pose estimation network models to extract enough pose features, which severely affects the detection performance of the algorithm and makes it impossible to accurately estimate human pose. Existing pose estimation network models usually downsample the features through pooling layers or strided convolutional layers. This downsampling method leads to the loss of fine-grained information, which severely limits the performance of the network model and makes it impossible to accurately predict the pose of the object.

[0113] from Figure 3 As can be seen from the two comparison figures, the low-resolution human pose estimation method provided by this invention significantly improves the prediction accuracy of the network, overcomes the problems of torso detection misalignment and inaccurate key point prediction caused by the lack of high-frequency information, and greatly improves the detection performance of low-resolution human pose estimation.

[0114] As shown in Table 1, the method of the present invention outperforms existing technologies in human pose estimation, with a smaller average joint error, indicating that the prediction results are closer to the actual annotations. The higher pose accuracy indicates more accurate prediction of human pose, demonstrating that the method of the present invention can improve pose estimation performance and more accurately predict human pose in low-resolution image environments.

[0115] Table 1 Comparison of Experimental Data

[0116] Mean joint error 2.34 pixels 3.21 pixels Attitude accuracy 92.30％ 87.50％ Key point positioning error 2.89 pixels 4.56 pixels

[0117] Example 3

[0118] Reference Figure 4 As an embodiment of the present invention, a low-resolution human pose estimation system based on high-frequency information enhancement is provided, including a spatial information to channel information module, a high-frequency enhancement fusion module, a high-frequency component decoupling module, a cross-domain feature fusion module, a human pose estimation model construction module, and a training and prediction module.

[0119] The spatial information to channel information module uses interval sampling to segment the original features into multiple sub-features, and merges the sub-features into new features according to the channel dimension; the high-frequency enhancement fusion module fuses the new features with the original features by performing discrete cosine transform, high-frequency enhancement coding, multi-frequency joint coding, and feature enhancement on the features; the high-frequency component decoupling module uses a high-frequency filter to decompose the high-frequency components in the original features to supplement the high-frequency information lost by interval sampling; the cross-domain feature fusion module linearly maps spatial domain features and frequency domain features to the same feature domain, and linearly fuses them to obtain the final downsampled features; the human pose estimation model construction module constructs a skeleton... The backbone network model replaces the pooling layers and stride convolutional layers in the core network model with a downsampling module that enhances high-frequency information, forming a backbone network model to extract human pose features from low-resolution images. The training and prediction module reads input samples from the training set, performs data preprocessing to obtain input data and labels, feeds the input data into the human pose estimation model to obtain output data, uses a loss function to measure the difference between the prediction result and the label, and obtains the trained model. The module reads input samples from the test set, performs data preprocessing to obtain input data, feeds the input data into the trained low-resolution human pose estimation model, and obtains the human pose estimation prediction result.

[0120] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0122] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0123] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0124] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A low-resolution human pose estimation method based on high-frequency information enhancement, characterized in that, include: A downsampling module for enhancing high-frequency information is constructed, including a spatial information to channel information unit, a high-frequency enhancement fusion unit, a high-frequency component decoupling unit, and a cross-domain feature fusion unit; Construct a low-resolution human pose estimation model based on high-frequency information enhancement; A low-resolution human pose estimation model based on high-frequency information augmentation is trained using an annotated human pose dataset to perform human pose estimation and prediction. The high-frequency information enhancement downsampling module includes a spatial information to channel information unit; The original features are segmented into multiple sub-features according to the scaling scale using interval sampling. These sub-features are then concatenated and recombined according to the channel dimension to form a new feature, represented as follows: in, To indicate Divided according to spatial dimensions, etc. Individual characteristics, This means concatenating and recombining the obtained sub-features into a completely new feature. This indicates that the splicing operation is performed along the channel dimension; The high-frequency information enhancement downsampling module also includes a high-frequency enhancement fusion unit; right Performing a discrete cosine transform yields the corresponding multi-frequency coefficients. , represented as: in, This is a discrete cosine operation; The high-frequency enhancement coefficients are obtained by high-frequency enhancement coding of the multi-frequency coefficients, and are represented as follows: in, For high-frequency information enhancement coefficients, This is a global max pooling operation. It is a multilayer perceptron consisting of two fully connected layers; The multi-frequency enhancement coefficients are obtained by performing multi-frequency joint encoding on the multi-frequency coefficients, and are expressed as follows: in, For multi-frequency information enhancement coefficients, This is a global average pooling operation; To enhance high-frequency information from multiple frequency bands of a feature, the high-frequency enhancement coefficient and the multi-frequency enhancement coefficient are added together to obtain the enhanced multi-frequency coefficient, expressed as: in, These are the coefficients after the high-frequency enhanced coding; The enhanced multi-frequency coefficient and The feature obtained by multiplication, enhanced with high-frequency information, is represented as: in, For the enhanced features, The final output features, To fuse features according to the channel dimension; The high-frequency information enhancement downsampling module also includes a high-frequency component decoupling unit; The initial weights of the high-frequency filter are expressed as: in, For three filters with fixed weights, The filter captures high-frequency information that a single high-pass filter cannot obtain; The filter extracts information from high-frequency components; The filter extracts the high-frequency characteristics of the high-frequency components; The initialized high-frequency filter is decoupled alternately along the longitudinal and transverse directions of the original features to extract all features containing high-frequency information. All features containing high-frequency information are then combined according to the channel dimension to form a high-frequency feature, represented as: in, , , For three filter layers with fixed weights, The splicing operation is performed along the channel dimension; By reducing the channel dimension of high-frequency features to reduce noise in the high-frequency features, the high-frequency components are obtained, represented as: in, To fuse features according to the channel dimension; The high-frequency information enhancement downsampling module also includes a cross-domain feature fusion unit; Linear encoding is performed on the spatial and frequency domain features respectively, mapping them to the same feature domain. The two features located in the same feature domain are then added together to obtain the final output feature, represented as: in, It is a linear coding layer.

2. The low-resolution human pose estimation method based on high-frequency information enhancement as described in claim 1, characterized in that: The low-resolution human pose estimation model includes a backbone network model; The pooling layers and stride convolutional layers in the backbone network model are replaced with downsampling modules that enhance high-frequency information, forming a backbone network model to extract human pose features from low-resolution images.

3. The low-resolution human pose estimation method based on high-frequency information enhancement as described in claim 1, characterized in that: The training based on a low-resolution human pose estimation model enhanced with high-frequency information and the prediction of human pose estimation include reading input samples from the training set, performing data preprocessing on the input samples to obtain input data and annotations; The input data is fed into a low-resolution human pose estimation model based on high-frequency information enhancement to obtain the output data; The loss function is used to measure the gap between the prediction results and the labels, and the parameters of the model are optimized to obtain the trained model. Input samples are read from the test set, and data preprocessing is used to obtain input data. The input data is then fed into the trained low-resolution human pose estimation model to obtain the human pose estimation prediction result.

4. A system employing the low-resolution human pose estimation method based on high-frequency information enhancement as described in any one of claims 1 to 3, characterized in that: It includes a spatial information to channel information module, a high-frequency enhancement fusion module, a high-frequency component decoupling module, a cross-domain feature fusion module, a human pose estimation model construction module, and a training and prediction module; The spatial information to channel information module uses interval sampling to segment the original feature into multiple sub-features, and then merges the sub-features into a new feature according to the channel dimension; The high-frequency enhancement fusion module fuses new features with original features by performing discrete cosine transform, high-frequency enhancement coding, multi-frequency joint coding, and feature enhancement on the features. The high-frequency component decoupling module uses a high-frequency filter to decompose the high-frequency components in the original features, supplementing the high-frequency information lost during interval sampling. The cross-domain feature fusion module linearly maps spatial domain features and frequency domain features to the same feature domain, and then linearly fuses them to obtain the final downsampled features. The human pose estimation model construction module constructs a backbone network model, replacing the pooling layer and stride convolution layer in the backbone network model with a downsampling module that enhances high-frequency information, and extracts human pose features from low-resolution images by forming a backbone network model. The training and prediction module reads input samples from the training set, performs data preprocessing to obtain input data and labels, feeds the input data into the human pose estimation model to obtain output data, uses a loss function to measure the difference between the prediction result and the label, and obtains the trained model. It also reads input samples from the test set, performs data preprocessing to obtain input data, feeds the input data into the trained low-resolution human pose estimation model, and obtains the human pose estimation prediction result.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the low-resolution human pose estimation method based on high-frequency information enhancement as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the low-resolution human pose estimation method based on high-frequency information enhancement as described in any one of claims 1 to 3.

Images