3D human body posture estimation method based on diffusion model and continuous frame time-frequency information fusion

Abstract

The invention provides a 3D human body posture estimation method based on a diffusion model and continuous frame time-frequency information fusion, and the method comprises the steps: inputting a 2D posture into an initial estimator for posture extraction, and obtaining an initial 3D human body posture; inputting the initial 3D human body posture into a diffusion model in an iteration form, and training in combination with noise to obtain an optimized 3D human body posture; performing mechanism processing on the long sequence data of the 3D human body skeleton by using an encoder of a space Transform module and a time frequency domain fusion module to obtain encoding features, and performing mechanism processing on the long sequence data of the 3D human body skeleton by using the time frequency domain fusion module; and the coding features are input into the time-frequency feature fusion module for mechanism processing, so that the accuracy of attitude estimation is improved. According to the method, the initial estimator is used for integrating constraints such as the bone length consistency and the joint connection relation, and fuzziness and ambiguity in 2D-3D inference can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a 3D human body posture estimation method based on a diffusion model and continuous frame time-frequency information fusion. Background Art

[0002] 3D human pose estimation is an important research direction in the field of computer vision. Its goal is to estimate the position of human joints in 3D space through 2D images or video data. Traditional methods mainly rely on deterministic models, but when processing multi-frame inputs, there are the following problems: fuzzy information is difficult to remove, and deterministic methods often cannot handle the uncertainty in the input data well, resulting in fuzziness and deviation in the generated 3D pose results. High computational cost. Although the traditional Transformer model can capture long sequence dependencies, the computational cost is too high when processing skeleton sequences, which is not suitable for real-time application scenarios with limited resources. Noise sensitivity. Noise is often present in 2D human joint point detection, which affects the accuracy of 3D pose estimation.

[0003] In the process of 2D to 3D human posture estimation, there are mainly two difficulties: The first is the problem of missing depth information and multiple solutions. When estimating 3D human posture from a single-frame 2D image, due to the lack of depth information, the same 2D joint point configuration may correspond to multiple different 3D posture solutions. This multiple solution problem makes the prediction results uncertain. In addition, complex backgrounds, lighting changes, and occlusions will introduce significant noise, making the 2D detection results unstable, further affecting the accuracy of 3D posture estimation. This depth ambiguity problem is the main challenge in 2D to 3D conversion. The second is the temporal continuity and computational overhead in dynamic scenes. In dynamic scenes, the temporal continuity of human motion provides important contextual information, but traditional methods need to repeat feature extraction in each frame when processing time series, which consumes huge computing resources. Especially in fast motion and complex posture changes, it becomes more difficult to capture the dynamic characteristics and continuity of the action. In addition, the existing methods are relatively conventional in modeling time domain features and fail to make full use of frequency domain information, resulting in insufficient estimation accuracy and efficiency of dynamic motion. Summary of the invention

[0004] In view of the above situation, the main purpose of the present invention is to propose a 3D human posture estimation method based on a diffusion model and continuous frame time-frequency information fusion to solve the above technical problems.

[0005] The present invention proposes a 3D human posture estimation method based on a diffusion model and continuous frame time-frequency information fusion, the method comprising the following steps: Step 1: Build a time-frequency feature fusion module based on the Transformer architecture, build a diffusion model based on the time-frequency feature fusion module, and build an initial estimator based on the diffusion model; Among them, the time-frequency feature fusion module includes a spatial Transformer module and a time-frequency domain fusion module, and the time-frequency domain fusion module includes a standard multi-layer perceptron and a frequency-domain multi-layer perceptron; Among them, the standard multi-layer perceptron and the frequency domain multi-layer perceptron share self-attention; Step 2: Input the 2D posture into the initial estimator for posture extraction to obtain the initial 3D human body posture; Step 3: Input the initial 3D human body posture into the diffusion model and train it in an iterative manner in combination with noise to obtain the target 3D human body posture; Get the trained diffusion model based on the target 3D human pose; Step 4: Use the diffused training model to recognize and process the initial 3D human body posture to obtain long sequence data of the 3D human body skeleton; In the spatial Transformer module, the 3D posture embedding model is used to perform center frame selection and frame processing operations on the long sequence data of the 3D human skeleton in turn to obtain the joint coordinates; Perform spatial position embedding operation on joint coordinates to obtain spatial embedded feature sequence; The spatially embedded feature sequence is input into the encoder of the spatial Transformer module for self-attention mechanism processing to obtain the encoded features; In the time-frequency domain fusion module, the long sequence data of the 3D human skeleton is subjected to discrete cosine transform operation combined with noise to obtain frequency domain features; The frequency domain features are subjected to low-pass filtering and information extraction operations in sequence to obtain frequency domain features that retain information; Perform linear projection operation on the frequency domain features that retain information to obtain feature representation in the frequency domain, and perform embedding operation on the feature representation in the frequency domain to generate the final frequency domain features; Combine the encoded features and the final frequency domain features to obtain the final input features; Step 5: Input the final input features into the time-frequency feature fusion module for mechanism processing to obtain the output center frame; The local details and global information focused by the diffusion model are improved based on the output center frame.

[0006] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention uses an initial estimator to estimate the approximate 3D human body posture. By directly inferring the preliminary 3D posture from the 2D posture, the initial estimator can integrate constraints such as bone length consistency and joint connection relationship, reducing the ambiguity and ambiguity in 2D to 3D inference; at the same time, the initial 3D posture provides a high-dimensional input for the subsequent model, so that the optimization network can focus on detail correction and noise suppression, rather than learning complex deep features from scratch; 2. The present invention designs a DR-TFF diffusion baseline refinement framework and applies the diffusion model (DM-TFF) to time-series 3D pose estimation, breaking through the limitations of traditional single-frame estimation. The initial estimated continuous multi-frame 3D pose is combined with Gaussian sampling noise and input into the diffusion model. The 3D pose estimation result is gradually denoised and optimized through the reverse diffusion process. 3. The present invention proposes a time-frequency feature fusion module (TFFormer) with a conditional mechanism, which integrates time step conditions with time-frequency domain information to enhance the model's ability to express complex dynamics.

[0007] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a flowchart of the steps of the 3D human posture estimation method based on the diffusion model and the fusion of continuous frame time-frequency information proposed by the present invention.

[0009] Figure 2 This is the overall network framework diagram of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention.

[0010] Figure 3 This is a schematic diagram of the overall architecture of the initial estimator of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention.

[0011] Figure 4 This is a schematic diagram of the overall architecture of the diffusion model of the 3D human posture estimation method based on the diffusion model and the fusion of continuous frame time-frequency information proposed in the present invention.

[0012] Figure 5 This is a schematic diagram of the time-frequency feature fusion module architecture of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention.

[0013] Figure 6 This is a spatial Transformer architecture diagram of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention.

[0014] Figure 7 This is a time-frequency domain fusion module architecture diagram of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention.

[0015] Figure 8 This is a schematic diagram of the frequency domain multi-layer perceptron network structure of the 3D human posture estimation method based on the diffusion model and continuous frame time-frequency information fusion proposed in the present invention. DETAILED DESCRIPTION

[0016] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout are the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0017] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to provide some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0018] See also Figure 1 The embodiment of the present invention proposes a 3D human posture estimation method based on a diffusion model and continuous frame time-frequency information fusion, the method comprising the following steps: Step 1: Build a time-frequency feature fusion module based on the Transformer architecture, build a diffusion model based on the time-frequency feature fusion module, and build an initial estimator based on the diffusion model; Among them, the time-frequency feature fusion module includes a spatial Transformer module and a time-frequency domain fusion module, and the time-frequency domain fusion module includes a standard multi-layer perceptron and a frequency-domain multi-layer perceptron; Among them, the standard multi-layer perceptron and the frequency domain multi-layer perceptron share self-attention.

[0019] Step 2: Input the 2D posture into the initial estimator for posture extraction to obtain the initial 3D human body posture.

[0020] See also Figure 2 and Figure 3 In step 2, the 2D posture is input into the initial estimator for posture extraction to obtain the initial 3D human body posture. The corresponding relationship in the process is as follows: ; in, represents the initial 3D human body posture, represents the initial estimator, represents the 2D pose input to the initial estimator.

[0021] Step 3: Input the initial 3D human body posture into the diffusion model and train it in an iterative manner in combination with noise to obtain the target 3D human body posture; The trained diffusion model is obtained based on the target 3D human pose.

[0022] See also Figure 4 In step 3, the initial 3D human body posture is input into the diffusion model, and the training is performed in an iterative form in combination with noise to obtain the target 3D human body posture, which specifically includes the following steps: The initial 3D human body posture is input into the diffusion model to give a training sample, and the training sample is obtained. Noise is added to the training sample for forward propagation to obtain the network estimation distribution of the noise, and the 3D human body posture with noise is obtained at the end of the forward propagation. Based on the time-frequency feature fusion module, the noisy 3D human posture is denoised using an optimization model combined with the network estimation distribution of the noise. After denoising, the target 3D human posture is obtained. Noise is re-added to the target 3D human posture, and the process is repeated in an iterative manner to complete the denoising. In the step of inputting the initial 3D human posture into the diffusion model to give a training sample, obtaining the training sample, adding noise to the training sample for forward propagation to obtain the network estimation distribution of the noise, and obtaining the 3D human posture with noise at the end of the forward propagation, the corresponding relationship of the process is as follows: ; in, represents the network estimated distribution of noise, represents a Gaussian distribution, Indicates at time 3D human pose with noise, Indicates at time Varying noise variance schedule, represents the matrix unit; In the step of denoising the noisy 3D human posture combined with the noisy network estimation distribution based on the time-frequency feature fusion module, the corresponding relationship of the process is as follows: ; in, represents the conditional probability distribution, represents the covariance predicted by the time-frequency feature fusion module, Represents the mean value predicted by the time-frequency feature fusion module.

[0023] Furthermore, at the end of the forward propagation process, the corresponding relationship between the edge distribution process of the noisy 3D human posture is as follows: ; in, Indicates that the original signal ratio is retained in each diffusion step. Represents the true attitude value.

[0024] The covariance predicted by the time-frequency feature fusion module is further expressed as follows: ; in, represents the target 3D human pose, represents the attenuation coefficient of each step.

[0025] The relationship between the mean setting of the time-frequency feature fusion module prediction is as follows: ; in, Indicates at time The noise variance.

[0026] After the initial estimator extracts the initial 3D human pose, it is input into the diffusion model and gradually denoised through multiple iterations. Each iteration generates multiple original true 3D poses from the Gaussian distribution, and after denoising, the target's 3D pose is generated, and the noise is re-added as input in the next iteration. Through multiple iterations and refinements, the 3D pose prediction is gradually optimized, and finally a high-precision target 3D pose is generated.

[0027] It should be noted that in the attached Figure 2 In the example, TFFormer module represents the time-frequency feature fusion module. It represents the process of extracting 2D pose by the initial estimator. It represents the process of optimizing the processed features of the long sequence data of the 3D human skeleton in the time-frequency feature fusion module to obtain the output central frame.

[0028] Step 4: Use the diffused training model to recognize and process the initial 3D human body posture to obtain long sequence data of the 3D human body skeleton; In the spatial Transformer module, the 3D posture embedding model is used to perform center frame selection and frame processing operations on the long sequence data of the 3D human skeleton in turn to obtain the joint coordinates; Perform spatial position embedding operation on joint coordinates to obtain spatial embedded feature sequence; The spatially embedded feature sequence is input into the encoder of the spatial Transformer module for self-attention mechanism processing to obtain the encoded features; In the time-frequency domain fusion module, the long sequence data of the 3D human skeleton is subjected to discrete cosine transform operation combined with noise to obtain frequency domain features; The frequency domain features are subjected to low-pass filtering and information extraction operations in sequence to obtain frequency domain features that retain information; Perform linear projection operation on the frequency domain features that retain information to obtain feature representation in the frequency domain, and perform embedding operation on the feature representation in the frequency domain to generate the final frequency domain features; The encoded features and the final frequency domain features are combined to obtain the final input features.

[0029] See also Figure 5 and Figure 6 In step 4, the initial 3D human body posture is recognized and processed using the diffused training model to obtain long sequence data of the 3D human body skeleton. The corresponding relationship of the process is as follows: ; in, Represents a given input sequence, Represents a collection of long sequence data, represents the sequence length, represents the number of joints, and 3 represents the 3D coordinates of each joint.

[0030] It should be noted that the 3D coordinates of each joint contain (x, y, z) components.

[0031] In the step of performing discrete cosine transform operation on long sequence data of 3D human skeleton combined with noise to obtain frequency domain features, the corresponding relationship of the process is as follows: ; in, represents the discrete cosine transform coefficient of each joint, represents the frequency domain index, represents the joint index, It means that after the cosine function processing, represents pi, Represents a time index.

[0032] In the step of performing low-pass filtering and information extraction operations on the frequency domain features in sequence to obtain the frequency domain features that retain the information, the corresponding relationship in the process is as follows: ; in, represents the frequency domain features that retain information, Indicates The joint Discrete cosine transform coefficients.

[0033] In the step of performing a linear projection operation on the frequency domain features of the retained information to obtain the feature representation of the frequency domain, and performing an embedding operation on the feature representation of the frequency domain to generate the final frequency domain features, the corresponding relationship of the process is as follows: ; in, represents the final frequency domain features, Represents the frequency domain position embedding operation.

[0034] In the step of combining the coding features and the final frequency domain features to obtain the final input features, the corresponding relationship between the process is as follows: ; in, represents the final input features, Represents the encoding feature.

[0035] Step 5: Input the final input features into the time-frequency feature fusion module for mechanism processing to obtain the output center frame; The local details and global information focused by the diffusion model are improved based on the output center frame.

[0036] See also Figure 7 and Figure 8 In step 5, the final input feature is input into the time-frequency feature fusion module for mechanism processing to obtain the output center frame, which specifically includes the following sub-steps: The final input features are input to the normalization layer of the time-frequency feature fusion module for processing to obtain the normalization layer processing features; The features processed by the normalization layer are processed by a multi-head self-attention mechanism to obtain self-attention features; Among them, the self-attention feature is obtained, and the corresponding relationship of the process is as follows: ; in, represents the self-attention feature, It means that it has been processed by the self-attention mechanism. Indicates Normalization processing features of the layer; The self-attention features and the normalization layer processing features are sequentially subjected to residual connection processing and normalization processing to obtain residual normalized features; Among them, the residual normalization feature is obtained, and the corresponding relationship of the process is as follows: ; in, represents the residual normalized feature, Indicates that it has been processed by layer normalization; The residual normalized features are segmented to obtain the features in the time domain and the features in the frequency domain; Among them, the characteristics of the time domain and the characteristics of the frequency domain are obtained, and the corresponding relationship between the process is as follows: ; in, Indicates The temporal characteristics of the layer, Indicates The frequency domain characteristics of the layer, Represents the previous step of taking residual normalized features elements, Represents the first step of taking residual normalization features The part that can be determined by an element; Input the time domain features into the frequency domain multilayer perceptron for mechanism processing to obtain the time domain output features; Input the frequency domain features into a standard multi-layer perceptron for processing to obtain frequency domain output features; Perform a splicing operation on the time domain output features and the frequency domain output features to obtain a splicing feature; Among them, the splicing features are obtained, and the corresponding relationship in the process is as follows: ; in, Represents the concatenation of the time domain output features and the frequency domain output features, Represents a splicing operation, It means that it has been processed by the frequency domain multilayer perceptron. Indicates that it has been processed by a standard multi-layer perceptron; The splicing features are regressed through a normalization layer to obtain a regression result, and the regression result is subjected to an inverse discrete cosine transform operation to obtain an output center frame; Among them, the output center frame is obtained, and the corresponding relationship of the process is as follows: ; in, Indicates the output center frame.

[0037] The time domain features are input into the frequency domain multilayer perceptron for mechanism processing to obtain the time domain output features, which specifically includes the following sub-steps: The time domain features are input into the frequency domain multilayer perceptron for discrete cosine transform operation to obtain the decomposed frequency domain features; Among them, the decomposed frequency domain characteristics are obtained, and the corresponding relationship in the process is as follows: ; in, represents the decomposed frequency domain features, Represents a discrete cosine transform operation; Perform time embedding initialization operation on the decomposed frequency domain features to obtain time embedded features; Among them, the characteristics of time embedding are obtained, and the corresponding relationship of the process is as follows: ; in, represents the time step embedding initialization, Indicates passing Function calculation processing, Indicates location, represents the embedding dimension, represents the embedding dimension index; The time-embedded features are sequentially subjected to time-embedded linear transformation operations, two-layer linear layer processing, and nonlinear layer activation function processing to obtain features processed by the nonlinear layer activation function; Among them, the characteristics of the nonlinear layer activation function processing are obtained, and the corresponding relationship of the process is as follows: ; in, represents the first linear layer, represents the dimension of the hidden layer, It means that it has been processed by the activation function. represents the second linear layer; Generate adjustment parameters using features processed by nonlinear layer activation functions; Among them, the relationship between the process of generating adjustment parameters and the existing ones is as follows: ; in, represents the third linear change, represents the translation parameter, represents the scaling parameter, Indicates splitting a tensor into a preset number of smaller tensors along the specified dimension; The frequency domain feature calculation is adjusted using adjustment parameters to obtain the output feature of the frequency domain rate; Among them, the output characteristics of the frequency domain rate are obtained, and the corresponding relationship of the process is as follows: ; in, Represents the adjusted frequency domain features.

[0038] Furthermore, the frequency domain rate multilayer perceptron, as a flexible frequency domain filter, can finely adjust the expression of human motion features in the frequency domain. Finally, the processed frequency domain features are mapped back to the time domain through the inverse discrete cosine transform, so that the features enhanced in the frequency domain can be re-adapted to the time domain representation. This step ensures that the frequency domain rate multilayer perceptron can not only capture the global dynamic information of human motion, but also retain the detailed features, thereby enhancing the accuracy of posture estimation.

[0039] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0040] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0041] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion, characterized in that: The method comprises the following steps: Step 1: Build a time-frequency feature fusion module based on the Transformer architecture, build a diffusion model based on the time-frequency feature fusion module, and build an initial estimator based on the diffusion model; Among them, the time-frequency feature fusion module includes a spatial Transformer module and a time-frequency domain fusion module, and the time-frequency domain fusion module includes a standard multi-layer perceptron and a frequency-domain multi-layer perceptron; Among them, the standard multi-layer perceptron and the frequency domain multi-layer perceptron share self-attention; Step 2: Input the 2D posture into the initial estimator for posture extraction to obtain the initial 3D human body posture; Step 3: Input the initial 3D human body posture into the diffusion model and train it in an iterative manner in combination with noise to obtain the target 3D human body posture; Get the trained diffusion model based on the target 3D human pose; Step 4: Use the diffused training model to recognize and process the initial 3D human body posture to obtain long sequence data of the 3D human body skeleton; In the spatial Transformer module, the 3D posture embedding model is used to perform center frame selection and frame processing operations on the long sequence data of the 3D human skeleton in turn to obtain the joint coordinates; Perform spatial position embedding operation on joint coordinates to obtain spatial embedded feature sequence; The spatially embedded feature sequence is input into the encoder of the spatial Transformer module for self-attention mechanism processing to obtain the encoded features; In the time-frequency domain fusion module, the long sequence data of the 3D human skeleton is subjected to discrete cosine transform operation combined with noise to obtain frequency domain features; The frequency domain features are subjected to low-pass filtering and information extraction operations in sequence to obtain frequency domain features that retain information; Perform linear projection operation on the frequency domain features that retain information to obtain feature representation in the frequency domain, and perform embedding operation on the feature representation in the frequency domain to generate the final frequency domain features; Combine the encoded features and the final frequency domain features to obtain the final input features; Step 5: Input the final input features into the time-frequency feature fusion module for mechanism processing to obtain the output center frame; The local details and global information focused by the diffusion model are improved based on the output center frame.

2. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 1 is characterized in that: In step 3, the initial 3D human body posture is input into the diffusion model, and the training is performed in an iterative form in combination with noise to obtain the target 3D human body posture, which specifically includes the following steps: The initial 3D human body posture is input into the diffusion model to give a training sample, and the training sample is obtained. Noise is added to the training sample for forward propagation to obtain the network estimation distribution of the noise, and the 3D human body posture with noise is obtained at the end of the forward propagation. Based on the time-frequency feature fusion module, the noisy 3D human posture combined with the network estimated distribution of the noise is denoised, and the target 3D human posture is obtained after denoising; Noise is re-added to the target 3D human pose and the process is repeated in an iterative manner to complete the denoising.

3. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 2 is characterized in that: In the step of inputting the initial 3D human body posture into the diffusion model to give a training sample, obtaining the training sample, adding noise to the training sample for forward propagation to obtain the network estimation distribution of the noise, and obtaining the 3D human body posture with noise at the end of the forward propagation, the corresponding relationship of the process is as follows: ； in, represents the network estimated distribution of noise, represents a Gaussian distribution, Indicates at time 3D human pose with noise, Indicates at time Varying noise variance schedule, represents the matrix unit; In the step of denoising the noisy 3D human posture combined with the noisy network estimation distribution based on the time-frequency feature fusion module, the corresponding relationship of the process is as follows: ； in, represents the conditional probability distribution, represents the covariance predicted by the time-frequency feature fusion module, Represents the mean value predicted by the time-frequency feature fusion module.

4. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 3 is characterized in that: In step 4, the initial 3D human body posture is recognized and processed to obtain the long sequence data of the 3D human body skeleton. The corresponding relationship of the process is as follows: ； in, Represents a given input sequence, Represents a collection of long sequence data, represents the sequence length, represents the number of joints, and 3 represents the 3D coordinates of each joint.

5. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 4 is characterized in that: In the step of performing discrete cosine transform operation on long sequence data of 3D human skeleton combined with noise to obtain frequency domain features, the corresponding relationship of the process is as follows: ； in, represents the discrete cosine transform coefficient of each joint, represents the frequency domain index, represents the joint index, It means that after the cosine function processing, represents pi, Represents a time index.

6. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 5, characterized in that: In the step of performing low-pass filtering and information extraction operations on the frequency domain features in sequence to obtain the frequency domain features that retain the information, the corresponding relationship in the process is as follows: ； in, represents the frequency domain features that retain information, Indicates The joint Discrete cosine transform coefficients.

7. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 6, characterized in that: In the step of performing a linear projection operation on the frequency domain features of the retained information to obtain the feature representation of the frequency domain, and performing an embedding operation on the feature representation of the frequency domain to generate the final frequency domain features, the corresponding relationship of the process is as follows: ； in, represents the final frequency domain features, Represents the frequency domain position embedding operation.

8. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 7, characterized in that: In the step of combining the coding features and the final frequency domain features to obtain the final input features, the corresponding relationship between the process is as follows: ； in, represents the final input features, Represents the encoding feature.

9. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 8, characterized in that: In step 5, the final input feature is input into the time-frequency feature fusion module for mechanism processing to obtain the output center frame, which specifically includes the following sub-steps: The final input features are input to the normalization layer of the time-frequency feature fusion module for processing to obtain the normalization layer processing features; The features processed by the normalization layer are processed by a multi-head self-attention mechanism to obtain self-attention features; Among them, the self-attention feature is obtained, and the corresponding relationship of the process is as follows: ； in, represents the self-attention feature, It means that it has been processed by the self-attention mechanism. Indicates Normalization processing features of the layer; The self-attention features and the normalization layer processing features are sequentially subjected to residual connection processing and normalization processing to obtain residual normalized features; Among them, the residual normalization feature is obtained, and the corresponding relationship of the process is as follows: ； in, represents the residual normalized feature, Indicates that it has been processed by layer normalization; The residual normalized features are segmented to obtain the features in the time domain and the features in the frequency domain; Among them, the characteristics of the time domain and the characteristics of the frequency domain are obtained, and the corresponding relationship between the process is as follows: ； in, Indicates The temporal characteristics of the layer, Indicates The frequency domain characteristics of the layer, Represents the previous step of taking residual normalized features elements, Represents the first step of taking residual normalization features The part that can be determined by an element; Input the time domain features into the frequency domain multilayer perceptron for mechanism processing to obtain the time domain output features; Input the frequency domain features into a standard multi-layer perceptron for processing to obtain frequency domain output features; Perform a splicing operation on the time domain output features and the frequency domain output features to obtain a splicing feature; Among them, the splicing features are obtained, and the corresponding relationship in the process is as follows: ； in, Represents the concatenation of the time domain output features and the frequency domain output features, Represents a splicing operation, It means that it has been processed by the frequency domain multilayer perceptron. Indicates that it has been processed by a standard multi-layer perceptron; The splicing features are regressed through a normalization layer to obtain a regression result, and the regression result is subjected to an inverse discrete cosine transform operation to obtain an output center frame; Among them, the output center frame is obtained, and the corresponding relationship of the process is as follows: ； in, Indicates the output center frame.

10. The 3D human posture estimation method based on diffusion model and continuous frame time-frequency information fusion according to claim 9, characterized in that: The time domain features are input into the frequency domain multilayer perceptron for mechanism processing to obtain the time domain output features, which specifically includes the following sub-steps: The time domain features are input into the frequency domain multilayer perceptron for discrete cosine transform operation to obtain the decomposed frequency domain features; Among them, the decomposed frequency domain characteristics are obtained, and the corresponding relationship in the process is as follows: ； in, represents the decomposed frequency domain features, Represents a discrete cosine transform operation; Perform time embedding initialization operation on the decomposed frequency domain features to obtain time embedded features; Among them, the characteristics of time embedding are obtained, and the corresponding relationship of the process is as follows: ； in, represents the time step embedding initialization, Indicates passing Function calculation processing, Indicates location, represents the embedding dimension, represents the embedding dimension index; The time-embedded features are sequentially subjected to time-embedded linear transformation operations, two-layer linear layer processing, and nonlinear layer activation function processing to obtain features processed by the nonlinear layer activation function; Among them, the characteristics of the nonlinear layer activation function processing are obtained, and the corresponding relationship of the process is as follows: ； in, represents the first linear layer, represents the dimension of the hidden layer, It means that it has been processed by the activation function. represents the second linear layer; Generate adjustment parameters using features processed by nonlinear layer activation functions; Among them, the relationship between the process of generating adjustment parameters and the existing ones is as follows: ； in, represents the third linear change, represents the translation parameter, represents the scaling parameter, Indicates splitting a tensor into a preset number of smaller tensors along the specified dimension; The frequency domain feature calculation is adjusted using adjustment parameters to obtain the output feature of the frequency domain rate; Among them, the output characteristics of the frequency domain rate are obtained, and the corresponding relationship of the process is as follows: ； in, Represents the adjusted frequency domain features.

Images