Human 3D skeleton extraction method with local-to-global spatio-temporal feature enhancement under low-light conditions

Abstract

The application discloses a human body 3D skeleton extraction method with local-to-global spatiotemporal feature enhancement under low light, and belongs to the technical field of computer vision. A local perception spatiotemporal robust modulation module is constructed, local time sequence motion and skeleton structure consistency modeling is performed on a skeleton sequence, and feature reliability evaluation and multiplicative modulation are completed through a double-gated adaptive mechanism; a global time sequence consistency distillation module is constructed, a teacher feature is constructed by adopting an exponential weighted moving average, and noise propagation is inhibited by combining time difference consistency constraints and confidence perception modulation; the two modules are embedded into a time sequence modeling system to realize adaptive re-labeling and smoothing of unstable time sequences and structure information. The application can effectively inhibit the noise, jitter and drift of skeleton features under low light, significantly improve the stability, continuity and structure rationality of 3D skeleton extraction in dark light scenes, and is suitable for complex low-light application scenarios such as night monitoring and dark light interaction.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low light conditions. Background Technology

[0002] 3D human skeleton extraction is a key technology in computer vision and deep learning, widely used in scenarios such as nighttime surveillance, vehicle sensing, and low-light human-computer interaction. Low-light environments lead to a decrease in image signal-to-noise ratio, blurring of joint features, and a surge in detection noise, causing significant jitter, drift, and structural distortion in skeleton sequences. Noise also propagates over time, severely reducing the stability and accuracy of 3D reconstruction. Existing methods largely rely on pose modeling assumptions under normal lighting conditions, failing to introduce explicit spatiotemporal constraints for low-light environments, making it difficult to simultaneously suppress local noise and maintain global motion consistency.

[0003] Current 3D skeleton extraction methods mostly employ graph convolution and Transformer architectures, enhancing pose representation capabilities through spatial topology modeling and temporal dependency learning. Some methods improve pose continuity through temporal smoothing or attention mechanisms, but when low illumination causes unreliable input joints, they are prone to misinterpreting noise as real motion, exacerbating pose jitter. Other methods attempt to integrate image enhancement preprocessing and pose estimation, but fail to establish robust modulation mechanisms at the skeleton feature layer, thus failing to fundamentally address the problems of feature distortion and error accumulation under low illumination.

[0004] Academic research has focused on robust pose estimation in low-light conditions. The ExLPose method improves 2D pose robustness under extremely low illumination through illumination-adaptive batch normalization and knowledge distillation; the Frequency Decoupling Framework enhances low-light texture features through dynamic illumination correction and high-frequency denoising; and the MHFCL framework utilizes Multi-granularity High-frequency Consistency Learning to recover low-light details. However, these methods primarily focus on image or 2D pose enhancement and do not incorporate local spatiotemporal robust modulation and global consistency distillation in the 3D skeleton stage, resulting in insufficient stability of the 3D skeleton in complex dynamic scenes.

[0005] Chinese patent CN113537393B (A 3D Human Pose Estimation Algorithm for Dark Scenes Based on Improved Transformer) improves pose modeling performance in low light by optimizing the Transformer structure; patent CN112560796B (Real-time Human Pose Detection Method) uses multi-scale feature fusion to improve the robustness of low-light detection. While these patents improve pose extraction in low-light environments, none of them design a dual-gated local modulation and exponential moving average global distillation mechanism, thus failing to achieve accurate noise suppression and ensure global temporal consistency. Summary of the Invention

[0006] Based on the aforementioned technical problems, this application discloses a method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low light conditions, specifically including:

[0007] S1: Construct a local perception spatiotemporal robust modulation module. By modeling the consistency between local temporal motion and skeleton structure, a dual-gated adaptive mechanism is used to evaluate the reliability of features and perform multiplicative modulation.

[0008] S2: Construct a global time-series consistency distillation module, use exponential weighted moving average to construct teacher features, and suppress noise propagation in the global time dimension through time difference consistency constraints and confidence-aware modulation;

[0009] S3: The local sensing spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded into the temporal modeling system to achieve adaptive recalibration and smoothing of unstable temporal and structural information at the feature layer.

[0010] Preferably, in step S1, a locally perceptive spatiotemporally robust modulation module is constructed. By modeling the consistency between local temporal motion and skeleton structure, a dual-gated adaptive mechanism is used to evaluate the reliability of features and perform multiplicative modulation. Specifically, this includes:

[0011] S100: Given the input skeleton sequence : ,in For the first Frame skeleton data, For time frames, The number of joints, For joint feature dimensions; the actual input Represented as: ,in, This is the noise term.

[0012] S101: Calculate the first-order and second-order velocities through local temporal motion modeling; where the first-order velocity... for: The second-order velocity is : ;

[0013] By concatenating displacement, velocity, and acceleration at the feature level, an enhanced local time series is obtained. The formula is: And through multilayer perceptron The formula is: ;

[0014] S102: Calculate the average joint value of the skeleton in each frame by modeling the consistency of the skeleton structure. The formula is: ,in For the first Each skeleton data point; each joint feature is represented as an offset relative to the center of the overall structure. The formula is: Through multilayer perceptron The formula is: ;

[0015] S103: The time-dimensional gating is calculated separately through a dual-gated adaptive modulation mechanism. With structural dimension gating Time-dimensional gating The formula is: Structural dimension gating The formula is: ,in, It is a fully connected layer. Using the Sigmoid activation function, the joint modulation coefficients are obtained through multiplicative fusion. : The original input features are subjected to element-wise weighted modulation to obtain the locally modulated enhanced version of the first feature. Frame skeleton features The formula is: .

[0016] Preferably, in S2, a global temporal consistency distillation module is constructed, which uses an exponentially weighted moving average to construct teacher features and suppresses noise propagation in the global time dimension through temporal difference consistency constraints and confidence-aware modulation. Specifically,

[0017] S200: Given the input sequence: Linear projection of the input features: in, Represents a linear mapping;

[0018] S201: Constructing Teacher Characteristics Using Exponentially Weighted Moving Average The formula is: ,in, The momentum coefficient;

[0019] S202: Calculate the feature difference between two adjacent frames , The formula is: , obtain the time correction term The formula is: ;

[0020] S203: Calculate the confidence value for each time step using a fully connected layer. The formula is: ,in, It is a nonlinear mapping. Using the Sigmoid activation function, the temporal correction term is multiplied by the confidence value and then applied to the original features to obtain the modulated features. : .

[0021] Preferably, in step S3, the local sensing spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded in the temporal modeling system. The two work together to eliminate noise in the local time range while maintaining feature consistency in the global time dimension, thereby achieving adaptive recalibration and smoothing of unstable temporal and structural information.

[0022] Compared with the prior art, the technical solution of this application has the following technical effects:

[0023] This invention constructs a locally perceptive spatiotemporally robust modulation module to model the displacement, velocity, and acceleration features of the input skeleton sequence, generating temporal and structural perceptive features adapted to low-light scenarios. By generating joint modulation coefficients through a dual-gating adaptive mechanism, the original features are weighted and modulated element by element, effectively suppressing local noise and jitter caused by low light and improving the local stability of the skeleton sequence under complex dark light conditions.

[0024] This invention uses a global temporal consistency distillation module to construct teacher features using an exponentially weighted moving average, calculates feature differences to obtain temporal correction terms, and combines confidence-perceived weighting to complete feature correction. This can suppress noise propagation and error accumulation in the global time dimension, ensure the motion continuity of the skeleton sequence over a long period of time, and reduce attitude drift problems caused by error accumulation in low-light environments.

[0025] This invention embeds a local sensing spatiotemporal robust modulation module and a global temporal consistency distillation module into a temporal modeling system to achieve collaborative work of local noise elimination and global consistency maintenance. Through adaptive recalibration and smoothing of the feature layer, unstable temporal and structural information is corrected, which can effectively improve the overall performance of the skeleton sequence in low-light occlusion and fast motion scenarios and enhance the structural rationality of the posture results.

[0026] This invention achieves refined processing and consistency assurance of skeleton features in low-light environments through coordinated constraints of local and global dimensions. It can complete noise suppression and error correction without additional supervision signals, significantly improving the overall robustness of the 3D skeleton extraction results. It provides stable and reliable technical support for human posture analysis in complex low-light scenes and meets the actual needs of various low-light application scenarios.

[0027] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0028] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0030] Based on the description of the figures and their corresponding technical content in the document, the titles of the figures are as follows:

[0031] Figure 1 A schematic diagram of the process for extracting a 3D human skeleton with enhanced spatiotemporal features from local to global under low light conditions;

[0032] Figure 2 A schematic diagram of a skeleton extraction network architecture for enhancing local to global spatiotemporal features under low-light conditions;

[0033] Figure 3 A schematic diagram of the dual-gated adaptive feature processing structure of the local sensing spatiotemporal robust modulation module;

[0034] Figure 4 A schematic diagram of the exponential weighting and confidence-aware correction structure of the global time-series consistency distillation module;

[0035] Figure 5 A schematic diagram comparing the skeleton extraction performance of the method in this application with existing methods under low light conditions. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0037] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0038] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0039] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0040] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0041] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0042] Example 1

[0043] This embodiment mainly describes a method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low light conditions, such as... Figure 1 As shown, it specifically includes:

[0044] S1: Construct a local perception spatiotemporal robust modulation module. By modeling the consistency between local temporal motion and skeleton structure, a dual-gated adaptive mechanism is used to evaluate the reliability of features and perform multiplicative modulation.

[0045] S2: Construct a global time-series consistency distillation module, use exponential weighted moving average to construct teacher features, and suppress noise propagation in the global time dimension through time difference consistency constraints and confidence-aware modulation;

[0046] S3: The local sensing spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded into the temporal modeling system to achieve adaptive recalibration and smoothing of unstable temporal and structural information at the feature layer.

[0047] like Figure 2 As shown, by constructing a local-aware spatiotemporal robust modulation module and a global temporal consistency distillation module, and embedding them into the temporal modeling system, adaptive enhancement of skeleton sequences under low-light imaging conditions is achieved. The local-aware spatiotemporal robust modulation module models the consistency between local temporal motion and skeleton structure, and uses a dual-gated adaptive mechanism to evaluate the reliability of features and perform multiplicative modulation. This effectively suppresses local jitter and structural deviations caused by illumination degradation, occlusion, or detection errors at the feature layer. The global temporal consistency distillation module constructs teacher features using an exponentially weighted moving average. Through temporal difference consistency constraints and confidence-aware modulation, it suppresses noise propagation in the global time dimension and avoids the accumulation and amplification of errors along the time axis. Through local and global dual constraints, the stability and robustness of skeleton sequences are improved without adding additional supervision, thereby improving the accuracy and reliability of 3D human pose estimation under complex imaging environments such as low light and fast motion.

[0048] Furthermore, a locally sensing spatiotemporally robust modulation module is constructed in S1, such as... Figure 3 As shown, by modeling the consistency between local temporal motion and skeleton structure, a dual-gated adaptive mechanism is used to evaluate the reliability of features and perform multiplicative modulation, specifically including:

[0049] S100: Given the input skeleton sequence : ,in For the first Frame skeleton data, For time frames, The number of joints, For joint feature dimensions; the actual input Represented as: ,in, This is the noise term.

[0050] S101: Calculate the first-order and second-order velocities through local temporal motion modeling; where the first-order velocity... for: The second-order velocity is : ;

[0051] By concatenating displacement, velocity, and acceleration at the feature level, an enhanced local time series is obtained. The formula is: And through multilayer perceptron The formula is: ;

[0052] S102: Calculate the average joint value of the skeleton in each frame by modeling the consistency of the skeleton structure. The formula is: ,in For the first Each skeleton data point; each joint feature is represented as an offset relative to the center of the overall structure. The formula is: Through multilayer perceptron The formula is: ;

[0053] S103: The time-dimensional gating is calculated separately through a dual-gated adaptive modulation mechanism. With structural dimension gating Time-dimensional gating The formula is: Structural dimension gating The formula is: ,in, It is a fully connected layer. Using the Sigmoid activation function, the joint modulation coefficients are obtained through multiplicative fusion. : The original input features are subjected to element-wise weighted modulation to obtain the locally modulated enhanced version of the first feature. Frame skeleton features The formula is: .

[0054] Furthermore, a global time-consistent distillation module is constructed in S2, such as... Figure 4 As shown, teacher features are constructed using an exponentially weighted moving average. Noise propagation is suppressed in the global time dimension through temporal difference consistency constraints and confidence-aware modulation. Specifically:

[0055] S200: Given the input sequence: Linear projection of the input features: in, Represents a linear mapping;

[0056] S201: Constructing Teacher Characteristics Using Exponentially Weighted Moving Average The formula is: ,in, The momentum coefficient;

[0057] S202: Calculate the feature difference between two adjacent frames , The formula is: , obtain the time correction term The formula is: ;

[0058] S203: Calculate the confidence value for each time step using a fully connected layer. The formula is: ,in, It is a nonlinear mapping. Using the Sigmoid activation function, the temporal correction term is multiplied by the confidence value and then applied to the original features to obtain the modulated features. : .

[0059] Furthermore, in S3, the local perceptual spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded into the temporal modeling system. The two work together to eliminate noise in the local time range while maintaining feature consistency in the global time dimension, thereby achieving adaptive recalibration and smoothing of unstable temporal and structural information.

[0060] This implementation constructs a local sensing spatiotemporally robust modulation module and a global temporal consistency distillation module to achieve local noise suppression and global temporal smoothing of skeleton sequences under low light conditions. This effectively improves joint jitter, structural distortion and error accumulation, enhances the continuity and structural rationality of posture sequences, and improves the robustness and stability of 3D skeleton extraction in low light scenes, providing technical support for human posture perception in dark environments.

[0061] Based on Example 1, this example details the verification of a method for extracting a 3D human skeleton with enhanced local-to-global spatiotemporal features under low-light conditions. Training was performed on a high-performance workstation equipped with an NVIDIA Quadro RTX 8000 graphics card using the PyTorch framework. A horizontal flipping data augmentation strategy was employed during both training and testing phases to improve the model's generalization ability. The training mini-batch size was set to 16 action sequences. Model parameters were optimized using the AdamW optimization algorithm, with a total of 80 training epochs and a weight decay coefficient of 0.01. The initial learning rate was... The model is gradually adjusted using a linear decay strategy with an exponential decay rate of 0.99. For the Human3.6M dataset, 2D keypoints predicted by the stacked hourglass network and 2D ground truth labeled data are used simultaneously during training; for the MPI-INF-3DHP dataset, 2D ground truth keypoints are used as model input according to the established evaluation criteria.

[0062] like Figure 5 As shown in Table 1, the proposed LTLGSTE model (input sequence length T=243) is compared with current mainstream methods on the Human3.6M dataset. Using CPN as a 2D pose detector and the predicted 2D pose as input, LTLGSTE achieves an average error of 38.2 mm under Protocol 1 and 32.0 mm under Protocol 2. Specifically, LTLGSTE achieves the best results in 5 of the 15 pose categories under Protocol 1, and performs best in 6 of the 15 categories under Protocol 2. Table 2 presents the quantitative comparison results based on real 2D pose input under Protocol 1. LTLGSTE achieves excellent results in all 15 action categories with an average error of 16.8 mm, fully validating its effectiveness.

[0063] Table 1. Quantitative comparison of 2D joints as input based on CPN estimation under Protocol #1 and Protocol #2.

[0064] Protocol#1 Dir. Disc Eat Greet Phone Photo Pose Purch. Sit SitD. Smoke Wait WalkD. Walk WalkT. Avg HSTFormer (arXiv’23) 39.5 42.0 39.9 40.8 44.4 50.9 40.9 41.3 54.7 58.8 43.6 40.7 43.4 30.1 30.4 42.7 HDFormer (IJCAI’23) 38.1 43.1 39.3 39.4 44.3 49.1 41.3 40.8 53.1 62.1 43.3 41.8 43.1 31.0 29.7 42.6 GLA - GCN (ICCV’23) 41.3 44.3 40.8 41.8 45.9 54.1 42.1 41.5 57.8 62.9 45.0 42.8 45.9 29.4 29.9 44.4 HopFIR (ICCV’23) 43.9 47.6 45.5 48.9 50.1 58.0 46.2 44.5 55.7 62.9 49.0 45.8 51.8 38.0 39.9 48.5 ConvFormer (TVC’24) 41.0 43.2 39.0 42.4 44.5 52.2 41.7 40.8 53.0 60.6 44.8 41.3 43.7 29.6 30.9 43.2 GLSTE (TMM’24) 38.1 41.7 39.1 39.5 43.7 49.7 40.4 39.7 54.2 58.7 42.8 42.1 42.0 28.7 28.5 41.9 KTPFormer (CVPR’24) 37.3 39.2 35.9 37.6 42.5 48.2 38.6 39.0 51.4 55.9 41.6 39.0 40.0 27.0 27.4 40.1 CrossFormer (SIVP’25) 40.7 44.1 40.8 41.5 45.8 52.8 41.2 40.8 55.3 61.9 44.9 41.8 44.6 29.2 31.1 43.7 Hipart (CVPR’25) 42.8 42.7 38.1 41.3 42.7 46.3 37.2 44.2 51.0 51.4 40.9 38.3 40.0 39.9 33.7 42.0 LTLGSTE (Ours) 36.0 38.7 36.3 33.1 40.3 46.7 37.5 34.8 51.0 54.7 39.8 36.9 25.5 35.8 26.0 38.2 Protocol#2 Dir. Disc Eat Greet Phone Photo Pose Purch. Sit SitD. Smoke Wait WalkD. Walk WalkT. Avg HSTFormer (arXiv’23) 31.1 33.7 33.0 33.2 33.6 38.8 31.9 31.5 43.7 46.3 35.7 31.5 33.1 24.2 24.5 33.7 HDFormer (IJCAI’23) 29.6 33.8 31.7 31.3 33.7 33.7 30.6 31.0 41.4 47.6 35.0 30.9 33.7 25.3 25.6 33.2 UpliftAndUp (WACV’23) 31.6 33.7 31.8 33.3 34.7 38.7 32.2 31.2 41.9 48.9 35.5 32.6 33.7 23.4 24.0 33.8 ConvFormer (TVC’24) 31.4 34.2 32.0 35.2 34.0 40.3 32.7 31.3 42.6 49.0 36.2 31.3 34.8 23.4 24.9 34.2 GLSTE (TMM’24) 30.8 34.6 32.4 32.9 34.0 39.5 31.5 31.0 44.2 48.7 35.2 32.4 33.5 23.1 23.5 33.8 KTPFormer (CVPR’24) 30.1 32.3 29.6 30.8 32.3 37.3 30.0 30.2 41.0 45.3 33.6 29.9 31.4 21.5 22.6 31.9 CrossFormer (SIVP’25) 31.4 34.6 32.6 33.7 34.3 39.7 31.6 31.0 44.3 49.3 35.9 31.3 34.4 23.4 25.5 34.3 Hipart (CVPR’25) 30.4 29.7 26.3 27.2 28.7 29.1 28.2 29.2 30.9 33.1 29.6 26.2 27.2 21.9 26.2 28.3 LTLGSTE (Ours) 29.7 31.7 31.2 27.9 33.1 37.0 30.7 30.0 43.0 47.7 34.5 29.8 21.8 30.2 22.7 32.0

[0065] Table 2 shows a quantitative comparison of 2D pose based on ground truth (GT) as input under Protocol #1.

[0066] Protocol #1 Dir. Disc Eat Greet Phone Photo Pose Purch. Sit SitD. Smoke Wait WalkD. Walk WalkT. Avg MixSTE (CVPR’22) 21.6 22.0 20.4 21.0 20.8 24.3 24.7 21.9 26.9 24.9 21.2 21.5 20.8 14.7 15.7 21.6 Stridedformer (TMM’22) 27.1 29.4 26.5 27.1 29.6 33.0 30.7 26.8 38.2 34.7 29.1 29.8 26.8 19.1 19.8 28.5 GLA - GCN (ICCV’23) 26.5 27.2 29.2 25.4 28.2 31.7 29.5 26.9 37.8 29.9 29.9 27.0 27.3 20.5 20.8 28.5 HopFIR (ICCV’23) 31.3 34.0 28.0 32.0 33.1 42.1 34.1 28.1 33.6 39.8 31.7 32.9 33.8 26.7 28.9 32.7 HSTFormer (arXiv’23) 24.9 27.4 28.1 25.9 28.2 33.5 28.9 26.8 33.4 38.2 27.2 26.7 27.1 20.4 20.8 27.8 ConvFormer (TVC’24) 28.9 31.8 28.0 28.2 29.5 33.0 32.9 30.1 36.8 37.4 29.8 29.6 28.2 21.7 30.9 43.2 GLSTE (TMM’24) 27.8 29.5 27.1 26.5 27.4 31.0 30.8 27.5 31.9 35.0 28.2 28.5 27.0 20.3 20.3 27.9 KTPFormer (CVPR’24) 19.6 18.6 18.5 18.1 18.7 22.1 20.8 18.3 22.8 22.4 18.8 18.1 18.4 13.9 15.2 19.0 CrossFormer (SIVP’25) 26.0 30.0 26.8 26.2 28.0 31.0 30.4 29.6 35.4 37.1 28.4 27.3 26.7 20.5 19.9 28.3 LTLGSTE (Ours) 15.8 17.3 17.4 15.5 17.2 18.4 18.1 18.3 21.3 22.6 16.8 16.3 10.8 16.2 11.5 16.8

[0067] The experimental results above show that the proposed method can maintain stable skeleton extraction performance on various standard test sets, effectively suppress feature noise and temporal jitter caused by low-light environments, and significantly improve the smoothness and structural rationality of the 3D skeleton sequence through the synergistic effect of local temporal control and global consistency distillation. This enables the model to still have strong robustness and generalization ability under complex dynamic actions and low-light imaging conditions, and can stably output continuous, accurate and structurally reliable human 3D skeleton results.

[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter changes made to these embodiments within the spirit and principles of the present invention, without departing from the principles and spirit of the present invention, through conventional substitutions or to achieve the same function, fall within the scope of protection of the present invention.

Claims

1. A method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low light conditions, characterized in that, include: S1: Construct a local perception spatiotemporal robust modulation module. By modeling the consistency between local temporal motion and skeleton structure, a dual-gated adaptive mechanism is used to evaluate the reliability of features and perform multiplicative modulation. S2: Construct a global time-series consistency distillation module, use exponential weighted moving average to construct teacher features, and suppress noise propagation in the global time dimension through time difference consistency constraints and confidence-aware modulation; S3: The local sensing spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded into the temporal modeling system to achieve adaptive recalibration and smoothing of unstable temporal and structural information at the feature layer.

2. The method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low illumination as described in claim 1, characterized in that, The S1 module constructs a local sensing spatiotemporal robust modulation module, which models the consistency between local temporal motion and skeleton structure, and uses a dual-gated adaptive mechanism to evaluate the reliability of features and perform multiplicative modulation. Specifically, it includes: S100: Given the input skeleton sequence : ,in For the first Frame skeleton data, For time frames, The number of joints, For joint feature dimensions; the actual input Represented as: ,in, This is the noise term. S101: Calculate the first-order and second-order velocities through local temporal motion modeling; where the first-order velocity... for: The second-order velocity is : ; By concatenating displacement, velocity, and acceleration at the feature level, an enhanced local time series is obtained. The formula is: And through multilayer perceptron The formula is: ; S102: Calculate the average joint value of the skeleton in each frame by modeling the consistency of the skeleton structure. The formula is: ,in For the first Each skeleton data point; each joint feature is represented as an offset relative to the center of the overall structure. The formula is: Through multilayer perceptron The formula is: ; S103: The time-dimensional gating is calculated separately through a dual-gated adaptive modulation mechanism. With structural dimension gating Time-dimensional gating The formula is: Structural dimension gating The formula is: ,in, It is a fully connected layer. Using the Sigmoid activation function, the joint modulation coefficients are obtained through multiplicative fusion. : The original input features are subjected to element-wise weighted modulation to obtain the locally modulated enhanced version of the first feature. Frame skeleton features The formula is: .

3. The method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low illumination as described in claim 2, characterized in that, The global temporal consistency distillation module is constructed in S2. It uses an exponentially weighted moving average to construct teacher features and suppresses noise propagation in the global time dimension through temporal difference consistency constraints and confidence-aware modulation. Specifically, S200: Given the input sequence: Linear projection of the input features: in, Represents a linear mapping; S201: Constructing Teacher Characteristics Using Exponentially Weighted Moving Average The formula is: ,in, The momentum coefficient; S202: Calculate the feature difference between two adjacent frames , The formula is: , obtain the time correction term The formula is: ; S203: Calculate the confidence value for each time step using a fully connected layer. The formula is: ,in, It is a nonlinear mapping. Using the Sigmoid activation function, the temporal correction term is multiplied by the confidence value and then applied to the original features to obtain the modulated features. : .

4. The method for extracting a 3D human skeleton with enhanced local to global spatiotemporal features under low illumination as described in claim 1, characterized in that, In S3, the local sensing spatiotemporal robust modulation module and the global temporal consistency distillation module are embedded in the temporal modeling system. The two work together to eliminate noise in the local time range while maintaining feature consistency in the global time dimension, thereby achieving adaptive recalibration and smoothing of unstable temporal and structural information.

Images