Load-bearing self-adaptive adjusting system and method based on posture recognition

Abstract

The invention discloses a weight self-adaptive adjusting system and method based on posture recognition, and particularly relates to the technical field of man-machine cooperation and wearable robotics.The method comprises the steps that firstly, multi-modal data are obtained through an IMU and a visual camera device, and PTP clock synchronization, coordinate system calibration and Kalman filtering processing are completed; then, the system measures the action rhythm and the sensing credibility according to indexes such as KLE, LMPL, BDR and SAC, self-adaptive switching is conducted between a constant window sliding window mechanism and a variable window sliding window mechanism, and the window length is calculated in combination with the rate level under the variable window mechanism; the time sequence modeling adopts an LSTM / GRU network supporting variable-length input and a mask mechanism, and outputs an attitude evolution trend, load response pre-judgment and sudden action risks; and the control end generates a feed-forward adjustment instruction of multi-actuator cooperation based on a prediction result, and realizes self-correction and individualized parameter self-adaption through feedback monitoring.

Description

Technical Field

[0001] The present invention relates to the field of human-machine collaboration and wearable robot technology, and more specifically, to a load-bearing adaptive adjustment system and method based on posture recognition. Background Art

[0002] As demands for human-machine collaboration and human safety continue to rise in industrial production, logistics, and rehabilitation medicine, exoskeleton-based load-bearing assistive devices are rapidly developing. Existing load-bearing adjustment systems typically rely on a single sensory modality, such as an inertial measurement unit (IMU), pressure sensor, or visual camera, to acquire human motion information. This information is then fed into a fixed-length sliding window, which then feeds a timing model to output assistance commands.

[0003] Existing exoskeletons or load-assisted systems often rely on a single modality for posture recognition. This is limited by occlusion, drift, and clock asynchrony, resulting in unstable perception results. Furthermore, the common fixed-length sliding window input can cause prediction lag during rapid posture changes and introduce redundant history during slow, stable movements, leading to poor generalization of models for behaviors of varying tempos, frequent misadjustments, and output jitter. Furthermore, some methods fail to quantitatively model the coupling relationship, hysteresis, and burstiness between load signals and kinematic responses, and lack online evaluation of multimodal fusion consistency, making it difficult to adjust input timescales and control strategies in a timely and reliable manner. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a system and method for adaptively adjusting weight based on posture recognition to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: The method for adaptively adjusting load based on posture recognition includes the following steps: The inertial measurement unit and visual camera equipment synchronously collect three-axis acceleration, three-axis angular velocity and skeleton image information under a unified clock. Through spatial calibration and filtering, interpolation and normalization processing, the skeleton key points, joint kinematics and load signals are fused into frame-level multimodal features and written into the memory ring buffer. Then, four dynamic indicators, namely KLE, LMPL, BDR and SAC, are calculated in the memory ring buffer. Based on the four dynamic indicators and the hysteresis rules of the preset Th threshold and Td threshold, the constant window timing encoding mechanism and the variable window timing control mechanism are switched; If the variable window timing control mechanism is adopted, the window length is determined according to the rate level output by the rate level discriminator, the BDR value, the load disturbance intensity, and the SAC value; if the constant window timing coding mechanism is adopted, a fixed window length is used; The multimodal feature sequence of corresponding length is then input into an LSTM or GRU network that supports variable-length input and masking mechanisms to obtain predictions of future posture evolution, load changes, and the risk of sudden movements. Finally, based on this prediction, a control vector containing the target adjustment position, direction, amplitude, and duration is generated to drive the exoskeleton actuator to implement assistance or buffering adjustments in advance, and the real-time feedback from the actuator is used to self-correct the control strategy.

[0006] In a preferred embodiment, the calculation of the KLE includes: constructing the instantaneous power sequence P of the main joint angular velocity and the load sensor output sequence F in a time window of length L frames; discretizing the sequences P and F into bins, setting N power intervals and M load intervals respectively, and counting the number of frames in which each pair of bins (i, j) appears simultaneously to form an N×M dimensional joint probability matrix ; Further calculate the marginal probability of the power sequence and the marginal probability of the load sequence, and obtain the mutual information value of the two sequences according to the mutual information formula; finally, take the minimum entropy of sequences P and F as the benchmark, and obtain KLE by performing minimum entropy normalization.

[0007] In a preferred embodiment, the calculation of the LMPL includes constructing a lumbar vertical pressure signal sequence and a pelvic vertical velocity signal sequence in a time window of length L frames; performing a short-time Fourier transform on the lumbar vertical pressure signal sequence and the pelvic vertical velocity signal sequence to obtain a spectrum that varies with time. and ; Further calculate the cross spectral density of the two spectra ; Select Maximum resonant frequency , extract the phase difference at this frequency ; and according to the formula: The average time lag Δt is obtained, and the Δt is the LMPL value.

[0008] In a preferred embodiment, the BDR is determined by identifying the jerk peak of the main joint angular velocity and establishing a reference with the basal metabolic level, and further measuring its time series distribution density.

[0009] In a preferred embodiment, the jerk peak is the sudden increase point of the angular acceleration change rate; and the basal metabolic level is the minimum jerk activity level of a person in a state of rest or slow, steady movement.

[0010] In a preferred embodiment, the SAC is a dynamic indicator for evaluating the fusion consistency between the visual sensor and the inertial sensor in real time.

[0011] In a preferred embodiment, the weighted moving average of the most recent KLE, LMPL, BDR, and SAC is read; then the comprehensive variable window score is calculated based on the weighted sum. ; By presetting two thresholds, Th (high threshold) and Td (low threshold); when When Th is exceeded, a variable window timing control mechanism is adopted; Lower than Td, maintain or switch to the constant window timing coding mechanism; if Falling between Th and Td, choose the mechanism that keeps what was already running in the previous frame.

[0012] In a preferred embodiment, the rate level discriminator is an experience-data hybrid driven rate level discriminator, which comprehensively maps the features in the current time window to discrete action rate levels; the rate level discriminator can be defined using a threshold map, a multidimensional rule tree, or a fuzzy logic model based on experience optimization.

[0013] In a preferred embodiment, the rate level corresponds to a sliding window candidate interval; a weighted compression mechanism is used to linearly weighted fuse the BDR, load disturbance intensity, SAC and rate level to finally output the window length.

[0014] In a preferred embodiment, the perception and feature construction module is used to collect three-axis acceleration, three-axis angular velocity and skeleton image information, and fuse skeleton key points, joint kinematics and load signals into frame-level multimodal features and write them into a memory ring buffer; The sliding window mechanism and rhythm control module is used to calculate the four dynamic indicators of KLE, LMPL, BDR and SAC. Based on the above four dynamic indicators and the hysteresis rules of the preset Th threshold and Td threshold, it switches between the constant window timing encoding mechanism and the variable window timing control mechanism. The sliding window mechanism and rhythm control module includes a window length calculation module; the window length calculation module determines the window length according to the rate level output by the rate level discriminator, the BDR value, the load disturbance intensity, and the SAC value; if the constant window timing coding mechanism is adopted, a fixed window length is adopted; The intelligent adjustment and feedback control module is used to input multimodal feature sequences of corresponding length into an LSTM or GRU network that supports variable-length input and masking mechanisms to obtain predictions of future posture evolution, load changes, and risks of sudden movements. Finally, based on the prediction, a control vector containing the adjustment target position, direction, amplitude, and duration is generated, and the control strategy is self-corrected using real-time feedback from the actuator. Dynamic adjustment operations are performed based on the prediction results, and the control effect is continuously optimized through the feedback mechanism.

[0015] Technical effects and advantages of the present invention: The present invention unifies PTP clock synchronization, spatial calibration and EKF fusion algorithms, supplemented by real-time SAC consistency evaluation, to precisely align the visual skeleton with IMU inertial information in the spatiotemporal dimensions. This not only significantly reduces occlusion drift and zero bias accumulation errors, but also greatly improves the estimation stability of key joint angular velocity, jerk peak and load signals. Based on this solid perception foundation, the system uses the KLE-LMPL-BDR-SAC four-dimensional dynamic index and dual-threshold hysteresis strategy to seamlessly switch between constant window timing encoding and variable window timing control, and uses rate level discrimination and weighted compression algorithm to online By adjusting the window length, the model can quickly tighten the window and shorten the prediction delay in high-BDR sudden movements, and extend the window and suppress jitter in low-speed and stable stages, thereby taking into account both real-time response and robust prediction. The LSTM / GRU network that supports variable-length input and masking mechanism still maintains the timing modeling accuracy under different window lengths. Combined with the dual-channel "prediction-fast template" control architecture, the exoskeleton actuator can output feedforward assistance or buffering actions, significantly reducing the load impact peak and improving the human-equipment coupling comfort, thus achieving efficient coordination of the entire perception-prediction-adjustment link. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 Schematic diagram of the flow of the method for adaptive weight adjustment based on posture recognition of the present invention; Figure 2 The figure is a structural diagram of the weight-bearing adaptive adjustment system based on posture recognition of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] Example 1 The present invention is based on the weight self-adaptive adjustment method of posture recognition, such as Figure 1 As shown, the following steps are included: An inertial measurement unit (IMU) is installed at the main load-bearing parts of the exoskeleton. This IMU integrates a three-axis accelerometer and a three-axis gyroscope to obtain real-time linear acceleration and angular velocity information of the body. Some models are also equipped with a magnetometer to enhance the stability of direction estimation. At the same time, network cameras with a sampling frequency synchronized with the IMU are deployed in the operating area to collect full-body image information of the wearer, facilitating visual skeleton recognition and posture estimation. All sensor devices are synchronized through hardware trigger signals or the Precision Time Protocol (PTP) to ensure that data from different modalities are strictly aligned on the acquisition timeline.

[0019] After completing equipment deployment, the system calibrates the camera's internal and external parameters, correcting for lens distortion and establishing the geometric relationship between the world coordinate system and the camera coordinate system. To achieve the spatial mapping relationship between the IMU and the skeleton coordinate system, the system sets a set of standard posture sequences, including "cross stance-forward bend-upright," which the subjects perform and simultaneously records images and raw IMU data. Using this calibration sequence, the system solves the posture mapping matrix from the IMU coordinate system to the pelvis-hip joint local reference system, achieving the fusion of visual and inertial data in a unified spatial coordinate system.

[0020] In terms of timing, the system performs static zero-bias calibration on the IMU and performs noise baseline identification and drift compensation on its output data to ensure the stability and reliability of the inertial information during subsequent calculations. All sensor outputs are entered into a memory ring buffer indexed by timestamps for subsequent posture estimation, feature localization, and dynamic sliding window construction modules.

[0021] Through the above steps, the system achieves high-precision alignment of visual and inertial multimodal sensors in spatial and temporal dimensions, laying an accurate and stable perception foundation for the subsequent establishment of frame-level multimodal feature vector sequences and the implementation of predictive adjustment based on time series modeling.

[0022] The video stream recorded by the visual acquisition device is input into the system's image processing module at a fixed frame rate. After pre-processing steps such as image cropping, denoising, and normalization, it is sent to the deep neural network model deployed on the GPU server. The system uses HRNet (High-Resolution Network), which has been fine-tuned for this task scenario, as the posture estimation backbone network. This network structure has multi-resolution parallel processing capabilities and can extract the key point positions of the human skeleton while maintaining high-resolution feature maps. After processing, each frame of the image outputs a set of two-dimensional human key point coordinates and their corresponding confidence heat maps to reflect the stability and reliability of each node detection. For key points with confidence levels below the set threshold, the system marks them as "missing" and performs interpolation reconstruction in the subsequent processing stage.

[0023] The system performs temporal differentiation of the extracted keypoint positions based on the skeleton structure, calculating kinematic characteristics such as angles, angular velocities, and angular accelerations between joints. To enhance signal stability and robustness, the system introduces an adaptive Kalman filter algorithm to smooth keypoint trajectories in the temporal domain, compensating for trajectory breaks caused by occlusion, blur, or transient recognition failures. Furthermore, for node positions with insufficient confidence, the system continuously completes them along the timeline using interpolation between previous and next frames, ensuring the structural integrity and dynamic consistency of the skeleton sequence.

[0024] In terms of inertial modality, the system synchronously receives linear acceleration and angular velocity data from the IMU. Based on the posture mapping relationship established in step 1, the IMU output is converted from the sensor body coordinate system to the pelvis-hip joint reference coordinate system, achieving spatial alignment of visual and inertial information. At the same time, the PTP time synchronization mechanism ensures that the IMU data corresponding to the current frame image is strictly consistent on the time axis, achieving multimodal information fusion at the frame level.

[0025] After spatially and temporally aligning the raw visual and inertial data, the system concatenates all key features extracted from the current frame, including skeleton position, joint angles, angular velocity, angular acceleration, IMU acceleration, and IMU angular velocity, into a unified frame-level multimodal feature vector. To eliminate discrepancies in numerical scales between feature channels, the system performs normalization on all feature dimensions, including coordinate normalization, angle mapping compression, amplitude normalization, and unit consistency for velocity and acceleration channels, to construct a structurally unified and numerically stable input feature tensor.

[0026] Each frame of feature vectors after the above processing is written into a memory ring buffer built inside the system in chronological order. This memory ring buffer stores continuous observation data over a period of time in a first-in-first-out manner.

[0027] A dynamic time window selection mechanism is introduced during sliding window construction, standardization, and label synchronization. Its core purpose is to allow the system to adaptively adjust the input sequence length based on the complexity, rhythm, suddenness, and sensor quality of the current weight-bearing movement. Compared with traditional fixed windows, dynamic sliding windows have the following significant advantages: Movements like turning quickly, squatting, and bending over are sudden. If a fixed window is used for observation, the prediction model will be slow to react. The advantage of a dynamic sliding window is that it can actively compress the window length, allowing the model to quickly detect changes in movement, improve prediction lead time, and trigger timely adjustments.

[0028] Slow movements, such as bending and standing up slowly, may have a steady rhythm, but any sensor jitter or occlusion can lead to misjudgment. The system uses long-window input to provide sufficient historical information to the LSTM / GRU, improving stability and anti-interference capabilities, and preventing "misadjustment" or jitter triggering.

[0029] Models trained with a fixed window are likely to be effective only for a single rhythm; once the behavior rate changes, they are difficult to generalize. Different rhythmic actions correspond to different window lengths, and the system automatically adjusts and inputs variable-length sequences, so that the model always receives input depth that matches the rhythm during training and deployment, improving prediction consistency and generalization capabilities under complex working conditions.

[0030] There are two window management mechanisms: a constant window timing encoding mechanism (fixed window length) and a variable window timing control mechanism (dynamic window length). The system decides whether to switch mechanisms based on the temporal evolution of four key parameters, each of which is calculated by the vision and inertial fusion system. They are: The KLE calculation process is as follows: In the example (length L frames), the system constructs two parallel one-dimensional time series: Instantaneous power sequence of main joint angular velocity: for all main joint angular velocities in frame t Calculate instantaneous power: ; in, To select the number of main joints, Represents the Euclidean norm of the angular velocity vector.

[0031] Load sensor output sequence: Synchronously read the load sensor output of the exoskeleton backplate or legs (e.g. vertical force): ; Get the load signal sequence .

[0032] A load signal is a real-time measurement of the external force or pressure acting on a part of the human body, collected by external sensors (typically mounted on an exoskeleton, wearable device, or body support structure) during system operation. It reflects the mechanical load experienced by the human body during activity, particularly physical interventions associated with tasks such as lifting heavy objects or climbing stairs.

[0033] The system performs the above two sequences 、 To estimate mutual information, the steps are as follows: because and It is a continuous sequence of values. The system needs to discretize them into a finite number of intervals (or categories) in order to estimate the probability. This step is called binning. Common methods include: Equal-width binning: divide the entire value range evenly into N intervals (e.g. [0,1] is divided into 5 intervals: 0–0.2, 0.2–0.4, etc.); Equal frequency binning: sort all samples by size to ensure that the number of samples in each interval is roughly the same, which is more suitable for skewed distribution; Custom binning: Set thresholds based on prior knowledge (such as low power, medium power, and high power).

[0034] remember Represents the power sequence The i-th bin of Represents the jth bin of the payload sequence F. The discretized sequence will be represented as a bin index pair (i, j) of a certain frame.

[0035] Step 2: Joint Probability Estimation The system then counts all In the box The frequency of simultaneous occurrence in is used to construct the joint probability distribution matrix.

[0036] Define the elements of the joint probability matrix as: ; In the sliding window How many frames satisfy the power value at the same time? ; Load value .but: ;in: It is a sliding window The number of samples in the i-th, j-th bin; L is the total number of frames in the sliding window.

[0037] Constructed It is a two-dimensional joint probability matrix, which is used for the next step of mutual information calculation.

[0038] Step 3: Calculate marginal probability In order to calculate the mutual information, the marginal probability distribution of the two signals is also required: ; (The instantaneous power sequence of the main joint angular velocity is in the Total probability of the interval); (Load sensor output sequence is in total probability of the interval).

[0039] They represent the overall probability of the power or load value falling into a certain bin.

[0040] Step 4: Calculation of mutual information formula Mutual Information The definition of is: ; N: is the number of categories obtained after binning the power series. For example, if power is divided into "low, medium, and high", then N=3; M: is the number of categories obtained after the load sequence is binned. For example, if the load is divided into "light, medium, and heavy", then M=3.

[0041] The above formula means: If P and F are completely independent, then , the mutual information is 0; if there is a dependency between P and F (for example, when the power is high, the load is also high), the mutual information is greater than 0, and the larger it is, the tighter the coupling.

[0042] Normalized Lesion Entropy (KLE) definition: Using minimum entropy normalization, KLE is defined as: ;in: , ; The above mentioned "time window "Length is L frames" means that we are observing a continuous time period, such as the most recent L=30 frames (i.e., the data for the last 1 second). And "calculating the instantaneous power for all main joint angular velocities in frame t1" does not mean only looking at frame t1, but rather that we calculate the instantaneous power once for each frame in this window.

[0043] That is to say, for arrive For all frames within this window, the same operation is performed: the angular velocities of all main joints in each frame are extracted, their squares are calculated, and the average is calculated to obtain the "instantaneous power" value of the frame. .

[0044] So the whole power series is expressed as follows: ; Each of these All of them are aimed at the The angular velocities of multiple joints within a frame are calculated comprehensively.

[0045] LMPL is a key dynamic parameter used in this system to describe the average time lag between the load disturbance signal and the motion response signal. Its physical meaning is that in each time window , the degree of lag between the vertical pressure change on the lumbar spine (reflecting the load impact) and the vertical movement response of the pelvis.

[0046] The core calculation method of this indicator is based on short-time Fourier cross-spectrum analysis. Its essence is to measure the relative phase relationship between two time-varying non-stationary signals in the frequency domain, and further quantify the phase difference into time delay, which is used to characterize the span of the causal mechanism on the time axis, thereby providing a decision-making basis for the sliding window strategy.

[0047] Specifically, the system first constructs two synchronized time series: It represents the vertical pressure value collected by the lumbar load sensor at time t (in N or Pa), reflecting the instantaneous impact of the external load on the body; = represents the vertical velocity of the pelvis (in m / s) obtained by fusing the pelvic position estimate with the inertial navigation system at the same time point, representing the body's dynamic response to the impact. Within a sliding window of length L frames, the system applies a Short-Time Fourier Transform (STFT) to each of the two signals to obtain their time-varying spectral representation. The STFT calculation formula is: ; where x(t) represents the signal sequence (here it can be or , w(τ) is the window function (such as Hanning window), and f is the frequency variable. Through this transformation, the system can obtain and , that is, the complex spectrum of the load and motion signal at each frequency.

[0048] Next, calculate the cross-spectral density function (Cross-Spectral Density) of these two spectra: ; Where * represents the complex conjugate, the obtained is a complex-valued function whose modulus is reflects the frequency domain amplitude correlation, while the phase angle This reveals that the two signals have a phase lag at this frequency.

[0049] Identification Maximum main frequency , that is, the strongest resonance frequency between the load signal and the motion response, and then extract the phase difference corresponding to this frequency point: ; Phase difference is a measure of radians, which can be further converted to an average time lag (in seconds): ; This Δt is the LMPL value, representing the average time delay between a load change and a substantial response in pelvic velocity. LMPL is one of the driving parameters of the sliding window mechanism, and its value directly determines the input window length that the model should adopt. When Δt is large, meaning the load response has significant lag, the model requires a longer historical context, so the system tends to maintain or switch to an adaptive sliding window mechanism to ensure that the LSTM can observe the complete input-output evolution path. Conversely, when Δt decreases significantly, the movement responds quickly to the load perturbation, and the system determines that a fixed window can be used for the current input sequence.

[0050] The lumbar compression force curve is obtained by directly reading it from the load sensor embedded in the exoskeleton system, reflecting the force applied to the human lumbar spine or backboard in the vertical direction.

[0051] The method for obtaining the vertical velocity curve of the pelvis is that the signal is obtained by the system performing the first-order differentiation (velocity estimation) on the position sequence of the pelvis.

[0052] BDR is a key dynamic indicator used by this system to characterize the frequency of high-intensity, sudden movements. It identifies the "jerk peak" of the angular velocity of the main joints (i.e., the sudden increase in the rate of change of angular acceleration) and establishes a reference with the basal metabolic rate ("basal metabolic rate" here specifically refers to the minimum jerk activity level of a person when at rest or in slow, steady movements). It further measures its temporal distribution density to assist in determining whether the time window needs to be tightened and the model's response sensitivity to sudden movements needs to be improved.

[0053] Jerk usually refers to the first-order derivative of velocity with respect to time, that is, the rate of change of acceleration. In this system, since the input signal is the angular velocity sequence of the skeleton joints, jerk is expressed as the second-order derivative of the angular velocity with respect to time, that is: ; However, since the data is a discretely sampled frame sequence, the actual implementation uses numerical difference estimation: ; Where j represents the main joint number. Usually, the hip joint, knee joint, ankle joint, etc. that have the greatest impact on system control are selected for monitoring.

[0054] The system is in a sliding time window Above, the jerk value of each main joint is calculated frame by frame, and then the norm of these jerk values ​​is calculated within the range of all joints, such as the Euclidean modulus or the maximum norm, which is recorded as: ; Then, a basal metabolic jerk threshold value obtained based on the user's resting state or basic movement is used. If a frame , then the frame is judged as a “jerk peak” and recorded as a sudden event frame.

[0055] The number of frames that meet this condition in the window is accumulated as , then BDR is defined as: ; This ratio reflects the density of bursty motion. A higher BDR indicates a denser distribution of high-jerk frames, making them more likely to represent explosive behaviors (such as quick jumps, sharp turns, and sudden squats). Conversely, a low BDR indicates slow and steady movements with little to no high-speed variation.

[0056] If the BDR at a certain moment rises significantly, it means the organism is experiencing a rapid dynamic response. Continuing to use a fixed window will cause the model to miss the starting point of the mutation, leading to prediction lags and delayed adjustments. Therefore, the system must quickly shrink the sliding window to a shorter length to allow the LSTM or GRU to quickly capture the mutation signature and issue prediction outputs. Conversely, when the BDR is low, it indicates a slow behavioral rhythm and does not require frequent dynamic window adjustments. Using a fixed sliding window strategy avoids wasting computing resources and helps enhance the temporal stability of the model output.

[0057] The analysis and use of BDR does not rely on inference results. Instead, it serves as a direct input to the rate-sensing module, enabling an external model-based preemptive judgment of whether the current motion is explosive. This metric works in conjunction with KLE, LMPL, SAC, and other metrics to provide the system with a multi-dimensional dynamic understanding of motion rhythm, thereby driving the adaptive switching of the sliding window mechanism. In other words, BDR is a temporal density estimator of motion burstiness and a core regulatory signal in the proposed method's prediction delay compression path.

[0058] SAC is a dynamic metric used to evaluate the consistency of the fusion between visual sensors (such as cameras) and inertial sensors (IMUs) in real time. Its core purpose is to determine the current perception credibility by observing the degree of coordination between these two types of sensors in joint state estimation. It then adjusts the time window mechanism accordingly to improve the system's robustness under abnormal conditions.

[0059] The system uses an extended Kalman filter (EKF) or other state estimation algorithm to fuse the position and velocity of human skeleton nodes. This process combines visual observation data (such as skeleton posture tracking results) with IMU output (acceleration and angular velocity) to output a state estimation sequence. The filter generates two key matrices at each moment: Prediction covariance matrix (Y): This matrix combines historical state and sensor measurement predictions to determine state uncertainty, reflecting how much the system "believes" its current position and velocity fluctuate. Observation error covariance matrix (R): Indicates the reliability of the current frame vision or IMU single observation, that is, the measurement error of a certain sensor itself in this frame.

[0060] The system defines the SAC value as: Where Tr represents the trace operation of the matrix (i.e., the sum of all variance terms on the diagonal). This ratio reflects the uncertainty of the current fusion estimate compared to the single-frame observation error.

[0061] Decision logic: Each time the system updates the window strategy, it reads the weighted moving average of the most recent KLE, LMPL, BDR, and SAC. It then calculates the comprehensive window change score based on the weighted sum: ; 、 、 、 is the weight coefficient; 、 、 It is the weighted moving average value of KLE, LMPL, BDR and SAC.

[0062] The four-dimensional real-time indicators are compressed into a comprehensive variable window score, and two thresholds, Th (high threshold) and Td (low threshold), are preset; when When Th exceeds Th, it indicates that the load-action coupling is tight, the hysteresis is short, the jerk peaks are dense, and the observation is stable. At this time, the model adopts a variable window timing control mechanism to capture the mutation characteristics in time and output the prediction in advance. If it is lower than Td, it indicates weak coupling, long lag, slow action and large sensor noise. Maintaining or switching to the constant window timing coding mechanism and using a longer fixed window to provide sufficient context for the network can ensure stable operation. If it falls between Th and Td, there is insufficient evidence to prove that the external environment or system status has changed significantly. To avoid frequent switching back and forth due to slight fluctuations in indicators, the system chooses to maintain the mechanism that was already running in the previous frame, and naturally suppresses jitter through the "hysteresis" effect in this interval.

[0063] If a variable window timing control mechanism is employed, the system continuously splices this data into multimodal feature frames on a frame-by-frame basis. Each frame contains key variables such as skeleton position, angle, angular velocity, and load status, and stores them in a circular memory buffer. As the feature sequence accumulates, raw feature variables are extracted from the most recent fixed short-term window (e.g., 0.5 seconds, typically 15–20 frames) and specific dynamic indicators are calculated. Specifically, the changing patterns of the following five variables are analyzed in real time: first, the vertical vibration frequency and amplitude of the human trunk, derived from the vertical velocity curve of the pelvis; second, the angular velocity of the main joints (typically those with dominant influence on movement, such as the hip, knee, and ankle) to observe the degree of limb rotational activity; third, the BDR, used to assess the occurrence frequency of sudden, highly dynamic behaviors; fourth, the rate of change of force per unit time, recorded by the exoskeleton backplate load sensor, used to monitor the strength of load perturbations; and fifth, the SAC, which measures the stability of the coordination between the IMU and visual observations within the current time window.

[0064] These signals are fed into a set of rate classifiers built within the system. The core of these rate classifiers is an empirically and data-driven hybrid, which comprehensively maps features in the current time window into discrete motion rate class intervals. This rate classifier can be defined using threshold maps, multidimensional rule trees, or empirically optimized fuzzy logic models. Its output is categorized into four classes: "stationary," "slow," "medium," and "fast." Each rate class corresponds to a set of preset sliding window length intervals. For example, the sliding window range for stationary motion is 45–60 frames, for slow motion 30–45 frames, for medium speed 20–30 frames, and for fast 10–20 frames. These ranges are not fixed lengths; rather, they provide boundary conditions for the subsequent fine-grained window allocation.

[0065] In a variable window mechanism, directly using continuous metrics to linearly infer the window length presents two key risks: When the action cadence is too fast, the sliding window may be compressed too short, resulting in insufficient model input context, reduced prediction lead, and increased jitter; When the action cadence is very slow but the load disturbance is high, the sliding window may still be compressed, leading to misadjustment or model misjudgment. Therefore, it is necessary to first implement a rate classification system to manage continuously changing features in segments and define the "window allowable range" within each rate segment.

[0066] According to BDR, load disturbance intensity, SAC and rate level, a specific window length is calculated from the candidate sliding window interval; according to the current rate level R, the system selects the corresponding sliding window candidate interval .

[0067] The system adopts a weighted compression mechanism to linearly weight multiple dynamic signals. ;in . Indicates the adaptive window length; 、 、 is the weight coefficient; load disturbance intensity , by calculating the fluctuation amplitude (variance or peak-to-peak difference) of the load signal within a short time window and normalizing it; To determine the window length.

[0068] Once the window length is finalized, a segment of the latest historical frames of equal length is extracted from the memory ring buffer as an input sample, a length mask is attached, and the sample is fed into the LSTM or GRU model. During training, the network uses variable-length inputs and masks for forward propagation and backward gradient propagation, with the loss function masking out invalid frames. During inference, each time the system outputs a prediction for the next frame, it includes the current window configuration for reference by subsequent decision-making or control modules. This adaptive sliding window process operates in parallel and is decoupled from the main inference logic, using only the predicted input as the interface. This ensures stable deployment and strong real-time performance. It also gives the network the ability to perceive the temporal rhythms of different behaviors, significantly improving prediction lead time and adjustment responsiveness under complex load-bearing behaviors.

[0069] Constant window time series encoding mechanism: To achieve fixed-length time series modeling, the system sets the window length parameter D (in frames) during initialization. This is usually the optimal parameter for the validation set during the training phase, such as 30 or 40 frames. A sliding queue structure is then constructed, which maintains multimodal observation data for the past D consecutive frames in a first-in, first-out manner. With the arrival of each new frame of data, the oldest frame is popped out and the current frame is added to the end of the queue, ensuring that the window content always contains the latest D-frame data sequence. This mechanism ensures that the input time series length is constant, provides a stable size structure for the input tensor of subsequent encoding networks such as LSTM or GRU, and maintains controllable model inference efficiency when computing resources are limited.

[0070] After the buffer window is constructed, the system performs unified feature encoding on the raw observation data of each frame within the window. The process is as follows: First, the skeleton tracking signal is normalized, including zero-centering of joint coordinates, modulus normalization, and primary joint screening to enhance the consistency and scale stability of temporal features. Next, the system strictly time-aligns the raw data from the visual and IMU signals, ensuring complete correspondence between observations at the same moment through interpolation or truncation synchronization. Fusion algorithms such as the extended Kalman filter (EKF) are then used to integrate the two types of information into a stable state estimate. Furthermore, the system concatenates multidimensional observations such as skeleton geometry, joint angular velocity, load response, and IMU features into a unified feature vector at the frame level. Finally, the feature vectors of consecutive D frames in the sliding window are stacked in chronological order to form a two-dimensional temporal input matrix, which serves as the input tensor for the fixed-window encoding model and is used by the downstream prediction or classification network for temporal modeling and behavior recognition.

[0071] Build an LSTM or GRU network model that supports variable-length input structures to achieve unified modeling of various different rhythmic weight-bearing behaviors. Because the sliding window length of the input sequence is adjusted in real time, the model must be able to adapt to changes in input duration during the training phase.

[0072] The system first generates a mask vector for each sliding window feature sequence, with the same number of frames as the sequence itself, marking the locations of valid data frames and padded frames. All sequences are then aligned to a uniform length (e.g., 60 frames) and zero-padded to ensure a uniform input tensor format. The masking mechanism also shields the padding from interfering with gradient propagation during training. To enhance the model's perception of temporal depth, the system embeds the current sliding window length as a one-dimensional numerical feature into the feature vector of each frame, helping the model understand the coupling between "input time span" and "forecast lead time."

[0073] The training network backbone utilizes a standard bidirectional or stacked LSTM / GRU architecture, supplemented by a variable-length input processing module supporting mask propagation, enabling effective modeling of multimodal features in the temporal dimension. The model output can be configured with different supervisory signals for frame-level posture evolution trends, sudden motion probability, and load response prediction. The system employs a key joint weighting mechanism in its loss function design, assigning higher error penalties to joints highly relevant to load regulation, such as the hip and knee joints, ensuring that the model focuses on the dynamic changes in the core load path.

[0074] To improve the model's generalization across various motion rhythms, the system evenly samples different rhythm segments, including static, slow, medium, and high-jerk bursts, within each training batch. A rhythm-aware learning rate scheduling strategy is also introduced, appropriately increasing the learning rate during training with highly dynamic, short-window samples, enhancing the model's ability to learn bursty signals. Ultimately, the trained model can flexibly accept input feature sequences of arbitrary window lengths and, using a masking mechanism, deliver accurate and efficient forward prediction and backward updates. This model exhibits cross-rhythm adaptability and deployment stability, providing a foundation for proactive motion judgment in subsequent regulatory control modules.

[0075] After the model outputs its predictions, the system immediately enters the regulatory control phase. Based on the current sliding window configuration and the model's outputs regarding motion trends, jerk risk, and load disturbance intensity, the system determines whether it has entered a "high-risk coupling state" and generates feedforward regulatory commands accordingly. If the prediction indicates an impending drastic load change or highly dynamic posture evolution, the system proactively triggers regulatory commands before the disturbance actually occurs, enabling predictive-driven dynamic assist or buffering.

[0076] The system constructs structured control vectors from analytical model outputs, clearly defining the target's action location, output direction, amplitude, duration, and output strategy. Control commands are distributed to multiple actuator modules (such as the backplate motor, leg assist assembly, and ankle stabilization structure). An internal scheduling module optimizes the execution order based on factors such as each actuator's current bandwidth, temperature rise, and power limits, achieving coordinated regulation among multiple actuators. This scheduling strategy also considers the coupling between joint actions to avoid mechanical interference caused by simultaneous adjustments at multiple points.

[0077] During the adjustment process, the system reads actuator feedback in real time, such as position change, current response, adjustment hysteresis, and residual error, and dynamically compares it with the predicted target. If the adjustment target deviates, the system automatically triggers a rapid correction mechanism, adjusting the adjustment amplitude, output curve, or hold time in the next cycle, forming a closed-loop control chain of "prediction-adjustment-feedback-re-prediction."

[0078] To enhance the system's robustness under sudden events, a dual-channel regulation architecture is designed in this step. The primary channel drives control based on LSTM / GRU predictions, while the secondary channel constantly runs a set of fast-response templates based on raw sensory data. When the system detects typical sudden events, such as a sudden jerk peak or a jump in pelvic velocity, the secondary channel independently initiates a short-term regulation response before the primary model reacts, ensuring the wearer's safety during sudden events.

[0079] Furthermore, the system collects the wearer's behavioral characteristics and adjustment feedback data over a long period of time to build an individual user behavior model. By dynamically fine-tuning the adjustment amplitude, trigger sensitivity, and rhythm judgment threshold during the deployment phase, it achieves personalized adaptation of the adjustment strategy, making the adjustment process more natural and smooth, thereby improving user wearing comfort and the practical value of the system.

[0080] Example 2 The design of the weight-bearing adaptive adjustment system based on posture recognition of the present invention is based on the method in Example 1, specifically as follows Figure 2 The following modules are shown: The perception and feature construction module is used to collect three-axis acceleration, three-axis angular velocity, and skeleton image information. Through spatial calibration and filtering, interpolation, and normalization, it fuses skeleton key points, joint kinematics, and load signals into frame-level multimodal features and writes them into a memory ring buffer. The sliding window mechanism and rhythm control module is used to calculate the four dynamic indicators of KLE, LMPL, BDR and SAC. Based on the above four dynamic indicators and the hysteresis rules of the preset Th threshold and Td threshold, it switches between the constant window timing encoding mechanism and the variable window timing control mechanism. The sliding window mechanism and rhythm control module includes a window length calculation module; the window length calculation module determines the window length according to the rate level output by the rate level discriminator, the BDR value, the load disturbance intensity, and the SAC value; if the constant window timing coding mechanism is adopted, a fixed window length is adopted; The intelligent adjustment and feedback control module is used to input multimodal feature sequences of corresponding length into an LSTM or GRU network that supports variable-length input and masking mechanisms to obtain predictions of future posture evolution, load changes, and risks of sudden movements. Finally, based on the prediction, a control vector containing the adjustment target position, direction, amplitude, and duration is generated, and the control strategy is self-corrected using real-time feedback from the actuator. Dynamic adjustment operations are performed based on the prediction results, and the control effect is continuously optimized through the feedback mechanism.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0083] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0085] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A weight-adaptive adjustment method based on posture recognition, characterized in that: The following steps are involved: The inertial measurement unit and visual camera equipment synchronously collect three-axis acceleration, three-axis angular velocity and skeleton image information under a unified clock. Through spatial calibration and filtering, interpolation and normalization processing, the skeleton key points, joint kinematics and load signals are fused into frame-level multimodal features and written into the memory ring buffer. Then, four dynamic indicators, namely KLE, LMPL, BDR and SAC, are calculated in the memory ring buffer. Based on the four dynamic indicators and the hysteresis rules of the preset Th threshold and Td threshold, the constant window timing encoding mechanism and the variable window timing control mechanism are switched; If the variable window timing control mechanism is adopted, the window length is determined according to the rate level output by the rate level discriminator, the BDR value, the load disturbance intensity, and the SAC value; if the constant window timing coding mechanism is adopted, a fixed window length is used; The multimodal feature sequence of corresponding length is then input into an LSTM or GRU network that supports variable-length input and masking mechanisms to obtain predictions of future posture evolution, load changes, and the risk of sudden movements. Finally, based on this prediction, a control vector containing the target adjustment position, direction, amplitude, and duration is generated to drive the exoskeleton actuator to implement assistance or buffering adjustments in advance, and the real-time feedback from the actuator is used to self-correct the control strategy.

2. The method for adaptive weight adjustment based on posture recognition according to claim 1, characterized in that: The calculation of KLE includes: constructing the instantaneous power sequence P of the main joint angular velocity and the load sensor output sequence F in a time window of length L frames; discretizing the sequences P and F into bins, setting N power intervals and M load intervals respectively, and counting the number of frames in which each pair of bins (i, j) appears simultaneously to form an N×M dimensional joint probability matrix ; Further calculate the marginal probability of the power sequence and the marginal probability of the load sequence, and obtain the mutual information value of the two sequences according to the mutual information formula; finally, take the minimum entropy of sequences P and F as the benchmark, and obtain KLE by performing minimum entropy normalization.

3. The method for adaptive weight adjustment based on posture recognition according to claim 1, characterized in that: The calculation of the LMPL includes constructing a lumbar vertical pressure signal sequence and a pelvic vertical velocity signal sequence in a time window of length L frames; performing a short-time Fourier transform on the lumbar vertical pressure signal sequence and the pelvic vertical velocity signal sequence to obtain a spectrum that changes with time. and ; Further calculate the cross spectral density of the two spectra ; Select Maximum resonant frequency , extract the phase difference at this frequency ; and according to the formula: The average time lag Δt is obtained, and the Δt is the LMPL value.

4. The method for adaptively adjusting load based on posture recognition according to claim 1, characterized in that: The BDR is determined by identifying the jerk peak of the main joint angular velocity and establishing a reference with the basal metabolic level, and further measuring its time series distribution density.

5. The method for adaptively adjusting load based on posture recognition according to claim 4, characterized in that: The jerk peak is the sudden increase point of the angular acceleration change rate; the basal metabolic rate is the minimum jerk activity level of a person when he is at rest or in a slow and steady movement state.

6. The method for adaptively adjusting load based on posture recognition according to claim 1, characterized in that: The SAC is a dynamic indicator used to evaluate the fusion consistency between visual sensors and inertial sensors in real time.

7. The method for adaptively adjusting load based on posture recognition according to claim 1, characterized in that: Read the weighted moving average of the most recent KLE, LMPL, BDR, and SAC; then calculate the comprehensive variable window score based on the weighted sum ; By presetting two thresholds, Th (high threshold) and Td (low threshold); when When Th is exceeded, a variable window timing control mechanism is adopted; Lower than Td, maintain or switch to the constant window timing coding mechanism; if Falling between Th and Td, choose the mechanism that keeps what was already running in the previous frame.

8. The method for adaptive weight adjustment based on posture recognition according to claim 1, characterized in that: The rate level discriminator is an experience-data hybrid driven rate level discriminator, which comprehensively maps the features in the current time window to discrete action rate levels; the rate level discriminator can be defined using a threshold map, a multidimensional rule tree, or a fuzzy logic model based on experience optimization.

9. The method for adaptive weight adjustment based on posture recognition according to claim 8, characterized in that: The rate level corresponds to the sliding window candidate interval; a weighted compression mechanism is used to linearly weight the BDR, load disturbance intensity, SAC, and rate level to finally output the window length.

10. The weight-adaptive adjustment system based on posture recognition is characterized by: The regulation system is based on the method according to any one of claims 1 to 9, and comprises the following modules: The perception and feature construction module is used to collect three-axis acceleration, three-axis angular velocity and skeleton image information, and fuse skeleton key points, joint kinematics and load signals into frame-level multimodal features and write them into the memory ring buffer; The sliding window mechanism and rhythm control module is used to calculate the four dynamic indicators of KLE, LMPL, BDR and SAC. Based on the above four dynamic indicators and the hysteresis rules of the preset Th threshold and Td threshold, it switches between the constant window timing encoding mechanism and the variable window timing control mechanism. The sliding window mechanism and rhythm control module includes a window length calculation module; the window length calculation module determines the window length according to the rate level output by the rate level discriminator, the BDR value, the load disturbance intensity, and the SAC value; if the constant window timing coding mechanism is adopted, a fixed window length is adopted; The intelligent adjustment and feedback control module is used to input multimodal feature sequences of corresponding length into an LSTM or GRU network that supports variable-length input and masking mechanisms to obtain predictions of future posture evolution, load changes, and risks of sudden movements. Finally, based on the prediction, a control vector containing the adjustment target position, direction, amplitude, and duration is generated, and the control strategy is self-corrected using real-time feedback from the actuator. Dynamic adjustment operations are performed based on the prediction results, and the control effect is continuously optimized through the feedback mechanism.

Images