Multi-mode sensing and motion state recognition method and device of intelligent wearable equipment

Abstract

The invention relates to the technical field of intelligent wearable devices, and discloses a multi-modal sensing and motion state recognition method and device for an intelligent wearable device, and the method comprises the steps: carrying out the anatomical coordinate transformation of original motion data collected by a wrist wearable device, and obtaining an acceleration sequence and an angular velocity sequence; performing sliding window segmentation and multi-scale wavelet packet decomposition to obtain a potential conversion period window set and corresponding multi-scale time-frequency features; constructing a conversion period feature vector for describing a conversion relation between the wrist movement and the whole body movement; inputting the conversion period feature vector into a hierarchical conversion identification model for processing to obtain a conversion type label and a conversion confidence coefficient; body part association mapping and lower limb action state inference are executed, a lower limb state sequence and a whole body action conversion type in the conversion process are obtained, real-time efficient recognition of action conversion is achieved, limitation of a single model is avoided, and the high recognition rate under low calculation complexity is kept.

Description

Technical Field

[0001] The present invention relates to the technical field of smart wearable devices, and particularly to a multimodal perception and motion state recognition method and device for smart wearable devices. Background Art

[0002] Currently, smart wearable devices mainly focus on steady-state action recognition, such as the classification of static activities like walking, running, sitting postures, etc., while less research has been done on the recognition of action transition periods (such as the transition processes from standing to walking, from walking to running, from flat ground to climbing stairs, etc.). As a bridge connecting different steady-state actions, the action transition period contains rich biomechanical and kinematic information, and its accurate recognition is crucial for all-weather human activity monitoring and refined motion assessment.

[0003] However, there are multiple technical challenges in recognizing the action transition period based on wrist-worn devices. Compared with the trunk position, the signal-to-noise ratio of the signals at the wrist position is significantly reduced, and the sensor data is easily interfered by random hand movements. Secondly, the action transition period usually lasts for a short time (about 100 - 350 milliseconds), and the features are not obvious. Traditional single-scale analysis methods are difficult to capture the transient signal features of the transition period. Thirdly, the wrist sensors are less sensitive to changes in lower limb movements, especially for transition types involving vertical movements such as going up and down stairs. It is difficult for sensors at a single position to accurately identify the whole-body action state. Summary of the Invention

[0004] The present invention provides a multimodal perception and motion state recognition method and device for smart wearable devices. The present invention realizes real-time and efficient recognition of action transitions, avoids the limitations of a single model, and maintains a high recognition rate under low computational complexity.

[0005] In a first aspect, the present invention provides a multimodal perception and motion state recognition method for a smart wearable device, and the multimodal perception and motion state recognition method for the smart wearable device includes: Performing anatomical coordinate transformation on the original motion data collected by a wrist-worn device to obtain an acceleration sequence and an angular velocity sequence; Performing sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi-scale time-frequency features; Constructing a transition period feature vector describing the relationship between wrist movement and whole-body action transition based on the set of potential transition period windows and their corresponding multi-scale time-frequency features; Inputting the transition period feature vector into a hierarchical transition recognition model for processing to obtain a transition type label and a transition confidence level; Perform body part association mapping and lower limb motion state inference based on the conversion type tag and the conversion confidence, and obtain the lower limb state sequence and the full body motion conversion type during the conversion process.

[0006] Combined with the first aspect, in the first implementation manner of the first aspect of the present invention, performing anatomical coordinate transformation on the original motion data collected by the wrist-worn device to obtain an acceleration sequence and an angular velocity sequence includes: Collect the original motion data through the triaxial acceleration sensor and the triaxial gyroscope in the wrist-worn device, and construct an original acceleration vector and an original angular velocity vector based on the original motion data; Use the anatomical coordinate transformation matrix parameterized by the wrist joint range of motion to perform transformation processing on the original acceleration vector and the original angular velocity vector to obtain mapping data from the local coordinate system to the human reference coordinate system; Calculate the dynamic consistency constraint value according to the mapping data, and judge the dynamic consistency constraint value based on a preset constraint threshold to obtain an outlier marker set; Perform cubic spline interpolation replacement on the data points marked in the outlier marker set to obtain an acceleration sequence and an angular velocity sequence.

[0007] Combined with the first aspect, in the second implementation manner of the first aspect of the present invention, performing sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential conversion period windows and their corresponding multi-scale time-frequency features includes: Perform sliding window segmentation on the acceleration sequence and the angular velocity sequence respectively to obtain an acceleration window sequence and an angular velocity window sequence; Calculate the first derivative change rate of the acceleration vector modulus for the acceleration window sequence, and calculate the standard deviation of the angular velocity modulus value for the angular velocity window sequence; Compare and judge the first derivative change rate with a first preset threshold, and at the same time compare and judge the standard deviation with a second preset threshold to obtain a set of potential conversion period windows; Perform four-layer multi-scale wavelet packet decomposition on the signal data in the set of potential conversion period windows to obtain a set of wavelet coefficients at four scales; Perform band signal reconstruction on the set of wavelet coefficients at the four scales to obtain multi-scale time-frequency features.

[0008] Combined with the first aspect, in the third implementation manner of the first aspect of the present invention, constructing a conversion period feature vector describing the conversion relationship between wrist motion and full body motion based on the set of potential conversion period windows and their corresponding multi-scale time-frequency features includes: Calculate a time-domain feature set including kurtosis, skewness, zero-crossing rate, signal vector amplitude, and signal amplitude area based on the signal data in the set of potential transition period windows; Calculate a frequency-domain feature set including energy ratio, band entropy, dominant frequency drift, power spectrum kurtosis, and power spectrum entropy based on the signals in each frequency band of the multi-scale time-frequency features; Obtain a target angular velocity sequence from the set of potential transition period windows, and calculate a joint angle feature set including wrist flexion / extension angle, ulnar / radial deviation angle, and pronation / supination angle in the target angular velocity sequence through quaternion integration; Perform normalized feature fusion based on the time-domain feature set, the frequency-domain feature set, and the joint angle feature set to obtain a transition period feature vector, which is used to describe the conversion relationship between wrist movement and whole-body movement.

[0009] Combined with the first aspect, in the fourth implementation manner of the first aspect of the present invention, inputting the transition period feature vector into a hierarchical transition recognition model for processing to obtain a transition type label and a transition confidence level includes: Input the transition period feature vector into a transition trigger detector in the hierarchical transition recognition model for lightweight decision tree judgment to obtain a transition trigger probability value; Perform a comparison and judgment process on the transition trigger probability value and a user variance adaptive threshold to obtain a transition trigger signal; Based on the transition trigger signal, input the transition period feature vector into a transition type classifier in the hierarchical transition recognition model for classification to obtain a probability distribution of each transition category; Select the category corresponding to the highest probability from the probability distribution of each transition category as the initial type label; Perform a temporal consistency verification on the initial type label to obtain a transition type label and a transition confidence level.

[0010] Combined with the first aspect, in the fifth implementation manner of the first aspect of the present invention, inputting the transition period feature vector into a transition type classifier in the hierarchical transition recognition model for classification to obtain a probability distribution of each transition category includes: Based on the transition trigger signal, input the transition period feature vector into a bidirectional LSTM layer in the transition type classifier for temporal modeling to obtain a bidirectional temporal feature vector; Input the bidirectional temporal feature vector into a wrist-whole body movement mapping residual network in the transition type classifier for transition relationship modeling, retain the combination of the original feature and the learning vector through residual connection, and at the same time calculate a dynamic attention gating coefficient using the transition period context information to obtain an enhanced residual feature vector; Input the enhanced residual feature vector into the softmax classification layer in the conversion type classifier to calculate the conversion type, and obtain the probability distribution of each conversion category, where the conversion categories include standing-sitting, sitting-standing, walking-running, walking-standing, standing-walking, running-walking, going upstairs-walking flat, going downstairs-walking flat, walking flat-going upstairs, walking flat-going downstairs, standing-running, running-standing.

[0011] Combined with the first aspect, in the sixth implementation manner of the first aspect of the present invention, the performing body part association mapping and lower limb motion state inference based on the conversion type label and the conversion confidence to obtain the lower limb state sequence and the whole body motion conversion type during the conversion includes: Extract the wrist azimuth angle and the wrist joint bending angle from the conversion period feature vector based on the conversion type label, and perform mapping transformation on the wrist azimuth angle and the wrist joint bending angle by using the body part association mapping matrix to obtain the initial lower limb state data; Select the corresponding mapping parameter set according to the conversion type label, and perform mapping on the initial lower limb state data based on the mapping parameter set to obtain the lower limb state parameters of different conversion types; Extract the initial swing phase identifier, the wrist vertical acceleration feature point and the frequency domain periodicity information from the conversion period feature vector, and perform gait analysis to obtain the gait cycle and the foot off / landing time data; Based on the gait cycle and the foot off / landing time data, use the lower limb joint angle estimation model to calculate the knee joint angle, the ankle joint angle and the hip joint angle to obtain the lower limb joint angle sequence; Adopt a time-varying mapping function to perform dynamic conversion processing on the lower limb joint angle sequence, and at the same time identify the vertical motion component of the conversion type to obtain the lower limb state sequence and the whole body motion conversion type during the conversion.

[0012] Combined with the first aspect, in the seventh implementation manner of the first aspect of the present invention, the based on the gait cycle and the foot off / landing time data, using the lower limb joint angle estimation model to calculate the knee joint angle, the ankle joint angle and the hip joint angle to obtain the lower limb joint angle sequence includes: Based on the foot off / landing time data, divide the gait cycle into a swing phase and a support phase, and perform weight allocation on the correlation between the wrist feature and the lower limb motion according to different phases to obtain the gait phase weight sequence; According to the gait phase weight sequence, weight the wrist flexion / extension angle, the ulnar deviation / radius deviation angle, the pronation / supination angle and the acceleration component in the conversion period feature vector to obtain the wrist feature time series data synchronized with the lower limb motion; Based on the wrist characteristic time series data, an initial lower limb joint angle value is obtained by applying a lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle, and hip joint angle; Biomechanical constraints are imposed on the initial lower limb joint angle value to ensure that the joint angle change rate conforms to the kinematic characteristics of the conversion type, and a lower limb joint angle sequence is obtained.

[0013] In a second aspect, the present invention provides a multi-modal perception and motion state recognition device for an intelligent wearable device. The multi-modal perception and motion state recognition device for the intelligent wearable device includes: A transformation module for performing anatomical coordinate transformation on the original motion data collected by the wrist-worn device to obtain an acceleration sequence and an angular velocity sequence; A decomposition module for performing sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi-scale time-frequency features; A construction module for constructing a transition period feature vector describing the conversion relationship between wrist movement and whole-body movement based on the set of potential transition period windows and their corresponding multi-scale time-frequency features; A processing module for inputting the transition period feature vector into a hierarchical conversion recognition model for processing to obtain a conversion type label and a conversion confidence level; An inference module for performing body part association mapping and lower limb motion state inference based on the conversion type label and the conversion confidence level to obtain a lower limb state sequence and a whole-body motion conversion type during the conversion process.

[0014] In the technical solution provided by the present invention, through the processing method combining anatomical coordinate transformation and dynamic consistency constraint, the problem of low signal-to-noise ratio of the wrist position sensor signal is effectively solved, ensuring the consistency of data in the anatomical sense. By adopting the strategy of combining sliding window segmentation and multi-scale wavelet packet decomposition, the transient features during the action transition period can be accurately captured, especially for the conversion signals with short duration and insignificant features, which has an enhanced recognition ability, avoiding the limitation that traditional single analysis in the time domain or frequency domain cannot distinguish short-time conversion from noise. By extracting time-domain features, frequency-domain features, joint angle features, and pose estimation features and fusing them, a multi-dimensional feature representation describing the relationship between wrist movement and full-body action conversion is constructed, significantly improving the mapping accuracy between wrist data and full-body action conversion. Based on the cascaded processing mechanism of the hierarchical conversion recognition architecture, which includes three modules: a conversion trigger detector, a conversion type classifier, and a conversion completion validator, the real-time and efficient recognition of action conversion is realized, avoiding the limitations of a single model and maintaining a high recognition rate with low computational complexity. A body part association mapping and lower limb action state inference mechanism is designed to effectively solve the problem that it is difficult for wrist sensors to directly capture lower limb actions, realizing an effective mapping from wrist actions to lower limb states, providing a technical guarantee for the complete representation of full-body action conversion.

[0015] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are realized and obtained by the structures specifically pointed out in the specification, claims, and drawings.

[0016] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specific preferred embodiments are given, and in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of an embodiment of the multi-modal perception and motion state recognition method of the intelligent wearable device in the embodiment of the present invention; Figure 2 It is a schematic diagram of an embodiment of the multi-modal perception and motion state recognition device of the intelligent wearable device in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0019] As used in the embodiments of the present invention, the terms "comprising", "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include other steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0020] To facilitate the understanding of this embodiment, first, a multi-modal perception and motion state recognition method for an intelligent wearable device disclosed in the embodiments of the present invention will be introduced in detail. As Figure 1 shown, the method includes the following steps: 101. Perform anatomical coordinate transformation on the original motion data collected by the wrist-worn device to obtain an acceleration sequence and an angular velocity sequence; It can be understood that the execution subject of the present invention can be a multi-modal perception and motion state recognition device of an intelligent wearable device, or a terminal or a server. Specifically, it is not limited here. In the embodiments of the present invention, the server is taken as an example of the execution subject for illustration.

[0021] Specifically, the triaxial acceleration sensor and triaxial gyroscope in the wrist-worn device are used to synchronously collect the original motion data. Under the condition of a sampling frequency of 100 Hz, a high-timeliness data stream is obtained and respectively constructed into a three-dimensional original acceleration vector and an original angular velocity vector. The acceleration vector is used to describe the linear motion state of the wrist in space, while the angular velocity vector depicts its rotational change characteristics. Since the wrist is one of the human end effectors, its sensor data is vulnerable to the influence of posture changes, device sliding, and wearing offset. Therefore, the data is standardized based on a coordinate calibration mechanism with anatomical basis. This coordinate transformation mechanism describes the three typical degrees of freedom of the wrist, namely flexion and extension, ulnar deviation and radial deviation, and pronation and supination, respectively, with angular variables by parameterizing the physiological activity range of the wrist, constructs a three-dimensional spatial rotation structure, and then combines the spatial displacement of the wrist relative to the human reference coordinate system to form an anatomical coordinate transformation matrix. This matrix can map the original acceleration vector and angular velocity vector at any moment from the local sensor coordinate system to the human global reference system, so that subsequent data analysis has cross-time and cross-individual comparability and reproducibility. After applying this transformation, the transformed acceleration vector and angular velocity vector sequences are obtained respectively. A consistency constraint calculation method based on a dynamic model is designed to verify the coupling characteristics of acceleration and angular velocity by introducing the rigid body motion relationship, that is, considering the influence of angular velocity change on linear acceleration in the ideal state, and comparing the deviation between the actually measured acceleration value and the theoretically deduced result to form a quantifiable error index, which can physically reflect whether the current motion state is reasonable. When the dynamic consistency constraint value exceeds the set acceleration threshold, it indicates that the current data point does not conform to the law of the human real motion trajectory, that is, it is marked as an outlier. All outliers are collected uniformly to form an outlier marked set. The marked abnormal data points are subjected to cubic spline interpolation operation, that is, a continuous and continuously differentiable smooth curve is constructed on the time axis based on the adjacent valid data segments before and after it, and the original outliers are replaced by interpolation, so as to eliminate the influence of mutations and noises while maintaining the overall dynamic trend, and the acceleration sequence and angular velocity sequence are obtained.

[0022] 102. Perform sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi-scale time-frequency features; Specifically, the acceleration sequence and the angular velocity sequence are processed by sliding window partitioning. This process adopts a fixed-length window sliding mechanism, where the time span of each window is set to 400 milliseconds and slides at an overlapping ratio of 75% to ensure that the continuous details during action transitions are not missed, respectively forming an acceleration window sequence and an angular velocity window sequence. Calculate the rate of change of the first derivative of the vector modulus value of the signal in each acceleration window on the time axis to measure the sudden change in the motion amplitude contained in this window. This index can sensitively reflect the short-term intense motion fluctuation characteristics generated when the body state transitions from one steady state to another. At the same time, calculate the standard deviation of the angular velocity modulus value in each window of the angular velocity window sequence as a discrete index to measure the intensity of rotational activities within this window. Subsequently, compare the rate of change of the derivative corresponding to each window with the first preset threshold, and at the same time compare the angular velocity standard deviation with the second preset threshold. Only when both exceed the threshold, mark this window as a potential action transition period window, thus forming a set of candidate transition period window sets. Perform four-layer multi-scale wavelet packet decomposition on the signal data in the potential transition period window set. Adopt a four-layer decomposition structure, starting from the widest frequency band, gradually divide the signal into more refined sub-bands in the frequency domain layer by layer, and complete the acquisition of decomposition coefficients with the wavelet packet basis function as the analysis core. In each layer of decomposition, the signal is split into twice as many sub-bands until the fourth layer forms a structure containing 16 sub-frequency bands, and the corresponding frequency range covers multiple sub-intervals from high frequency to low frequency. Since different action transition types exhibit different energy distribution characteristics in the frequency domain, four-layer wavelet packet decomposition can capture local high-frequency perturbations, stability changes, and continuous characteristics during action transitions at multiple scales. Reconstruct the frequency band signals from the four-layer wavelet coefficient set to restore the frequency domain component signals corresponding to each scale. These frequency band signals still have the same length as the original signal on the time axis, but in the spectral structure, they have been decomposed into several non-overlapping sub-frequency bands, thus forming a set of multi-scale time-frequency characteristics with time-frequency joint expression ability.

[0023] 103. Construct a transition period feature vector describing the relationship between wrist movement and whole-body action transitions based on the potential transition period window set and its corresponding multi-scale time-frequency characteristics; Specifically, based on the set of potential transition period windows and their corresponding multi-scale time-frequency features as input, multi-dimensional signal features with high discriminative ability are extracted. Among them, starting from the time domain, statistical feature analysis is performed on the acceleration signals in each potential transition period window. The kurtosis index is calculated to characterize the sharpness of the signal waveform, the skewness is used to measure the offset degree of the signal on the mean axis, and the zero-crossing rate is combined to judge the oscillation frequency of the signal in a short time, so as to capture the dynamic change trend of the signal. At the same time, to characterize the overall movement amplitude of the signal in three-dimensional space, the signal vector magnitude is calculated as a unified quantization index for the acceleration magnitude, and the signal amplitude area is calculated to reflect the total energy accumulation of the acceleration within a given time window. After the time-domain features are constructed, frequency-domain feature extraction is performed on the multi-scale frequency band signals decomposed in the previous stage. Specifically, the energy ratio of each frequency band signal is calculated respectively to form an energy ratio vector, reflecting the distribution trend of the signal energy at different scales; the band entropy of each frequency band is calculated through the normalized frequency distribution to quantify the discreteness and complexity of the frequency distribution, and at the same time identify whether there is a significant activation of the concentrated frequency band; to reflect the dynamic drift of the signal spectrum during the action conversion process, the offset of the main frequency between adjacent windows is tracked and the main frequency drift value is recorded to reflect whether the rhythmic action has a mutation; at the same time, the power spectrum kurtosis is calculated as a reference for judging the energy concentration degree, and the power spectrum is normalized and then the spectral entropy is calculated to obtain the disorder index of the entire frequency-domain energy distribution. This frequency-domain feature set expands the understanding of the internal rhythmic structure of the signal during the conversion process and enhances the system's ability to represent short-time and non-stationary signal changes. The description of the joint movement dimension is introduced to strengthen the connection between the local movement of the wrist and the evolution of the whole body posture. The angular velocity sequence is extracted from the potential transition period window, and the angular velocity signal is converted into angular displacement information through the quaternion integration method to construct a set of joint angle feature sets including wrist flexion and extension angles, ulnar or radial deviation angles, and pronation or supination angles. These angle indicators describe the specific posture evolution trajectory of the wrist during the conversion from the perspective of kinematic anatomy. At the same time, combining the change trends of joint angular velocity and angular acceleration, an angle-velocity phase space distribution map is constructed to capture the posture adjustment behavior occurring during the action start, stop, or transition stage. Normalization processing is performed based on the time-domain feature set, frequency-domain feature set, and joint angle feature set to ensure the comparability of each dimension index on the same numerical scale. Subsequently, through the feature fusion mechanism, the multi-source data are merged into a high-dimensional transition period feature vector with a unified structure, reflecting the mapping relationship between the local movement pattern of the wrist and the whole body action conversion.

[0024] 104. Input the transition period feature vector into the hierarchical transition recognition model for processing to obtain the transition type label and transition confidence; Specifically, the conversion period feature vector is input into a hierarchical conversion recognition architecture, which consists of three parts: a conversion trigger detector, a conversion type classifier, and a conversion completion validator, forming an interconnected and clearly defined recognition link. By inputting the feature vector into the conversion trigger detector for preliminary analysis, the detector is constructed using a lightweight decision tree model. With a small amount of computing resources, it can evaluate whether the current feature has the possibility of a movement conversion based on key indicators such as signal vector amplitude, main frequency drift, joint angular acceleration, and attitude change rate in the feature, and thus calculate a quantified conversion trigger probability value, which is used to represent that the current time segment is very likely to be the critical point of a certain posture transition. The conversion trigger probability value is compared with a threshold adaptively generated based on user behavior characteristics. This threshold is jointly composed of a basic reference value and the variance of the user's individual behavior changes. Through a dynamic adjustment mechanism, the model can not only adapt to the physiological differences between users but also self-adapt to the fluctuations of a user's behavior characteristics over time. When the conversion trigger probability value is higher than this threshold, the system determines that the conversion behavior has been activated and then issues a conversion trigger signal. Based on the conversion trigger signal, the same conversion period feature vector is transmitted to the conversion type classifier in the hierarchical conversion recognition model for category determination. The classifier uses an improved bidirectional long short-term memory network with a residual structure and an attention mechanism introduced. In the network structure, the original feature vector is not only used as input for time series modeling but also has a residual connection with the intermediate hidden state, thus retaining the key structural information in the input feature and enhancing the model's expressive ability. At the same time, through the attention gating mechanism, the contribution degree of different time slices in the input sequence is dynamically regulated, making the model pay more attention to the key change area between the start and end of the conversion action. The classifier outputs a probability distribution covering all preset conversion categories, representing the likelihood weights of the current sample belonging to each specific action conversion type. The category corresponding to the highest probability is selected from the probability distributions of each conversion category as the initial type label, which represents the model's most confident judgment of the action category at present. The initial type label is verified for temporal consistency. Based on the Markov logic network, by introducing four types of constraint mechanisms, including the time continuity constraint between conversion actions, the duration range limit, the physical consistency constraint of the posture at the conversion boundary, and the dynamic feasibility judgment that conforms to the amplitude of human physiological changes, the rationality and consistency of the initial label in the conversion sequence are comprehensively judged. If the verification passes, the label is confirmed as the final conversion type label, and the value corresponding to it in the probability distribution is output as the conversion confidence; if the verification fails, the label value is adjusted through a dynamic correction mechanism, or the output is delayed in the case of logical conflicts.

[0025] Based on the conversion trigger signal, the conversion period feature vector is input into the bidirectional LSTM layer in the conversion type classifier for temporal modeling. The bidirectional LSTM structure learns the temporal trend of wrist movement changes from the forward sequence, and at the same time combines the information of the reverse sequence to supplement the lag effect in short-term actions, thereby forming a set of bidirectional temporal feature vectors that integrate forward and reverse time contexts, reflecting the dynamic evolution process of feature dimensions during the conversion period. The bidirectional temporal feature vectors are input into the wrist-body action mapping residual network in the conversion type classifier to deeply model the non-linear correspondence between wrist local action features and body-wide action patterns. This residual network designs multiple transformation paths, performs spatial mapping on the input feature vectors through the feed-forward weight matrix, and at the same time constructs a bypass connection to directly add the initial input features to the intermediate result of network calculation, so as to retain the original structural information and suppress the over-smoothing effect brought by the deep network. At the same time, in order to strengthen the model's ability to focus on key conversion state segments, a dynamic attention gating mechanism based on the conversion period context is introduced. This mechanism generates a set of attention weight coefficients by calculating the semantic correlation between different time segments, and applies these weights to the temporal feature vectors through a gating function, thereby amplifying the expression intensity at key conversion nodes, suppressing redundant background interference, and enabling the enhanced residual feature vectors to have cross-temporal identification consistency while containing local fine changes. The enhanced residual feature vectors are input into the softmax output layer of the classifier structure for the final conversion type determination. The softmax classification layer generates a set of normalized probability distributions by performing a normalized exponential operation on all output dimensions, representing the belonging probability values of the current conversion period features in all predefined conversion types. This probability distribution includes twelve types of action conversion modes, including transitions such as standing to sitting, sitting to standing, walking to running, walking to standing, standing to walking, running to walking, going upstairs to walking flat, going downstairs to walking flat, walking flat to going upstairs, walking flat to going downstairs, standing to running, and running to standing.

[0026] 105. Perform body part association mapping and lower limb action state inference based on the conversion type label and conversion confidence to obtain the lower limb state sequence and the whole body action conversion type during the conversion process.

[0027] Specifically, based on the conversion type tag, the wrist azimuth angle and the wrist joint bending angle are extracted from the conversion period feature vector. These two types of parameters respectively reflect the attitude direction of the wrist in space and the local physiological structure state. On this basis, the preset body part association mapping matrix is called to perform a non-linear space mapping on the wrist azimuth angle and the wrist joint bending angle, so as to project the local feature information of the wrist area into the whole body state space and generate the initial lower limb state data representing the overall movement trend of the human body. According to the currently recognized conversion type tag, the mapping parameter set corresponding to the specific action is determined, and the type-specific mapping operation is performed on the initial lower limb state data through this parameter set, so that the system can introduce a discriminative lower limb state adjustment mechanism when processing complex conversion modes such as "walk-run" or "flat walk-upstairs". By selecting different types of conversion mapping parameters, the typical changes in the lower limb state during each action switch are adapted to obtain more detailed and accurate lower limb state parameters. In order to restore the complete dynamic gait structure, the initial swing phase identifier, the key feature points (such as local extreme points) of the wrist vertical acceleration, and the periodic peak information reflected in the frequency domain analysis are extracted from the current conversion period feature vector. By synthesizing this information, gait pattern analysis is performed, and the current gait cycle and the time nodes of foot takeoff and landing are identified. These gait elements provide time anchors for deriving the lower limb angle change process. Based on the recognized gait cycle and foot event moments, the lower limb joint angle estimation model is called, and combined with the known wrist angle data and acceleration characteristics, the angle change curves of the knee joint, ankle joint, and hip joint during the entire conversion period are derived to form a set of lower limb joint angle sequences with physiological interpretation significance. This estimation model controls the contribution ratio of different input variables to the target angle through weight coefficients and corrects the model output by combining the bias term, so that it can accurately restore the movement trajectories of the main lower limb joints during the conversion process without relying on lower limb sensors. A time-varying mapping function is introduced to perform dynamic transformation processing on the lower limb joint angle sequence, complete the smooth transition modeling from the starting state to the ending state by introducing the time variable, and improve the continuity and physical rationality of the conversion process through cubic function form time interpolation. While the dynamic mapping is being executed, it is identified whether the current conversion type includes a vertical motion component. By analyzing statistical indicators such as the integral value, maximum value, and standard deviation of the vertical acceleration, a vertical motion discriminator is constructed to identify complex motion patterns involving changes in the body center of gravity, such as "stand-upstairs" or "flat walk-downstairs". When the system determines that there is an obvious vertical motion trend, the dynamic change amplitude and time distribution of the knee, ankle, hip and other joint angles are adjusted to improve the adaptation accuracy for special actions. By synthesizing all the dynamic estimation and motion compensation results, the lower limb state sequence during the conversion process is obtained, and the final determination result of the whole body action conversion type is obtained.

[0028] Based on the data at the moment of foot lift-off / landing, the gait cycle is structurally segmented. According to the classical division of the gait cycle in biomechanics, a complete cycle is divided into two major parts: the swing phase and the stance phase. The swing phase corresponds to the time period from foot lift-off to the next touchdown, while the stance phase covers the stage when the foot contacts the ground and bears weight. To improve the contribution accuracy of wrist data to the inference of lower limb states, the correlation weight between wrist features and lower limb movements is dynamically allocated according to the current gait phase, and a gait phase weight sequence that changes over time is constructed. This weight sequence enhances the influence of wrist posture changes in the swing phase and relies more on the stable fluctuation law of acceleration in the stance phase, thereby achieving precise control of the contribution differences of signal features in different stages. Based on this dynamic weight structure, various key indicators in the feature vector of the transition period are weighted, including wrist flexion / extension angle, ulnar deviation or radial deviation angle, pronation or supination angle, and acceleration components in three directions, to form a wrist feature sequence that is highly consistent with the timing of lower limb movements. This sequence is organized with time as the main axis, aligning the local movement trajectory of the wrist with different phases in the entire lower limb movement process, thereby enhancing the mapping stability and physical consistency of the inference model. Based on the wrist feature time series data, a lower limb joint angle estimation model is applied. A set of mapping weights and non-linear transformation structures are embedded in this model, which can comprehensively consider the change trends of wrist angles and accelerations to calculate the angle estimation values of the knee joint, ankle joint, and hip joint at the corresponding time points. These estimations initially restore the movement path of the lower limb during the entire transition period. Biomechanical constraints are imposed on the initial lower limb joint angle values for physical rationality verification and adjustment. This biomechanical constraint is mainly controlled from two dimensions. On the one hand, the range of each joint angle in space is restricted to ensure that it fluctuates within the physiologically allowed angle range, avoiding the output of non-human possible movement angles. On the other hand, restrictions are imposed on the temporal continuity of angle changes, that is, the angle change rate of joints such as the knee, ankle, and hip at consecutive time points cannot exceed a certain set threshold, and the coordination relationship between different joints is required to conform to the kinematic characteristics of the current movement transition type. For example, during the "walk-run" transition, the flexion / extension rate of the knee joint needs to increase significantly, while during the "flat walk-downstairs" transition, it is required that the angle adjustment dominated by the ankle joint has an obvious buffering trend. Through physical and physiological restrictions, dynamic adjustment is performed on the initial angle sequence to obtain the lower limb joint angle sequence.

[0029] In the embodiments of the present invention, through a processing method that combines anatomical coordinate transformation and dynamic consistency constraint, the problem of low signal-to-noise ratio of the wrist position sensor signal is effectively solved, ensuring the consistency of data in the anatomical sense. By adopting a strategy that combines sliding window segmentation and multi-scale wavelet packet decomposition, transient features in the action transition period can be accurately captured, especially for conversion signals with short duration and insignificant features, enhancing the recognition ability and avoiding the limitation that traditional single-domain analysis in the time domain or frequency domain cannot distinguish short-time transitions from noise. By extracting time-domain features, frequency-domain features, joint angle features, and pose estimation features and fusing them, a multi-dimensional feature representation that describes the relationship between wrist movement and full-body action conversion is constructed, significantly improving the mapping accuracy between wrist data and full-body action conversion. Based on the cascaded processing mechanism of the hierarchical conversion recognition architecture, which includes three modules: a conversion trigger detector, a conversion type classifier, and a conversion completion validator, real-time and efficient recognition of action conversion is achieved, avoiding the limitations of a single model and maintaining a high recognition rate with low computational complexity. A body part association mapping and lower limb action state inference mechanism is designed to effectively solve the problem that it is difficult for wrist sensors to directly capture lower limb actions, realizing an effective mapping from wrist actions to lower limb states and providing technical support for the complete representation of full-body action conversion.

[0030] In a specific embodiment, the process of executing step 101 may specifically include the following steps: Collect raw motion data through the triaxial acceleration sensor and triaxial gyroscope in the wrist-worn device, and construct a raw acceleration vector and a raw angular velocity vector based on the raw motion data; Use the anatomical coordinate transformation matrix parameterized by the wrist joint range of motion to perform transformation processing on the raw acceleration vector and the raw angular velocity vector to obtain mapping data from the local coordinate system to the human reference coordinate system; Calculate the dynamic consistency constraint value according to the mapping data, and make a determination on the dynamic consistency constraint value based on a preset constraint threshold to obtain an outlier marker set; Perform cubic spline interpolation replacement on the data points marked in the outlier marker set to obtain an acceleration sequence and an angular velocity sequence.

[0031] Specifically, through the triaxial acceleration sensor and triaxial gyroscope module arranged on the wrist part, continuous high-frequency sampling is carried out on the limb dynamic process of the wearer. The change trajectories of linear acceleration and angular velocity in three-dimensional space are recorded in real time at a sampling frequency of 100 Hz. Among them, the acceleration sensor obtains the linear acceleration components of the wrist in the X, Y, and Z directions, while the gyroscope outputs the angular velocity components of the wrist rotating around the three coordinate axes. By synchronously collecting these two groups of data, the original acceleration vector and the original angular velocity vector are respectively constructed to describe the spatial dynamic information of the wrist in the motion state. The original vectors are spatially normalized through a coordinate transformation mechanism based on anatomical structure. In the coordinate normalization stage, the sensor output data is rotated and position-corrected based on the three-degree-of-freedom range of motion of the wrist anatomical structure. The palmar flexion and dorsiflexion, ulnar deviation and radial deviation, pronation and supination of the wrist constitute a three-dimensional movement angle space, which are respectively parameterized as rotation angle variables. The system defines a set of coordinate transformation matrices under anatomical constraints according to these angles. This matrix integrates both rotation and displacement information and aims to map the data collected by the local sensors on the wrist to the human reference coordinate system based on the torso. During this process, all the original acceleration vectors and original angular velocity vectors are uniformly transformed through this transformation matrix, making them anatomically consistent in space and avoiding deviations in data interpretation caused by different wearing postures or wrist structure differences between individuals. A constraint function based on the rigid body dynamics model is introduced to construct a calculation mechanism for verifying the logical consistency of acceleration and angular velocity. This mechanism jointly analyzes the transformed acceleration vector and angular velocity vector. Under the ideal rigid body model, the linear acceleration of the wrist is jointly composed of the gravity term, the centripetal acceleration term generated by the change in angular velocity, and the tangential acceleration under the action of angular acceleration. The system constructs a dynamic consistency constraint value based on this, that is, calculates the difference between the ideal acceleration and the actually observed acceleration, reflecting whether the current motion state conforms to physical laws. When this constraint value exceeds the preset threshold, it indicates that this data point is affected by non-physiological interference and cannot accurately describe the true motion state of the wrist. This point is marked as an outlier and recorded in the outlier marking set. The data points marked in the outlier marking set are replaced by cubic spline interpolation. Several normal data points before and after each outlier are selected on the time axis, and the abnormal position is filled by constructing a cubic spline interpolation function to obtain the acceleration sequence and the angular velocity sequence.

[0032] In a specific embodiment, the process of executing step 102 may specifically include the following steps: The acceleration sequence and the angular velocity sequence are respectively segmented by a sliding window to obtain an acceleration window sequence and an angular velocity window sequence; Calculate the first derivative change rate of the acceleration vector modulus for the acceleration window sequence, and calculate the standard deviation of the angular velocity modulus for the angular velocity window sequence; Compare and judge the change rate of the first derivative with a first preset threshold, and at the same time compare and judge the standard deviation with a second preset threshold to obtain a set of potential conversion period windows; Perform four-layer multi-scale wavelet packet decomposition on the signal data in the set of potential conversion period windows to obtain a set of wavelet coefficients at four scales; Perform band signal reconstruction on the set of wavelet coefficients at four scales to obtain multi-scale time-frequency features.

[0033] Specifically, the acceleration sequence and the angular velocity sequence are respectively segmented by a sliding window to construct an acceleration window sequence and an angular velocity window sequence with temporal locality and continuity. In the sliding window design, a window parameter with a fixed time length is selected, and an appropriate overlap ratio is set to ensure that key features are not missed in the locally varying region of the time series. The typical parameter settings are a window width of 400 milliseconds and a window overlap ratio of 75%, so that each action segment is covered by multiple windows to enhance the perception ability of short-term dynamic features. After the window segmentation is completed, the original continuous time series is divided into subsequences in the form of multiple time segments, and each subsequence contains three-dimensional acceleration or three-dimensional angular velocity data, respectively constituting the acceleration window sequence and the angular velocity window sequence. Feature quantization analysis is performed on the signals within these windows to detect the signs of action transitions. For the acceleration window sequence, the time change rate of the acceleration vector modulus is calculated within each window, and a first-order derivative change rate index is constructed to reflect the sudden change trend of the overall acceleration amplitude in the time dimension. The higher the derivative change rate of the acceleration modulus, the more significant the change in the motion state in the current time period, which is manifested as posture switching, sudden speed increase, or center of gravity transfer in actual action transitions. For the angular velocity window sequence, the standard deviation of the angular velocity modulus within the window is calculated to measure the rotation intensity and instability degree of the wrist during this time period. The larger the standard deviation, the more intense the action amplitude, which is also a signal of a drastic change in the motion state. The acceleration derivative change rate is compared with a preset first threshold, and at the same time, the angular velocity standard deviation is compared with a second threshold. Only when the two indicators of a certain window exceed the threshold range at the same time, it is marked as a candidate window that may be in the action transition state. The dual judgment mechanism improves the accuracy and reliability of the transition detection and effectively reduces the misjudgment caused by individual signal noise or short-term fluctuations. Through the joint judgment process, the time segments that meet the conditions are extracted from all windows and assembled into a set of potential transition period windows. Four-layer multi-scale wavelet packet decomposition is performed on the signal data in the set of potential transition period windows. The wavelet packet decomposition has the ability to perform local analysis of signals in both the time domain and the frequency domain, and can sensitively capture the short-term mutation features in non-stationary signals. The number of layers of the wavelet packet decomposition is set to four. In each layer, the original signal is divided into sub-bands with double the number, and until the fourth layer, 16 subdivided band regions are formed. These bands cover multiple scales from high frequency to low frequency and can effectively cover the typical frequency components in common action transition signals. Through each layer of decomposition, the wavelet packet algorithm extracts the coefficient sets corresponding to multiple sub-bands, respectively records the signal energy distribution and structural changes at different scales, and the system outputs these coefficient sets as the intermediate frequency domain representation, constituting a wavelet coefficient set of four scales to express the signal structure change trajectory during the action transition period.Perform the band reconstruction operation on the above four-layer coefficient sets respectively, and recombine the wavelet functions corresponding to each band coefficient back to the original signal time length, so as to restore the sub-band time-domain signals at each scale. These reconstructed band signals together constitute the multi-scale time-frequency feature set of the transition period window, which can reveal the changing trends of different frequency components on the time axis while retaining the time information of the original signal.

[0034] In a specific embodiment, the process of performing step 103 may specifically include the following steps: Calculate the time-domain feature set including kurtosis, skewness, zero-crossing rate, signal vector amplitude and signal amplitude area according to the signal data in the potential transition period window set; Calculate the frequency-domain feature set including energy ratio, band entropy, main frequency drift, power spectrum kurtosis and power spectrum entropy based on the band signals of the multi-scale time-frequency features; Obtain the target angular velocity sequence from the potential transition period window set, and calculate the joint angle feature set including wrist flexion / extension angle, ulnar / radial deviation angle and pronation / supination angle in the target angular velocity sequence through quaternion integration; Perform normalized feature fusion based on the time-domain feature set, frequency-domain feature set and joint angle feature set to obtain the transition period feature vector, which is used to describe the conversion relationship between wrist movement and whole-body movement.

[0035] Specifically, in the time-domain feature extraction stage, statistical analysis is performed on the three-axis acceleration and angular velocity signals contained in each potential transition period window. A time-domain description vector is constructed by analyzing the distribution structure and fluctuation characteristics of the signals on the time axis. The kurtosis of the signal is calculated to measure the sharpness of the signal segment and reflect whether there are sudden high-amplitude events. At the same time, the skewness index is calculated. This value represents the asymmetry of the signal about its mean and can reflect whether there are continuous directional changes during the movement, such as rotating or deviating in a certain direction for a long time. To evaluate the degree of frequent signal direction changes, its zero-crossing rate is statistically analyzed. The oscillation behavior is indirectly measured by recording the number of times the signal jumps from positive to negative or from negative to positive. Especially in the action transition edge region, this index has extremely strong discriminative ability. To more intuitively quantify the overall movement amplitude and energy release level of the wrist, the signal vector magnitude and the signal amplitude area are calculated. The signal vector magnitude represents the modulus of the three-axis acceleration vector at each sampling moment, and the overall movement intensity of the wrist is described by taking the time average. The signal amplitude area reflects the overall accumulation of signal energy during the time period by summing the absolute values in three directions. Frequency-domain feature extraction is performed on the signal based on multi-scale time-frequency features. At each decomposition scale, the energy ratio of the reconstructed band signal is calculated respectively to construct an energy ratio vector, which reflects whether the signal energy distribution is concentrated or dispersed at each scale. At the same time, the frequency components in each frequency band are normalized, and its band entropy is calculated as an energy dispersion metric index for the signal in this frequency band. To characterize the sensitivity of the rhythm change during the action transition, the offset amplitude of the dominant frequency is tracked in adjacent windows, and the dominant frequency drift value is used as an important feature reflecting the action rhythm stability. At the same time, in the entire power spectrum structure, the sharpness of the peak is extracted to form the power spectrum kurtosis index, and the spectral entropy is calculated after normalizing the power spectral density to measure the disorder and complexity of the overall frequency distribution. The system enters the joint angle feature extraction stage. The core task of this stage is to obtain the target angular velocity sequence from the set of potential transition period windows and construct a time-continuous attitude representation trajectory based on this sequence through quaternion integration. During the processing, the angular velocity is input into the integration module, and the time-series evolution of the rotation attitude is established in quaternion form. The quaternion sequence describing the wrist direction change is obtained through step-by-step integration, and then it is converted into Euler angle parameters with anatomical significance, so as to calculate the flexion-extension angle, ulnar deviation or radial deviation angle, and pronation or supination angle of the wrist during the transition period respectively. In addition to the static angle values, the first derivative and the second derivative of each angle on the time axis are calculated to obtain the angular velocity and angular acceleration information, constituting the dynamic feature representation of the joint angle change. These angle information not only have physical interpretability but also can directly reflect the structural connection between the current wrist movement and the whole body posture transition.After the above three types of feature extraction are completed, all feature vectors are normalized to ensure that feature indicators with different dimensions and different units have the same scale and comparability in subsequent model inputs. The normalization method selects the maximum-minimum scaling or Z-score standardization method, and the parameters are dynamically adjusted according to the distribution of features in the training set. The normalized time-domain feature set, frequency-domain feature set, and joint angle feature set are concatenated and fused to form a feature vector during the transition period. This feature vector encompasses multi-source information describing the local movement trajectory of the wrist, the evolution process of the signal spectrum, and the physiological joint posture state.

[0036] In a specific embodiment, the process of executing step 104 may specifically include the following steps: Input the feature vector during the transition period into the transition trigger detector in the hierarchical transition recognition model for lightweight decision tree judgment to obtain a transition trigger probability value; Compare and judge the transition trigger probability value with the user variance adaptive threshold to obtain a transition trigger signal; Based on the transition trigger signal, input the feature vector during the transition period into the transition type classifier in the hierarchical transition recognition model for classification to obtain the probability distribution of each transition category; Select the category corresponding to the highest probability from the probability distribution of each transition category as the initial type label; Verify the temporal consistency of the initial type label to obtain the transition type label and the transition confidence level.

[0037] Specifically, each conversion period feature vector to be judged is input into the conversion trigger detector for analysis. This detector is constructed using a lightweight decision tree structure and has characteristics such as fast operation speed, clear decision-making logic, and strong adaptability. Its judgment basis comes from several core indicators with high discrimination in the feature vector, such as signal vector amplitude, main frequency drift amplitude, joint angular acceleration, and attitude change rate, etc. During the training stage, the system selects several features with the strongest trigger prediction ability through the feature importance scoring method as the main judgment nodes of the decision tree. When the conversion period feature vector is input into this lightweight decision tree, the model makes a hierarchical determination of the signal state through node condition splitting layer by layer. At each branch node, the input vector is classified into different paths according to whether a specific feature value meets the set conditions. At the leaf node, the probability value that the current feature vector is judged to "contain a conversion action" is output, that is, the conversion trigger probability value. This value represents the likelihood of an action conversion occurring within the current window, and its magnitude is controlled by the comprehensive performance of the features and is optimized through prior probability, tree structure depth, and sample distribution diversity. The conversion trigger probability value is compared with a dynamically calculated threshold to determine whether to confirm the trigger conversion signal. This threshold is constructed based on the behavioral fluctuation data accumulated by the user during long-term use and has an adaptive characteristic. The specific implementation method is that the system calculates the probability fluctuation variance of the user in the non-conversion state in the historical data and fuses it with the basic probability threshold through weighting to form a user-specific dynamic threshold, ensuring that true conversion signals and natural fluctuation behaviors can be accurately distinguished under different users, different times, and different motion modes. When the conversion trigger probability value is greater than this adaptive threshold, the system confirms that there is a real state conversion behavior at the current moment and generates a conversion trigger signal. After the conversion trigger signal is confirmed to be valid, the same conversion period feature vector is input into the second stage of the hierarchical conversion recognition model, that is, the conversion type classifier, for category determination. This classifier is constructed using a deep structure, and its core is a bidirectional long short-term memory network structure that introduces residual connections and attention mechanisms, which can simultaneously model the time-dependent relationship and local feature activation state during the action conversion process. In this model, the input conversion period feature vector enters the bidirectional LSTM layer, and the phased changes in the action evolution process are extracted separately in the forward and backward time series directions, and a hidden representation with time context awareness is formed in the memory unit; then this hidden state is connected with the original input features through a residual connection, so that the final expression vector not only retains the original structural information but also integrates time dependence and context dynamic features. In order to enhance the model's perception ability of key state nodes, a dynamic attention module based on the gating mechanism is introduced to weight the importance of different positions in the time series, thereby amplifying the feature response in the action change burst period and suppressing the influence of irrelevant or stable intervals, and outputting a probability distribution covering all action conversion categories.Each item in the probability distribution corresponds to a specific conversion type, such as standing - sitting, walking - running, walking on flat ground - going downstairs, etc. The probability value of each category reflects the matching degree of the current input feature vector to the action type. The system selects the category corresponding to the maximum probability as the current initial type label, which is the action conversion state that the model currently believes is most likely to occur. Perform temporal consistency verification on the initial type label. By constructing a Markov logic network model, the conversion path between the current label and the historical label sequence is constrained and verified. Its core mechanism lies in introducing a set of predefined temporal rules and logical relationships, including the duration constraint of the conversion action, the amplitude limit of the difference between adjacent labels, the judgment of pose physical consistency, and the evaluation of the acceleration / angular velocity change trend that conforms to physiological feasibility. Trace back the initial label on the time axis and compare its logical continuity with adjacent labels, and calculate the energy function value generated in the logical network. If the energy value is lower than the preset cost threshold, it is determined that the label passes the verification and is finally output as the official conversion type label; if it fails, label correction or a suspension confirmation mechanism may be performed to prevent incorrect labels from flowing into the subsequent action recognition link. At the same time, jointly calculate the conversion confidence based on the original maximum probability value output by the classifier and the temporal consistency evaluation result in the validator. This confidence reflects the signal matching strength of the model's current judgment result and reflects its reasonableness in the time logic and physical motion chain.

[0038] In a specific embodiment, the process of inputting the conversion - period feature vector into the conversion - type classifier in the hierarchical conversion recognition model based on the conversion trigger signal for classification to obtain the probability distribution of each conversion category may specifically include the following steps: Based on the conversion trigger signal, input the conversion - period feature vector into the bidirectional LSTM layer in the conversion - type classifier for temporal modeling to obtain a bidirectional temporal feature vector; Input the bidirectional temporal feature vector into the wrist - whole - body action mapping residual network in the conversion - type classifier for conversion relationship modeling. Through residual connection, retain the combination of the original feature and the learned vector, and at the same time calculate the dynamic attention gating coefficient using the conversion - period context information to obtain an enhanced residual feature vector; Input the enhanced residual feature vector into the softmax classification layer in the conversion - type classifier for conversion - type calculation to obtain the probability distribution of each conversion category, where the conversion categories include standing - sitting, sitting - standing, walking - running, walking - standing, standing - walking, running - walking, going upstairs - walking on flat ground, going downstairs - walking on flat ground, walking on flat ground - going upstairs, walking on flat ground - going downstairs, standing - running, running - standing.

[0039] Specifically, based on the conversion trigger signal, the conversion period feature vector is input into the bidirectional LSTM layer in the conversion type classifier for temporal modeling to capture the dynamic change law of features in the time dimension, and information about past states and future trends is obtained simultaneously through the recursive calculation paths in the forward and backward directions. The forward LSTM encodes the evolution direction of the current feature vector in time, gradually accumulating state information from the initial moment to the current moment; while the backward LSTM traces back from the current moment to the past to model the influence of future states on the current node. By concatenating the hidden state outputs in the two directions, a set of bidirectional temporal feature vectors that integrate the front and back time information is formed. The bidirectional temporal feature vectors are input into the wrist-to-whole body motion mapping residual network inside the classifier for conversion relationship modeling. The design of this residual network adopts a multi-layer feed-forward structure and introduces a residual connection mechanism, enabling the direct superposition of the original input features and the high-order semantic vectors obtained from deep learning in structure to form a set of combined features. This mechanism can effectively prevent the problem of feature degradation or gradient disappearance in the training of deep networks while retaining the key information in the original wrist features. The system embeds a context-aware module in this residual network, calculates the semantic weights of features at each time step through a dynamic attention gating mechanism, and the gating coefficients are dynamically allocated according to the correlation between the features at the current time point and the global context, so that the model can automatically identify the key points of action conversion and enhance its response, while suppressing the stable sections before and after the conversion, obtaining a set of structured enhanced residual feature vectors. These vectors structurally integrate temporal dynamic information, context attention weights, and residual path outputs, and semantically contain multi-dimensional information such as conversion intention, action amplitude, and posture change trend. The enhanced residual feature vectors are input into the softmax classification layer of the classifier for the final determination of the action type probability. The softmax layer, as a multi-classification output module, performs exponential mapping on each output node after receiving high-dimensional input and normalizes all output values into a probability distribution, making the output value of each category fall between 0 and 1, and the sum of the probabilities of all categories being 1. The matching probabilities of each action conversion category are calculated respectively according to the fitting degree between the enhanced residual features and each category, and finally a probability vector containing 12 conversion types is output. The 12 action conversion categories preset in this classifier are standing-sitting, sitting-standing, walking-running, walking-standing, standing-walking, running-walking, going upstairs-walking flat, going downstairs-walking flat, walking flat-going upstairs, walking flat-going downstairs, standing-running, and running-standing, and each category represents a common dynamic transition mode of a human body posture in real-life scenarios. The system learns the typical feature representations of various conversions through a large amount of labeled data during the training phase, so that in the prediction phase, corresponding probability values can be given based on the similarity of the enhanced feature vectors in each category.For example, when the system detects a rapid forward tilt of the wrist posture, a sharp downward change in acceleration accompanied by a sudden change in joint angle, the model tends to output a high probability of the standing-sitting category. When it is recognized that the angular velocity rises sharply and the frequency peak moves forward after the smooth movement rhythm is interrupted, it is more likely to be judged as the walking-running type. According to the probability distribution output by the softmax classification layer, the category corresponding to the maximum probability is selected as the initial recognition result of the current action conversion, and the probability value corresponding to this category is used as a confidence index to participate in subsequent temporal verification and fusion judgment.

[0040] In a specific embodiment, the process of executing step 105 may specifically include the following steps: Extract the wrist azimuth angle and wrist joint bending angle from the transition period feature vector based on the transition type label, and perform mapping transformation on the wrist azimuth angle and wrist joint bending angle using the body part association mapping matrix to obtain the initial lower limb state data; Select the corresponding mapping parameter set according to the transition type label, and map the initial lower limb state data based on the mapping parameter set to obtain the lower limb state parameters of different transition types; Extract the initial swing phase identifier, wrist vertical acceleration feature point and frequency domain periodicity information from the transition period feature vector, and perform gait analysis to obtain the gait cycle and foot off / landing time data; Based on the gait cycle and foot off / landing time data, use the lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle and hip joint angle to obtain the lower limb joint angle sequence; Perform dynamic conversion processing on the lower limb joint angle sequence using a time-varying mapping function, and at the same time identify the vertical motion component of the conversion type to obtain the lower limb state sequence and the whole body action conversion type during the conversion process.

[0041] Specifically, based on the conversion type tag, the wrist azimuth angle and the wrist joint flexion angle are extracted from the conversion period feature vector. These two types of angle information are obtained through quaternion integration of the angular velocity sequence and have clear anatomical significance and temporal continuity. These two types of angle data are input into a pre-constructed body part association mapping matrix, which is constructed by supervised learning from a large number of multi-source labeled samples. Its structure defines the non-linear mapping relationship between wrist angle changes and lower limb state responses. Taking the wrist azimuth angle and the wrist joint angle as three-dimensional input vectors, the corresponding initial lower limb state data is obtained through matrix operations. According to the currently recognized conversion type tag, the mapping parameter set corresponding to this type of action is called in the parameter library. Different conversion types have differences in spatial path, angle amplitude, and action rhythm. Therefore, each set of conversion types needs to be equipped with its independent parameter model to ensure the accuracy and adaptability of the mapping. Using the previously obtained initial lower limb state data as input, and combining the mapping weight matrix, bias vector, and transformation coefficient corresponding to the current type to perform feature transformation. Through this process, the structural but unclassified lower limb basic state data is mapped into lower limb state parameters with action discrimination features. Taking "stand-run" and "stand-walk" as examples, although the wrist angle change trends are similar, the system applies amplitude modulation and rhythm regulation in the output space through different types of mapping parameters, making the two actions show significant differences in the lower limb state expression, such as the knee joint flexion rate, ankle joint release angle, etc. After obtaining the lower limb state parameters specific to the action type, information features related to gait modeling are extracted from the conversion period feature vector for deriving the gait cycle and the critical moments of foot lift-off / landing. By analyzing the acceleration structure of the wrist during the conversion process, local maximum and minimum points in the vertical acceleration are identified, and these points correspond highly to the foot lift-off or touchdown actions in the gait. Secondly, the action rhythm pattern is extracted by combining the main frequency peak information identified in the frequency domain analysis, thereby constructing an estimation function of the gait cycle. At the same time, an initial swing phase identifier is constructed by combining the wrist flexion and extension angle and the acceleration change trend. This indicator is used to mark the gait sub-phase (swing period or support period) where the current window is located, thereby marking the accurate time points of foot lift-off and landing on the time axis. Combining the above three information sources, a gait time series template is established, and the gait cycle length and foot event timestamp data are output. Based on the obtained gait cycle structure and foot critical moment data, the time-synchronized wrist signal and gait time series are jointly input into the lower limb joint angle estimation model. This model is a multi-input multi-output non-linear prediction system. Internally, the wrist angle feature, acceleration amplitude feature, gait cycle position, support / swing state, etc. are used as joint input variables through a weighted fusion method to output the predicted angle values of the knee joint, ankle joint, and hip joint at the current time point.The system considers the coordinated movement relationships between different joints in the model and introduces physical constraint rules to prevent non-physiological mutations in the output angles at consecutive time points. For example, the rotation speed of the ankle joint cannot exceed a certain threshold, and the hip joint angle cannot change reversely in a jump. During the entire movement transition period, this angle estimation process is continuously executed, thereby generating angle sequences of the knee joint, ankle joint, and hip joint in the time dimension, and performing multi-segment fitting and interpolation processing on them, so that the angle trajectory curve has structural continuity and biomechanical rationality. To improve the adaptability of the lower limb angle sequence to the dynamic characteristics of movement transition, a time-varying mapping function operation is executed after the angle sequence is generated. This function introduces a dynamic adjustment factor on the time axis, enabling the joint angles to smoothly transition from the starting state to the ending state, and at the same time adjusting the angle change rate at each moment according to the progress of the conversion completion. The system uses a cubic interpolation function to construct the time coefficient, enabling the conversion process to be completed under the natural rhythm of slow start in the early stage, rapid change in the middle stage, and gradual convergence in the later stage, thereby enhancing the modeling ability of the nonlinear characteristics during the movement transition process. Moreover, in order to distinguish special conversion types involving vertical changes in the center of gravity, such as "flat walking - going upstairs" or "standing - squatting", a vertical movement discrimination mechanism is also introduced at this stage. By jointly analyzing indicators such as the integral, maximum value, and mean square deviation of the vertical acceleration to identify whether there is a significant vertical displacement trend currently, if so, the angle sequence is readjusted to reflect the actual movement structures such as hip elevation and knee folding. The lower limb joint angles corresponding to each time point during the entire movement transition process are output as a set of continuous structural data, constituting the lower limb state sequence, and the whole body movement conversion type in which the current movement is located is output in combination with the conversion type label.

[0042] In a specific embodiment, the process of performing the steps of calculating the knee joint angle, ankle joint angle, and hip joint angle using the lower limb joint angle estimation model based on the gait cycle and foot off - ground / landing time data to obtain the lower limb joint angle sequence may specifically include the following steps: Based on the foot off - ground / landing time data, the gait cycle is divided into a swing phase and a support phase, and weight distribution is performed on the correlation between the wrist characteristics and the lower limb movement according to different phases to obtain a gait phase weight sequence; According to the gait phase weight sequence, weight the wrist flexion / extension angle, ulnar deviation / radioulnar deviation angle, pronation / supination angle, and acceleration component in the transition period feature vector to obtain wrist feature time - series data synchronized with the lower limb movement; Based on the wrist feature time - series data, apply the lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle, and hip joint angle to obtain the initial lower limb joint angle values; Perform biomechanical constraints on the initial lower limb joint angle values to ensure that the joint angle change rate conforms to the kinematic characteristics of the conversion type, and obtain the lower limb joint angle sequence.

[0043] Specifically, based on the data of the foot-off / foot-strike moments, the gait cycle is divided into a swing phase and a stance phase. The swing phase refers to the period from when the foot lifts off the ground until it touches the ground again. During this phase, the lower limb is in a non-weight-bearing state, characterized by dynamic movement features such as rapid flexion and extension of the knee joint and adjustment of the ankle joint direction. The stance phase corresponds to the period when the foot is in contact with the ground. During this phase, the lower limb mainly bears the weight of the torso to maintain body stability, with relatively smaller movement amplitudes and stronger rhythmicity. According to the positions of foot events in the gait cycle, the range of each phase on the time axis is calibrated, and a gait phase time mapping table is constructed accordingly. Weight assignments are made to the correlation between wrist features and lower limb movements according to different phases. The upper-limb-lower-limb coordination relationships in the swing phase and the stance phase of the human body are not the same. For example, in the swing phase, the wrist angular velocity more sensitively reflects the initiation, swing, and acceleration changes of the lower limb, while in the stance phase, it shows a rhythmic vibration response. Therefore, to improve the contribution accuracy of wrist signals to the inference of lower limb states, different wrist feature weight parameters are set for the two gait phases respectively, forming a set of temporally structured gait phase weight sequences. The gait phase weight sequences are applied to multiple key parameters in the transition period feature vector that are highly correlated with the spatial movement state, including three types of angular features: wrist flexion / extension angle, ulnar deviation / radius deviation angle, and pronation / supination angle, as well as the acceleration components in the X, Y, and Z directions. By multiplying the above feature sequences by the weight factors corresponding to their respective phases at each time point, a set of wrist feature time series data that is aligned with the lower limb gait rhythm and has enhanced structural correlation under the guidance of phase information is generated. The wrist feature time series data is input into the lower limb joint angle estimation model, which is a composite function module that integrates physiological structure relationships, dynamic feature extraction, and non-linear regression capabilities. The model aligns the angle sequences, acceleration change rates, and dynamic parameters of each phase in the input features with the lower limb posture changes in the human biomechanical model through a mapping function, and combines the pre-trained multi-dimensional regression weight matrix and bias parameters to output the initial angle values of the knee joint, ankle joint, and hip joint in the current gait cycle. This estimation process is executed point by point based on time, forming an initial lower limb joint angle sequence that is the same length as the entire transition period duration. A biomechanical constraint mechanism is applied to the initial angle sequence. A change rate model for each joint is constructed, and its first derivative, i.e., the angular velocity, is dynamically monitored. A maximum allowable change rate threshold is set. When the angle change in a certain time period exceeds this threshold, the system automatically enters the correction mode, and methods such as time-weighted smoothing, bilateral constraint adjustment, or non-linear curvature adjustment are used to pull the angle change amplitude back to a reasonable range. At the same time, an inter-joint coordination constraint model is introduced to ensure that the mutual relationships between the knee, ankle, and hip joints conform to the movement patterns of specific action transitions. For example, in the "stand-sit" transition, the knee joint and the hip joint should maintain a highly coordinated flexion / extension trajectory, while in the "walk-run" process, it is required that the ankle joint releases before the hip and shows the maximum angle swing in the gait cycle.The system extracts typical angular change templates for various types of actions from the historical action library, and uses this as a reference to perform multi-segment matching and dynamic adjustment on the current initial sequence, thereby maintaining the integrity of the lower limb structure and the biological consistency of the actions throughout the conversion process. The finally output lower limb joint angle sequence has time continuity, amplitude rationality and structural coordination, and realizes a closed-loop mapping mechanism for driving lower limb modeling from upper limb signals through high coupling with wrist signals.

[0044] The multi-modal perception and motion state recognition method of the intelligent wearable device in the embodiment of the present invention is described above. Next, the multi-modal perception and motion state recognition device of the intelligent wearable device in the embodiment of the present invention will be described. Please refer to Figure 2 , an embodiment of the multi-modal perception and motion state recognition device of the intelligent wearable device in the embodiment of the present invention includes: A transformation module 201, configured to perform anatomical coordinate transformation on the original motion data collected by the wrist-worn device to obtain an acceleration sequence and an angular velocity sequence; A decomposition module 202, configured to perform sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi-scale time-frequency features; A construction module 203, configured to construct a transition period feature vector describing the conversion relationship between wrist movement and full-body movement based on the set of potential transition period windows and their corresponding multi-scale time-frequency features; A processing module 204, configured to input the transition period feature vector into a hierarchical transition recognition model for processing to obtain a transition type label and a transition confidence level; An inference module 205, configured to perform body part association mapping and lower limb motion state inference based on the transition type label and the transition confidence level to obtain a lower limb state sequence and a full-body motion conversion type during the conversion process.

[0045] Through the collaborative cooperation of the above-mentioned various components, by combining anatomical coordinate transformation with dynamic consistency constraints, the problem of low signal-to-noise ratio of the wrist position sensor signal is effectively solved, ensuring the anatomical consistency of the data. By adopting a strategy that combines sliding window segmentation with multi-scale wavelet packet decomposition, transient features during the action transition period can be accurately captured. In particular, it has an enhanced recognition ability for conversion signals with short duration and insignificant features, avoiding the limitation that traditional single-domain analysis in the time domain or frequency domain cannot distinguish short-time transitions from noise. By extracting time-domain features, frequency-domain features, joint angle features, and pose estimation features and fusing them, a multi-dimensional feature representation describing the relationship between wrist movement and full-body action transition is constructed, significantly improving the mapping accuracy between wrist data and full-body action transition. Based on the cascaded processing mechanism of the hierarchical transition recognition architecture, which includes three modules: a transition trigger detector, a transition type classifier, and a transition completion validator, real-time and efficient recognition of action transitions is achieved, avoiding the limitations of a single model and maintaining a high recognition rate with low computational complexity. A body part association mapping and lower limb action state inference mechanism is designed to effectively solve the problem that it is difficult for wrist sensors to directly capture lower limb actions, realizing an effective mapping from wrist actions to lower limb states, providing technical support for the complete representation of full-body action transitions.

[0046] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, systems, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0047] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0048] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-modal perception and motion state recognition method for an intelligent wearable device, characterized in that Comprising: Performing anatomical coordinate transformation on the original motion data collected by the wrist - worn device to obtain an acceleration sequence and an angular velocity sequence; Performing sliding window segmentation and multi - scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi - scale time - frequency features; Constructing a transition period feature vector that describes the conversion relationship between wrist movement and whole - body movement based on the set of potential transition period windows and their corresponding multi - scale time - frequency features; Inputting the transition period feature vector into a hierarchical conversion recognition model for processing to obtain a conversion type label and a conversion confidence; Performing body part association mapping and lower - limb movement state inference based on the conversion type label and the conversion confidence to obtain a lower - limb state sequence and a whole - body movement conversion type during the conversion process.

2. The multi-modal perception and motion state recognition method of the intelligent wearable device according to claim 1, wherein, The performing anatomical coordinate transformation on the original motion data collected by the wrist - worn device to obtain an acceleration sequence and an angular velocity sequence includes: Collecting original motion data through a tri - axis acceleration sensor and a tri - axis gyroscope in the wrist - worn device, and constructing an original acceleration vector and an original angular velocity vector based on the original motion data; Using an anatomical coordinate transformation matrix parameterized by the wrist joint range of motion to perform transformation processing on the original acceleration vector and the original angular velocity vector to obtain mapping data from the local coordinate system to the human reference coordinate system; Calculating a dynamic consistency constraint value according to the mapping data, and making a determination on the dynamic consistency constraint value based on a preset constraint threshold to obtain an outlier marker set; Performing cubic spline interpolation replacement on the data points marked in the outlier marker set to obtain an acceleration sequence and an angular velocity sequence.

3. The multi-modal perception and motion state recognition method of the intelligent wearable device according to claim 1, wherein The performing sliding window segmentation and multi - scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi - scale time - frequency features includes: Performing sliding window segmentation on the acceleration sequence and the angular velocity sequence respectively to obtain an acceleration window sequence and an angular velocity window sequence; Calculating the first - order derivative change rate of the acceleration vector modulus for the acceleration window sequence and calculating the standard deviation of the angular velocity modulus value for the angular velocity window sequence; Comparing and judging the first - order derivative change rate with a first preset threshold, and at the same time comparing and judging the standard deviation with a second preset threshold to obtain a set of potential transition period windows; Performing four - layer multi - scale wavelet packet decomposition on the signal data in the set of potential transition period windows to obtain a set of wavelet coefficients at four scales; Performing frequency - band signal reconstruction on the set of wavelet coefficients at the four scales to obtain multi - scale time - frequency features.

4. The multimodal perception and motion state recognition method of the smart wearable device according to claim 1, characterized in that, The constructing a transition period feature vector that describes the conversion relationship between wrist movement and whole - body movement based on the set of potential transition period windows and their corresponding multi - scale time - frequency features includes: Calculating a set of time - domain features including kurtosis, skewness, zero - crossing rate, signal vector amplitude, and signal amplitude area according to the signal data in the set of potential transition period windows; Calculating a set of frequency - domain features including energy ratio, frequency - band entropy, main frequency drift, power spectrum kurtosis, and power spectrum entropy based on the frequency - band signals of the multi - scale time - frequency features; Obtain a target angular velocity sequence from the set of potential transition period windows, and calculate a set of joint angle features including wrist flexion / extension angle, ulnar deviation / radioulnar deviation angle, and pronation / supination angle in the target angular velocity sequence through quaternion integration; Perform normalized feature fusion based on the time-domain feature set, the frequency-domain feature set, and the joint angle feature set to obtain a transition period feature vector, which is used to describe the conversion relationship between wrist movement and whole-body movement.

5. The multi-modal perception and motion state recognition method of the intelligent wearable device according to claim 1, characterized in that, Inputting the transition period feature vector into a hierarchical transition recognition model for processing to obtain a transition type label and a transition confidence, including: Input the transition period feature vector into a transition trigger detector in the hierarchical transition recognition model for lightweight decision tree judgment to obtain a transition trigger probability value; Compare and judge the transition trigger probability value with the user variance adaptive threshold to obtain a transition trigger signal; Based on the transition trigger signal, input the transition period feature vector into a transition type classifier in the hierarchical transition recognition model for classification to obtain a probability distribution of each transition category; Select the category corresponding to the highest probability from the probability distribution of each transition category as the initial type label; Perform temporal consistency verification on the initial type label to obtain a transition type label and a transition confidence.

6. The multimodal perception and motion state recognition method of the intelligent wearable device according to claim 5, characterized in that, Based on the transition trigger signal, input the transition period feature vector into a transition type classifier in the hierarchical transition recognition model for classification to obtain a probability distribution of each transition category, including: Based on the transition trigger signal, input the transition period feature vector into a bidirectional LSTM layer in the transition type classifier for temporal modeling to obtain a bidirectional temporal feature vector; Input the bidirectional temporal feature vector into a wrist-whole body movement mapping residual network in the transition type classifier for conversion relationship modeling. Through residual connection, the combination of the original feature and the learned vector is retained, and at the same time, the dynamic attention gating coefficient is calculated using the context information of the transition period to obtain an enhanced residual feature vector; Input the enhanced residual feature vector into a softmax classification layer in the transition type classifier for transition type calculation to obtain a probability distribution of each transition category, where the transition categories include stand-sit, sit-stand, walk-run, walk-stand, stand-walk, run-walk, upstairs-walk, downstairs-walk, walk-upstairs, walk-downstairs, stand-run, run-stand.

7. The multimodal perception and motion state recognition method of the smart wearable device according to claim 1, characterized in that, Based on the transition type label and the transition confidence, perform body part association mapping and lower limb movement state inference to obtain a lower limb state sequence and a whole body movement conversion type during the conversion, including: Extract the wrist azimuth angle and the wrist joint bending angle from the transition period feature vector based on the transition type label, and perform mapping transformation on the wrist azimuth angle and the wrist joint bending angle using a body part association mapping matrix to obtain initial lower limb state data; Select a corresponding mapping parameter set according to the transition type label, and map the initial lower limb state data based on the mapping parameter set to obtain lower limb state parameters of different conversion types; Extract the initial swing phase identifier, wrist vertical acceleration feature points, and frequency-domain periodicity information from the transition period feature vector, and perform gait analysis to obtain gait cycle and foot-off / foot-strike time data; Based on the gait cycle and the foot-off / foot-strike time data, use the lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle, and hip joint angle to obtain a lower limb joint angle sequence; Perform dynamic transformation processing on the lower limb joint angle sequence using a time-varying mapping function, and at the same time identify the vertical motion component of the transformation type to obtain the lower limb state sequence and the whole body motion transformation type during the transformation process.

8. The multimodal perception and motion state recognition method for the intelligent wearable device according to claim 7, wherein, The step of, based on the gait cycle and the foot-off / foot-strike time data, using the lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle, and hip joint angle to obtain a lower limb joint angle sequence, includes: Based on the foot-off / foot-strike time data, divide the gait cycle into a swing phase and a stance phase, and perform weight assignment on the correlation between the wrist features and the lower limb movement according to different phases to obtain a gait phase weight sequence; According to the gait phase weight sequence, weight the wrist flexion / extension angle, ulnar deviation / radioulnar deviation angle, pronation / supination angle, and acceleration component in the transition period feature vector to obtain wrist feature time series data synchronized with the lower limb movement; Based on the wrist feature time series data, apply the lower limb joint angle estimation model to calculate the knee joint angle, ankle joint angle, and hip joint angle to obtain the initial lower limb joint angle values; Perform biomechanical constraints on the initial lower limb joint angle values to ensure that the joint angle change rate conforms to the kinematic characteristics of the transformation type to obtain a lower limb joint angle sequence.

9. A multi-modal perception and motion state recognition device for an intelligent wearable device, characterized in that, A multi-modal perception and motion state recognition method for an intelligent wearable device for performing the method according to any one of claims 1-8, the multi-modal perception and motion state recognition device of the intelligent wearable device includes: A transformation module for performing anatomical coordinate transformation on the original motion data collected by the wrist wearable device to obtain an acceleration sequence and an angular velocity sequence; A decomposition module for performing sliding window segmentation and multi-scale wavelet packet decomposition on the acceleration sequence and the angular velocity sequence to obtain a set of potential transition period windows and their corresponding multi-scale time-frequency features; A construction module for constructing a transition period feature vector describing the relationship between wrist movement and whole body motion transformation based on the set of potential transition period windows and their corresponding multi-scale time-frequency features; A processing module for inputting the transition period feature vector into a hierarchical transition recognition model for processing to obtain a transition type label and a transition confidence; An inference module for performing body part association mapping and lower limb motion state inference based on the transition type label and the transition confidence to obtain a lower limb state sequence and a whole body motion transformation type during the transformation process.

Images