Otolithiasis self-resetting auxiliary system based on position sensor

Abstract

The invention belongs to the field of intelligent medical assistance, and particularly relates to an otolithiasis self-resetting assistance system based on a position sensor, which comprises a sensor module, an image acquisition module, an intelligent resetting analysis module, a feedback interaction module and a central control module, according to the method, the otolith position, the head posture and the limb posture of the patient are combined, the head posture characteristics and the limb posture characteristics during otolith resetting are matched, the limb posture of the patient is guided, and the resetting efficiency is improved; according to the method, a state space model based on partition optimization is creatively adopted, and partition features of different partitions of human limbs are captured in a human body partition mode on the basis of comprehensive multi-direction limb posture dynamic feature analysis by combining channel division, segmentation and zero-order maintenance-based state space model processing; and the partition features and the overall features are fused and optimized based on a channel mechanism, so that complex limb feature recognition and accurate action guidance are realized, and accurate resetting is realized.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent medical assistance, and in particular is an otolithiasis self-resetting assistance system based on a position sensor. Background Art

[0002] Otolithiasis is an inner ear disease, usually caused by the detachment or displacement of otoliths (a calcium crystal-like structure in the vestibular organ of the inner ear) in the semicircular canals, which causes dizziness when the head position changes. Self-repositioning technology is a method commonly used to treat otolithiasis. Through specific head postures and movements, it helps to reposition the detached or displaced otoliths to their normal position, thereby reducing or eliminating the symptoms of dizziness. Existing self-repositioning methods for otolithiasis usually require auxiliary repositioning through specific equipment or the patient's own posture adjustment for repositioning. The repositioning efficiency is low and it is difficult to reposition accurately, resulting in a high probability of recurrence of otolithiasis. Summary of the invention

[0003] In order to overcome the defects of the prior art and address the problem of low resetting efficiency of the existing self-resetting methods for otolithiasis, the present invention, through the idea of ​​computer vision, combines the patient's otolith position, head posture and limb posture, matches the head posture characteristics during otolith resetting with the limb posture characteristics, and guides the patient's limb posture with the matching results, thereby effectively improving the resetting efficiency; in view of the problem that the prior art is difficult to accurately reposition, resulting in a high probability of recurrence of otolithiasis, the present invention creatively adopts a state-space model based on partition optimization, realizes efficient parallel processing of different limb movement characteristics through channel division, introduces channel segmentation and state-space model processing based on zero-order hold, and on the basis of comprehensive multi-directional limb posture dynamic feature analysis, performs rapid feature extraction through dimensionality reduction, and at the same time, captures the partition features of different partitions of the human limbs in a human body partitioning manner, and fuses the partition features with the overall features and optimizes the processing based on the channel mechanism, thereby realizing the recognition of complex limb features, accurately guiding the patient's limb movements, and achieving accurate resetting.

[0004] The present invention provides a position sensor-based otolithiasis self-reset auxiliary system, comprising a sensor module, an image acquisition module, an intelligent reset analysis module, a feedback interaction module and a central control module;

[0005] The sensor module uses an inertial sensor to collect the posture changes of the patient's head in real time and send the head posture changes to the intelligent reset analysis module;

[0006] The image acquisition module acquires the patient's overall image video and otolith position in real time through a camera, and sends the overall image video and otolith position to the intelligent repositioning analysis module;

[0007] The intelligent resetting analysis module uses a state space model based on partition optimization to extract the human body dynamic features in the patient's overall image video, and adopts a spatiotemporal feature extraction method to extract the corresponding spatiotemporal features of the head posture change. When the otolith position is reset, the human body dynamic features and the corresponding spatiotemporal features of the head posture change are recorded to obtain the corresponding relationship between the overall image video and the head posture change when the otolith is reset, and the corresponding relationship is sent to the feedback interaction module;

[0008] The feedback interaction module reminds the patient how to adjust his / her limb posture through the corresponding relationship and the real-time changes in the patient's head posture;

[0009] The central control module directly controls all modules and stores information.

[0010] Furthermore, in the intelligent reset analysis module, a processing of a state space model based on partition optimization specifically includes the following steps:

[0011] Step S1: skeleton sequence extraction, using the human key point detection model to extract the human key point sequence in the overall image video , ,in, represents the sequence length, represents the number of key points, Represents the key point coordinates;

[0012] Step S2: Feature extraction and segmentation, using three linear layers to transform the human key point sequence Projection is performed to obtain a high-dimensional embedding vector ,in Represents the number of channels of the high-dimensional embedding vector;

[0013] Step S3: Spatiotemporal Mamba module processing based on partition optimization to process high-dimensional embedding vectors Perform L-layer partition-optimized spatiotemporal Mamba module processing to obtain an optimized embedding vector;

[0014] Step S4: multi-layer optimization processing, performing multi-level partition-optimized spatiotemporal Mamba module processing on the optimized embedding vector and performing feature activation to obtain human body dynamic features.

[0015] Further, in step S3, the spatiotemporal Mamba module processing based on partition optimization specifically includes the following steps:

[0016] Step S31: Channel segmentation, high-dimensional embedding vector Perform channel segmentation to obtain three segmentation vectors : [ X s , X m , X t ]=Split(Linear( X C )) ;

[0017] In the formula, represents the fully connected layer processing, represents the channel split operation, Represents the segmentation vector, and its number of channels is , , ;

[0018] Step S32: Based on the multi-head attention mechanism processing of the partition-optimized Mamba module, the three segmentation vectors Perform multi-head attention mechanism processing based on partitioned Mamba module: ; ; ;

[0019] In the formula, Represents a single-layer GCN network with a learnable matrix, Represents partition optimization Mamba module processing, represents a one-dimensional convolution operation, , and are the three segmentation vectors after processing;

[0020] Step S33: channel splicing, splicing the three processed segmentation vectors to obtain a splicing vector: ;

[0021] In the formula, represents the channel splicing function, represents the concatenation vector;

[0022] Step S34: Based on the feature refinement operation optimized by the feedforward network mechanism, the concatenated vector is subjected to the feature refinement operation optimized by the feedforward network mechanism to obtain the optimized embedding vector: ;

[0023] In the formula, represents the layer normalization operator, GELU represents the activation function based on the Gaussian error function, Represents optimized embedding vector.

[0024] Further, in step S32, the partition-based optimization Mamba module processing specifically includes the following steps:

[0025] Step S321: Skeleton key point division, segmentation vector Perform division along the channel dimension to obtain four tensors with the same channel dimension: [ X t→s , X s→t , X t←s , X s←t ]=Split( X m ) ;

[0026] In the formula, represents the resulting tensor;

[0027] Step S322: Zero-order hold-based state space model processing is performed on four tensors with the same channel dimension respectively. ; ; ; ;

[0028] In the formula, , represents the evolution matrix with time scale parameter, represents the general evolution matrix, represents the natural exponential function treatment, is the zero-order hold parameter, Represents the hidden state at the previous moment, Represents the current hidden state, represents the input sequence, i.e., a tensor, Represents the predicted dynamic characteristics;

[0029] Step S323: predict dynamic feature splicing: ;

[0030] In the formula, represents the state-space model processing based on zero-order hold, Represents a splicing operation, represents a 2D bidirectional modeling operation, i.e., the processing of step S321 to step S323, Represents the overall dynamic characteristics of the human body;

[0031] Step S324: human body partitioning, decomposing the human body key points into 6 partitions, corresponding to key body parts: arms, legs and torso, and related combination partitions: arms-legs, arms-torso and legs-torso;

[0032] Step S325: Perform a partition-based 2D bidirectional modeling operation on the six combined partitions to obtain the dynamic characteristics of each partition: X p =C2DSS M p ( b p + X m [:, L p ]) ;

[0033] In the formula, Represents an index set, represents a learnable partition label, Represents a partition-based 2D bidirectional modeling operation; Represents the dynamic characteristics of each partition, An integer from 1 to 6, representing the partition order number;

[0034] Step S326: The overall-partition fusion strategy is to fuse the dynamic features of all partitions with the overall dynamic features to obtain the global optimized dynamic features: ;

[0035] In the formula, and represents the learnable parameters, Represents the overall dynamic characteristics, that is , Represents the global optimization dynamic characteristics;

[0036] Step S327: Channel mechanism optimization processing, global optimization of dynamic features and segmentation vectors Perform optimization processing based on the channel mechanism to obtain the results of partition optimization Mamba module processing : ; ;

[0037] In the formula, represents average pooling processing, represents the Sigmoid activation function, Representing Hadamard, The partitioning vector after processing by the partition optimization Mamba module.

[0038] The beneficial results achieved by the present invention using the above scheme are as follows:

[0039] (1) In order to solve the problem of low resetting efficiency of existing self-resetting methods for otolithiasis, the present invention combines the patient's otolith position, head posture and limb posture through the idea of ​​computer vision, matches the head posture characteristics with the limb posture characteristics during otolith resetting, and uses the matching results to guide the patient's limb posture, thereby effectively improving the resetting efficiency;

[0040] (2) In order to solve the problem that the existing technology is difficult to accurately reposition, resulting in a high probability of recurrence of otolithiasis, the present invention creatively adopts a state-space model based on partition optimization, realizes efficient parallel processing of different limb movement characteristics through channel division, introduces channel segmentation and state-space model processing based on zero-order hold, and performs rapid feature extraction through dimensionality reduction on the basis of comprehensive multi-directional limb posture dynamic feature analysis. At the same time, the partition features of different partitions of the human limb are captured in a human body partitioning manner, and the partition features are fused with the overall features and optimized based on the channel mechanism to realize the recognition of complex limb features, accurately guide the patient's limb movements, and achieve accurate repositioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A module diagram of a position sensor-based otolithiasis self-reset auxiliary system provided by the present invention;

[0042] Figure 2 Schematic diagram of the processing flow of a state-space model based on partition optimization.

[0043] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0045] Example 1, see Figure 1 , the present invention provides a position sensor-based otolithiasis self-reset auxiliary system, including a sensor module, an image acquisition module, an intelligent reset analysis module, a feedback interaction module and a central control module;

[0046] The sensor module uses an inertial sensor to collect the posture changes of the patient's head in real time and send the head posture changes to the intelligent reset analysis module;

[0047] The image acquisition module acquires the patient's overall image video and otolith position in real time through a camera, and sends the overall image video and otolith position to the intelligent repositioning analysis module;

[0048] The intelligent resetting analysis module uses a state space model based on partition optimization to extract the human body dynamic features in the patient's overall image video, and adopts a spatiotemporal feature extraction method to extract the corresponding spatiotemporal features of the head posture change. When the otolith position is reset, the human body dynamic features and the corresponding spatiotemporal features of the head posture change are recorded to obtain the corresponding relationship between the overall image video and the head posture change when the otolith is reset, and the corresponding relationship is sent to the feedback interaction module;

[0049] The feedback interaction module reminds the patient how to adjust his / her limb posture through the corresponding relationship and the real-time changes in the patient's head posture;

[0050] The central control module directly controls all modules and stores information.

[0051] By performing the above operations, in order to solve the problem of low resetting efficiency of existing self-resetting methods for otolithiasis, the present invention, through the idea of ​​computer vision, combines the patient's otolith position, head posture and limb posture, and matches the head posture characteristics during otolith resetting with the limb posture characteristics. The matching results are used to guide the patient's limb posture, thereby effectively improving the resetting efficiency.

[0052] Example 2, see Figure 2 This embodiment is based on the above embodiment. In the intelligent reset analysis module, a state space model based on partition optimization is processed, which specifically includes the following steps:

[0053] Step S1: skeleton sequence extraction, using the human key point detection model to extract the human key point sequence in the overall image video , ,in, represents the sequence length, represents the number of key points, Represents the key point coordinates;

[0054] Step S2: Feature extraction and segmentation, using three linear layers to transform the human key point sequence Projection is performed to obtain a high-dimensional embedding vector ,in Represents the number of channels of the high-dimensional embedding vector;

[0055] Step S3: Spatiotemporal Mamba module processing based on partition optimization to process high-dimensional embedding vectors Perform L-layer partition-optimized spatiotemporal Mamba module processing to obtain an optimized embedding vector;

[0056] Step S4: multi-layer optimization processing, performing multi-level partition-optimized spatiotemporal Mamba module processing on the optimized embedding vector and performing feature activation to obtain human body dynamic features.

[0057] By executing the above operations, in order to solve the problem that the existing technology is difficult to accurately reposition, resulting in a high probability of recurrence of otolithiasis, the present invention creatively adopts a state-space model based on partition optimization, and realizes efficient parallel processing of different limb movement characteristics through channel division. It introduces channel segmentation and state-space model processing based on zero-order hold, and performs rapid feature extraction through dimensionality reduction on the basis of comprehensive multi-directional limb posture dynamic feature analysis. At the same time, the partition features of different partitions of human limbs are captured in a human body partitioning manner, and the partition features are fused with the overall features and optimized based on the channel mechanism to realize the recognition of complex limb features, accurately guide the patient's limb movements, and achieve precise repositioning.

[0058] Embodiment 3: This embodiment is based on the above embodiment. In step S3, the spatiotemporal Mamba module processing based on partition optimization specifically includes the following steps:

[0059] Step S31: Channel segmentation, high-dimensional embedding vector Perform channel segmentation to obtain three segmentation vectors : [ X s , X m , X t ]=Split(Linear( X C )) ;

[0060] In the formula, represents the fully connected layer processing, represents the channel split operation, Represents the segmentation vector, and its number of channels is , , ;

[0061] Step S32: Based on the multi-head attention mechanism processing of the partition-optimized Mamba module, the three segmentation vectors Perform multi-head attention mechanism processing based on partitioned Mamba module: ; ; ;

[0062] In the formula, Represents a single-layer GCN network with a learnable matrix, Represents partition optimization Mamba module processing, represents a one-dimensional convolution operation, , and are the three segmentation vectors after processing;

[0063] Step S33: channel splicing, splicing the three processed segmentation vectors to obtain a splicing vector: ;

[0064] In the formula, represents the channel splicing function, represents the concatenation vector;

[0065] Step S34: Based on the feature refinement operation optimized by the feedforward network mechanism, the concatenated vector is subjected to the feature refinement operation optimized by the feedforward network mechanism to obtain the optimized embedding vector: ;

[0066] In the formula, represents the layer normalization operator, GELU represents the activation function based on the Gaussian error function, Represents optimized embedding vector.

[0067] Embodiment 4: This embodiment is based on the above embodiment. In step S32, the partition-based optimization Mamba module processing specifically includes the following steps:

[0068] Step S321: Skeleton key point division, segmentation vector Perform division along the channel dimension to obtain four tensors with the same channel dimension: [ X t→s , X s→t , X t←s , X s←t ]=Split( X m ) ;

[0069] In the formula, represents the resulting tensor;

[0070] Step S322: Zero-order hold-based state space model processing is performed on four tensors with the same channel dimension respectively. ; ; ; ;

[0071] In the formula, , represents the evolution matrix with time scale parameter, represents the general evolution matrix, represents the natural exponential function treatment, is the zero-order hold parameter, Represents the hidden state at the previous moment, Represents the current hidden state, represents the input sequence, i.e., a tensor, Represents the predicted dynamic characteristics;

[0072] Step S323: predict dynamic feature splicing: ;

[0073] In the formula, represents the state-space model processing based on zero-order hold, Represents a splicing operation, represents a 2D bidirectional modeling operation, i.e., the processing of step S321 to step S323, Represents the overall dynamic characteristics of the human body;

[0074] Step S324: human body partitioning, decomposing the human body key points into 6 partitions, corresponding to key body parts: arms, legs and torso, and related combination partitions: arms-legs, arms-torso and legs-torso;

[0075] Step S325: Perform a partition-based 2D bidirectional modeling operation on the six combined partitions to obtain the dynamic characteristics of each partition: X p =C2DSS M p ( b p + X m [:, L p ]) ;

[0076] In the formula, Represents an index set, represents a learnable partition label, Represents a partition-based 2D bidirectional modeling operation; Represents the dynamic characteristics of each partition, An integer from 1 to 6, representing the partition order number;

[0077] Step S326: The overall-partition fusion strategy is to fuse the dynamic features of all partitions with the overall dynamic features to obtain the global optimized dynamic features: ;

[0078] In the formula, and represents the learnable parameters, Represents the overall dynamic characteristics, that is , Represents the global optimization dynamic characteristics;

[0079] Step S327: Channel mechanism optimization processing, global optimization of dynamic features and segmentation vectors Perform optimization processing based on the channel mechanism to obtain the results of partition optimization Mamba module processing : ; ;

[0080] In the formula, represents average pooling processing, represents the Sigmoid activation function, Representing Hadamard, The partitioning vector after processing by the partition optimization Mamba module.

[0081] Embodiment 5: Based on the above embodiment, this embodiment extracts the corresponding spatiotemporal features of head posture changes in the intelligent reset analysis module, which specifically includes the following steps:

[0082] Data collection:

[0083] Use position sensors to collect head position data, including head posture, rotation angle, acceleration and other information;

[0084] Data preprocessing:

[0085] Pre-process the collected data, including denoising, data alignment, data correction, etc., to ensure the quality and accuracy of the data;

[0086] Feature extraction:

[0087] According to the head position data, spatiotemporal features are extracted, including the following aspects:

[0088] Time domain characteristics: statistical characteristics of mean, variance, and standard deviation;

[0089] Frequency domain features: frequency-related features are extracted through methods such as Fourier transform;

[0090] Time-frequency domain features: combining time-domain and frequency-domain features through wavelet transform;

[0091] Spatial features: angle and rotation matrix of head posture;

[0092] Motion characteristics: changes in acceleration, speed, and angular velocity;

[0093] Feature Selection:

[0094] Screen and select the extracted features, retain the most representative and informative features, reduce the dimension and improve the model effect;

[0095] Combination of spatiotemporal features:

[0096] The BiLSTM model is used to combine the features of different time points and different spatial dimensions to construct spatiotemporal features;

[0097] Model building and optimization:

[0098] The extracted spatiotemporal features are input into the neural network for training and used for the recognition and analysis of head position changes.

[0099] Embodiment 6: This embodiment is based on the above embodiment. In step S32, a single-layer GCN network with a learnable matrix specifically includes the following data processing steps:

[0100] Graph data representation:

[0101] The graph data It is represented as an adjacency matrix and a node feature matrix, where the adjacency matrix represents the connection relationship between nodes and the node feature matrix represents the feature vector of each node;

[0102] Parameter initialization:

[0103] Initialize a learnable weight matrix, which is used to transform the features of nodes in GCN;

[0104] Feature Propagation:

[0105] Perform feature propagation operations to calculate new node feature representations;

[0106] Feature transformation:

[0107] Perform feature transformation on the propagated node features to obtain the final node feature representation;

[0108] Non-linear activation:

[0109] After feature transformation, a nonlinear activation function ReLU activation function is applied to the final node feature representation to introduce nonlinear transformation and enhance the expressive power of the network.

[0110] Embodiment 7, based on the above embodiment, this embodiment adopts different human feature extraction models for reset guidance, including convolutional neural network CNN, human posture estimation based on deep learning OPENPOSE, deep high-resolution network HRNET and posture detection model POSENET, and calculates reset efficiency indicators, including root mean square error RMSE, mean absolute percentage error MAPE and mean absolute error MAE:

[0111] CNN volume: RMSE: 0.454; MAPE: 0.080; MAE: 0.446;

[0112] OPENPOSE: RMSE: 0.321; MAPE: 0.021; MAE: 0.342;

[0113] HRNET: RMSE: 0.313; MAPE: 0.015; MAE: 0.301;

[0114] POSENET: RMSE: 0.203; MAPE: 0.012; MAE: 0.210;

[0115] A state-space model based on partition optimization:

[0116] RMSE: 0.102; MAPE: 0.009; MAE: 0.098;

[0117] The present invention is a more commonly used human body dynamic characteristic model. It has a more obvious resetting effect in guiding the resetting of otolithiasis, and the resetting efficiency and accuracy are improved to a certain extent.

[0118] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0119] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0120] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A self-repositioning auxiliary system for otolithiasis based on a position sensor, characterized in that: It includes sensor module, image acquisition module, intelligent reset analysis module, feedback interaction module and central control module; The sensor module uses an inertial sensor to collect the posture changes of the patient's head in real time and send the head posture changes to the intelligent reset analysis module; The image acquisition module acquires the patient's overall image video and otolith position in real time through a camera, and sends the overall image video and otolith position to the intelligent repositioning analysis module; The intelligent resetting analysis module uses a state space model based on partition optimization to extract the human body dynamic features in the patient's overall image video, and adopts a spatiotemporal feature extraction method to extract the corresponding spatiotemporal features of the head posture change. When the otolith position is reset, the human body dynamic features and the corresponding spatiotemporal features of the head posture change are recorded to obtain the corresponding relationship between the overall image video and the head posture change when the otolith is reset, and the corresponding relationship is sent to the feedback interaction module; The feedback interaction module reminds the patient how to adjust his / her limb posture through the corresponding relationship and the real-time changes in the patient's head posture; The central control module directly controls all modules and stores information.

2. The position sensor-based otolithiasis self-repositioning auxiliary system according to claim 1, characterized in that: In the intelligent reset analysis module, a processing of a state space model based on partition optimization specifically includes the following steps: Step S1: Extract skeleton sequence, extract the human key point sequence in the whole image video , ; Step S2: Feature extraction and segmentation, converting the human body key point sequence Projection is performed to obtain a high-dimensional embedding vector ; Step S3: Processing the spatiotemporal Mamba module based on partition optimization to obtain an optimized embedding vector; Step S4: multi-layer optimization processing, performing multi-level partition-optimized spatiotemporal Mamba module processing on the optimized embedding vector and performing feature activation to obtain human body dynamic features.

3. The position sensor-based otolithiasis self-repositioning auxiliary system according to claim 2, characterized in that: In step S3, the spatiotemporal Mamba module processing based on partition optimization specifically includes the following steps: Step S31: Channel segmentation, high-dimensional embedding vector Perform channel segmentation to obtain three segmentation vectors ; Step S32: Three segmentation vectors Perform multi-head attention mechanism processing based on partitioned Mamba module: ； ； ； In the formula, Represents a single-layer GCN network with a learnable matrix, Represents partition optimization Mamba module processing, represents a one-dimensional convolution operation, , and are the three segmentation vectors after processing; Step S33: perform a splicing operation on the three processed segmentation vectors to obtain a splicing vector ; Step S34: Perform feature refinement operation based on feedforward network mechanism optimization on the concatenated vector to obtain an optimized embedding vector: ； In the formula, represents the layer normalization operator, GELU represents the activation function based on the Gaussian error function, Represents optimized embedding vector.

4. The position sensor-based otolithiasis self-repositioning auxiliary system according to claim 3, characterized in that: In step S32, the Mamba module processes the specific The following steps are involved: Step S321: Segmentation vector Perform division along the channel dimension to obtain four tensors with the same channel dimension: ； In the formula, represents the resulting tensor; Step S322: Perform state space model processing based on zero-order hold on four tensors with the same channel dimension respectively: ； ； ； ； In the formula, , represents the evolution matrix with time scale parameter, represents the general evolution matrix, represents the natural exponential function treatment, is the zero-order hold parameter, Represents the hidden state at the previous moment, Represents the current hidden state, represents the input sequence, i.e., a tensor, Represents the predicted dynamic characteristics; Step S323: predict dynamic feature splicing: ； In the formula, represents the state-space model processing based on zero-order hold, Represents a splicing operation, represents a 2D bidirectional modeling operation, i.e., the processing of step S321 to step S323, Represents the overall dynamic characteristics of the human body; Step S324: decomposing the key points of the human body into 6 partitions, corresponding to the key body parts: arms, legs and torso, and related combination partitions: arms-legs, arms-torso and legs-torso; Step S325: Perform a partition-based 2D bidirectional modeling operation on the six combined partitions to obtain the dynamic characteristics of each partition: ； In the formula, Represents an index set, represents a learnable partition label, Represents a partition-based 2D bidirectional modeling operation; Represents the dynamic characteristics of each partition, An integer from 1 to 6, representing the partition order number; Step S326: overall-partition fusion strategy, fusing the dynamic features of all partitions with the overall dynamic features to obtain the global optimized dynamic features; Step S327: Channel mechanism optimization processing, global optimization of dynamic features and segmentation vectors Perform optimization processing based on the channel mechanism to obtain the results of partition optimization Mamba module processing .

Images