Computer visual recognition and guidance system for spinal rehabilitation training actions

Abstract

The invention relates to the technical field of spine rehabilitation training, and discloses a computer vision recognition and guidance system for spine rehabilitation training actions. The system comprises a data acquisition module for acquiring user spine motion three-dimensional dynamic image data by using a multi-camera array; the action recognition module tracks bone joint points based on an improved algorithm to generate a real-time action sequence; the posture evaluation module is used for extracting spine curvature and joint angle features through a multi-scale convolutional neural network; the guidance instruction generation module inputs the features into a reinforcement learning model to generate a personalized instruction; and the feedback execution module constructs a fuzzy self-adaptive control model to correct the motion trail of the spine. According to the system, training actions can be accurately recognized, personalized guidance is provided, real-time correction is achieved, the spine rehabilitation training effect is effectively improved, subjectivity and errors of manual evaluation are reduced, and rehabilitation requirements of different patients are met.

Description

Technical Field

[0001] The present invention relates to the technical field of spinal rehabilitation training, and in particular to a computer vision recognition and guidance system for spinal rehabilitation training movements. Background Art

[0002] With the accelerating pace of life and changing work styles, the incidence of spinal diseases is on the rise, and more and more people are requiring spinal rehabilitation training to restore their health. Spinal rehabilitation training is crucial for improving spinal function, alleviating pain, and enhancing quality of life. However, current spinal rehabilitation training faces numerous challenges.

[0003] When it comes to monitoring and evaluating training movements, traditional manual assessment methods rely on the experience of professional rehabilitation therapists. Rehabilitation therapists rely on visual observation of patients' training movements, a method that is not only inefficient but also highly subjective. Different rehabilitation therapists may have different judgments on the same movement, resulting in a lack of accuracy and consistency in the assessment results. For example, when determining whether the spinal curvature is correct, manual observation makes it difficult to accurately quantify the degree of deviation, making it difficult for patients to receive accurate training feedback, which in turn affects rehabilitation effectiveness.

[0004] From the perspective of personalized training guidance, existing rehabilitation training programs are often standardized and fail to fully consider the individual differences of each patient. Each patient's spinal condition, physical foundation, and rehabilitation goals are different. For example, some patients have mild scoliosis caused by long-term sitting, while others have spinal injuries caused by trauma. The rehabilitation training movements and intensity required for the two are very different. However, traditional programs are difficult to provide customized guidance for these differences, which may cause some patients to delay their recovery process due to improper training, or even cause secondary injuries.

[0005] Current technology also has significant shortcomings in providing real-time feedback and correction during training. Patients are unable to promptly determine whether their movements are correct during training. By the time the therapist discovers a problem, the incorrect movement may have been repeated multiple times, which not only reduces the effectiveness of the training but may also reinforce incorrect movement patterns. Furthermore, existing feedback methods, which are mostly verbal or simple visual cues, lack precise, real-time movement correction mechanisms, making them ineffective in guiding patients through correct rehabilitation training movements.

[0006] Computer vision technology has made significant progress in recent years and is widely used in fields such as human posture recognition. However, its application in the recognition and guidance of spinal rehabilitation training exercises faces numerous technical challenges. The complexity of spinal motion makes precise tracking of skeletal joints difficult. Furthermore, accurately extracting spinal curvature deviations and joint angle features, and generating effective guidance instructions, requires overcoming challenges in algorithm optimization and model training.

[0007] Therefore, it is urgent to develop a system that can accurately identify spinal rehabilitation training movements, provide personalized guidance, and correct movements in real time. This has important practical significance for improving the quality and effectiveness of spinal rehabilitation training and promoting patient recovery. Summary of the Invention

[0008] The purpose of the present invention is to provide a computer vision recognition and guidance system for spinal rehabilitation training movements to solve the problems raised in the above background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a computer vision recognition and guidance system for spinal rehabilitation training movements, the system comprising:

[0010] Data acquisition module: used to collect three-dimensional dynamic image data of the user's spinal movement through a multi-camera array;

[0011] Action recognition module: tracks the skeleton joints of the dynamic image data based on an improved three-dimensional posture estimation algorithm to generate a real-time action sequence;

[0012] Posture assessment module: uses a multi-scale convolutional neural network to perform feature mapping on the real-time motion sequence and extract spinal curvature deviation features and joint angle features;

[0013] Guidance instruction generation module: inputs the spinal curvature deviation characteristics and joint angle characteristics into the pre-trained reinforcement learning model to generate personalized rehabilitation training guidance instructions;

[0014] Feedback execution module: constructs a fuzzy adaptive control model based on the guidance instructions, and corrects the user's spinal motion trajectory in real time through a flexible actuator array.

[0015] Preferably, the collecting of three-dimensional dynamic image data of the user's spinal motion by a multi-camera array includes:

[0016] The multi-camera array includes binocular infrared cameras, depth sensors, and wide-angle motion capture cameras;

[0017] Perform spatiotemporal registration of binocular infrared camera data and depth sensor data to construct a three-dimensional skeletal motion trajectory map; perform optical flow field analysis on wide-angle motion capture camera data to generate local muscle group motion characteristics;

[0018] Constructing a dual-branch feature fusion network, the first branch uses a spatiotemporal graph convolutional network to extract the spatiotemporal correlation features of the three-dimensional skeletal motion trajectory graph, and the second branch uses an attention mechanism recurrent neural network to extract the temporal variation pattern of the local muscle group motion features;

[0019] The spatiotemporal correlation features and temporal variation patterns are fused through a cross-modal feature alignment mechanism to generate comprehensive motion features including spinal displacement, joint mobility and muscle synergy.

[0020] Preferably, the improved three-dimensional pose estimation algorithm adopts a cascade residual learning framework, including:

[0021] Construct a hierarchical skeletal topology graph, where nodes include spinal vertebrae, pelvic nodes, and limb joints. Node attributes include 3D coordinates, rotation angles, and movement speeds.

[0022] A hybrid loss function is used. In the first stage, the geometric constraint loss function is used to optimize the consistency of bone length, and in the second stage, the dynamic constraint loss function is used to optimize the smoothness of joint motion.

[0023] The trajectory of missing joints is repaired based on the adaptive keyframe interpolation algorithm, and the relative rotation matrix of each spinal segment is calculated using the inverse kinematics algorithm.

[0024] Preferably, the multi-scale convolutional neural network comprises parallel convolution channels and a feature pyramid structure, including:

[0025] The input action sequence is divided into short-term segments and long-term sequences, which are input into the time dimension convolution layer and the space dimension convolution layer respectively;

[0026] In the feature pyramid structure, the high-level network extracts the overall curvature features of the spine, and the low-level network extracts the micro-motion features between vertebrae;

[0027] The receptive field is expanded by using a dilated convolution kernel to capture the local distortion pattern of the scoliosis area, and a channel attention mechanism is introduced to perform weighted focus on abnormal motion features.

[0028] Preferably, the reinforcement learning model adopts a hierarchical decision-making architecture, including:

[0029] Construct a state-action value function network, where the input layer integrates spinal health assessment indicators and historical training data, and the output layer generates a multi-dimensional action instruction vector;

[0030] Design a dual reward mechanism, where the first reward function calculates immediate rewards based on the progress of spinal curvature correction, and the second reward function calculates delayed rewards based on long-term training stability;

[0031] The network parameters are updated through the policy gradient algorithm, and the experience replay buffer is used to store high-value training samples to improve the efficiency of policy exploration.

[0032] Preferably, the fuzzy adaptive control model comprises a dynamic rule base and a parameter self-tuning module, including:

[0033] The spinal motion trajectory error is decomposed into lateral displacement error and axial rotation error, and independent fuzzy inference subsystems are established for each. An adaptive membership function generator is constructed to dynamically adjust the coverage and overlapping areas of the fuzzy set according to the error change rate.

[0034] The fuzzy output is converted into driving torque instructions for the actuator array through a weighted average defuzzification algorithm.

[0035] Preferably, the spatiotemporal graph convolutional network adopts a relative motion encoding mechanism, including:

[0036] Define the relative motion vector between adjacent bone nodes, including displacement difference, velocity difference and acceleration difference;

[0037] Mapping the relative motion vector to a weight coefficient of a graph convolution kernel through a nonlinear transformation layer;

[0038] The weight coefficient is multiplied by the node attribute matrix to generate a spatiotemporal correlation feature map.

[0039] Preferably, the adaptive keyframe interpolation algorithm adopts a Bezier curve optimization strategy, including:

[0040] Detect abnormal joint point jump areas in the skeleton topology and calculate the kinematic parameters of adjacent valid key frames;

[0041] Construct a cubic Bezier curve model to fit the missing trajectory segments, and the control point positions are determined by the acceleration continuity conditions of adjacent key frames;

[0042] The interpolated trajectory is aligned with the actual kinematic constraints through an iterative closest point algorithm to ensure the integrity of the spinal kinematic chain.

[0043] Preferably, the state-action value function network adopts a double-delayed deep deterministic policy gradient framework, including:

[0044] Construct a dual network structure of the main network and the target network. The main network is responsible for generating the current strategy, and the target network provides stable value estimation.

[0045] In the policy improvement stage, a deterministic policy gradient algorithm is used to update the action generation network parameters, the main network and target network parameters are synchronized through a soft update mechanism, and an entropy regularization term is introduced to enhance the policy exploration capability.

[0046] Preferably, the dynamic rule base adopts an incremental rule generation method, including:

[0047] Collect the error distribution and execution effect data of the historical correction process to build a rule sample set, divide the feature space through the fuzzy clustering algorithm, and automatically generate the initial fuzzy rule base;

[0048] Design a rule confidence evaluation mechanism to dynamically eliminate inefficient rules and add new rules based on the actual correction effect.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] For data collection, the system uses a multi-camera array that integrates a binocular infrared camera, a depth sensor, and a wide-angle motion capture camera. Through spatiotemporal registration of data from different devices, optical flow analysis, and dual-branch feature fusion network processing, it can comprehensively and accurately acquire comprehensive motion characteristics including spinal displacement, joint range of motion, and muscle coordination. Compared with traditional single-device data collection methods, this greatly improves data integrity and accuracy. This provides a solid and reliable data foundation for subsequent motion recognition and posture assessment, avoiding misjudgments due to missing or inaccurate data and providing strong support for precise rehabilitation training.

[0051] The motion recognition module, based on an improved 3D posture estimation algorithm, achieves precise tracking of skeletal joints and generates highly accurate real-time motion sequences by constructing a hierarchical skeletal topology map, employing a hybrid loss function, and employing an adaptive keyframe interpolation algorithm. This precise motion recognition capability clearly captures subtle changes in the spine during movement, promptly identifying irregular movements. This provides an accurate basis for subsequent posture assessment and guidance instruction generation, effectively preventing the accumulation of incorrect movements and reducing the risk of secondary injury to patients.

[0052] The posture assessment module utilizes the parallel convolution channels and feature pyramid structure of a multi-scale convolutional neural network, combined with a dilated convolution kernel and a channel attention mechanism, to deeply analyze movement sequences and accurately extract spinal curvature deviation and joint angle characteristics. This multi-dimensional, refined feature extraction method is more sensitive to abnormalities in spinal movement than traditional, simple assessment methods. It can accurately identify both overall spinal curvature problems and micro-motion abnormalities between vertebrae, providing a scientific and precise reference for adjustments in rehabilitation training.

[0053] The instruction generation module leverages a pre-trained reinforcement learning model, integrates spinal health assessment indicators with historical training data, and employs a hierarchical decision-making architecture and a dual reward mechanism to generate highly personalized rehabilitation training instructions. This fully accounts for individual patient differences, ensuring that training plans are tailored to their specific needs and enhancing their relevance and effectiveness. Patients who follow personalized instructions can achieve their rehabilitation goals faster, shorten the recovery cycle, and enhance their rehabilitation experience.

[0054] The fuzzy adaptive control model constructed by the feedback execution module decomposes and processes spinal motion trajectory errors based on guidance instructions. Through a dynamic rule library and parameter self-tuning module, it achieves precise control of the flexible actuator array and corrects the user's spinal motion trajectory in real time. This ensures that patients receive timely and correct guidance during training, avoids repeating incorrect movements, effectively strengthens correct movement patterns, and improves the quality and efficiency of rehabilitation training.

[0055] The system's integrated modules work together to form a closed-loop rehabilitation training guidance system. From data acquisition, motion recognition, and posture assessment to instruction generation and feedback execution, each link works closely together to continuously optimize the training process. This not only improves the intelligence level of rehabilitation training and reduces reliance on professional rehabilitation therapists, but also provides patients with a safer, more efficient, and personalized rehabilitation training environment, playing a significant role in promoting the development of spinal rehabilitation training technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a working principle diagram of the computer vision recognition and guidance system for spinal rehabilitation training movements according to the present invention;

[0057] Figure 2 This is the working principle diagram of the data acquisition module;

[0058] Figure 3 This is a working principle diagram of the improved 3D pose estimation algorithm;

[0059] Figure 4 This is a diagram of the working principle of the reinforcement learning model. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] See also Figures 1-4 The present invention relates to a computer vision recognition and guidance system for spinal rehabilitation training movements, and its overall implementation scheme is as follows:

[0062] The data acquisition module uses a multi-camera array to collect three-dimensional dynamic image data of the user's spinal motion. This multi-camera array includes various types of cameras and sensors, such as binocular infrared cameras, depth sensors, and wide-angle motion capture cameras. These devices collect data on the user's spinal motion from different angles and using different principles, providing the foundation for subsequent precise analysis.

[0063] The motion recognition module uses an improved 3D pose estimation algorithm to track skeletal joints in the captured dynamic image data, generating real-time motion sequences. This algorithm, employing a specific framework and methodology, accurately determines the position and posture changes of the spine and related joints during movement, providing accurate motion data for subsequent posture assessment.

[0064] The Posture Assessment Module uses a multi-scale convolutional neural network to perform feature mapping on real-time motion sequences, extracting spinal curvature deviation and joint angle features. This allows the user's spinal and joint movements to be quantified during rehabilitation training, determining whether they meet correct training standards.

[0065] The guidance generation module inputs extracted spinal curvature deviation and joint angle features into a pre-trained reinforcement learning model to generate personalized rehabilitation training instructions. Based on the user's specific situation, this model takes into account factors such as spinal health assessment indicators and historical training data to provide targeted training recommendations to improve rehabilitation training effectiveness.

[0066] The feedback execution module builds a fuzzy adaptive control model based on the guidance instructions, using the flexible actuator array to provide real-time correction of the user's spinal motion trajectory. Based on deviations during training, this module promptly adjusts the flexible actuator's movements, helping the user correct incorrect motion trajectories and ensuring the accuracy and safety of training movements.

[0067] The technical solution of the present invention is further described below with reference to specific embodiments.

[0068] Example 1:

[0069] In this embodiment, the process of collecting data and generating comprehensive motion features by a multi-camera array in the data acquisition module is described in detail. Specifically, it includes:

[0070] The binocular infrared cameras in the multi-camera array use infrared technology to capture image information of the user's spinal movement, while the depth sensors measure the distance between various body parts and the sensors. The binocular infrared camera data and the depth sensor data are temporally and spatially aligned using a specific algorithm to ensure temporal and spatial consistency between the two data types, thereby constructing a three-dimensional skeletal motion trajectory map. In this map, each skeletal node has corresponding three-dimensional coordinate information, which accurately reflects the positional changes of the spine in space. Simultaneously, a wide-angle motion capture camera captures the movement of the user's local muscle groups and generates local muscle group motion characteristics through optical flow analysis. Optical flow analysis is an analysis method based on pixel motion in an image. It can detect the displacement of pixels during muscle group movement, thereby inferring the muscle group's motion characteristics.

[0071] Construct a dual-branch feature fusion network. The first branch adopts a spatiotemporal graph convolutional network. Its principle is to extract the spatiotemporal correlation features of the three-dimensional skeletal motion trajectory graph by defining the relative motion vector between adjacent skeletal nodes. The relative motion vector includes the displacement difference Δd, the velocity difference Δv and the acceleration difference Δa. These relative motion vectors are mapped to the weight coefficient w of the graph convolution kernel through the nonlinear transformation layer. The nonlinear transformation layer here can use common nonlinear functions such as the ReLU function. Then the weight coefficient w is tensor-multiplied with the node attribute matrix M to generate the spatiotemporal correlation feature graph G, that is, in Represents a tensor product operation. The second branch uses an attention mechanism recurrent neural network to extract the temporal variation pattern of local muscle group motion characteristics. The attention mechanism allows the network to pay more attention to important muscle motion information, while the recurrent neural network is capable of processing muscle motion data with temporal characteristics. Finally, a cross-modal feature alignment mechanism is used to fuse spatiotemporal correlation features with temporal variation patterns to generate comprehensive motion features that include spinal displacement, joint range of motion, and muscle synergy. This comprehensive motion feature comprehensively reflects the user's spinal motion and provides a rich and accurate data foundation for subsequent motion recognition and posture assessment.

[0072] Example 2:

[0073] This embodiment details the specific implementation of the improved three-dimensional pose estimation algorithm. The improved three-dimensional pose estimation algorithm adopts a cascade residual learning framework. First, a hierarchical skeletal topology graph is constructed. The nodes in the graph include key parts such as spinal vertebrae nodes, pelvic nodes, and limb joints. Each node has attributes such as three-dimensional coordinates (x, y, z), rotation angle θ, and motion speed v. These attributes can accurately describe the position, posture, and motion state of the skeletal node in space.

[0074] In the optimization process, a hybrid loss function is used. In the first stage, the geometric constraint loss function is used to optimize the consistency of bone length. The geometric constraint loss function can be expressed as Where n is the number of bones, l i is the actual length of the i-th bone, is the standard bone length. By minimizing this loss function, the bone length is made as close to the standard value as possible to ensure the rationality of the bone structure. In the second stage, the smoothness of joint movement is optimized through the dynamic constraint loss function. The dynamic constraint loss function can be expressed as Where m is the number of joints, α j is the actual motion parameter of the joint, β j is the desired joint motion parameter. This loss function can make the joint motion smoother and conform to the laws of human kinematics.

[0075] In the case of missing joints, trajectory repair is performed based on an adaptive keyframe interpolation algorithm. This algorithm uses a Bezier curve optimization strategy. First, it detects abnormal joint jump areas in the skeletal topology and calculates the kinematic parameters of adjacent valid keyframes, such as displacement, velocity, and acceleration. Then, a cubic Bezier curve model is constructed to fit the missing trajectory segment. The control point positions of the cubic Bezier curve model are determined by the acceleration continuity conditions of adjacent keyframes. Finally, the interpolated trajectory is aligned with the actual kinematic constraints through an iterative closest point algorithm to ensure the integrity of the spinal motion chain. At the same time, the relative rotation matrix of each spinal segment is calculated in combination with the kinematic inverse solution algorithm, providing a basis for accurate analysis of spinal motion posture.

[0076] Example 3:

[0077] This embodiment focuses on the application of multi-scale convolutional neural networks in posture assessment modules. The multi-scale convolutional neural network contains parallel convolution channels and feature pyramid structures. First, the input action sequence is divided into short-term segments and long-term sequences, which are input into the time dimension convolution layer and the space dimension convolution layer respectively. Short-term segments focus on the instantaneous changes of the action, while long-term sequences focus more on the overall trend of the action. The time dimension convolution layer can capture the temporal change characteristics of the action, while the spatial dimension convolution layer can extract the spatial structural characteristics of the action.

[0078] In the feature pyramid structure, the high-level network extracts the overall curvature features of the spine, and the low-level network extracts the micro-motion features between vertebrae. The high-level network can grasp the curvature of the spine from a macroscopic perspective and determine whether there are overall problems such as scoliosis; the low-level network can accurately analyze the tiny movement changes between vertebrae and detect potential abnormal micro-motions. The receptive field is expanded by using a dilated convolution kernel to capture the local distortion pattern in the scoliosis area. The dilated convolution kernel's dilation rate can be adjusted according to actual conditions. For example, when the dilation rate is r, the convolution kernel will skip r-1 pixels during the convolution process, which can expand the receptive field without increasing parameters. At the same time, the channel attention mechanism is introduced to perform weighted focus on abnormal motion features. The channel attention mechanism weights the features of different channels so that the network pays more attention to the feature channels related to abnormal motion, thereby improving the recognition ability of abnormal motion. For example, by calculating the importance weight w of each channel c , perform weighted processing on the feature map F to obtain the weighted feature map F ′ =w c ·F, thereby highlighting abnormal motion features.

[0079] Example 4:

[0080] This embodiment details the working process of the reinforcement learning model. The reinforcement learning model adopts a hierarchical decision-making architecture. First, a state-action value function network is constructed, and the input layer integrates spinal health assessment indicators and historical training data. Spinal health assessment indicators can include spinal curvature, range of motion of joints, etc., and historical training data contains information such as the user's previous training action sequence and training effect. The output layer generates multi-dimensional action instruction vectors. These vectors contain specific training action suggestions for the user's current situation, such as the direction and amplitude of movement.

[0081] Design a dual reward mechanism, the first reward function calculates the immediate reward based on the spinal curvature correction progress. Assume that the current spinal curvature is C t , the target spinal curvature is C target , the immediate reward function can be expressed as R1 = -|C t -C target |, through this function, the closer the spinal curvature is to the target curvature, the greater the reward, encouraging the model to guide the user towards the correct spinal curvature. The second reward function calculates a delayed reward based on long-term training stability. Long-term training stability can be measured by factors such as the consistency of actions and the fluctuation of errors during training. For example, if the standard deviation σ of the action error over a period of time is used to represent stability, the delayed reward function can be expressed as R2 = -σ. A smaller standard deviation means a more stable training process, resulting in a larger reward.

[0082] The network parameters are updated using a policy gradient algorithm. This algorithm adjusts the network parameters based on the current reward and policy, aligning the model with the goal of achieving greater rewards. An experience replay buffer is also used to store high-value training samples to improve policy exploration efficiency. This buffer can store successful experiences and high-value samples from previous training sessions. These samples can then be randomly sampled for learning during subsequent training, preventing the model from becoming overly dependent on the current training data and improving its generalization capabilities.

[0083] Example 5:

[0084] This embodiment mainly describes the specific implementation of the fuzzy adaptive control model in the feedback execution module. The fuzzy adaptive control model includes a dynamic rule base and a parameter self-tuning module. First, the spinal motion trajectory error is decomposed into the lateral displacement error e x and axial rotation error e y , establish independent fuzzy reasoning subsystems respectively. For the lateral displacement error e x , construct an adaptive membership function generator, according to the error change rate Δe x Dynamically adjust the coverage and overlapping areas of fuzzy sets. For example, when the error change rate Δe xWhen the error rate of change Δe is large, the coverage of the fuzzy set should be appropriately expanded to respond to the error change more quickly. x When it is smaller, the coverage of the fuzzy set is reduced and the control accuracy is improved.

[0085] Similarly, for the axial rotation error e y , is also processed using a similar adaptive membership function generator. Through the fuzzy inference subsystem, fuzzy outputs are generated based on the error and error change rate. The fuzzy outputs are then converted into driving torque instructions for the actuator array using a weighted average defuzzification algorithm. Assume that the fuzzy outputs are y1, y2, …, y n , the corresponding weights are w1,w2,…,w n , then the driving torque command The dynamic rule base uses an incremental rule generation approach, collecting error distribution and execution performance data from historical correction processes to construct a rule sample set. A fuzzy clustering algorithm is then used to partition the feature space and automatically generate an initial fuzzy rule base. As the correction process progresses, a rule confidence assessment mechanism is designed to dynamically eliminate inefficient rules and add new ones based on actual correction results. This allows the rule base to continuously adapt to the needs of different users and training stages, improving control accuracy and effectiveness.

[0086] Example 6:

[0087] This example uses a user performing spinal extension rehabilitation training as an example to comprehensively demonstrate the collaborative working process of the system's various modules. When the user begins spinal extension training, the data acquisition module's multi-camera array immediately begins operation. A binocular infrared camera, a depth sensor, and a wide-angle motion capture camera collect data from different angles during the user's training. After spatiotemporal alignment, optical flow field analysis, and dual-branch feature fusion network processing, a comprehensive motion feature containing spinal displacement, joint range of motion, and muscle coordination is generated and transmitted to the motion recognition module.

[0088] The action recognition module uses an improved 3D pose estimation algorithm to track skeletal joints in this data and generate real-time action sequences. This algorithm accurately determines the motion trajectory and posture changes of the user's spine and related joints during training by constructing a hierarchical skeletal topology graph, employing a hybrid loss function, and an adaptive keyframe interpolation algorithm.

[0089] After receiving the real-time motion sequence, the posture assessment module uses a multi-scale convolutional neural network for feature mapping, extracting spinal curvature deviation and joint angle features. By combining parallel convolution channels and a feature pyramid structure with a dilated convolution kernel and a channel attention mechanism, it accurately analyzes the user's training posture and determines whether there are any spinal curvature anomalies or joint angle deviations.

[0090] The guidance instruction generation module feeds the extracted spinal curvature deviation and joint angle features into a pre-trained reinforcement learning model. Based on a hierarchical decision-making architecture, this model combines spinal health assessment indicators with historical training data to generate a multidimensional action instruction vector via a state-action value function network. Furthermore, a dual reward mechanism is employed to calculate rewards based on spinal curvature correction progress and long-term training stability, continuously optimizing the model's output instructions.

[0091] The feedback execution module constructs a fuzzy adaptive control model based on the guidance instructions. The spinal motion trajectory error is decomposed into lateral displacement error and axial rotation error. Through an independent fuzzy inference subsystem and adaptive membership function generator, combined with a dynamic rule base and parameter self-tuning module, the fuzzy output is converted into a driving torque instruction for the flexible actuator array. The flexible actuator array corrects the user's spinal motion trajectory in real time based on these instructions, helping the user adjust the training movements to ensure the safety and effectiveness of the training. Throughout the training process, the system continuously cycles the above process, dynamically adjusts and optimizes according to the user's real-time movement conditions, and provides users with personalized and accurate spinal rehabilitation training guidance.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0093] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A computer vision recognition and guidance system for spinal rehabilitation training movements, characterized in that: include: Data acquisition module: used to collect three-dimensional dynamic image data of the user's spinal movement through a multi-camera array; Action recognition module: tracks the skeleton joints of the dynamic image data based on an improved three-dimensional posture estimation algorithm to generate a real-time action sequence; Posture assessment module: uses a multi-scale convolutional neural network to perform feature mapping on the real-time motion sequence and extract spinal curvature deviation features and joint angle features; Guidance instruction generation module: inputs the spinal curvature deviation characteristics and joint angle characteristics into the pre-trained reinforcement learning model to generate personalized rehabilitation training guidance instructions; Feedback execution module: constructs a fuzzy adaptive control model based on the guidance instructions, and corrects the user's spinal motion trajectory in real time through a flexible actuator array.

2. The spinal rehabilitation training system according to claim 1, characterized in that: The three-dimensional dynamic image data of the user's spinal movement collected by the multi-camera array includes: The multi-camera array includes binocular infrared cameras, depth sensors, and wide-angle motion capture cameras; Perform spatiotemporal registration of binocular infrared camera data and depth sensor data to construct a three-dimensional skeletal motion trajectory map; perform optical flow field analysis on wide-angle motion capture camera data to generate local muscle group motion characteristics; Constructing a dual-branch feature fusion network, the first branch uses a spatiotemporal graph convolutional network to extract the spatiotemporal correlation features of the three-dimensional skeletal motion trajectory graph, and the second branch uses an attention mechanism recurrent neural network to extract the temporal variation pattern of the local muscle group motion features; The spatiotemporal correlation features and temporal variation patterns are fused through a cross-modal feature alignment mechanism to generate comprehensive motion features including spinal displacement, joint mobility and muscle synergy.

3. The spinal rehabilitation training system according to claim 1, characterized in that: The improved 3D pose estimation algorithm adopts a cascaded residual learning framework, including: Construct a hierarchical skeletal topology graph, where nodes include spinal vertebrae, pelvic nodes, and limb joints. Node attributes include 3D coordinates, rotation angles, and movement speeds. A hybrid loss function is used. In the first stage, the geometric constraint loss function is used to optimize the consistency of bone length, and in the second stage, the dynamic constraint loss function is used to optimize the smoothness of joint motion. The trajectory of missing joints is repaired based on the adaptive keyframe interpolation algorithm, and the relative rotation matrix of each spinal segment is calculated using the inverse kinematics algorithm.

4. The spinal rehabilitation training system according to claim 1, characterized in that: The multi-scale convolutional neural network includes parallel convolution channels and a feature pyramid structure, including: The input action sequence is divided into short-term segments and long-term sequences, which are input into the time dimension convolution layer and the space dimension convolution layer respectively; In the feature pyramid structure, the high-level network extracts the overall curvature features of the spine, and the low-level network extracts the micro-motion features between vertebrae; The receptive field is expanded by using a dilated convolution kernel to capture the local distortion pattern of the scoliosis area, and a channel attention mechanism is introduced to perform weighted focus on abnormal motion features.

5. The spinal rehabilitation training system according to claim 1, characterized in that: The reinforcement learning model adopts a hierarchical decision-making architecture, including: Construct a state-action value function network, where the input layer integrates spinal health assessment indicators and historical training data, and the output layer generates a multi-dimensional action instruction vector; Design a dual reward mechanism, where the first reward function calculates immediate rewards based on the progress of spinal curvature correction, and the second reward function calculates delayed rewards based on long-term training stability; The network parameters are updated through the policy gradient algorithm, and the experience replay buffer is used to store high-value training samples to improve the efficiency of policy exploration.

6. The spinal rehabilitation training system according to claim 1, characterized in that: The fuzzy adaptive control model includes a dynamic rule base and a parameter self-tuning module, including: The spinal motion trajectory error is decomposed into lateral displacement error and axial rotation error, and independent fuzzy inference subsystems are established for each. An adaptive membership function generator is constructed to dynamically adjust the coverage and overlapping areas of the fuzzy set according to the error change rate. The fuzzy output is converted into driving torque instructions for the actuator array through a weighted average defuzzification algorithm.

7. The spinal rehabilitation training system according to claim 2, characterized in that: The spatiotemporal graph convolutional network adopts a relative motion encoding mechanism, including: Define the relative motion vector between adjacent bone nodes, including displacement difference, velocity difference and acceleration difference; Mapping the relative motion vector to a weight coefficient of a graph convolution kernel through a nonlinear transformation layer; The weight coefficient is multiplied by the node attribute matrix to generate a spatiotemporal correlation feature map.

8. The spinal rehabilitation training system according to claim 3, characterized in that: The adaptive keyframe interpolation algorithm adopts a Bezier curve optimization strategy, including: Detect abnormal joint point jump areas in the skeleton topology and calculate the kinematic parameters of adjacent valid key frames; Construct a cubic Bezier curve model to fit the missing trajectory segments, and the control point positions are determined by the acceleration continuity conditions of adjacent key frames; The interpolated trajectory is aligned with the actual kinematic constraints through an iterative closest point algorithm to ensure the integrity of the spinal kinematic chain.

9. The spinal rehabilitation training system according to claim 5, characterized in that: The state-action value function network adopts a dual-delay deep deterministic policy gradient framework, including: Construct a dual network structure of the main network and the target network. The main network is responsible for generating the current strategy, and the target network provides stable value estimation. In the policy improvement stage, a deterministic policy gradient algorithm is used to update the action generation network parameters, the main network and target network parameters are synchronized through a soft update mechanism, and an entropy regularization term is introduced to enhance the policy exploration capability.

10. The spinal rehabilitation training system according to claim 6, characterized in that: The dynamic rule base adopts an incremental rule generation method, including: Collect the error distribution and execution effect data of the historical correction process to build a rule sample set, divide the feature space through the fuzzy clustering algorithm, and automatically generate the initial fuzzy rule base; Design a rule confidence evaluation mechanism to dynamically eliminate inefficient rules and add new rules based on the actual correction effect.

Images