Rehabilitation training action specification degree intelligent identification system based on computer vision (g06t7*)

Abstract

The application discloses a rehabilitation training action specification degree intelligent identification system based on computer vision, relates to the cross field of computer vision and rehabilitation medicine, and aims to solve the problems of strong subjectivity, feedback lag and single evaluation dimension in the rehabilitation training action specification determination depending on doctors. The system contains image acquisition, preprocessing, human body posture extraction, standard action library, specification degree evaluation, feedback output and data storage modules. An adaptive light adjustment algorithm is adopted to realize clear acquisition in the environment of 100-1000 lux; background interference and noise are removed through the combination of Gaussian filtering, histogram equalization and interframe difference method; 14 joint three-dimensional coordinates, angles and motion parameters are accurately extracted; a standard action library containing a clinical verification template is constructed and self-defined adaptation is supported; the specification degree is calculated in multiple dimensions through a dynamic model, and real-time feedback is realized in multiple modes such as text, voice and interface. The system can reduce the dependence on professional doctors, improve the scientific nature of training, and is suitable for limb rehabilitation and other scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision application technology, specifically relating to an intelligent recognition system for the standardization of rehabilitation training movements based on G06T7* visual detection technology. It is applicable to scenarios such as rehabilitation medicine and sports rehabilitation training, and is used to realize real-time standardization assessment and feedback of rehabilitation training movements. Background Technology

[0002] Exercise rehabilitation training is a crucial step in restoring physical function for patients with limb dysfunction. The accuracy of movement directly impacts rehabilitation outcomes and can even lead to secondary injuries. Currently, the assessment of movement accuracy in rehabilitation training primarily relies on the manual observation of professional rehabilitation physicians, which has the following technical shortcomings: (1) High subjectivity: Different physicians have different judgment criteria, making it difficult to form a unified and objective evaluation result; (2) High dependence on resources: The number of professional rehabilitation physicians is limited, which cannot meet the real-time guidance needs of home rehabilitation, community rehabilitation and other scenarios; (3) Lagging feedback: Manual assessment makes it difficult to provide real-time feedback during the movement process, and trainees cannot adjust their movements in a timely manner; (4) Single evaluation dimension: It focuses only on the similarity of the appearance of the movement and ignores the quantitative evaluation of key parameters such as joint angle and movement rhythm.

[0003] In existing technologies, some sensor-based motion detection systems require trainees to wear specialized equipment, which is cumbersome to operate, uncomfortable, and costly. Meanwhile, conventional computer vision recognition systems are mostly used for motion posture capture, lacking specialized adaptation for rehabilitation training movements, resulting in insufficient recognition accuracy and scenario adaptability. Therefore, developing an intelligent recognition system for the standardization of rehabilitation training movements that requires no additional equipment, provides objective accuracy, and offers real-time feedback has become an urgent technical problem to be solved. Summary of the Invention

[0004] 3.1 Purpose of the Invention The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent recognition system for the standardization of rehabilitation training movements based on computer vision (G06T7*), which realizes non-contact real-time detection, multi-dimensional quantitative evaluation and instant feedback of rehabilitation training movements, reduces the dependence on professional physicians and improves the scientificity and convenience of rehabilitation training.

[0005] 3.2 Technical Solution The core technical solution of this invention is as follows, and the specific implementation details of each module are as follows.

[0006] 3.2.1 Image Acquisition Module: (1) Hardware configuration: It adopts a USB3.0 interface high-definition camera, the sensor type is CMOS, the pixel size is 1.4μm×1.4μm, the maximum resolution is 3840×2160, and it supports stable output at a frame rate of 60fps; the lens focal length is 3.6mm, the field of view is 120°, and the distortion rate is ≤1%; (2) Adaptive light adjustment algorithm: The ambient light intensity is collected in real time by the built-in light sensor of the camera (sampling frequency 10Hz). When the light intensity is <300 lux, the sensitivity is automatically increased (ISO800-1600) and the exposure time is extended (1 / 30s-1 / 15s); when the light intensity is >800 lux, the sensitivity is automatically decreased (ISO100-400) and the exposure time is shortened (1 / 120s-1 / 60s); at the same time, the automatic white balance algorithm (color temperature range 2700K-6500K) is enabled to ensure accurate reproduction of human skin color under different lighting conditions. (3) Image transmission: Image compression is performed using H.264 encoding format, with a transmission bandwidth of ≤10Mbps. It supports wired (USB3.0) or wireless (Wi-Fi5) transmission modes, with a wireless transmission delay of ≤50ms.

[0007] 3.2.2 Preprocessing Module: (1) Gaussian filtering for noise reduction: The image is convolved using a 5×5 Gaussian kernel with a standard deviation of σ=1.2, which effectively removes Gaussian noise and salt-and-pepper noise, and the peak signal-to-noise ratio (PSNR) is ≥35dB. (2) Histogram equalization: Adaptive histogram equalization (CLAHE) algorithm is used to divide the image into 8×8 pixel sub-blocks and limit the contrast threshold to 2.0 to avoid noise amplification caused by overall equalization and enhance local image details (such as joint contours). (3) Motion region extraction: Based on the three-frame difference method, the motion region mask is obtained by calculating the pixel difference of three consecutive frames (threshold T=20). Then, small noise areas (area < 50 pixels) are removed by morphological opening operation (3×3 rectangular structuring element). Finally, the complete motion region of the trainee is extracted, and the background removal accuracy is ≥ 98%.

[0008] 3.2.3 Human Pose Extraction Module: Improved G06T7* visual inspection algorithm: (1) Feature extraction stage: The G06T7* edge detection algorithm is used to extract human contour features by calculating the gradient magnitude and direction of the image. The gradient operator is the Sobel operator (3×3), and the gradient threshold is adaptively adjusted (range 50-150). (2) Candidate region selection: The non-maximum suppression (NMS) algorithm is used on the edge detection results with a suppression threshold of 0.5 to select human candidate regions and eliminate background interference; (3) Skeletal key point localization: The MobilePose lightweight model is integrated. This model is based on a convolutional neural network (CNN) and contains 8 convolutional layers, 3 pooling layers and 2 fully connected layers. The input image size is 256×256 pixels, and the output is the two-dimensional coordinates of 14 key joints. Then, the coordinates are converted into three-dimensional coordinates by triangulation. The localization error is ≤1cm. (4) Coordinate calibration: The camera is calibrated using the Zhang Zhengyou calibration method to obtain the intrinsic parameter matrix and distortion coefficients. The extracted joint coordinates are then distorted and corrected. The accuracy of joint angle calculation after correction is ±1°. (5) Motion parameter estimation: Based on the inter-frame difference method, the change in the coordinates of the joint points between two adjacent frames is calculated to obtain the joint motion velocity (unit: cm / s) and acceleration (unit: cm / s²), with an estimation error ≤5%; (6) Definition of key joints: including head (1), neck (2), left shoulder (3), right shoulder (4), left elbow (5), right elbow (6), left wrist (7), right wrist (8), left hip (9), right hip (10), left knee (11), right knee (12), left ankle (13), right ankle (14), a total of 14 core nodes, covering the main joints of human movement.

[0009] 3.2.4 Standard Action Library Module: (1) Template construction: Standard rehabilitation movement data of 100 healthy adults were collected and verified and labeled by 3 senior rehabilitation physicians to establish a standard movement template library, covering 20 common rehabilitation movements in three categories: upper limb rehabilitation (elbow flexion and extension, shoulder abduction / adduction, wrist rotation), lower limb rehabilitation (knee flexion and extension, hip adduction / abduction, ankle dorsiflexion / plantar flexion), and trunk rehabilitation (trunk rotation, lumbar flexion and extension); (2) Template parameters: Each action template contains the following quantization parameters: 1) Joint angle range: The dynamic angle range of each key joint (e.g., the angle range for elbow flexion and extension training is 30°-120°). 2) Motion trajectory curve: The three-dimensional motion trajectory coordinate sequence of each joint (sampling interval 1 / 60s); 3) Duration of movement: The range of time required to complete one standard movement (e.g., the standard duration for knee flexion and extension training is 2-3 seconds). 4) Movement speed threshold: the maximum movement speed of each joint (e.g., the maximum speed of shoulder abduction ≤ 30cm / s); 5) Customizable adaptation function: Allows physicians or trainees to input parameters such as height, weight, rehabilitation stage, and limb range of motion to automatically adjust parameters such as the angle range and speed threshold of the standard movement template to adapt to the training needs of different individuals.

[0010] 3.2.5 Standardization Assessment Module: Multi-dimensional comparison model: (1) Joint angle similarity: For each key joint, calculate the overlap between the real-time angle and the standard angle range. The formula is: similarity = 1 - |real-time angle - standard angle mean| / standard angle standard deviation. The similarity value range is 0-1. (2) Motion trajectory overlap: The dynamic time warping (DTW) algorithm is used to align the real-time motion trajectory with the standard trajectory and calculate the warping distance. The overlap = 1 - warping distance / standard trajectory length, with a value range of 0-1. (3) Action rhythm matching degree: Calculate the deviation rate between the real-time action duration and the standard duration. Matching degree = 1 - |real-time duration - standard duration| / standard duration, with a value range of 0-1; (4) Dynamic weighting algorithm: The weights of each dimension are determined by the analytic hierarchy process (AHP), a judgment matrix is ​​constructed and a consistency test is performed (CR < 0.1). The final weight allocation is as follows: joint angle similarity 0.4, motion trajectory overlap 0.35, and movement rhythm matching degree 0.25. (5) Calculation of comprehensive score: Comprehensive standardization score = joint angle similarity × 0.4 + motion trajectory overlap × 0.35 + movement rhythm matching degree × 0.25, the score range is 0-100 points, of which ≥80 points is qualified, 60-79 points is basically qualified, and <60 points is unqualified. (6) Abnormal detection mechanism: When the real-time angle of a joint exceeds 20% of the standard range, or the movement speed exceeds 30% of the threshold, the system automatically marks it as an abnormal action and triggers immediate feedback.

[0011] 3.2.6 Feedback Output Module: Multimodal feedback methods: (1) Text prompts: The visual interface displays the judgment results of "qualified", "basically qualified" and "unqualified" in real time, and marks the specific problem joints and parameter deviations (such as "left knee bending angle 45°, standard range 60°-100°"). (2) Voice broadcast: TTS speech synthesis technology is used, the voice broadcast frequency is 1 time / 2s, the broadcast content is consistent with the text prompt, and the volume adjustment (0-100%) is supported. (3) Visualized motion comparison: Real-time motion skeleton diagram and standard motion skeleton diagram are displayed synchronously in the interface, and different colors are used to distinguish them (real-time is blue and standard is red). Joints with large deviations are marked (blinking prompts). Feedback delay: The total delay from the completion of posture data extraction to the output of feedback information is ≤100ms, ensuring that the trainee can adjust the movement in a timely manner.

[0012] 3.2.7 Data Storage Module (1) Stored content: including trainee basic information (name, age, rehabilitation type, training plan), movement sequence data for each training session (joint coordinates, angles, speed), standardization score records, feedback suggestions, etc.; (2) Storage format: Action data is stored in JSON format, and scoring records are stored in CSV format, supporting batch export and data analysis; (3) Storage strategy: Local storage uses SQLite database and supports data encryption (AES-256 encryption algorithm) to protect the privacy of trainers; it also supports cloud backup (user authorization required), and the backup frequency can be set to real-time, daily or weekly.

[0013] 3.3 Beneficial Effects (Supplemented and Detailed) Compared with the prior art, the beneficial effects of the present invention are as follows: (1) Strong objectivity: Based on the improved G06T7* algorithm, the quantitative evaluation of parameters such as joint angle and motion trajectory is realized. The angle calculation accuracy is ±1° and the motion trajectory positioning error is ≤1cm. The subjectivity of manual evaluation is avoided and the evaluation results are repeatable and traceable. (2) Non-contact detection: No need for trainees to wear sensors, electrode pads or other equipment, detection can be achieved with just a regular camera. It is convenient to operate and highly comfortable, and is suitable for rehabilitation trainees with sensitive skin and limited limb movement. (3) Real-time feedback: The total delay from image acquisition to feedback output is ≤100ms, which can remind trainees to adjust their movements in real time during training, avoid the solidification of incorrect movements, and improve the effect of rehabilitation training; (4) Multi-dimensional assessment: Breaking through the traditional assessment model that only focuses on the appearance of movement, it conducts a comprehensive assessment from three core dimensions: joint angle, movement trajectory, and movement rhythm, covering the key technical indicators of rehabilitation training; (5) Wide scene adaptability: The standard movement library covers 20 common rehabilitation movements and supports custom adaptation function. It can meet the training needs of different age groups, different rehabilitation stages and different types of limb injuries. It is suitable for various scenarios such as home rehabilitation, community rehabilitation center, and hospital rehabilitation department. (6) Low cost and easy deployment: The hardware only requires a regular computer and a high-definition camera, and the software supports multiple operating systems such as Windows, Linux, and Android. The deployment cost is low and it is easy to promote and apply on a large scale. Detailed Implementation

[0014] 4.1 System Hardware Configuration (1) Image acquisition device: USB 3.0 high-definition camera (model: Logitech C920e), resolution 3840×2160, frame rate 60fps, field of view 120°, distortion rate ≤1%; (2) Processing device: CPU is Intel Core i5-12400H (6 cores and 12 threads, 2.5GHz), GPU is NVIDIA-GTX1650 (4GB video memory), memory is 16GB-DDR4-3200MHz, storage is 512GB NVMe-SSD; (3) Output device: 27-inch monitor (resolution 2560×1440), built-in speaker (frequency response range 20Hz-20kHz); (4) Auxiliary equipment: tripod (height adjustable range 0.8-1.5m) is used to fix the camera and ensure stable acquisition angle.

[0015] 4.2 Software Implementation Process.

[0016] 4.2.1 System Initialization: (1) Hardware calibration: After the camera is started, the Zhang Zhengyou calibration program is automatically executed to acquire 10 chessboard images from different angles, calculate the intrinsic parameter matrix (focal length fx, fy, principal point coordinates cx, cy) and distortion coefficients (k1, k2, p1, p2) to complete the calibration; (2) Template loading: When the user selects the target rehabilitation movement (such as "knee flexion and extension training"), the system loads the corresponding movement template from the standard movement library, including parameters such as joint angle range (60°-100°), movement trajectory curve, and standard duration (2.5s); (3) Parameter settings: The user inputs height (175cm), weight (70kg), rehabilitation stage (recovery period), and the system automatically adjusts the angle range (55°-95°) and maximum movement speed (≤25cm / s) of the standard movement template; (4) Image acquisition: The camera acquires images of the trainee's action sequence in real time, compresses them using H.264 encoding format, and transmits them to the processing device with a transmission delay of ≤50ms.

[0017] 4.2.2 Image Preprocessing: (1) Gaussian filtering: The image is convolved using a 5×5 Gaussian kernel with σ=1.2 to remove Gaussian noise; (2) Adaptive histogram equalization: Divide the image into 8×8 sub-blocks, set a contrast threshold of 2.0, and enhance joint contour details; (3) Motion region extraction: Pixel difference is calculated using the three-frame difference method (T=20), morphological opening operation (3×3 rectangular structuring element) is used to remove small noise regions, and motion region mask is extracted.

[0018] 4.2.3 Pose Extraction: (1) Feature extraction: The G06T7* edge detection algorithm calculates the image gradient and extracts human contour features; (2) Candidate region screening: Non-maximum suppression (NMS, threshold 0.5) is used to screen human candidate regions; (3) Key point localization: The MobilePose model outputs the two-dimensional coordinates of 14 key points, which are then converted into three-dimensional coordinates using triangulation. (4) Coordinate calibration: The coordinates of the joints are distorted using the camera intrinsic parameter matrix and distortion coefficients; (5) Parameter calculation: The joint angle is calculated based on the vector dot product method (e.g., knee joint angle = arc cos[(thigh vector・lower leg vector) / (|thigh vector|×|lower leg vector|)]), and the motion velocity is calculated based on the inter-frame difference method.

[0019] Standardization assessment: (6) Joint angle similarity: Calculate the similarity between the real-time angle of the left knee (75°) and the standard mean (77.5°) = 1 - |75 - 77.5| / 12.5 = 0.8; (7) Trajectory overlap: The DTW algorithm calculates the normalized distance between the real-time trajectory and the standard trajectory, and the overlap is 0.85; (8) Matching degree of movement rhythm: The matching degree between real-time duration (2.3s) and standard duration (2.5s) = 1 - |2.3 - 2.5| / 2.5 = 0.92; (9) Overall score: 0.8×0.4+0.85×0.35+0.92×0.25=0.32+0.2975+0.23=0.8475, that is, 84.75 points (pass).

[0020] 4.2.4 Feedback Output: (1) Text prompt: The interface displays "Qualified, left knee bending angle 75° (standard range 55°-95°)"; (2) Voice broadcast: "Your movements are correct, please continue to maintain them"; (3) Visual comparison: The interface synchronously displays the blue real-time skeleton diagram and the red standard skeleton diagram, without any flickering prompts; (4) Data storage: The system automatically stores the joint coordinates, angles, speed data and the score record of 84.75 points for this training, and generates a training log (file name: 20240520_knee flexion and extension_qualified.log).

[0021] 4.3 System debugging and optimization.

[0022] 4.3.1 Light Adaptability Adjustment: Under three lighting conditions—100 lux (low light), 500 lux (natural light), and 1000 lux (strong light)—50 knee flexion and extension movements were tested respectively. The recognition accuracy rates were 88.5%, 94.2%, and 93.7%, with fluctuations ≤3%, meeting the needs of different scenarios. (1) Individual adaptation adjustment: 20 trainees of different heights (155-190cm) and different rehabilitation stages (acute phase, recovery phase, sequelae phase) were selected to test the system's adaptability. After custom adjustment, the recognition accuracy rate was ≥89%; (2) Algorithm optimization: The inference speed of the MobilePose model was optimized by GPU acceleration (CUDA11.0), reducing the pose extraction latency from 120ms to 87ms to meet the real-time feedback requirements; (3) Stability test: The system was run continuously for 24 hours, and 10,000 frames of motion images were collected. There were no stutters or crashes, and the data storage integrity rate was 100%. Attached Figure Description

[0023] Appendix Figure 1 The system architecture diagram shows the hardware composition and data interaction relationship of each module, and marks key parameters such as data transmission rate and storage capacity.

[0024] Appendix Figure 2 A schematic diagram of key human joints, labeled with the joint number, world coordinate system definition, and coordinate accuracy.

[0025] Appendix Figure 3 The flowchart of the standardization assessment process shows the multi-dimensional comparison algorithm, dynamic weight calculation and scoring logic.

[0026] Appendix Figure 4 The flowchart of the improved G06T7* algorithm shows the complete process of feature extraction, key point localization, and coordinate calibration.

Claims

1. A computer vision-based (G06T7*) intelligent recognition system for the standardization of rehabilitation training movements, characterized in that, The system includes an image acquisition module, a preprocessing module, a human posture extraction module, a standard action library module, a standardization assessment module, a feedback output module, and a data storage module. The image acquisition module acquires images of the rehabilitation trainees' movement sequences, employing an adaptive lighting adjustment algorithm to suit lighting environments ranging from 100 to 1000 lux. The preprocessing module uses a combination of Gaussian filtering, histogram equalization, and inter-frame differencing to remove background interference and noise. The human posture extraction module, based on an improved G06T7* visual detection algorithm and incorporating the MobilePose lightweight model, extracts the 3D coordinates, joint angles, and motion parameters of 14 key joints of the human body. The standard action library module stores clinically validated rehabilitation training action templates and supports custom adaptation. The standardization assessment module calculates a comprehensive standardization score through multi-dimensional comparison and a dynamic weighting algorithm. The feedback output module provides real-time multimodal feedback. The data storage module stores training data and assessment records.

2. The system according to claim 1, characterized in that, The image acquisition module uses a high-definition camera with a USB 3.0 interface, a maximum resolution of 3840×2160, a frame rate of 30-60fps, a field of view of 120°, and a distortion rate of ≤1%. The adaptive lighting adjustment algorithm dynamically adjusts the sensitivity (ISO100-1600) and exposure time (1 / 120s-1 / 15s) by collecting ambient light intensity in real time to ensure clear images under different lighting conditions.

3. The system according to claim 1, characterized in that, In the preprocessing module, Gaussian filtering uses a 5×5 Gaussian kernel (σ=1.2), histogram equalization uses an adaptive algorithm (8×8 sub-blocks, contrast threshold 2.0), motion region extraction uses three-frame difference method (threshold T=20) and morphological opening operation (3×3 rectangular structuring element), and background removal accuracy is ≥98%.

4. The system according to claim 1, characterized in that, The human posture extraction module extracts 14 key joints, including the head, neck, shoulder, elbow, wrist, hip, knee, and ankle. The joint coordinate accuracy is ≤1cm, the joint angle calculation accuracy is ±1°, and the motion parameters include velocity (error ≤5%) and acceleration. The improved G06T7* algorithm includes five steps: feature extraction, candidate region screening, key point localization, coordinate calibration, and parameter calculation. The posture extraction delay is ≤100ms.

5. The system according to claim 1, characterized in that, The standard movement library module contains 20 common rehabilitation movement templates in three categories: upper limb rehabilitation, lower limb rehabilitation, and trunk rehabilitation. Each template includes four quantitative parameters: joint angle range, movement trajectory curve, movement duration, and movement speed threshold. It supports automatic adjustment of template parameters to adapt to different individuals by inputting parameters such as height, weight, and rehabilitation stage.

6. The system according to claim 1, characterized in that, The multi-dimensional comparison model of the standardization evaluation module includes joint angle similarity (cosine similarity algorithm), motion trajectory overlap (DTW algorithm, normalized window size of 5), and movement rhythm matching degree (deviation rate calculation). The dynamic weights are determined by the analytic hierarchy process as joint angle 0.4, motion trajectory 0.35, and movement rhythm 0.

25. The comprehensive score ranges from 0 to 100 points, with ≥80 points being qualified, 60-79 points being basically qualified, and <60 points being unqualified. The misjudgment rate is ≤3%.

7. The system according to claim 1, characterized in that, The feedback output module adopts a multimodal feedback method with text prompts, voice broadcasts, and visual action comparisons. The text prompts indicate the specific problem joints and parameter deviations, the voice broadcasts use TTS technology (frequency 1 time / 2s), and the visual interface synchronously displays real-time and standard action skeleton diagrams (different colors to distinguish them). The total feedback delay is ≤100ms.

8. The system according to claim 1, characterized in that, The data storage module uses an SQLite database, supports AES-256 encrypted storage, and has a maximum storage capacity of ≥10GB per user. The stored content includes trainee basic information, action data, scoring records, and feedback suggestions. It supports local storage and cloud backup (with user authorization).

9. The system according to claim 1, characterized in that, The system supports Windows, Linux, and Android operating systems. The hardware only requires a regular computer and a high-definition camera, resulting in low deployment costs. It is suitable for scenarios such as home rehabilitation, community rehabilitation centers, and hospital rehabilitation departments.

Images