Musculoskeletal rehabilitation posture correction method and system based on visual recognition

Abstract

The application provides a vision-identification-based musculoskeletal rehabilitation posture correction method and system. First, posture video data of a rehabilitation subject in a musculoskeletal rehabilitation training is collected, covering the joint movement, muscle contour change and body balance state change process. The posture video data is analyzed to generate joint movement trajectory, muscle force correlation and body balance state characteristics, which are corresponded to a preset musculoskeletal rehabilitation stage target to establish a multi-dimensional adaptation relationship. Hierarchical and staged correction guidance information is generated according to the multi-dimensional adaptation relationship. The joints, muscles and balance adjustment layers are prioritized and specific adjustment methods are provided. New posture video data after adjustment is obtained for repeated operation. The correction guidance information is updated according to the deviation change, precise and dynamic musculoskeletal rehabilitation posture correction is realized, and the rehabilitation effect and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and system for musculoskeletal rehabilitation posture correction based on visual recognition. Background Technology

[0002] In the field of musculoskeletal rehabilitation, effective rehabilitation training is crucial for patients' functional recovery. Traditional musculoskeletal rehabilitation posture correction methods mainly rely on the manual observation and experience of rehabilitation therapists. Rehabilitation therapists visually observe patients' training movements, using their professional knowledge and clinical experience to assess whether the patient's posture is correct and provide corresponding corrective suggestions.

[0003] However, the above methods have many limitations. On the one hand, human observation is subjective; different rehabilitation therapists may have different judgments about the same posture, making it difficult to guarantee the accuracy and consistency of corrective suggestions. On the other hand, human observation cannot comprehensively and accurately capture the subtle changes in the rehabilitation subject during training. For example, minute movement paths of joints, subtle changes in muscle contours, and slight shifts in the body's center of gravity are crucial for accurately assessing the rehabilitation subject's posture and developing targeted corrective plans, but human observation often cannot accurately identify these details.

[0004] Furthermore, while existing rehabilitation equipment and technologies can provide some data support, they mostly focus on monitoring single indicators, such as joint angles or muscle strength, lacking a comprehensive and multi-dimensional analysis of the rehabilitation subject's posture. This one-sided monitoring method cannot fully reflect the overall state of the rehabilitation subject during training, making it difficult to formulate scientific and effective corrective strategies, thus affecting the effectiveness and efficiency of musculoskeletal rehabilitation. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a musculoskeletal rehabilitation posture correction method based on visual recognition, the method comprising:

[0006] Collect postural video data of rehabilitation subjects during musculoskeletal rehabilitation training. The postural video data includes the joint movement process, muscle contour change process, and body balance change process of rehabilitation subjects under different training movements.

[0007] The posture video data is analyzed to generate joint motion trajectory features, muscle force correlation features, and body balance features of the rehabilitation subject. The joint motion trajectory features reflect the movement path and smoothness of the joints in the training movements. The muscle force correlation features reflect the dynamic correspondence between muscle contour changes and joint movements. The body balance features reflect the trend of changes in the body's center of gravity during the execution of the training movements.

[0008] The joint motion trajectory features, muscle exertion correlation features, and body balance state features are all mapped to preset musculoskeletal rehabilitation stage goals, and a multi-dimensional adaptation relationship is established between them and the musculoskeletal rehabilitation stage goals. The multi-dimensional adaptation relationship includes the deviation correlation between each type of feature and the corresponding standard feature, as well as the mutual influence relationship between the deviations of different features.

[0009] Based on the multi-dimensional adaptation relationship, hierarchical and phased correction guidance information is generated for the current training movement of the rehabilitation subject. The hierarchical and phased correction guidance information is divided into joint adjustment layer, muscle adjustment layer and balance adjustment layer according to the correction priority. Each adjustment layer contains the specific adjustment method for the corresponding training movement stage.

[0010] The system obtains video data of the new posture performed by the rehabilitation subject according to the hierarchical and phased correction guidance information. It then repeatedly performs parsing and multi-dimensional adaptation operations on the new posture video data to generate a new multi-dimensional adaptation relationship. Based on the difference in deviation between the new multi-dimensional adaptation relationship and the original multi-dimensional adaptation relationship, it updates the adjustment level priority and specific adjustment method of the hierarchical and phased correction guidance information.

[0011] Furthermore, embodiments of the present invention also provide a musculoskeletal rehabilitation posture correction system based on visual recognition, comprising:

[0012] A processor; a machine-readable storage medium for storing machine-executable instructions of the processor; wherein the processor is configured to perform the above-described vision-based musculoskeletal rehabilitation posture correction method by executing the machine-executable instructions.

[0013] In another aspect, embodiments of the present invention also provide a computer program product, the computer program product including machine-executable instructions, the machine-executable instructions being stored in a computer-readable storage medium, a processor of a computer device reading the machine-executable instructions from the computer-readable storage medium, the processor executing the machine-executable instructions, causing the computer device to perform the above-described musculoskeletal rehabilitation posture correction method based on visual recognition.

[0014] Based on the above, by collecting postural video data of rehabilitation subjects during musculoskeletal rehabilitation training, it is possible to obtain the joint movement process, muscle contour changes, and body balance changes of the subjects under different training movements. Multi-dimensional features generated by analyzing the postural video data include joint movement trajectory features, muscle force correlation features, and body balance features, comprehensively depicting the postural state of the rehabilitation subjects during training from different perspectives. These features are mapped to preset musculoskeletal rehabilitation stage goals, establishing a multi-dimensional adaptation relationship. This not only considers the deviation correlation between each type of feature and its corresponding standard feature but also deeply analyzes the mutual influence relationship between deviations of different features, achieving a multi-dimensional, comprehensive, and accurate assessment of the rehabilitation subjects' posture. Based on this multi-dimensional adaptation relationship, hierarchical and staged correction guidance information is generated, divided into joint adjustment layer, muscle adjustment layer, and balance adjustment layer according to correction priority. Each adjustment layer provides specific adjustment methods for the corresponding training movement stage, making the correction guidance more targeted and systematic. By acquiring video data of the new postures of rehabilitation subjects after adjustment and repeatedly analyzing and performing multi-dimensional adaptation operations, the adjustment priority and specific adjustment methods of the hierarchical and phased correction guidance information are dynamically updated based on the deviation changes between the new and original multi-dimensional adaptation relationships. This achieves real-time optimization and personalized adjustment of the correction process. Thus, it overcomes the limitations of traditional manual observation and single-indicator monitoring, assessing and correcting the postures of rehabilitation subjects from a global and comprehensive perspective. This significantly improves the accuracy, scientific rigor, and effectiveness of musculoskeletal rehabilitation posture correction, contributing to enhanced rehabilitation outcomes and efficiency. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the execution flow of the musculoskeletal rehabilitation posture correction method based on visual recognition provided in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram of exemplary hardware and software components of the musculoskeletal rehabilitation posture correction system based on visual recognition provided in an embodiment of the present invention. Detailed Implementation

[0017] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating a visual recognition-based musculoskeletal rehabilitation posture correction method according to an embodiment of the present invention. The following is a detailed description of this visual recognition-based musculoskeletal rehabilitation posture correction method.

[0018] Step S110: Collect postural video data of the rehabilitation subject during the musculoskeletal rehabilitation training process. The postural video data includes the joint movement process, muscle contour change process, and body balance change process of the rehabilitation subject under different training movements.

[0019] In this embodiment, the application scenario is set as posture correction for straight leg raises during post-knee surgery rehabilitation training. The rehabilitation subject lies supine on a rehabilitation training mat with both legs naturally straight. Two image acquisition devices are used, one positioned directly in front of the subject's body and the other to the left, to ensure complete capture of the entire movement process from preparing to raise the leg to raising it to the designated height and then slowly lowering it. After the image acquisition devices are turned on, the training movements of the rehabilitation subject are recorded in real time. The acquired video data clearly shows the movement of the hip, knee, and ankle joints during the movement, the contraction and relaxation changes of muscles such as the quadriceps and hamstrings as the movement progresses, and the changes in body balance reflected by the overall positional changes of the body on the training mat.

[0020] During the acquisition process, the video data is encrypted in real time using a symmetric encryption algorithm. The key is automatically generated by the system and stored in the security module to prevent unauthorized access to the data during transmission and storage.

[0021] Step S120: Analyze the posture video data to generate joint motion trajectory features, muscle force correlation features and body balance features of the rehabilitation subject. The joint motion trajectory features reflect the movement path and smoothness of the joint in the training movement. The muscle force correlation features reflect the dynamic correspondence between muscle contour changes and joint movements. The body balance features reflect the trend of changes in the body's center of gravity during the execution of the training movement.

[0022] Step S121: Perform adaptive frame segmentation processing on the posture video data, dynamically adjust the frame interval according to the changes in the amplitude of the training action, and obtain a set of continuous video frames with dynamic temporal density.

[0023] The acquired posture video data is imported into the frame segmentation processing module. This module first analyzes the changes in the rehabilitation subject's movements in the video. When it detects that the subject begins to lift their leg and the amplitude of the movement gradually increases, the frame interval is shortened, increasing the number of video frames per unit time. When the movement reaches its peak and remains stable, the frame interval is increased, decreasing the number of video frames. When the subject begins to lower their leg and the amplitude of the movement changes again, the frame interval is shortened again. Through this method, the video data is segmented into continuous video frames. The temporal distribution density of these video frames is dynamically adjusted according to the amplitude of the movement, forming a continuous set of video frames.

[0024] Step S122: Extract the key joint positions of the rehabilitation object from each video frame in the continuous video frame set, confirm the correspondence of the same key joint in different video frames, record the two-dimensional or three-dimensional coordinates of each key joint in each video frame, and form a key joint coordinate sequence.

[0025] Step S1221: Select the first video frame from the set of continuous video frames as the reference frame. In the reference frame, identify the key joints of the rehabilitation subject through the human posture recognition model. The key joints include the shoulder joint, elbow joint, wrist joint, hip joint, knee joint, ankle joint, cervical spine joint, and lumbar spine joint. Assign a unique joint identifier to each key joint.

[0026] In a continuous set of video frames, the first frame before the rehabilitation subject begins to lift their leg is designated as the baseline frame. This baseline frame is input into a human posture recognition model, which processes the image and identifies key joints such as the shoulder, elbow, wrist, hip, knee, ankle, cervical spine, and lumbar spine. Each identified key joint is assigned a unique identifier; for example, the shoulder joint is represented by SJ, the elbow by ZJ, the wrist by WJ, the hip by GJ, the knee by XJ, the ankle by HGJ, the cervical spine by JJGJ, and the lumbar spine by YZJGJ.

[0027] Step S1222: Perform local image cropping on the region of each key joint in the reference frame to obtain a joint local image. Extract the texture and shape features of the joint local image as the reference feature template of the key joint. The reference feature template corresponds one-to-one with the joint identifier.

[0028] A square image region is cropped centered on the center point of each key joint identified in the baseline frame; this region is the joint local image. The joint local image is then converted to grayscale to contain only brightness information. Texture features are extracted from the joint local image by calculating the grayscale difference between each pixel and its surrounding pixels; shape features are extracted by analyzing the contour lines of the joint region and the shape of the area enclosed by the contour. The extracted texture and shape features are combined to form the baseline feature template for that key joint, and each baseline feature template is associated with a corresponding joint identifier, such as the baseline feature template for the shoulder joint (SJ) and the baseline feature template for the elbow joint (ZJ).

[0029] Step S1223: Select the next video frame after the reference frame as the current frame. In the current frame, use the same human pose recognition model to initially identify the candidate positions of each key joint and assign a temporary identifier to each candidate position.

[0030] After processing the reference frame, the next video frame is taken as the current frame. The same human pose recognition model used to process the reference frame is applied to the current frame. The model identifies multiple possible locations for key joints within the current frame; these locations are called candidate locations. Each candidate location is assigned a temporary name, such as candidate location 1, candidate location 2, etc., to distinguish between different candidate locations.

[0031] Step S1224: Perform local image cropping on the region of each candidate position in the current frame to obtain candidate local images, and extract the texture and shape features of the candidate local images as candidate feature templates.

[0032] For each candidate location of a key joint in the current frame, a square image region of the same size as the local joint image is cropped, centered on the center point of the candidate location; this is the candidate local image. Using the same method as extracting the baseline feature template, the candidate local image is converted to grayscale, and its texture and shape features are extracted to form the candidate feature template.

[0033] Step S1225: Compare the similarity of each candidate feature template with the baseline feature templates of all key joints, and calculate the feature similarity value.

[0034] Each candidate feature template is compared with the baseline feature templates of all previously obtained key joints. During the comparison, the similarity between the candidate feature template and the baseline feature template is measured by calculating their distance in the feature space. The smaller the distance, the higher the similarity, and the larger the feature similarity value obtained.

[0035] Step S1226: The candidate position corresponding to the candidate feature template with the highest similarity value that exceeds the preset similarity threshold is determined as the key joint position of the joint identifier corresponding to the reference feature template. If the similarity value between all candidate feature templates and any reference feature template does not exceed the preset similarity threshold, the position of the key joint in the current frame is predicted by the adjacent frame joint position prediction algorithm. The adjacent frame joint position prediction algorithm uses the predicted position as the position of the key joint based on the position of the key joint in the reference frame and the motion trend of the previous frame.

[0036] For each candidate feature template, the baseline feature template with the highest similarity value is identified. If this highest similarity value is greater than a pre-set standard value (similarity threshold), then the candidate position corresponding to this candidate feature template is determined as the position of the key joint identified by the joint of the baseline feature template in the current frame. If the similarity value between any candidate feature template and any baseline feature template does not reach the similarity threshold, the most likely position of the key joint in the current frame is inferred based on its position in the baseline frame and its movement direction and speed in the previous frame. This inferred position is then taken as the position of the key joint in the current frame.

[0037] Step S1227: Record the two-dimensional or three-dimensional coordinates of each key joint in the current frame.

[0038] After determining the position of each key joint in the current frame, the corresponding horizontal and vertical coordinates (i.e., two-dimensional coordinates) are read in the image coordinate system of the current frame. If the acquisition device supports three-dimensional coordinate acquisition, its three-dimensional coordinates will also be recorded. These coordinate values ​​are recorded, such as the coordinates of the shoulder joint SJ in the current frame being (x1, y1), and the coordinates of the knee joint XJ being (x2, y2), etc.

[0039] Step S1228: Set the current frame as the new reference frame, update the reference feature template of each key joint. The update of the reference feature template is based on the local image features of the key joint in the current frame. Repeat the above steps of initially identifying the candidate positions of each key joint in the current frame using the same human pose recognition model and assigning a temporary identifier to each candidate position until all video frames in the continuous video frame set have been processed.

[0040] After determining the key joint positions and recording the coordinates of the current frame, the current frame is used as the new reference frame. Following the method in step S1222, the local image features of each key joint in the new reference frame are extracted again, and the previous reference feature template is updated. Then, the next video frame is taken as the new current frame, and the operations of candidate position recognition, candidate feature template extraction, similarity comparison, joint position determination, and coordinate recording are repeated. This process is continued frame by frame until all video frames in the continuous video frame set have been processed.

[0041] Step S1229: Organize the coordinates of each key joint in all video frames according to the joint identifier to form a coordinate sequence for each key joint. The coordinate sequence includes the video frame number, joint identifier, two-dimensional or three-dimensional coordinate value and coordinate determination method. Integrate the coordinate sequences of all key joints to form the key joint coordinate sequence.

[0042] After processing all video frames, the coordinate values ​​of each key joint in different video frames are aggregated according to the joint identifier. For example, the coordinate values ​​of the shoulder joint SJ in frame 1, frame 2, ..., frame n are arranged sequentially to form the coordinate sequence of the shoulder joint SJ. Each coordinate sequence contains the corresponding video frame number, such as frame 1, frame 2, etc.; the joint identifier, such as SJ; the two-dimensional or three-dimensional coordinate value of the joint in the corresponding video frame; and the coordinate determination method, i.e., whether it is determined by similarity comparison or by a prediction algorithm. Finally, the coordinate sequences of all key joints are integrated together to form the key joint coordinate sequence.

[0043] Step S123: Connect the coordinates of the same key joint in adjacent video frames in sequence to form the initial motion trajectory of the key joint. Perform path smoothness analysis on the initial motion trajectory, calculate the rate of change of the angle between adjacent line segments on the initial motion trajectory, remove the trajectory points corresponding to abnormal angles, and obtain the optimized key joint motion trajectory.

[0044] From the key joint coordinate sequence, extract the coordinate values ​​of the same key joint in adjacent video frames, such as (x1, y1) in frame 1, (x2, y2) in frame 2, and (x3, y3) in frame 3. Connect these coordinate points sequentially with line segments; the resulting line is the initial motion trajectory of the key joint. Analyze the initial motion trajectory to check its smoothness. Specifically, on the initial motion trajectory, take three consecutive coordinate points A, B, and C, connect AB and BC to form two adjacent line segments, and calculate the angle between these two line segments. Then, take points B, C, and D, calculate the angle between BC and CD, and so on, obtaining a series of angle values. Calculate the ratio of the difference between two adjacent angles to their corresponding time interval; this is the angle change rate. If the angle change rate differs significantly from the surrounding angle change rates, it indicates that the trajectory point corresponding to this angle may be abnormal, and this trajectory point is removed from the initial motion trajectory. After the above processing, the obtained trajectory is the optimized key joint motion trajectory.

[0045] Step S124: Collect the optimized motion trajectories of all key joints, classify and integrate them according to the body parts to which the joints belong, label the training action segments corresponding to each optimized motion trajectory, and form the joint motion trajectory features. The joint motion trajectory features include joint classification identifiers, action segment identifiers, coordinate sequences, and trajectory smoothness parameters.

[0046] The optimized motion trajectories of all key joints are collected. These joints are then categorized according to their body part: for example, shoulder (SJ), elbow (ZJ), and wrist (WJ) are classified as upper limbs; hip (GJ), knee (XJ), and ankle (HGJ) as lower limbs; and cervical spine (JJGJ) and lumbar spine (YZJGJ) as trunk. The straight leg raise movement is then divided into four phases: preparation, leg raise, maintenance, and lowering. Each key joint's optimized motion trajectory is labeled with its position within the leg raise phase (e.g., hip GJ in the leg raise phase, knee XJ in the maintenance phase). The trajectory smoothness parameter is calculated by averaging the rate of change of the angle between all adjacent line segments on the optimized trajectory. Combining the joint classification labels (e.g., upper limb, lower limb, trunk), movement phase labels (e.g., preparation, leg raise), coordinate sequences (coordinate point sequences of the optimized trajectory), and trajectory smoothness parameters creates the joint motion trajectory characteristics.

[0047] Step S125: Identify the main muscle regions of the rehabilitation subject from each video frame in the continuous video frame set, extract the contour boundaries of each main muscle region, mark the feature points on the contour boundaries, record the coordinates of the feature points in the video frames, and form a muscle contour feature point sequence.

[0048] Step S1251: Select representative video frames from the set of continuous video frames. The representative video frames include the starting frame of the training action, the frame with the maximum range of motion, and the ending frame of the action. In the representative video frames, identify the main muscle regions of the rehabilitation subject through a muscle region segmentation model. The main muscle regions include the biceps brachii region, triceps brachii region, quadriceps femoris region, hamstring region, latissimus dorsi region, and rectus abdominis region. Assign a unique muscle identifier to each main muscle region.

[0049] In a continuous set of video frames, the frame in which the rehabilitation subject begins preparing to perform a straight leg raise is selected as the starting frame of the training exercise; the frame in which the leg is raised to its highest position is selected as the frame with the maximum range of motion; and the frame in which the leg is fully lowered back to the initial position is selected as the ending frame. These three video frames are used as representative video frames. A muscle region segmentation model is used to process the representative video frames. The model can identify the regions containing the major muscles of the rehabilitation subject in the video frames, such as the biceps brachii, triceps brachii, quadriceps femoris, hamstrings, latissimus dorsi, and rectus abdominis. Each identified major muscle region is assigned a unique identifier, such as GSQ for the quadriceps femoris region and GHS for the hamstrings region.

[0050] Step S1252: Annotate the contours of each major muscle region in representative video frames to obtain standard muscle contours. Store the standard muscle contours in association with muscle identifiers as a reference for muscle region recognition.

[0051] In representative video frames, based on the main muscle regions identified by the muscle region segmentation model, the outline of the muscle regions is manually or automatically drawn along their edges. This outline is the standard muscle outline. The information of the standard muscle outline and the corresponding muscle identifier are stored together. For example, the standard muscle outline of the quadriceps femoris region GSQ is associated with GSQ, serving as a reference for subsequent identification of the muscle region in other video frames.

[0052] Step S1253: Preprocess each video frame in the continuous video frame set. The preprocessing includes grayscale processing, Gaussian filtering for noise reduction, and contrast enhancement processing. The preprocessed video frames retain the grayscale difference between the muscle area and the surrounding skin area.

[0053] For each video frame in the continuous video frame set, grayscale processing is first performed, converting the color image to a black and white image so that the brightness of each pixel in the image is represented by a grayscale value. Then, Gaussian filtering is used to denoise the grayscale image, reducing noise interference in the image. Next, the image contrast is adjusted to make the difference in brightness between the muscle area and the surrounding skin area more obvious, so as to better identify the contour of the muscle area in subsequent steps.

[0054] Step S1254: Use an edge detection algorithm to extract image edges from the preprocessed video frames to obtain edge images. In the edge images, based on the standard muscle contours in representative video frames, use a template matching algorithm to locate the region position of each major muscle region and determine the candidate contour region of each major muscle region in the current video frame.

[0055] An edge detection algorithm is applied to the preprocessed video frames. This algorithm detects pixels with drastic changes in grayscale values ​​and connects these pixels to form edge lines, resulting in an edge image. Within this edge image, a standard muscle contour from a representative video frame is used as a template. A template matching algorithm searches for regions in the edge image that are similar in shape to the template. The region with the highest similarity is identified as a candidate contour region for the main muscle area in the current video frame.

[0056] Step S1255: Extract the contour for each candidate contour region and use a contour tracking algorithm to obtain the pixel sequence of the candidate contour. The contour tracking algorithm includes contour tracking based on 8 neighborhoods, removing short contour segments with a length less than a preset length threshold from the pixel sequence, and retaining long contour segments as the muscle contour boundary.

[0057] For each candidate contour region, an 8-neighborhood-based contour tracking algorithm is used for contour extraction. Starting from a starting pixel on the edge of the candidate contour region, pixels in the eight directions surrounding that pixel (up, down, left, right, upper left, upper right, lower left, lower right) are examined sequentially. If an edge pixel is found, tracking continues until the starting pixel is reached, forming a closed sequence of contour pixels. Then, the number of pixels in this sequence is calculated, which is the length of the contour segment. If the length is less than a pre-set threshold, the short contour segment is removed; if the length is greater than or equal to the threshold, the long contour segment is retained as the muscle contour boundary.

[0058] Step S1256: Feature point filtering is performed on the pixels on the muscle contour boundary. The filtering rules include the inflection point of the contour boundary, the endpoint of the contour boundary, and the point on the contour boundary that is more than a preset distance threshold from the adjacent feature point. The inflection point of the contour boundary is the point whose curvature value exceeds a preset curvature threshold. The filtered points are used as muscle contour feature points.

[0059] Feature points are selected from the pixel sequence of the muscle contour boundary. First, the endpoints on the contour boundary are identified, that is, the start and end points of the contour. Then, the curvature value of each pixel on the contour boundary is calculated. The curvature value represents the degree of curvature of the curve at that pixel. When the curvature value exceeds a preset curvature threshold, that pixel is the inflection point of the contour boundary. In addition, if the distance between a point on the contour boundary and its adjacent feature points exceeds a preset distance threshold, that pixel will also be selected as a feature point. The points obtained after this selection are the muscle contour feature points.

[0060] Step S1257: Assign a feature point number to each muscle contour feature point. The feature point numbers are arranged sequentially along the muscle contour boundary in a clockwise or counterclockwise direction. Record the two-dimensional coordinates of each feature point in the current video frame. The two-dimensional coordinates are based on the pixel coordinate system of the video frame.

[0061] Starting from one endpoint of the muscle contour boundary in a clockwise direction, each muscle contour feature point is assigned a number along the boundary, such as 1, 2, 3, etc. Then, in the pixel coordinate system of the current video frame, the horizontal and vertical coordinate values ​​of each feature point, that is, the two-dimensional coordinates, are read and recorded, such as the coordinates of feature point 1 being (a1, b1), the coordinates of feature point 2 being (a2, b2), and so on.

[0062] Step S1258: Organize the coordinates of the muscle contour feature points of each major muscle region in the current video frame according to the muscle identifier to form a subset of the contour feature points of the muscle region in the current video frame. The subset of contour feature points includes the muscle identifier, feature point number and corresponding coordinates.

[0063] Based on the muscle identifier, the coordinates of the muscle contour feature points of each major muscle region in the current video frame are organized. For example, for the quadriceps muscle region GSQ, the indices and corresponding coordinates of all its feature points are organized together to form a subset. This subset contains information such as the muscle identifier GSQ, feature point indices 1 and their coordinates (a1, b1), feature point indices 2 and their coordinates (a2, b2), etc.

[0064] Step S1259: Perform the above preprocessing, edge detection, contour localization, contour extraction, feature point filtering and coordinate recording operations on all video frames in the continuous video frame set. Integrate the contour feature point subsets of each major muscle region in order of video frame number to form the muscle contour feature point sequence. The muscle contour feature point sequence includes video frame number, muscle identifier, feature point number and feature point coordinates.

[0065] For each video frame in the continuous video frame set, preprocessing, edge detection, contour localization, contour extraction, feature point selection, and coordinate recording are performed sequentially. Then, starting from video frame number 1, the subsets of contour feature points for each major muscle region in different video frames are integrated. For example, the subsets of contour feature points for the quadriceps GSQ region in video frame 1, the subsets of contour feature points for the quadriceps GSQ region in video frame 2, etc., are arranged in order of video frame number to form the muscle contour feature point sequence. This muscle contour feature point sequence includes the video frame number (e.g., 1, 2, 3...), muscle identifier (e.g., GSQ, GHS, etc.), feature point number (e.g., 1, 2, 3...), and feature point coordinates (e.g., (a1, b1), (a2, b2), etc.).

[0066] Step S126: Compare the contour feature point sequences of the same main muscle region in adjacent video frames, calculate the coordinate offset of the corresponding feature points, and calculate the mean and variance of the coordinate offsets of all feature points. Use the mean as the amplitude of muscle contour change and the variance as the stability parameter of muscle contour change to form dynamic change data of muscle contour.

[0067] Take two consecutive video frames, such as frame i and frame i+1, and for the same major muscle region, such as the quadriceps femoris region (GSQ), compare the contour feature point sequence of this region in frame i with the contour feature point sequence of this region in frame i+1. For feature points with the same index, calculate the difference between them in the horizontal and vertical coordinates, that is, the coordinate offset. For example, the offset of the first feature point in the x-direction is a(i+1, 1) - a(i, 1), and the offset in the y-direction is b(i+1, 1) - b(i, 1). Then, calculate the average value of the coordinate offsets of all feature points. This average value is the amplitude of muscle contour change. Calculate the variance of the coordinate offsets of all feature points. The variance reflects the dispersion of the offsets and is used as a stability parameter of muscle contour change. Integrating muscle identification, video frame number, muscle contour change amplitude, and change stability parameter together forms the dynamic change data of muscle contour.

[0068] Step S127: Perform correlation analysis on the movement distance and speed of each key joint in the joint movement trajectory features in the corresponding time period with the change amplitude and stability parameters of the muscle contour of the associated muscle region in the same time period, and determine the correlation coefficient between the joint movement parameters and the muscle contour change parameters.

[0069] From the joint motion trajectory features, the motion trajectory of each key joint within a certain time period is extracted, and the distance of joint movement within that time period, i.e., the length of the trajectory, is calculated; the motion speed is the motion distance divided by the duration of that time period. Simultaneously, the muscle regions associated with the key joint are identified, such as the quadriceps region GSQ associated with the knee joint (XJ). Within the same time period, the amplitude and stability parameters of muscle contour changes in this muscle region are obtained from the dynamic change data of muscle contours. Then, by calculating the correlation between joint motion distance and the amplitude of muscle contour changes, the correlation between joint motion speed and the amplitude of muscle contour changes, the correlation between joint motion distance and the stability parameters, and the correlation between joint motion speed and the stability parameters, the correlation coefficients between joint motion parameters and muscle contour change parameters are obtained by combining these correlations.

[0070] Step S128: Divide the joint-muscle association groups according to the correlation coefficient. Each joint-muscle association group includes one or more key joints and corresponding associated muscle regions. Record the corresponding rules of joint movement and muscle contour changes within each joint-muscle association group to form the muscle force association features. The muscle force association features include association group identifiers, correlation coefficients, and descriptions of joint-muscle correspondence rules.

[0071] A standard value for the correlation coefficient is set. When the correlation coefficient between joint motion parameters and muscle contour change parameters is greater than or equal to this standard value, the key joint and its corresponding muscle region are grouped into a joint-muscle association group. For example, the knee joint (XJ) and the quadriceps femoris region (GSQ) have a high correlation coefficient, so they are grouped into one association group. Each association group has a unique association group identifier, such as G1, G2, etc. Within each joint-muscle association group, the correspondence between joint motion and muscle contour change is analyzed. For example, how does the amplitude of muscle contour change change when the joint motion distance increases? How does the stability parameter of muscle contour change change when the joint motion speed increases? The association group identifier, correlation coefficient, and description of the joint-muscle correspondence are combined to form the muscle force exertion correlation characteristics.

[0072] Step S129: Extract the body contour of the rehabilitation object from each video frame in the continuous video frame set, and determine the projection coordinates of the body center of gravity in the video frame by the geometric center calculation method of the body contour to form a body center of gravity coordinate sequence.

[0073] For each video frame in a continuous set of video frames, an edge detection algorithm is used to extract the body contour of the rehabilitation subject. The body contour is the outer edge line encompassing the entire body of the subject. Then, the geometric center of the body contour is calculated by summing the horizontal and vertical coordinates of all pixels on the contour and dividing by the total number of pixels. The average of these coordinates is the projected coordinate of the body's center of gravity in the video frame. The projected coordinates of the body's center of gravity obtained from each video frame are arranged in sequence according to the video frame number, forming a sequence of body center of gravity coordinates.

[0074] Step S1210: Analyze the trajectory of the body center of gravity coordinate sequence in consecutive video frames, calculate the offset distance, offset direction and offset speed of the center of gravity trajectory, and count the proportion of time the center of gravity stays in the preset stable area. Integrate the center of gravity trajectory parameters and the proportion of time spent in the stable area to form the body balance state feature. The body balance state feature includes the center of gravity coordinate sequence, trajectory offset parameters and the proportion of time spent in the stable area.

[0075] Based on the body's center of gravity coordinate sequence, the center of gravity coordinates in adjacent video frames are connected sequentially to form a trajectory of center of gravity change. The distance between any two adjacent center of gravity coordinates on this trajectory is calculated; this is the offset distance of the center of gravity trajectory. The offset direction is determined by calculating the vector direction from the starting center of gravity coordinate to the ending center of gravity coordinate. The offset speed is the offset distance divided by the time interval between adjacent video frames. In the image coordinate system of the video frames, a stable region is preset. This stable region is a range defined based on the position of the rehabilitation subject's center of gravity in a static state. The proportion of video frames in the body's center of gravity coordinate sequence whose center of gravity coordinates fall within this stable region is counted out of the total number of video frames; this is the stable region dwell percentage. The trajectory offset parameters include offset distance, offset direction, and offset speed. Integrating the center of gravity coordinate sequence, trajectory offset parameters, and stable region dwell percentage together forms the body balance state characteristics.

[0076] Step S130: The joint motion trajectory features, muscle force correlation features, and body balance state features are mapped to preset musculoskeletal rehabilitation stage goals, and a multi-dimensional adaptation relationship is established with the musculoskeletal rehabilitation stage goals. The multi-dimensional adaptation relationship includes the deviation correlation between each type of feature and the corresponding standard feature, as well as the mutual influence relationship between the deviations of different features.

[0077] Step S131: Obtain a preset musculoskeletal rehabilitation stage target library. The musculoskeletal rehabilitation stage target library is divided into multiple rehabilitation stages according to the rehabilitation progress. Each rehabilitation stage corresponds to a set of standard features. The set of standard features includes standard joint motion trajectory features, standard muscle force correlation features, and standard body balance state features.

[0078] The musculoskeletal rehabilitation stage target database is pre-established, dividing the rehabilitation progress into multiple stages such as early, middle, and late stages based on the rehabilitation time and recovery status of the rehabilitation subjects. Each rehabilitation stage has a corresponding set of standard features. Standard joint motion trajectory features specify the motion trajectory coordinate sequence and trajectory smoothness parameters that key joints should have in each movement segment of that stage; standard muscle force correlation features specify the standard correlation coefficient and joint-muscle correspondence rules of the joint-muscle association group; standard body balance state features specify the standard coordinate sequence of the body center of gravity, standard trajectory offset parameters, and standard stable area dwell percentage.

[0079] Step S132: Based on the rehabilitation diagnosis report, training duration and historical training effect data of the rehabilitation subject, select the target musculoskeletal rehabilitation stage target that matches the current rehabilitation level of the rehabilitation subject from the musculoskeletal rehabilitation stage target library, and extract the standard joint motion trajectory features, standard muscle force correlation features and standard body balance state features corresponding to the target musculoskeletal rehabilitation stage target.

[0080] The rehabilitation diagnosis report records the specific details of the patient's knee injury, surgical procedure, etc.; training duration refers to the total time from the start of post-operative knee rehabilitation training to the present; historical training effect data includes deviations between the joint movement trajectory characteristics, muscle exertion correlation characteristics, and body balance characteristics from the standard characteristics of each stage during previous training. Based on this information, the patient's current rehabilitation level is assessed. Then, the rehabilitation stage that best matches the assessment result is found in the musculoskeletal rehabilitation stage target database and designated as the target musculoskeletal rehabilitation stage. The standard joint movement trajectory characteristics, standard muscle exertion correlation characteristics, and standard body balance characteristics corresponding to this target stage are extracted.

[0081] Step S133: Compare the joint motion trajectory features with the standard joint motion trajectory features joint by joint and movement segment by movement segment. Calculate the spatial deviation value between the actual motion trajectory of each key joint and the standard motion trajectory in each movement segment. Statistically identify trajectory segments with spatial deviation values ​​exceeding a preset deviation threshold and mark them as joint deviation trajectory segments. Record the deviation type and duration of each joint deviation trajectory segment.

[0082] Step S1331: Extract the joint identifiers of all key joints, the corresponding training action segment identifiers, and the joint motion trajectories within each action segment from the joint motion trajectory features. The joint motion trajectory includes a coordinate sequence and trajectory smoothness parameters.

[0083] From the joint motion trajectory features, relevant information for each key joint is extracted, such as joint identifier (e.g., XJ represents the knee joint), the training action segment identifier of the joint in the straight leg raise exercise (e.g., L represents the leg raise segment), and the coordinate sequence and trajectory smoothness parameters of the joint motion trajectory within that action segment.

[0084] Step S1332: Extract the standard joint motion trajectory corresponding to each joint identifier and each action segment identifier from the standard joint motion trajectory features. The standard joint motion trajectory includes a standard coordinate sequence and a standard trajectory smoothness parameter.

[0085] In the standard joint motion trajectory features, the corresponding standard joint motion trajectory is found based on the joint identifier and the motion segment identifier. The standard joint motion trajectory includes a standard coordinate sequence (i.e., the coordinate point sequence of the joint under the standard motion) and a standard trajectory smoothness parameter.

[0086] Step S1333: For each joint identifier, select any motion segment identifier corresponding to the joint identifier, and time-align the actual joint motion trajectory within the motion segment corresponding to the motion segment identifier with the standard joint motion trajectory. On the time-aligned trajectory, select the corresponding actual coordinate points and standard coordinate points at the same time interval, and calculate the characteristic distance between each pair of corresponding coordinate points as the spatial deviation value of the time point corresponding to the time interval.

[0087] Taking a joint identifier and its corresponding motion segment identifier as an example, such as the leg-lifting segment L of the knee joint XJ, the actual motion trajectory and the standard motion trajectory of this joint in this motion segment are aligned on the time axis so that their start and end times are consistent. Then, on the aligned trajectory, at fixed time intervals, such as every equal number of video frames, corresponding coordinate points are selected from both the actual and standard motion trajectories. The straight-line distance between these actual coordinate points and the standard coordinate points is calculated; this distance is the spatial deviation value of the time point corresponding to that time interval.

[0088] Step S1334: If the joint motion trajectory is a three-dimensional trajectory, calculate the feature distance between the three-dimensional coordinate points; if it is a two-dimensional trajectory, calculate the feature distance between the two-dimensional coordinate points.

[0089] If the joint motion trajectory is three-dimensional, then the coordinate point contains coordinate values ​​in three directions: x, y, and z. The spatial straight-line distance between the actual three-dimensional coordinate point and the standard three-dimensional coordinate point is calculated as the feature distance. If it is a two-dimensional trajectory, the coordinate point only has coordinate values ​​in two directions: x and y. The planar straight-line distance between the actual two-dimensional coordinate point and the standard two-dimensional coordinate point is calculated as the feature distance.

[0090] Step S1335: Compare the spatial deviation value at each time point with the preset deviation threshold. If the spatial deviation value at all time points in any time period exceeds the preset deviation threshold, then mark the actual joint motion trajectory segment in the time period as the joint deviation trajectory segment, and record the joint identifier, action segment identifier, start time point, end time point, and average spatial deviation value in the time period corresponding to the joint deviation trajectory segment.

[0091] The preset deviation threshold is set according to the target musculoskeletal rehabilitation stage, and the deviation threshold may be different for different joints and different movement segments. The spatial deviation value calculated at each time point is compared with the preset deviation threshold. If the spatial deviation value at all time points within a certain continuous time period is greater than the preset deviation threshold, then the actual joint movement trajectory within this period is marked as a joint deviation trajectory segment. At the same time, the joint identifier (e.g., XJ), movement segment identifier (e.g., L), start time point (the sequence number of the first video frame of this time period), end time point (the sequence number of the last video frame of this time period), and the average value of the spatial deviation values ​​at all time points within this time period are recorded, i.e., the average spatial deviation value.

[0092] Step S1336: Analyze the trajectory morphology of the joint deviation trajectory segment and determine the deviation type. By collecting information on all joint deviation trajectory segments, a set of joint deviation trajectory segments is formed. Each entry in the set of joint deviation trajectory segments includes a joint identifier, a motion segment identifier, a trajectory segment time range, an average spatial deviation value, a deviation type, and a trajectory morphology description.

[0093] In this embodiment, the shape of the joint deviation trajectory segment is observed to determine whether it has become curved, offset, or discontinuous compared to the standard motion trajectory. For example, if the actual trajectory segment curves upward more than the standard trajectory segment, the deviation type is excessive curvature; if the actual trajectory segment deviates to the left from the standard trajectory segment as a whole, the deviation type is overall offset.

[0094] For each movement segment of each key joint, the processes of trajectory comparison, spatial deviation calculation, marking of deviation trajectory segments, and determination of deviation type are repeated. The information of all obtained joint deviation trajectory segments is summarized into a set. Each entry in the set contains the joint identifier, movement segment identifier, time range of the trajectory segment (start and end time points), average spatial deviation value, deviation type, and a detailed description of the trajectory morphology (e.g., slight bending, severe deviation, etc.).

[0095] Step S134: Compare the muscle exertion correlation features with the standard muscle exertion correlation features group by group, calculate the difference between the actual correlation coefficient and the standard correlation coefficient of each joint-muscle correlation group, identify correlation groups whose correlation coefficient difference exceeds the preset coefficient threshold, mark them as muscle correlation deviation groups, analyze the deviation performance of the corresponding law of joint movement and muscle contour change in each muscle correlation deviation group, and record the deviation performance type.

[0096] For each joint-muscle association group in the muscle exertion association features, such as association group G1 which includes the knee joint (XJ) and the quadriceps femoris region (GSQ), the actual correlation coefficient of this association group is obtained from the muscle exertion association features, and the corresponding standard correlation coefficient is obtained from the standard muscle exertion association features. The difference between the actual correlation coefficient and the standard correlation coefficient is calculated, and the absolute value of the difference is taken. If the absolute value exceeds a preset coefficient threshold, the association group is marked as a muscle association deviation group. Then, the deviations between the corresponding laws of joint movement and muscle contour changes in this deviation group and the standard corresponding laws are analyzed. For example, the standard law states that the amplitude of muscle contour change should increase when the joint movement speed increases, but in reality, the amplitude increase is not significant. The above deviation type is recorded as insufficient amplitude increase.

[0097] Step S135: Compare the body balance state characteristics with the standard body balance state characteristics, calculate the difference between the actual center of gravity trajectory offset parameters and the standard center of gravity trajectory offset parameters, count the difference between the actual stable area stay percentage and the standard stable area stay percentage, determine the time period when the difference value or the difference value exceeds the preset balance threshold, mark it as the balance deviation time period, and record the center of gravity offset direction, offset amplitude and stability loss degree in each balance deviation time period.

[0098] The actual center of gravity trajectory offset parameters (offset distance, offset direction, offset velocity) and the actual stable region dwell percentage are obtained from the body balance state characteristics. The corresponding standard center of gravity trajectory offset parameters and standard stable region dwell percentage are obtained from the standard body balance state characteristics. The differences between the actual offset distance and the standard offset distance, the actual offset velocity and the standard offset velocity, etc., are calculated; these are the differences between the actual center of gravity trajectory offset parameters and the standard parameters. The difference between the actual stable region dwell percentage and the standard stable region dwell percentage is also calculated. Preset balance thresholds include offset distance difference thresholds, offset velocity difference thresholds, and dwell percentage difference thresholds. If any of the above differences exceeds the corresponding preset balance threshold within a certain time period, that time period is marked as a balance deviation time period. The offset direction of the center of gravity (e.g., left, right, forward, backward), offset magnitude (actual offset distance), and degree of stability loss (determined based on the magnitude of the dwell percentage difference; the larger the difference, the higher the degree of stability loss) are recorded within this time period.

[0099] Step S136: Analyze the temporal correlation between the joint deviation trajectory segment, the muscle-related deviation group, and the balance deviation time period, determine the combination of deviation types that occur simultaneously within the same time period, mark them as synergistic deviation combinations, calculate the influence weight of each deviation type in the synergistic deviation combination on the overall rehabilitation effect, and determine the influence weight based on the importance of the standard features corresponding to each deviation type in the musculoskeletal rehabilitation stage goals.

[0100] Examine the time range of the joint deviation trajectory segment, the time period of deviation occurrence in the muscle-related deviation group, and the time period of balance deviation, and identify overlapping time periods. If the joint deviation trajectory segment, muscle-related deviation group, and balance deviation time period occur simultaneously within the same time period, then combine these three deviation types together and label them as a synergistic deviation combination. The goals of musculoskeletal rehabilitation clearly define the importance of each standard feature (joint movement trajectory, muscle force association, and body balance state). Based on these definitions, assign an influence weight to each deviation type in the synergistic deviation combination; the higher the importance, the greater the influence weight.

[0101] Step S137: Based on the deviation information of the joint deviation trajectory segment, the deviation information of the muscle-related deviation group, the deviation information of the balance deviation time period, and the influence weight of the synergistic deviation combination, construct a multi-dimensional adaptation relationship table. The multi-dimensional adaptation relationship table includes deviation identifier, deviation type, deviation time period, associated deviation type, influence weight, and preliminary adjustment direction suggestion.

[0102] Each deviation in the joint deviation trajectory segment, muscle-related deviation group, and balance deviation time period is assigned a unique deviation identifier, such as B1, B2, B3, etc. Deviation types are categorized as joint deviation, muscle-related deviation, and balance deviation. The deviation time period refers to the time range in which each deviation occurs. Related deviation types refer to other deviation types that occur simultaneously with the given deviation in the synergistic deviation combination. The influence weight is the weight value corresponding to each deviation type in the synergistic deviation combination. Preliminary adjustment direction suggestions are general adjustment recommendations based on the deviation type; for example, for joint deviations, adjusting the joint movement trajectory is recommended, and for muscle-related deviations, strengthening muscle exertion is suggested. All of the above information is compiled into a table, forming a multi-dimensional adaptation relationship table.

[0103] Step S140: Generate hierarchical and phased correction guidance information for the current training movement of the rehabilitation subject based on the multi-dimensional adaptation relationship. The hierarchical and phased correction guidance information is divided into joint adjustment layer, muscle adjustment layer and balance adjustment layer according to the correction priority. Each adjustment layer contains the specific adjustment method for the corresponding training movement stage.

[0104] Step S141: Analyze the collaborative deviation combination and the influence weight of each deviation type in the multi-dimensional adaptation relationship, sort the deviation types from largest to smallest influence weight, and take the adjustment requirement corresponding to the deviation type with the largest influence weight as the highest priority adjustment layer, and determine the subsequent priority adjustment layers in turn.

[0105] The influence weights of the synergistic deviation combinations and each deviation type in the multi-dimensional adaptation relationship table are analyzed. The influence weights of joint deviation, muscle association deviation, and balance deviation within the synergistic deviation combinations are compared and arranged in descending order. The deviation type with the largest influence weight corresponds to the most urgent adjustment need and is designated as the highest priority adjustment layer; the next highest priority adjustment layer is designated as the second highest priority adjustment layer; and the lowest priority adjustment layer is designated as the lowest priority adjustment layer. For example, if the joint deviation has the largest influence weight, then the joint adjustment layer is the highest priority adjustment layer, followed by the muscle adjustment layer, and then the balance adjustment layer.

[0106] Step S142: For the deviation type corresponding to the highest priority adjustment layer, extract the training action steps involved in the deviation type, and divide the training action steps into multiple consecutive correction stages in chronological order. Each correction stage corresponds to a sub-action process in the training action.

[0107] Taking the joint adjustment layer as the highest priority adjustment layer as an example, the training movement segments involved in the joint deviation type are identified from the multi-dimensional adaptation relationship table, such as the leg raising and lowering segments in the straight leg raise exercise. These movement segments are then subdivided according to their chronological order. The leg raising segment is divided into the initial leg raising phase (the leg just begins to lift), the continuous leg raising phase (the leg steadily lifts upward), and the leg raising to the highest point phase (the leg reaches the specified height). The lowering segment is divided into the initial lowering phase (the leg just begins to move downward), the continuous lowering phase (the leg steadily moves downward), and the complete lowering phase (the leg returns to its initial position). Each subdivided phase is a correction phase, and each correction phase corresponds to a sub-movement process within the training movement.

[0108] Step S143: Set specific adjustment parameters for each correction stage. If the highest priority adjustment layer is the joint adjustment layer, the adjustment parameters include the target motion trajectory coordinates, trajectory smoothness target value, joint motion speed range, and joint motion coordination requirements with adjacent joints for each key joint in the correction stage.

[0109] Step S1431: Extract the key joint identifiers and corresponding action segment identifiers involved in the correction stage from the collaborative deviation combination corresponding to the joint adjustment layer with the highest priority adjustment layer, and determine the range of key joints that need to be adjusted.

[0110] Within the synergistic deviation combination corresponding to the joint adjustment layer, examine which key joints are involved in the combination and the corresponding movement segment identifiers for these joints. For example, in the sustained leg-raising phase of the leg-raising movement, the synergistic deviation combination involves the knee joint (XJ) and the hip joint (GJ). Therefore, the key joints that need adjustment are the knee joint (XJ) and the hip joint (GJ).

[0111] Step S1432: Extract the standard motion trajectory coordinate sequence corresponding to each key joint to be adjusted in the correction stage from the standard joint motion trajectory features, and use the standard motion trajectory coordinate sequence as the target motion trajectory coordinate of the key joint to be adjusted. The target motion trajectory coordinate contains the standard coordinate value of each time point in the correction stage.

[0112] In the standard joint motion trajectory features, find the standard motion trajectory coordinate sequence of each key joint to be adjusted in the current correction stage. For example, the standard coordinate sequence of the knee joint XJ in the continuous leg raising stage is (x1, y1), (x2, y2), ..., (xn, yn). Use this coordinate sequence as the target motion trajectory coordinate of the knee joint XJ in this correction stage, where each coordinate value corresponds to a time point within the correction stage.

[0113] Step S1433: Based on the rehabilitation stage goals corresponding to the correction stage, determine the trajectory smoothness target value for each key joint to be adjusted. The trajectory smoothness target value is obtained by multiplying the smoothness parameter of the standard trajectory by a correction coefficient. The correction coefficient is set based on the current rehabilitation level of the rehabilitation subject. The correspondence between the rehabilitation level and the correction coefficient is preset. The higher the rehabilitation level, the closer the corresponding correction coefficient is to 1.

[0114] The rehabilitation goals corresponding to the correction stage are as follows: For example, in the early stage, the smoothness parameter of the standard trajectory is the trajectory smoothness parameter of that joint in the corresponding correction stage, as defined in the standard joint motion trajectory characteristics. Based on the assessment results of the patient's current rehabilitation level, the corresponding correction coefficient is found from the pre-set correspondence between rehabilitation level and correction coefficient. The higher the rehabilitation level, the closer the correction coefficient is to 1. Multiplying the smoothness parameter of the standard trajectory by this correction coefficient yields the target trajectory smoothness value of the key joint to be adjusted in the current correction stage.

[0115] Step S1434: Analyze the distance difference between adjacent coordinate points within the correction stage of the standard motion trajectory, and calculate the standard motion speed range, which is the minimum and maximum value of the standard speed.

[0116] In the coordinate sequence of the standard motion trajectory, the distance between two adjacent coordinate points is calculated to obtain the distance difference. Since the time interval between adjacent coordinate points is known (determined based on the frame interval), the standard velocity is the distance difference divided by the time interval. The minimum and maximum values ​​of all standard velocities within the correction phase are calculated; these two values ​​constitute the standard motion velocity range.

[0117] Step S1435: Adjust the standard exercise speed range according to the rehabilitation subject's exercise ability level. If the rehabilitation subject's exercise ability level is limited, expand the upper or lower limit of the standard exercise speed range according to the preset expansion rules. The expanded speed range shall not exceed the preset safe exercise speed range.

[0118] The rehabilitation patient's motor ability level is determined by the rehabilitation therapist based on their daily training performance and physical condition, and is divided into normal level, slightly limited level, and limited level. If the assessment is limited level, the preset expansion rule may be to lower the lower limit of the standard exercise speed range by a certain percentage to accommodate the patient's insufficient motor ability. However, the upper and lower limits of the expanded speed range must not exceed the preset safe exercise speed range. This safe exercise speed range is set according to human exercise safety standards to ensure that the body will not be injured due to excessive speed or slow speed.

[0119] Step S1436: Determine the adjacent joints of each key joint to be adjusted. The adjacent joints are predefined according to the human joint kinematic chain model. The adjacent joints of the elbow joint are defined as the shoulder joint and wrist joint, and the adjacent joints of the knee joint are defined as the hip joint and ankle joint.

[0120] The human joint kinetic chain model defines the connections between various joints, and based on this model, the adjacent joints of each key joint can be determined. For example, for the knee joint XJ, its adjacent joints are the hip joint GJ (above) and the ankle joint HGJ (below); for the elbow joint ZJ, its adjacent joints are the shoulder joint SJ (above) and the wrist joint WJ (below).

[0121] Step S1437: Extract the standard motion trajectories of the key joint to be adjusted and adjacent joints in the correction stage from the standard joint motion trajectory features, and analyze the synchronicity of their motion time and the correlation of their motion amplitude. The synchronicity of motion time includes the time difference between the start and end of the motion of adjacent joints, and the correlation of motion amplitude includes the proportional relationship between the motion amplitude of the joint to be adjusted and the motion amplitude of adjacent joints.

[0122] Extract the standard motion trajectory of the key joint to be adjusted and its adjacent joints during the current correction phase. Analyze the synchronicity of motion time: examine the difference between the start time of the joint to be adjusted and the start time of adjacent joints, as well as the difference between the end time of the motion. Analyze the correlation of motion amplitude: calculate the ratio of the motion amplitude of the joint to be adjusted (e.g., the height of the leg lift) to the motion amplitude of adjacent joints (e.g., the rotation angle of the hip joint) during the correction phase.

[0123] Step S1438: Based on the synchronization of movement time and the correlation of movement amplitude, formulate the requirements for coordinated movement of joint movement and adjacent joints. The requirements for coordinated movement include: the time difference between the start of movement of the joint to be adjusted and the adjacent joint is controlled within a preset time difference range; the change in the movement amplitude of the joint to be adjusted and the change in the movement amplitude of the adjacent joint maintain a preset proportional relationship; and the trend of the change in the movement speed of the joint to be adjusted is consistent with the trend of the change in the movement speed of the adjacent joint.

[0124] The preset time difference range is set based on the results of motion time synchronization analysis. For example, the starting time difference between the joint to be adjusted and adjacent joints should be controlled within a small time range to ensure coordinated movement. The preset proportional relationship is determined based on the results of motion amplitude correlation analysis, ensuring that the change in motion amplitude of the joint to be adjusted maintains a fixed proportion with the change in motion amplitude of adjacent joints. A consistent trend in motion speed means that when the joint to be adjusted accelerates, adjacent joints should also accelerate, and when the joint to be adjusted decelerates, adjacent joints should also decelerate, to ensure the continuity and coordination of joint movement.

[0125] Step S1439: For each key joint to be adjusted, integrate the target motion trajectory coordinates, trajectory smoothness target value, joint motion speed range and coordinated motion requirements to form a subset of joint adjustment parameters for the key joint to be adjusted in the correction stage.

[0126] The target motion trajectory coordinates (standard coordinate sequence), trajectory smoothness target value (standard trajectory smoothness parameter multiplied by correction coefficient), adjusted joint motion speed range (range adjusted according to the level of motor ability), and coordination requirements with adjacent joints (time synchronization, amplitude correlation, and consistent speed trend) of each key joint to be adjusted in the current correction stage are integrated to form a subset of joint adjustment parameters for that key joint in this correction stage.

[0127] Step S14310: Collect all subsets of joint adjustment parameters for key joints to be adjusted during the correction phase, classify and sort them according to joint identifiers, and label the correction phase identifier and action segment identifier corresponding to each subset of joint adjustment parameters to form complete joint adjustment parameters for the correction phase. The complete joint adjustment parameters also include the applicable conditions of the parameters and the tolerance range of parameter adjustment. The applicable conditions of the parameters include the rehabilitation subject's motor ability level.

[0128] Collect all subsets of joint adjustment parameters for key joints requiring adjustment within the current correction phase, and categorize and sort them according to the alphabetical or numerical order of their joint identifiers. Label each subset of joint adjustment parameters with its corresponding correction phase identifier (e.g., continuous leg raise phase) and movement segment identifier (e.g., leg raise segment). The complete set of joint adjustment parameters should also specify the applicable conditions of the parameters, i.e., whether the parameters are suitable for rehabilitation subjects with limited mobility; and the tolerance range for parameter adjustment, such as allowing fluctuations in the target movement trajectory coordinates within a certain range, and allowing minor deviations in joint movement speed from the adjusted speed range.

[0129] Step S144: For subsequent priority adjustment layers, repeat the operation of dividing the correction stage and setting the adjustment parameters to make the correction stage of the subsequent priority adjustment layer completely aligned with the correction stage of the highest priority adjustment layer in the time dimension, and at the same time ensure that the adjustment parameters of the subsequent priority adjustment layer do not conflict with the adjustment parameters of the highest priority adjustment layer.

[0130] For the second-highest priority adjustment layers (such as the muscle adjustment layer) and the lowest priority adjustment layers (such as the balance adjustment layer), the correction phases are divided and adjustment parameters are set according to the methods in steps S142 and S143. When dividing the correction phases, ensure that the time range of these correction phases is exactly the same as that of the highest priority adjustment layer. For example, if the continuous leg-raising phase of the highest priority adjustment layer is from frame 10 to frame 20, then the correction phase of the subsequent adjustment layers should also be from frame 10 to frame 20. When setting the adjustment parameters, check whether these parameters conflict with the adjustment parameters of the highest priority adjustment layer. For example, the muscle adjustment layer requires rapid muscle exertion, while the joint adjustment layer requires slow joint movement. Such conflicts need to be avoided to ensure that the adjustment parameters of each adjustment layer cooperate with each other to jointly promote the correction of rehabilitation movements.

[0131] Step S145: Generate textual and visual guidance information for each correction stage of each adjustment layer. The textual guidance information uses natural language to describe the specific operation of the adjustment action, and the visual guidance information includes a schematic diagram of the target action outline, deviation position markers, and adjustment direction arrows.

[0132] For each adjustment layer and each correction stage, both textual and visual guidance information is generated. The textual guidance uses simple language to describe how the patient needs to adjust their movements, such as, "During the continuous leg raise stage, please ensure that the knee joint's movement trajectory matches the target trajectory displayed on the screen. Do not move too quickly, and coordinate with hip joint movements." The visual guidance, generated using computer graphics technology, displays a schematic diagram of the target movement, marks the deviation between the current movement and the target movement on the patient's real-time image, and indicates the direction of adjustment (e.g., up, down, left, right) with arrows.

[0133] Step S146: Arrange the correction stages of each adjustment layer in chronological order, integrate the adjustment parameters, textual guidance information and visual guidance information corresponding to each correction stage, mark the start and end time of each correction stage, and form the layered and phased correction guidance information. The layered and phased correction guidance information also includes the priority identifier of each adjustment layer and the transition and connection suggestions between correction stages.

[0134] All correction stages of the joint adjustment layer, muscle adjustment layer, and balance adjustment layer are arranged chronologically. For each correction stage, the adjustment parameters, textual instructions, and visual guidance information for each adjustment layer within that stage are integrated. The start and end times (corresponding video frame numbers) of each correction stage are marked. In the layered and phased correction guidance information, priority indicators are marked for each adjustment layer, such as "Joint Adjustment Layer (Priority 1)" and "Muscle Adjustment Layer (Priority 2)". Simultaneously, transition suggestions between correction stages are provided, such as gradually reducing the leg-raising speed and maintaining body balance when transitioning from the continuous leg-raising stage to the stage of raising the leg to the highest point.

[0135] Step S150: Obtain video data of the new posture of the rehabilitation subject after adjustment according to the hierarchical and phased correction guidance information, repeatedly perform parsing and multi-dimensional adaptation operations on the new posture video data to generate a new multi-dimensional adaptation relationship, and update the adjustment level priority and specific adjustment method of the hierarchical and phased correction guidance information according to the difference in deviation between the new multi-dimensional adaptation relationship and the original multi-dimensional adaptation relationship.

[0136] For example, in step S151: continuously collect video data of the rehabilitation subject performing training actions after receiving the layered and phased correction guidance information through a visual acquisition device, and use the video data as new posture video data. By setting the acquisition parameters of the visual acquisition device, the acquisition angle and acquisition resolution of the new posture video data are kept consistent with those of the original posture video data.

[0137] The rehabilitation participants practiced straight leg raises according to the tiered and phased correction guidance information. The visual acquisition devices (i.e., the two image acquisition devices mentioned earlier) continuously recorded their training process, obtaining video data of the new posture. During the acquisition process, it was ensured that the position, angle (acquisition perspective), and image resolution of the image acquisition devices were exactly the same as when the original posture video data was acquired. This ensures the comparability of the new and original posture video data and reduces errors caused by different acquisition conditions.

[0138] Step S152: Analyze the new posture video data to generate new joint motion trajectory features, new muscle force correlation features, and new body balance state features.

[0139] The new posture video data is analyzed according to the method in step S120, including adaptive frame segmentation processing, key joint position extraction, joint motion trajectory generation, muscle region recognition, muscle force correlation analysis, body center of gravity calculation, etc. Finally, new joint motion trajectory features, muscle force correlation features and body balance state features are generated, which are respectively called new joint motion trajectory features, new muscle force correlation features and new body balance state features.

[0140] Step S153: Obtain the target musculoskeletal rehabilitation stage target that is the same as the original adaptation operation, and map the new joint motion trajectory features, the new muscle force association features, and the new body balance state features to the target musculoskeletal rehabilitation stage target. Repeat the multi-dimensional adaptation operation to generate a new multi-dimensional adaptation relationship.

[0141] Using the target musculoskeletal rehabilitation stage target determined in step S132, the new joint motion trajectory characteristics, new muscle force association characteristics, and new body balance state characteristics are compared with the standard characteristics corresponding to the target. The multi-dimensional adaptation operation from step S133 to step S137 is repeated, including calculating the deviation, analyzing the combination of synergistic deviations, etc., to generate a new multi-dimensional adaptation relationship table, i.e., the new multi-dimensional adaptation relationship.

[0142] Example Implementation Section:

[0143] Step S154: Extract the original influence weights, original deviation degrees, and original collaborative deviation combinations for each deviation type from the original multi-dimensional adaptation relationship. The original deviation degree includes the total length of the joint deviation trajectory segment, the total number of muscle-related deviation groups, and the total duration of the balance deviation time period.

[0144] From the original multi-dimensional adaptation relationship table, the corresponding original influence weights are extracted according to the deviation type (joint deviation, muscle-related deviation, balance deviation). For the original deviation degree, the total length of the joint deviation trajectory segment is obtained by summing the frame difference between the end time and start time of all joint deviation trajectory segments (i.e., the number of frames the trajectory segment lasts) and multiplying it by the frame interval; the total number of muscle-related deviation groups is obtained by counting the number of entries marked as "muscle-related deviation group" in the original multi-dimensional adaptation relationship table; the total duration of the balance deviation time period is obtained by summing the frame difference between the end time and start time of all balance deviation time periods and multiplying it by the frame interval. The original collaborative deviation combination is directly extracted from the deviation type combination information recorded in the "Collaborative Deviation Combination" field of the original multi-dimensional adaptation relationship table, such as "Joint Deviation - Muscle-Related Deviation" and "Joint Deviation - Balance Deviation".

[0145] Step S155: Extract the new influence weights, new deviation degrees, and new collaborative deviation combinations for each deviation type from the new multi-dimensional adaptation relationship.

[0146] Using the same extraction method as in step S154, new influence weights for joint deviation, muscle association deviation, and balance deviation are extracted from the new multi-dimensional adaptation relationship table. The calculation method for the new deviation degree is consistent with the original deviation degree, that is, the total length of the new joint deviation trajectory segment is the sum of the number of continuous frames of all joint deviation trajectory segments in the new multi-dimensional adaptation relationship table multiplied by the frame interval; the total number of new muscle association deviation groups is the statistical number of entries in the "muscle association deviation group" in the new table; and the total duration of the new balance deviation time period is the sum of the number of continuous frames of all balance deviation time periods in the new table multiplied by the frame interval. The new collaborative deviation combination extracts the deviation type combination information recorded in the "collaborative deviation combination" field of the new table.

[0147] Step S156: Calculate the rate of change of the degree of deviation for each type of deviation, and compare the rate of change of the degree of deviation for each type of deviation with the preset optimization threshold.

[0148] For joint deviation types, the formula for calculating the rate of change in deviation severity is (total length of the new joint deviation trajectory segment - total length of the original joint deviation trajectory segment) / total length of the original joint deviation trajectory segment. For muscle-related deviation types, the rate of change in deviation severity is (total number of new muscle-related deviation groups - total number of original muscle-related deviation groups) / total number of original muscle-related deviation groups. For balance deviation types, the rate of change in deviation severity is (total duration of the new balance deviation time period - total duration of the original balance deviation time period) / total duration of the original balance deviation time period. The calculated rate of change in deviation severity for each deviation type is compared with a preset optimization threshold (this threshold is set according to the rehabilitation progress requirements of the target musculoskeletal rehabilitation stage; for example, -0.2 indicates that the deviation severity can be increased by a maximum of 20%, and -0.5 indicates that the deviation severity needs to be reduced by at least 50%) to determine the improvement status of each deviation type.

[0149] Step S157: Compare the values ​​of the original influence weight and the new influence weight, and determine whether the ranking order of the influence weights of each deviation type has changed according to the preset priority sorting rules.

[0150] The original and new influence weights for joint deviation, muscle-related deviation, and balance deviation are compared numerically one by one. The preset priority ranking rule is: the larger the influence weight value, the higher the priority; if the values ​​are equal, joint deviation takes precedence over muscle-related deviation, and muscle-related deviation takes precedence over balance deviation. By comparing the magnitude of the new influence weights and the ranking rule, it is determined whether the priority ranking of each deviation type is consistent with the original ranking (based on the original influence weight). For example, if the original ranking is joint deviation (0.6) > muscle-related deviation (0.3) > balance deviation (0.1), and the new influence weight is muscle-related deviation (0.5) > joint deviation (0.4) > balance deviation (0.1), then the ranking order changes.

[0151] Step S158: If the new impact weight value of the deviation type corresponding to the original highest priority adjustment layer is lower than its original impact weight value, and there are other deviation types with new impact weight values ​​higher than the new impact weight value of this deviation type, then update the adjustment layer priority according to the new ranking order, and set the adjustment layer corresponding to the deviation type with the highest new impact weight value as the new highest priority adjustment layer.

[0152] Assume the original highest priority adjustment layer was the joint adjustment layer (corresponding to the joint deviation type, with an original influence weight of 0.6). If the new influence weight for the joint deviation type drops to 0.4, while the new influence weight for the muscle association deviation type is 0.5, and 0.5 > 0.4, then according to the new ranking order (muscle association deviation > joint deviation > balance deviation), the muscle adjustment layer corresponding to the muscle association deviation type will be updated to the new highest priority adjustment layer, the joint adjustment layer will be downgraded to the second highest priority, and the balance adjustment layer will remain the lowest priority.

[0153] Step S159: For deviation types where the rate of change of deviation degree is greater than zero but less than the preset optimization threshold, adjust the adjustment parameters of the corresponding adjustment layer in the relevant correction stage. The adjustment operations include reducing the allowable range of target deviation of joint motion trajectory, increasing the target correlation coefficient requirement of muscle association group, and reducing the allowable range of target coordinates of body center of gravity.

[0154] For example, if the deviation rate of a joint deviation type is 0.1 (greater than zero, indicating an increase in the degree of deviation, but less than the preset optimization threshold of -0.2), then in the relevant correction stage of the joint adjustment layer (such as the continuous leg-raising stage of the leg-raising segment), the allowable range of the target deviation of the joint motion trajectory will be reduced from ±5% to ±3%; if the deviation rate of a muscle association deviation type is 0.15 (greater than zero and less than the optimization threshold), then the target correlation coefficient requirement for this association group will be increased from 0.8 times the original standard correlation coefficient to 0.9 times; if a similar situation occurs with the balance deviation type, then the allowable range of the target coordinates of the body center of gravity will be reduced from ±8 pixels to ±5 pixels.

[0155] Step S1510: For deviation types where the rate of change of deviation degree is less than or equal to zero, check whether its adjustment parameters conflict with the adjustment parameters of other adjustment layers; if a conflict is detected, re-divide the correction stages involved in the conflict adjustment layer, or adjust the value range of the conflict parameters to ensure that the adjustment operations of each adjustment layer are coordinated with each other.

[0156] If the deviation rate of a joint deviation type is -0.3 (less than zero, indicating improvement in deviation), but the adjustment parameters require the joint movement speed range to be [V1, V2], while the adjustment parameters of the muscle adjustment layer require the muscle exertion speed to correspond to the joint movement speed range to be [V3, V4], and [V1, V2] and [V3, V4] do not overlap (there is a conflict), then the correction stage should be redefined, separating the rapid joint movement stage from the slow muscle exertion stage in time, or adjusting the lower limit of the joint movement speed range to V3, while simultaneously adjusting the upper limit of the muscle exertion speed range to V2, so that the two overlap [V3, V2], ensuring synergy.

[0157] Step S1511: Compare the new collaborative deviation combination with the original collaborative deviation combination to identify the newly emerging collaborative deviation type. For the identified new collaborative deviation type, generate a cross-layer adjustment instruction in its corresponding correction stage. This cross-layer adjustment instruction is used to coordinate the adjustment operations of different adjustment layers in the correction stage to jointly address the newly emerging collaborative deviation.

[0158] The original collaborative bias combination was "joint bias - muscle association bias". A new type of collaborative bias combination, "joint bias - balance bias", appears in the new collaborative bias combination, and this combination corresponds to the initial leg-raising stage of the leg-raising process. For this new type of collaborative bias, cross-layer adjustment instructions are generated, for example: "In the initial leg-raising stage (frames 5-10), the joint adjustment layer needs to control the hip joint movement trajectory to shift inward by 2 pixels, and the balance adjustment layer needs to simultaneously adjust the body's center of gravity coordinate outward by 1 pixel to counteract the center of gravity imbalance caused by the joint shift. The adjustment ratio between the two is 2:1, and the adjustment start time difference is controlled within 1 frame."

[0159] Step S1512: Integrate the updated results of the adjustment hierarchy priority, the modification results of the adjustment parameters of each adjustment layer, and the generated cross-layer adjustment instructions to regenerate the updated hierarchical and phased correction guidance information; the updated hierarchical and phased correction guidance information modifies the correction stage division, adjustment parameter settings, and guidance content in the original guidance information based on the aforementioned comparison, detection, and identification results, and retains the correction stage adjustment method in the original guidance information where the rate of change of the corresponding deviation type of deviation is greater than or equal to the preset optimization threshold.

[0160] The new adjustment level priority determined in step S158 (e.g., the muscle adjustment layer is the highest priority), the adjustment parameters modified in steps S159 and S1510 (e.g., the reduced target deviation allowable range, the speed range after conflict adjustment), and the cross-layer adjustment instructions generated in step S1511 (e.g., the coordinated adjustment ratio of joints and balance) are integrated. For correction stages in the original guidance information where the rate of change of deviation is greater than or equal to the preset optimization threshold (e.g., the complete placement stage of the placement phase, where the rate of change of joint deviation is -0.3, meeting the optimization threshold of -0.2), their original adjustment methods are retained. In the final updated guidance information, each correction stage is arranged in the new priority order, the adjustment parameters correspond one-to-one with the cross-layer instructions, and an "updated" label is added to distinguish it from the original guidance information.

[0161] Based on the same inventive concept, please refer to Figure 2 The diagram shows a schematic block diagram of a musculoskeletal rehabilitation posture correction system 100 based on visual recognition, provided in an embodiment of this application, for performing the above-described musculoskeletal rehabilitation posture correction method based on visual recognition. The musculoskeletal rehabilitation posture correction system 100 based on visual recognition may include a communication unit 110, a machine-readable storage medium 120, and a processor 130.

[0162] In this embodiment, both the machine-readable storage medium 120 and the processor 130 are located within the vision-based musculoskeletal rehabilitation posture correction system 100 and are separately configured. However, it should be understood that the machine-readable storage medium 120 may also be independent of the vision-based musculoskeletal rehabilitation posture correction system 100 and may be accessed by the processor 130 via a bus interface. Alternatively, the machine-readable storage medium 120 may be integrated into the processor 130 and may communicate with external systems via the communication unit 110.

[0163] The processor 130 is the control center of the vision-based musculoskeletal rehabilitation posture correction system 100. It connects various parts of the system via various interfaces and lines, and performs overall monitoring of the system by running or executing software programs and / or modules stored in the machine-readable storage medium 120 and by calling data stored in the machine-readable storage medium 120. Optionally, the processor 130 may include one or more processing cores; for example, it may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, and the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. The machine-readable storage medium 120 is used to store machine-executable instructions for executing the scheme of this application, and the processor 130 is used to execute the machine-executable instructions stored in the machine-readable storage medium 120 to implement the musculoskeletal rehabilitation posture correction method based on visual recognition provided in the aforementioned method embodiments.

[0164] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A musculoskeletal rehabilitation posture correction method based on visual recognition, characterized in that, The method includes: Collect postural video data of rehabilitation subjects during musculoskeletal rehabilitation training. The postural video data includes the joint movement process, muscle contour change process, and body balance change process of rehabilitation subjects under different training movements. The posture video data is analyzed to generate joint movement trajectory features, muscle force correlation features, and body balance features of the rehabilitation subject; The joint motion trajectory features, muscle force correlation features, and body balance state features are all mapped to preset musculoskeletal rehabilitation stage goals, and a multi-dimensional adaptation relationship is established with the musculoskeletal rehabilitation stage goals respectively. Based on the multi-dimensional adaptation relationship, hierarchical and phased correction guidance information is generated for the current training movement of the rehabilitation subject. The hierarchical and phased correction guidance information is divided into joint adjustment layer, muscle adjustment layer and balance adjustment layer according to the correction priority. Each adjustment layer contains the specific adjustment method for the corresponding training movement stage. The system obtains video data of the new posture of the rehabilitation subject after adjustment according to the hierarchical and phased correction guidance information. It repeatedly performs parsing and multi-dimensional adaptation operations on the new posture video data to generate a new multi-dimensional adaptation relationship. Based on the difference in deviation between the new multi-dimensional adaptation relationship and the original multi-dimensional adaptation relationship, it updates the adjustment level priority and specific adjustment method of the hierarchical and phased correction guidance information. The process involves mapping the joint motion trajectory features, muscle exertion correlation features, and body balance state features to preset musculoskeletal rehabilitation stage goals, establishing multi-dimensional adaptation relationships between these goals and the predefined targets. This includes: Obtain a preset musculoskeletal rehabilitation stage target library, which is divided into multiple rehabilitation stages according to the rehabilitation progress. Each rehabilitation stage corresponds to a set of standard features, which includes standard joint motion trajectory features, standard muscle force correlation features, and standard body balance state features. Based on the rehabilitation diagnosis report, training duration and historical training effect data of the rehabilitation subjects, target musculoskeletal rehabilitation stage goals that match the current rehabilitation level of the rehabilitation subjects are selected from the musculoskeletal rehabilitation stage goal library, and standard joint motion trajectory features, standard muscle force correlation features and standard body balance state features corresponding to the target musculoskeletal rehabilitation stage goals are extracted. The joint motion trajectory features are compared with the standard joint motion trajectory features joint by joint and movement segment by movement segment. The spatial deviation value between the actual motion trajectory of each key joint and the standard motion trajectory in each movement segment is calculated. The trajectory segments with spatial deviation values ​​exceeding the preset deviation threshold are counted and marked as joint deviation trajectory segments. The deviation type and deviation duration of each joint deviation trajectory segment are recorded. The muscle exertion correlation features are compared with the standard muscle exertion correlation features group by group. The difference between the actual correlation coefficient and the standard correlation coefficient of each joint-muscle correlation group is calculated. The correlation groups whose correlation coefficient difference exceeds the preset coefficient threshold are identified and marked as muscle correlation deviation groups. The deviation performance of the corresponding law of joint movement and muscle contour change in each muscle correlation deviation group is analyzed and the deviation performance type is recorded. The body balance state characteristics are compared with the standard body balance state characteristics. The difference between the actual center of gravity trajectory offset parameters and the standard center of gravity trajectory offset parameters is calculated. The difference between the actual stable area stay percentage and the standard stable area stay percentage is calculated. The time period when the difference value or the difference value exceeds the preset balance threshold is determined and marked as the balance deviation time period. The center of gravity offset direction, offset amplitude and stability loss degree in each balance deviation time period are recorded. Analyze the temporal correlation between the joint deviation trajectory segment, the muscle-related deviation group, and the balance deviation time period, determine the deviation type combination that occurs simultaneously within the same time period, mark it as a synergistic deviation combination, calculate the influence weight of each deviation type in the synergistic deviation combination on the overall rehabilitation effect, and determine the influence weight based on the importance of the standard characteristics corresponding to each deviation type in the musculoskeletal rehabilitation stage goal. Based on the deviation information of joint deviation trajectory segments, the deviation information of muscle-related deviation groups, the deviation information of balance deviation time periods, and the influence weight of synergistic deviation combinations, a multi-dimensional adaptation relationship table is constructed. The multi-dimensional adaptation relationship table includes deviation identifier, deviation type, deviation time period, associated deviation type, influence weight, and preliminary adjustment direction suggestions.

2. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 1, characterized in that, The process of parsing the posture video data to generate joint movement trajectory features, muscle exertion correlation features, and body balance status features of the rehabilitation subject includes: The posture video data is subjected to adaptive frame segmentation processing, and the frame interval is dynamically adjusted according to the changes in the amplitude of the training action to obtain a set of continuous video frames with dynamic temporal density. Extract the key joint positions of the rehabilitation object from each video frame in the continuous video frame set, confirm the correspondence of the same key joint in different video frames, and record the two-dimensional or three-dimensional coordinates of each key joint in each video frame to form a key joint coordinate sequence. The coordinates of the same key joint in adjacent video frames are connected sequentially to form the initial motion trajectory of the key joint. The path smoothness of the initial motion trajectory is analyzed, the rate of change of the angle between adjacent line segments on the initial motion trajectory is calculated, and the trajectory points corresponding to abnormal angles are removed to obtain the optimized key joint motion trajectory. Collect the optimized motion trajectories of all key joints, classify and integrate them according to the body parts to which the joints belong, and label the training action segments corresponding to each optimized motion trajectory to form the joint motion trajectory features. The joint motion trajectory features include joint classification identifiers, action segment identifiers, coordinate sequences, and trajectory smoothness parameters. Identify the main muscle regions of the rehabilitation subject from each video frame in the continuous video frame set, extract the contour boundaries of each main muscle region, mark the feature points on the contour boundaries, record the coordinates of the feature points in the video frames, and form a muscle contour feature point sequence. By comparing the contour feature point sequences of the same main muscle region in adjacent video frames, the coordinate offset of the corresponding feature points is calculated. The mean and variance of the coordinate offset of all feature points are statistically analyzed. The mean is used as the amplitude of muscle contour change, and the variance is used as the stability parameter of muscle contour change, thus forming dynamic change data of muscle contour. The correlation analysis is performed on the movement distance and speed of each key joint in the joint movement trajectory features in the corresponding time period, and the change amplitude and stability parameters of the muscle contour of the associated muscle region in the same time period to determine the correlation coefficient between the joint movement parameters and the muscle contour change parameters. Based on the correlation coefficient, joint-muscle association groups are divided. Each joint-muscle association group contains one or more key joints and corresponding associated muscle regions. The correspondence between joint movement and muscle contour changes within each joint-muscle association group is recorded to form the muscle force association feature. The muscle force association feature includes association group identifier, correlation coefficient, and description of joint-muscle correspondence. Extract the body contour of the rehabilitation object from each video frame in the continuous video frame set, and determine the projection coordinates of the body center of gravity in the video frame by the geometric center calculation method of the body contour to form a body center of gravity coordinate sequence. The trajectory of the body's center of gravity coordinate sequence in consecutive video frames is analyzed, and the offset distance, offset direction, and offset speed of the center of gravity trajectory are calculated. The percentage of time the center of gravity spends within a preset stable region is statistically analyzed. The center of gravity trajectory parameters and the percentage of time spent in the stable region are integrated to form the body balance state feature. The body balance state feature includes the center of gravity coordinate sequence, trajectory offset parameters, and percentage of time spent in the stable region.

3. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 2, characterized in that, The process of extracting the key joint positions of the rehabilitation subject from each video frame in the continuous video frame set, confirming the correspondence of the same key joint in different video frames, and recording the two-dimensional or three-dimensional coordinates of each key joint in each video frame to form a key joint coordinate sequence includes: The first video frame is selected from the set of continuous video frames as the reference frame. In the reference frame, the key joints of the rehabilitation object are identified by the human posture recognition model. The key joints include the shoulder joint, elbow joint, wrist joint, hip joint, knee joint, ankle joint, cervical joint, and lumbar joint. A unique joint identifier is assigned to each key joint. Local image cropping is performed on the region of each key joint in the reference frame to obtain a joint local image. The texture and shape features of the joint local image are extracted as the reference feature template of the key joint. The reference feature template corresponds one-to-one with the joint identifier. The next video frame after the reference frame is selected as the current frame. In the current frame, the candidate positions of each key joint are initially identified using the same human pose recognition model, and a temporary identifier is assigned to each candidate position. For each candidate location, perform local image cropping on the region in the current frame to obtain candidate local images. Extract the texture and shape features of the candidate local images as candidate feature templates. Compare the similarity of each candidate feature template with the baseline feature templates of all key joints, and calculate the feature similarity value. The candidate position corresponding to the candidate feature template with the highest similarity value that exceeds the preset similarity threshold is determined as the key joint position of the joint identifier corresponding to the reference feature template. If the similarity value between all candidate feature templates and any reference feature template does not exceed the preset similarity threshold, the position of the key joint in the current frame is predicted by the adjacent frame joint position prediction algorithm. The adjacent frame joint position prediction algorithm uses the predicted position as the position of the key joint based on the position of the key joint in the reference frame and the motion trend of the previous frame. Record the two-dimensional or three-dimensional coordinates of each key joint in the current frame; Set the current frame as the new reference frame, update the reference feature template of each key joint. The update of the reference feature template is based on the local image features of the key joint in the current frame. Repeat the above steps of initially identifying the candidate positions of each key joint in the current frame using the same human pose recognition model and assigning a temporary identifier to each candidate position until all video frames in the continuous video frame set have been processed. The coordinates of each key joint in all video frames are organized according to the joint identifier to form a coordinate sequence for each key joint. The coordinate sequence includes the video frame number, joint identifier, two-dimensional or three-dimensional coordinate value, and coordinate determination method. The coordinate sequences of all key joints are integrated to form the key joint coordinate sequence.

4. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 2, characterized in that, The process of identifying the main muscle regions of the rehabilitation subject from each video frame in the continuous video frame set, extracting the contour boundaries of each main muscle region, marking feature points on the contour boundaries, recording the coordinates of the feature points in the video frames, and forming a muscle contour feature point sequence includes: Representative video frames are selected from the continuous video frame set. The representative video frames include the starting frame of the training action, the frame with the maximum range of motion, and the ending frame of the action. The main muscle regions of the rehabilitation subject are identified in the representative video frames using a muscle region segmentation model. The main muscle regions include the biceps brachii region, triceps brachii region, quadriceps femoris region, hamstring region, latissimus dorsi region, and rectus abdominis region. A unique muscle identifier is assigned to each main muscle region. The contours of each major muscle region in representative video frames are annotated to obtain standard muscle contours. The standard muscle contours are then associated with and stored with muscle identifiers as a reference for muscle region recognition. Each video frame in the continuous video frame set is preprocessed, including grayscale processing, Gaussian filtering noise reduction processing, and contrast enhancement processing. The preprocessed video frame retains the grayscale difference between the muscle area and the surrounding skin area. The Canny edge detection algorithm is used to extract the image edges of the preprocessed video frames to obtain edge images. In the edge images, based on the standard muscle contours in representative video frames, the template matching algorithm is used to locate the region position of each major muscle region and determine the candidate contour region of each major muscle region in the current video frame. Contour extraction is performed on each candidate contour region, and a contour tracking algorithm is used to obtain the pixel sequence of the candidate contour. The contour tracking algorithm includes contour tracking based on 8 neighborhoods, removing short contour segments with a length less than a preset length threshold from the pixel sequence, and retaining long contour segments as muscle contour boundaries. Feature point filtering is performed on the pixels on the muscle contour boundary. The filtering rules include the inflection point of the contour boundary, the endpoint of the contour boundary, and the point on the contour boundary that is more than a preset distance threshold from the adjacent feature point. The inflection point of the contour boundary is the point whose curvature value exceeds a preset curvature threshold. The filtered points are used as muscle contour feature points. Each muscle contour feature point is assigned a feature point number, which is arranged sequentially along the muscle contour boundary in a clockwise or counterclockwise direction. The two-dimensional coordinates of each feature point in the current video frame are recorded, and the two-dimensional coordinates are based on the pixel coordinate system of the video frame. The coordinates of the muscle contour feature points of each major muscle region in the current video frame are organized according to the muscle identifier to form a subset of the contour feature points of the muscle region in the current video frame. The subset of contour feature points includes the muscle identifier, feature point number and corresponding coordinates. The above-mentioned preprocessing, edge detection, contour localization, contour extraction, feature point filtering and coordinate recording operations are performed on all video frames in the continuous video frame set. The contour feature point subsets of each major muscle region are integrated in order of video frame number to form the muscle contour feature point sequence. The muscle contour feature point sequence includes video frame number, muscle identifier, feature point number and feature point coordinates.

5. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 1, characterized in that, The process involves comparing the joint motion trajectory features with the standard joint motion trajectory features joint by joint and movement segment by movement segment. The spatial deviation value between the actual motion trajectory of each key joint and the standard motion trajectory at each movement segment is calculated. Trajectory segments with spatial deviation values ​​exceeding a preset deviation threshold are identified and marked as joint deviation trajectory segments. The deviation type and duration of each joint deviation trajectory segment are recorded, including: Extract the joint identifiers of all key joints, the corresponding training action segment identifiers, and the joint motion trajectories within each action segment from the joint motion trajectory features. The joint motion trajectory includes a coordinate sequence and trajectory smoothness parameters. Extract the standard joint motion trajectory corresponding to each joint identifier and each motion segment identifier from the standard joint motion trajectory features. The standard joint motion trajectory includes a standard coordinate sequence and a standard trajectory smoothness parameter. For each joint identifier, select any motion segment identifier corresponding to the joint identifier, and time-align the actual joint motion trajectory within the motion segment corresponding to the motion segment identifier with the standard joint motion trajectory. On the time-aligned trajectory, select the corresponding actual coordinate points and standard coordinate points at the same time interval, and calculate the characteristic distance between each pair of corresponding coordinate points as the spatial deviation value of the time point corresponding to the time interval. If the joint motion trajectory is a three-dimensional trajectory, then calculate the feature distance between the three-dimensional coordinate points; If it is a two-dimensional trajectory, then calculate the feature distance between the two-dimensional coordinate points; The spatial deviation value at each time point is compared with a preset deviation threshold. If the spatial deviation value at all time points in any time period exceeds the preset deviation threshold, the actual joint movement trajectory segment in the time period is marked as a joint deviation trajectory segment. The joint identifier, action segment identifier, start time point, end time point, and average spatial deviation value in the time period corresponding to the joint deviation trajectory segment are recorded. The trajectory morphology of joint deviation trajectory segments is analyzed to determine the deviation type. By collecting information on all joint deviation trajectory segments, a set of joint deviation trajectory segments is formed. Each entry in the set of joint deviation trajectory segments includes a joint identifier, a motion segment identifier, a trajectory segment time range, an average spatial deviation value, a deviation type, and a trajectory morphology description.

6. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 1, characterized in that, The step of generating hierarchical and phased correction guidance information for the current training movements of the rehabilitation subject based on the multi-dimensional adaptation relationship includes: The collaborative deviation combination and the influence weight of each deviation type in the multi-dimensional adaptation relationship are analyzed. The deviation types are sorted from largest to smallest influence weight. The adjustment requirement corresponding to the deviation type with the largest influence weight is taken as the highest priority adjustment layer, and the subsequent priority adjustment layers are determined in turn. For the deviation type corresponding to the highest priority adjustment layer, the training action steps involved in the deviation type are extracted, and the training action steps are divided into multiple consecutive correction stages in chronological order. Each correction stage corresponds to a sub-action process in the training action. Specific adjustment parameters are set for each correction stage. If the highest priority adjustment layer is the joint adjustment layer, the adjustment parameters include the target motion trajectory coordinates, trajectory smoothness target value, joint motion speed range, and joint motion and the coordination requirements with adjacent joints for each key joint in the correction stage. If the highest priority adjustment layer is the muscle adjustment layer, the adjustment parameters include the target correlation coefficient of each muscle-related group in the correction stage, the range of muscle contour change, the target value of muscle contour change stability, and the time synchronization requirements between muscle exertion and corresponding joint movement. If the highest priority adjustment layer is the balance adjustment layer, the adjustment parameters include the target coordinate range of the body's center of gravity during the correction phase, the upper limit of the center of gravity trajectory offset, the target value of the proportion of the stable area, and the requirements for the coordination of center of gravity adjustment with joint movement and muscle exertion. For subsequent priority adjustment layers, repeat the operations of dividing the correction stage and setting adjustment parameters to ensure that the correction stage of the subsequent priority adjustment layer is fully aligned with the correction stage of the highest priority adjustment layer in the time dimension, and at the same time ensure that the adjustment parameters of the subsequent priority adjustment layer do not conflict with the adjustment parameters of the highest priority adjustment layer. For each adjustment layer and each correction stage, textual and visual guidance information is generated. The textual guidance information uses natural language to describe the specific operation of the adjustment action, while the visual guidance information includes a schematic diagram of the target action outline, deviation position markers, and adjustment direction arrows. The correction stages of each adjustment layer are arranged in chronological order. The adjustment parameters, textual instructions, and visual guidance information corresponding to each correction stage are integrated. The start and end times of each correction stage are marked to form the hierarchical and phased correction guidance information. The hierarchical and phased correction guidance information also includes the priority identifier of each adjustment layer and suggestions for transition between correction stages.

7. The musculoskeletal rehabilitation posture correction method based on visual recognition according to claim 6, characterized in that, The specific adjustment parameters are set for each correction stage. If the highest priority adjustment layer is the joint adjustment layer, the adjustment parameters include the target motion trajectory coordinates, target trajectory smoothness value, joint motion speed range, and coordination requirements between joint motion and adjacent joints for each key joint within the correction stage. From the synergistic deviation combination corresponding to the joint adjustment layer with the highest priority adjustment layer, extract the key joint identifiers and corresponding action segment identifiers involved in the correction stage to determine the range of key joints that need to be adjusted. From the standard joint motion trajectory features, extract the standard motion trajectory coordinate sequence corresponding to each key joint to be adjusted in the correction stage, and use the standard motion trajectory coordinate sequence as the target motion trajectory coordinate of the key joint to be adjusted. The target motion trajectory coordinate contains the standard coordinate value of each time point in the correction stage. Based on the rehabilitation stage goals corresponding to the correction stage, the target value of trajectory smoothness for each key joint to be adjusted is determined. The target value of trajectory smoothness is obtained by multiplying the smoothness parameter of the standard trajectory by a correction coefficient. The correction coefficient is set based on the current rehabilitation level of the rehabilitation subject. The correspondence between the rehabilitation level and the correction coefficient is preset. The higher the rehabilitation level, the closer the corresponding correction coefficient is to 1. Analyze the distance difference between adjacent coordinate points within the correction phase of the standard motion trajectory, and calculate the standard motion speed range, which is the minimum and maximum value of the standard speed. The standard exercise speed range is adjusted according to the rehabilitation subject's exercise ability level. If the rehabilitation subject's exercise ability level is limited, the upper or lower limit of the standard exercise speed range is expanded according to the preset expansion rules. The expanded speed range shall not exceed the preset safe exercise speed range. The adjacent joints of each key joint to be adjusted are determined. The adjacent joints are predefined according to the human joint kinematic chain model. The adjacent joints of the elbow joint are defined as the shoulder joint and wrist joint, and the adjacent joints of the knee joint are defined as the hip joint and ankle joint. Extract the standard motion trajectories of the key joint to be adjusted and adjacent joints during the correction phase from the standard joint motion trajectory characteristics, and analyze the synchronicity of their motion time and the correlation of their motion amplitude. The synchronicity of motion time includes the time difference between the start and end of the motion of adjacent joints, and the correlation of motion amplitude includes the proportional relationship between the motion amplitude of the joint to be adjusted and the motion amplitude of adjacent joints. Based on the synchronization of movement time and the correlation of movement amplitude, requirements for the coordinated movement of joints and adjacent joints are formulated. These requirements include: The time difference between the start of movement of the joint to be adjusted and the adjacent joint is controlled within a preset time difference range; the change in the amplitude of movement of the joint to be adjusted and the change in the amplitude of movement of the adjacent joint maintain a preset proportional relationship; and the trend of the change in the speed of movement of the joint to be adjusted is consistent with the trend of the change in the speed of movement of the adjacent joint. For each critical joint to be adjusted, the target motion trajectory coordinates, trajectory smoothness target value, joint motion speed range and coordinated motion requirements are integrated to form a subset of joint adjustment parameters for the critical joint to be adjusted in the correction stage. Collect a subset of joint adjustment parameters for all key joints to be adjusted during the correction phase, classify and sort them according to joint identifiers, and label the correction phase identifier and movement segment identifier corresponding to each subset of joint adjustment parameters to form complete joint adjustment parameters for the correction phase. The complete joint adjustment parameters also include the applicable conditions of the parameters and the tolerance range of parameter adjustment. The applicable conditions of the parameters include the rehabilitation subject's motor ability level.

8. A musculoskeletal rehabilitation posture correction system based on visual recognition, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the visual recognition-based musculoskeletal rehabilitation posture correction method according to any one of claims 1 to 7 by executing the machine-executable instructions.

9. A computer program product, characterized in that, The computer program product includes machine-executable instructions stored in a computer-readable storage medium. A processor of a computer device reads the machine-executable instructions from the computer-readable storage medium and executes the machine-executable instructions, causing the computer device to perform the musculoskeletal rehabilitation posture correction method based on visual recognition as described in any one of claims 1 to 7.

Images