Rehabilitation training detection method and system based on artificial intelligence

Abstract

The invention relates to the technical field of rehabilitation training detection, in particular to a rehabilitation training detection method and system based on artificial intelligence, a standard action library is constructed through standard action videos shot at multiple angles, and track and angle features of key joints are extracted for user training comparison; the skeleton key points of the user are extracted in real time through a MoveNet network, and efficient posture recognition in a home scene is achieved; analyzing position difference, angle change and acceleration characteristics by combining a space-time sequence matching algorithm, generating a dynamic matching degree index, and positioning a deviation joint to generate a correction prompt; introducing an attention mechanism model, learning the contribution degree of each joint to cycle recognition, dynamically selecting a dominant joint for action counting, recognizing starting and ending points of an action cycle through an acceleration curve, and finishing effective action statistics in combination with a dynamic threshold value, so that the counting accuracy and the self-adaptive capability are improved; therefore, the training cost is reduced, the evaluation credibility is enhanced, and accurate statistics and analysis of rehabilitation training data are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of rehabilitation training detection, in particular to a rehabilitation training detection method and system based on artificial intelligence. BACKGROUND

[0002] Home training programs for common rehabilitation needs such as bone and joint injury, postoperative recovery, and nerve injury are gradually popularizing. In this process, the current widely used rehabilitation training evaluation methods mainly include manual video training tracking and training counting. Among them, manual video guidance relies on remote monitoring by rehabilitation therapists, which is not only costly and inefficient; while wearable sensor devices such as electromyography devices can provide relatively accurate motion trajectory data, but they have high requirements for user wearing habits and body adaptability, and the device cost and maintenance complexity are high, making it difficult to be widely promoted.

[0003] The prior art mainly uses image recognition to assist in training tracking, and mostly uses a fixed threshold strategy for action counting to realize quantitative evaluation of rehabilitation training, but it ignores the dynamic relationship between action rhythm, amplitude and individual differences, which is easy to misjudge or miss, affecting the authenticity of training effect, the credibility of evaluation, and the accuracy of rehabilitation training data. SUMMARY

[0004] To solve the above problems, the present application provides a rehabilitation training detection method and system based on artificial intelligence.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is:

[0006] A rehabilitation training detection method based on artificial intelligence, comprising the following steps:

[0007] S1, based on a plurality of angle recorded rehabilitation action videos, extracting the skeletal key point time sequence features and joint angle change features of each complete action period, and constructing a standard action library;

[0008] S2, based on the real-time video images collected by the camera in the user training process, using the MoveNet network to extract the coordinates and confidence values of the skeletal key points in each frame of image, and generating a user skeletal key point time sequence;

[0009] S3, spatial and time sequence matching the user skeletal key point time sequence with the standard action library, calculating the skeletal key point position difference, joint angle change rate difference and trajectory acceleration difference, generating a dynamic matching degree index, and generating a correction prompt;

[0010] S4, based on the user skeleton key point time sequence, extracting the spatial trajectory of each joint in the continuous frame and inputting the attention mechanism model to analyze the contribution of each joint to the action cycle recognition, selecting the dominant counting joint and extracting its acceleration curve, dividing the action cycle stage based on the peak value and turning feature of the acceleration change, and combining the amplitude and length threshold to count the number of effective rehabilitation training actions.

[0011] Further, the S1 comprises the following steps:

[0012] S11, based on the rehabilitation action video recorded by several angles, performing image framing processing on each video, extracting the human body skeleton region in each frame image, and performing skeleton point recognition on each frame image to obtain a skeleton key point coordinate sequence;

[0013] S12, according to the skeleton key point coordinate sequence, calculating the joint angle change between the continuous frames in time sequence to obtain the joint angle change time sequence corresponding to each video;

[0014] S13, based on the skeleton key point coordinate sequence and the joint angle change time sequence, constructing the skeleton motion trajectory and angle change template of the standard action, and storing it in the standard action library according to the action category.

[0015] Further, the skeleton key point coordinate sequence obtained by performing skeleton point recognition on each frame image comprises:

[0016] Based on each frame image after image framing processing, input into the MoveNet skeleton recognition network, and extract the initial two-dimensional coordinates and corresponding confidence of 17 skeleton key points in each frame image;

[0017] Based on the initial two-dimensional coordinates and confidence data, combining the maximum response position of the heat map and the offset correction parameter, performing sub-pixel level coordinate regression and confidence screening on each skeleton key point to obtain a high-confidence skeleton key point set;

[0018] Based on the high-confidence skeleton key point set, the coordinate data of each frame is spliced in time axis according to the frame order to generate a skeleton key point coordinate time sequence.

[0019] Further, the S12 comprises the following steps:

[0020] Based on the skeleton key point coordinate time sequence, selecting joint associated point pairs, calculating the included angle between the vectors formed by each pair of key points in adjacent frames to obtain the instantaneous joint angle sequence between the continuous frames;

[0021] Based on the instantaneous joint angle sequence, the sliding window filtering method is used to smooth the angle curve, remove local jitter and abnormal fluctuations, and obtain the angle change trajectory in the continuous action process;

[0022] According to the angle change trajectory in the continuous action process, the joint angle change feature vector in each complete action period is extracted.

[0023] Further, the S3 comprises the following steps:

[0024] Based on the user skeleton key point time sequence and the target action template in the standard action library, frame-level time alignment processing is performed to construct the key point correspondence relationship between the user action and the standard action in the same behavior period.

[0025] Based on the aligned skeleton key point sequence, the Euclidean position difference of the corresponding key points, the angle change rate difference of the joints, and the acceleration difference of the key point trajectory are calculated frame by frame to obtain the error feature matrix in the space and time dimensions.

[0026] Based on the error feature matrix, the dynamic matching index is generated by weighted fusion of the spatial difference, the angle rate difference, and the trajectory acceleration difference, and the correction prompt is generated.

[0027] Further, the generation of the correction prompt comprises:

[0028] Based on the error feature matrix in the dynamic matching index, the joint position with an error amplitude exceeding a preset threshold is located and visually labeled at the coordinate position in the image frame, which is used to prompt the body part with posture deviation.

[0029] Further, the S4 comprises the following steps:

[0030] S41, based on the user skeleton key point time sequence, the spatial position trajectory of each joint in the continuous frame is extracted, which is input into the pre-constructed attention mechanism model to calculate the feature response degree of each joint to the action period change, and the weight distribution of the key joint is obtained.

[0031] S42, based on the weight distribution of the key joint, a group of dominant joints with the highest weight is selected, the spatial trajectory data thereof is extracted and the first and second derivatives thereof are calculated to obtain the velocity curve and the acceleration curve respectively.

[0032] S43, according to the velocity curve and the acceleration curve, the corresponding acceleration segment, peak segment and deceleration segment of each dominant joint in the training process are determined, and the start and end boundaries of the action period are extracted through the acceleration extreme point and the zero-crossing point.

[0033] S44, according to the start and end boundaries of the action period, the effective rehabilitation training action frequency is calculated in combination with the preset amplitude threshold and time length threshold.

[0034] Further, the attention mechanism model is constructed by the following steps:

[0035] Based on the rehabilitation action video recorded by the rehabilitation therapist from multiple angles, the spatial trajectory and acceleration characteristics of each key point in the complete action cycle are extracted and labeled, and the key joint behavior feature sequence is constructed;

[0036] The key joint behavior feature sequence and the corresponding labeled action cycle label are input into the multi-head attention neural network, and the response weight of different key joints to the action rhythm boundary is learned through training;

[0037] In the training process, the cross-entropy loss function is used to optimize the attention weight distribution, so as to make the model identify the dominant counting joints sensitive to the cycle change;

[0038] After training, the model parameters are solidified to complete the construction of the attention mechanism model.

[0039] Further, the cross-entropy loss function is as follows:

[0040]

[0041] Wherein, L is the cross-entropy loss value; N is the number of training samples; K is the total number of key joints; y ij is the true label of the i-th sample in the j-th class; is the predicted probability value of the i-th sample in the j-th class by the attention mechanism network.

[0042] A rehabilitation training detection system based on artificial intelligence is applied to the rehabilitation training detection method based on artificial intelligence, comprising:

[0043] The standard library module is used to extract the key point time sequence features and joint angle change features of each complete action cycle based on the rehabilitation action video recorded by the rehabilitation therapist from multiple angles, and construct a standard action library;

[0044] The real-time training data module is based on the real-time video images collected by the camera during the user training process, and uses the MoveNet network to extract the coordinates and confidence values of the key points in each frame of image, and generates the user key point time sequence;

[0045] The training correction module matches the user key point time sequence with the standard action library in space and time, calculates the position difference of the key points, the change rate difference of the joint angle, and the trajectory acceleration difference, generates a dynamic matching degree index, and generates a correction prompt in the user video screen;

[0046] A frequency counting module extracts spatial trajectories of each joint in consecutive frames based on a user skeleton key point time sequence, inputs an attention mechanism model to analyze contribution degrees of each joint to action cycle recognition, selects a dominant counting joint and extracts an acceleration curve thereof, divides action cycle stages based on peak values and turning features of acceleration changes, and combines amplitude and length thresholds to count effective rehabilitation training action frequencies.

[0047] The present application has the advantages that the present application constructs a standard action library by standard action videos recorded by a rehabilitation therapist from multiple angles, extracts time sequence trajectories and angle change features of key joints, and provides a comparison reference for user training; a lightweight MoveNet skeleton recognition network is used to extract skeleton key point sequences in user actions in real time, to realize efficient and unobtrusive posture tracking in a home scenario; a space-time matching algorithm is used to align multi-dimensional features of user actions and standard templates, to comprehensively analyze position differences, angle rates and trajectory accelerations, to generate a dynamic matching degree index, to locate deviating joints, to generate correction prompts in user videos in real time, and to improve training accuracy. An attention mechanism model is introduced to learn response degrees of period features of different joints, to automatically identify key joints that play a leading role in period division, and to avoid the limitations of relying on a single angle or fixed joints for action counting. Acceleration, peak and deceleration stages of actions are identified through acceleration change curves of dominant joints, period boundary recognition and effective action counting are performed in combination with a dynamic threshold strategy, individual action rhythm and amplitude differences are adaptively recognized, the accuracy and robustness of rehabilitation training action counting are greatly improved, the costs of training tracking and training counting are effectively reduced, and the authenticity of training effects and the reliability of evaluation are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0048] Fig. 1 is a step flowchart of a rehabilitation training detection method based on artificial intelligence in the present application.

[0049] Fig. 2 is a step flowchart of step S4 in the present application. DETAILED DESCRIPTION

[0050] Referring to Figs. 1-2 The present application relates to a rehabilitation training detection method based on artificial intelligence, and comprises the following steps:

[0051] S1, based on rehabilitation action videos recorded from several angles, extracting skeleton key point time sequence features and joint angle change features of each complete action cycle, constructing a standard action library;

[0052] S2, based on the real-time video images collected by the camera during the user training process, the MoveNet network is used to extract the coordinates and confidence values of the skeletal key points in each frame of image, and a time sequence of user skeletal key points is generated;

[0053] S3, the time sequence of user skeletal key points is matched with the standard action library in space and time sequence, the position difference of skeletal key points, the change rate difference of joint included angle and the trajectory acceleration difference are calculated, the dynamic matching degree index is generated, and the correction prompt is generated;

[0054] S4, based on the time sequence of user skeletal key points, the spatial trajectories of each joint in continuous frames are extracted and input into an attention mechanism model to analyze the contribution of each joint to the action cycle recognition, the dominant counting joint is selected and its acceleration curve is extracted, the action cycle stage is divided based on the peak value and turning feature of acceleration change, and the effective rehabilitation training action times are counted combined with the amplitude and time length threshold.

[0055] In some embodiments, first, the multi-angle camera device is used by a rehabilitation therapist to take a full cycle of standard rehabilitation movements, and the video processing module is used to perform frame decomposition and posture extraction on each movement video, and the coordinates of 17 standard key points are extracted based on a skeleton recognition algorithm. To ensure the wide adaptability of the standard movements, the extracted features include not only the spatial displacement sequence of the key points between frames, but also the change function of the angle between adjacent joints, and further, the timing vector and angle change feature template of the standard movements are constructed and stored in the standard movement library as the basis for subsequent comparison. This step ensures that the standard movements have a dynamic and structured skeleton expression method, rather than traditional static frame matching, improving the expression ability of the subsequent matching stage for timing integrity and rhythm features. During the user's rehabilitation training process, the system calls the front camera to collect video streams in real time, and uses an improved lightweight MoveNet neural network model to estimate the human posture of each frame of image. Through a multi-channel output structure, the model outputs the two-dimensional coordinates and corresponding confidence scores of 17 skeletal key points for each frame. To reduce noise and misidentification risk, the system filters the key point data through a confidence threshold and dynamically corrects the key point data through a sub-pixel level coordinate regression mechanism, and generates the user's skeletal key point time sequence in chronological order. Unlike the existing technology which only judges the posture based on spatial static deviation, the embodiment adopts a space-time dual-domain matching mechanism to dynamically align the user's skeletal key point time sequence with the template in the standard movement library, and calculates the corresponding Euclidean position difference, joint angle change rate difference, and acceleration difference based on the second derivative of the curve at each time. On this basis, the system constructs a dynamic matching degree index that fuses error weights, which is used as a measure of the fitting degree of the user's current training movement and the target movement in the whole process. This matching degree is not only used to quantitatively evaluate the movement quality, but also drives the subsequent correction module to locate the key joints with the most significant deviation, and prompts the user to adjust the specific limb part in the form of graphical and audio prompts, achieving high-response and fine-grained movement correction. In the movement counting process, to solve the problem that the fixed threshold strategy in traditional technology does not adapt to individual differences and rhythm changes, the embodiment innovatively introduces an attention mechanism model to deeply analyze the periodic features in the user's skeletal time sequence. The system inputs the trajectory curves of all joints in the continuous frames into the pre-trained multi-head attention structure, and the model automatically learns the response degree of each joint to the action period switching feature, thereby generating the attention distribution weight of the key joints. The weight ordering result indicates which joints best represent the start and end change trend of the movement, and the system selects the dominant counting joint accordingly and extracts its speed and acceleration curves on the time axis. By extracting features from the acceleration curve, the peak points and zero-crossing points are identified, which are used to mark the acceleration, peak, and deceleration segments of the complete movement period, and combined with the adaptive amplitude and duration threshold, the effective period that meets the movement standard is selected.Finally, the system outputs the valid action count result of the user in the current training. Further, the S1 comprises the following steps.

[0056] S11, based on the rehabilitation action video recorded at several angles, performing image frame processing on each video, extracting the human body skeleton region in each frame image, and performing skeleton point recognition on each frame image to obtain a skeleton key point coordinate sequence;

[0057] S12, according to the skeleton key point coordinate sequence, calculating the joint angle change between consecutive frames in time sequence to obtain a joint angle change time sequence feature corresponding to each video;

[0058] S13, based on the skeleton key point coordinate sequence and the joint angle change time sequence feature, constructing a standard action skeleton motion trajectory and an angle change template, and storing the template into a standard action library according to the action category.

[0059] In some embodiments, first, a video of a standard rehabilitation exercise motion is received, which is recorded by a multi-angle camera device used by a rehabilitation therapist, each video covering a complete cycle of the motion. To ensure the time continuity and posture integrity of subsequent feature extraction, the system performs image frame processing on the original video stream, converting the video into a time-ordered sequence of static frames. Subsequently, a human pose estimation network (such as HRNet or an improved version of OpenPose) is used to identify the skeletal points of each frame of image, extract two-dimensional coordinate data of 17 key joints, and form an initial skeletal key point coordinate sequence. In practical applications, to improve recognition accuracy and robustness, the system performs heat map smoothing and neighborhood regression completion operations on skeletal points with confidence lower than a threshold, ensuring the availability of complete skeletal structures for each frame. Next, the system processes the skeletal key point coordinate sequence in chronological order. By defining joint-related point pairs (such as shoulder-elbow, hip-knee, etc.), the system uses the vectors formed by each pair of key points in adjacent frames as the basis to calculate the angle changes between the vectors using the cosine theorem, thereby obtaining the instantaneous joint angle sequence between consecutive frames. To improve the stability and anti-interference ability of the time sequence features, the system introduces a sliding window filtering algorithm to perform weighted smoothing on the angle sequence, removing jitter and extreme values to obtain high-quality angle change time sequence features. This process not only reflects the trend of changes in the macro rhythm of the motion, but also retains the micro control characteristics of the key joints, making it suitable for subsequent individual matching and cycle division. In the standard motion template construction stage, the system combines the spatial trajectory of the skeletal key points and the angle change time sequence features to form a high-dimensional, multi-channel standard motion feature representation. The skeletal trajectory represents the spatial path of the motion with a sequence of coordinates for each key point, while the angle change sequence reflects the rhythm and amplitude rules of the motion in the time dimension. The system uses a feature alignment mechanism to synchronize the standard motions extracted from different angle videos on a unified time axis, eliminates the deviations introduced by perspective differences through feature averaging and outlier rejection algorithms, and finally generates the skeletal motion trajectory template and angle change template corresponding to the motion category, which are stored uniformly in the standard motion library.

[0060] Further, the skeletal point recognition of each frame of image to obtain the skeletal key point coordinate sequence comprises:

[0061] Based on each frame of image processed by image frame processing, input into the MoveNet skeletal recognition network to extract the initial two-dimensional coordinates and corresponding confidence of 17 skeletal key points in each frame of image;

[0062] Based on the initial two-dimensional coordinates and confidence data, combined with the maximum response position of the heat map and the offset correction parameter, perform sub-pixel level coordinate regression and confidence screening on each skeletal key point to obtain a high-confidence skeletal key point set;

[0063] Based on the high-confidence skeletal key point set, the coordinate data of each frame is spliced in time axis in sequence to generate a skeletal key point coordinate time sequence.

[0064] In some embodiments, the input training video is first subjected to image frame processing, converting the continuous video sequence into frame-by-frame static image data. Each frame of image is then fed into a pre-trained MoveNet skeletal recognition network, which is based on a lightweight deep convolutional neural network architecture and can efficiently extract two-dimensional coordinates of 17 standard skeletal key points and their corresponding confidence scores in each frame of image. The initial extracted skeletal point positions are integer pixel level, with a certain amount of quantization error, and some key points have low confidence due to occlusion, complex posture or poor image quality. To improve the spatial resolution and reliability of the posture data, the system further performs sub-pixel level coordinate regression optimization on the initial recognition results. Specifically, based on the feature heat map output by MoveNet, the system first locates the maximum response position of each key point as a rough positioning point. Then, by combining local area gradient and offset correction parameters, the residual position offset value is optimized through back propagation, and the coordinates of each key point are corrected using sub-pixel interpolation method, so that the final key point position can be accurately to decimal pixel coordinates. This sub-pixel level regression technique significantly improves the spatial continuity of skeletal points in high-frequency dynamic motion. After the key point coordinate optimization is completed, all skeletal point confidence values are screened. A confidence threshold (such as 0.5) is set, and only the key points higher than the threshold are retained as the high-confidence skeletal key point set, and the low-confidence points are removed to reduce the false recognition rate. In practical applications, for the locally missing skeletal points, the system performs interpolation completion based on the trajectory extrapolation of historical frames and the geometric constraints of adjacent points to ensure the integrity of the skeletal topology. Finally, the high-confidence skeletal key point coordinates extracted in each frame of image are spliced in time sequence to construct the skeletal key point coordinate time sequence of the user's complete training process. This time sequence retains the spatial trajectory changes of each joint in the user's whole motion, providing high-resolution and robust time sequence input features for subsequent posture matching, rhythm analysis and cycle recognition.

[0065] Further, the S12 comprises the following steps:

[0066] Based on the skeletal key point coordinate time sequence, the joint associated point pairs are selected, the angles between the vectors formed by each pair of key points in adjacent frames are calculated, and the instantaneous joint angle sequence between consecutive frames is obtained;

[0067] Based on the instantaneous joint angle sequence, a sliding window filtering method is used to smooth the angle curve, remove local jitter and abnormal fluctuations, and obtain the angle change trajectory in the continuous motion process;

[0068] According to the angle change trajectory in the continuous action process, a joint angle change feature vector in each complete action period is extracted.

[0069] Specifically, first, based on the identified skeleton key point coordinate time sequence, a plurality of key point pairs with joint structure significance, such as shoulder-elbow, elbow-wrist, hip-knee, etc., are selected as joint related point pairs. By calculating the angle change between the limb vectors formed by each group of key point pairs in adjacent two frames of images, an instantaneous joint angle sequence between consecutive frames is formed. The angle sequence can reflect the dynamic posture change of the user's limbs at each moment in the training process, and is an important basis for subsequent analysis of the action regularity and periodic characteristics. In order to improve the stability of the angle curve and suppress the high-frequency noise introduced by image jitter, recognition error or individual detail fluctuation, after obtaining the instantaneous angle sequence, the embodiment uses a sliding window filtering method to smooth the angle data. Specifically, in the time sequence, a fixed length sliding window is used to traverse each frame, and the average or weighted average of the angle values in the window is taken as the smoothed angle of the current frame, so as to eliminate local unnatural mutations and short-term fluctuations, and make the angle curve more in line with the actual rhythm change of the limb action in the macroscopic. After completing the smoothing processing, the angle change trajectory features in the continuous action process are further extracted. This process identifies the significant angle change patterns in the action period, such as periodically appearing maximum flexion and extension, turning points, turning sections of angle change rate, etc., and extracts a group of time sequence feature vectors that can represent the action rhythm and amplitude characteristics. These feature vectors not only have strong discriminability, but also can adapt to individual differences in amplitude and frequency of different rehabilitation action types.

[0070] Further, the S3 comprises the following steps:

[0071] Based on the user skeleton key point time sequence and the target action template in the standard action library, frame-level time alignment processing is performed to build the key point correspondence relationship between the user action and the standard action in the same behavior period.

[0072] Based on the aligned skeleton key point sequence, the Euclidean position difference of the corresponding key points, the angle change rate difference of the joints, and the acceleration difference of the key point trajectories are calculated frame by frame to obtain an error feature matrix in the space and time dimensions.

[0073] Based on the error feature matrix, a dynamic matching index is generated by weighted fusion of the spatial difference, the angle rate difference and the trajectory acceleration difference, and a correction prompt is generated.

[0074] In some embodiments, first, a frame-level time alignment process is performed based on the time series of skeletal key points extracted by the user during the training process and the target action template in the standard action library. This process is based on the dynamic time warping (DTW) algorithm, which identifies the optimal alignment path between the user's action and the standard action in the time sequence dimension while considering the speed variation of the user's action execution. Through this path, a one-to-one correspondence between the key points of the user's action and the standard action at the same behavior stage is established, ensuring that the subsequent difference calculation has temporal consistency and semantic comparability. After completing the time alignment, the spatial and temporal feature differences between the corresponding key points are calculated frame by frame. Specifically, it includes three types of differences: first, the Euclidean position difference of each pair of corresponding key points in the same frame is calculated to measure the degree of posture deviation; second, based on the angle change between consecutive frames, the angle change rate difference of the corresponding joints is calculated to evaluate the dynamic control level of the user's action; finally, the trajectory acceleration curve of the key points in the time sequence is extracted, and the acceleration difference with the corresponding trajectory in the standard action is calculated to reflect the deviation of the action rhythm and intensity. These three differences together form an error feature matrix in the spatial and temporal dimensions, describing the multi-dimensional performance of the user's action deviating from the standard template. Further based on the error feature matrix, a weighted fusion operation is performed according to the preset feature weight coefficients to generate a dynamic matching index representing the matching degree of the current action. This index comprehensively reflects the spatial consistency, angle change rationality, and dynamic rhythm matching degree of the user's action, and can effectively quantify the overall action fit between the user and the standard. Based on the abnormal areas in the dynamic matching index, the key joints with errors exceeding the tolerance threshold are located, and correction prompts such as red highlights, action feedback arrows, etc. are generated in the form of graphical annotations in the user's video screen to guide the user to adjust the posture in time.

[0075] Further, the generating the correction prompt comprises:

[0076] Based on the error feature matrix in the dynamic matching index, the joint positions with error amplitudes exceeding the preset threshold are located and visually annotated at the coordinate positions in the image frames to prompt the body parts with posture deviations.

[0077] Specifically, the three types of error values of each key point are jointly judged. If a key point exceeds the corresponding preset threshold in any one or more types of errors, it is considered that the joint has a posture execution deviation. Subsequently, the system takes the coordinates of the key point in the image frame as the target position and performs visual labeling on the image. The labeling methods include but are not limited to drawing a red circle at the deviation joint, an arrow direction guide, a transparent highlight area superimposition, and the like, to intuitively prompt the user of the abnormal posture part of the current body. In addition, to enhance the user's perception and correction efficiency, the system can also automatically adjust the prompt intensity according to the absolute value or relative amplitude of the error value, for example, the greater the error, the more prominent the prompt label, and even guide the user how to adjust the angle or force direction by voice or text prompts.

[0078] Further, the S4 comprises the following steps:

[0079] S41, based on the time sequence of user skeletal key points, the spatial position trajectory of each joint in consecutive frames is extracted, the pre-constructed attention mechanism model is input, the feature response degree of each joint to the action cycle change is calculated, and the weight distribution of the key joint is obtained;

[0080] S42, based on the weight distribution of the key joint, a group of dominant joints with the highest weight is selected, the spatial trajectory data thereof is extracted and the first and second derivatives thereof are calculated, and the speed curve and acceleration curve are obtained respectively;

[0081] S43, according to the speed curve and the acceleration curve, the corresponding acceleration segment, peak segment and deceleration segment of each dominant joint in the training process are determined, and the start and end boundaries of the action cycle are extracted by the acceleration extreme point and the zero-crossing point;

[0082] S44, according to the start and end boundaries of the action cycle, the effective rehabilitation training action frequency is calculated in combination with the preset amplitude threshold and time length threshold.

[0083] In some embodiments, first, based on the user's skeletal key point time series, the three-dimensional or two-dimensional spatial position trajectories of all skeletal joints in consecutive frames are extracted, and the trajectory data is input into a pre-trained multi-head attention mechanism model. The attention mechanism model is built with the Transformer structure as the core, and in the training stage, the key point trajectory sequence with action cycle label is used for supervised learning, so as to learn the response ability of each joint in the cycle rhythm identification process. Through the attention weight distribution output by the model, the contribution degree of each joint to the action cycle change feature in the time series can be obtained. Based on the above weight distribution, the system selects a group of dominant joints with the highest weight, such as elbows, knees, ankles and other joints with high action response, and further extracts their spatial trajectory data in the training process. First-order difference and second-order difference operations are performed on these trajectories to obtain velocity curves and acceleration curves, respectively. The velocity curve is used to capture the overall trend of the action, and the acceleration curve is used to depict the rhythm details and stage transition characteristics of the action. Then, based on the extreme points (acceleration peak and deceleration valley) and zero-crossing points (turning points of acceleration from positive to negative or from negative to positive) in the acceleration curve, the system automatically divides the starting and ending boundaries of the action cycle. In order to avoid misidentification, the system introduces a filtering mechanism and a duration constraint to exclude false cycles that are too fast or too slow, thereby improving the accuracy and robustness of the cycle boundary division. Finally, according to the amplitude change range (such as the displacement amplitude of the dominant joint or the acceleration peak) and the duration of each action cycle identified, the system judges whether the cycle constitutes an effective rehabilitation action in combination with the preset amplitude threshold and time length threshold. Only when the amplitude and time length meet the conditions at the same time, the cycle is counted as the effective training number.

[0084] Further, the attention mechanism model is built by the following steps:

[0085] Based on the rehabilitation action videos recorded from multiple angles by the rehabilitation therapist, the spatial trajectory and acceleration features of each skeletal key point in the complete action cycle in each sample are extracted and labeled, and the key joint behavior feature sequence is constructed;

[0086] The key joint behavior feature sequence and the corresponding labeled action cycle label are input into the multi-head attention neural network, and the response weight of different key joints to the action rhythm boundary is learned through training;

[0087] In the training process, the cross-entropy loss function is used to optimize the attention weight distribution, so that the model can identify the dominant counting joints sensitive to the cycle change;

[0088] After the training is completed, the model parameters are fixed to complete the construction of the attention mechanism model.

[0089] In some embodiments, the construction process of the attention mechanism model mainly faces the identification of the dominant joint of the rehabilitation training action cycle to improve the accuracy and adaptability of the cycle boundary division. First, the system takes the standard action video recorded from multiple angles by a rehabilitation therapist as the training sample input. For each complete action cycle, the MoveNet skeleton recognition network is used to extract 17 skeleton key point coordinates in each frame of image to generate a skeleton key point space trajectory sequence. For each key point, the spatial first-order derivative and second-order derivative between consecutive frames are calculated, representing the velocity vector and acceleration vector of the key point in the time period, respectively, to obtain the complete skeleton motion behavior characteristics. To enhance the response sensitivity of the model to different joints in the action rhythm change, further construct the key joint behavior feature sequence: each frame contains a high-dimensional feature vector composed of the coordinates, velocity, acceleration, velocity direction angle, acceleration modulus and its change rate of each joint. For each action sample, the start and end frame indexes of the action cycle are manually labeled to form a cycle boundary label sequence. These labels are used to supervise the model training, so that the model has clear target guidance when identifying the cycle stage (start, proceed, end) corresponding to each frame. Then, the constructed key joint behavior feature sequence and action cycle boundary label are input into a neural network model with a multi-head attention structure (Multi-Head Attention). The model introduces position encoding (Positional Encoding) in the encoding layer to maintain the time sequence information, so that the attention mechanism not only pays attention to the feature value of each time, but also considers the feature change pattern over time. The multi-head attention mechanism performs weighted learning on the joint behavior features in different subspaces in parallel, calculates the response weight of each joint to all frame time points through self-attention operation, and forms a frame-joint level weight distribution tensor. The model uses cross-entropy loss function as the training target. During training, the learning rate is dynamically adjusted, dropout mechanism is added to prevent overfitting, and batch normalization is used to improve the training stability. After the model is trained, its accuracy, recall rate and F1 value in identifying the cycle boundary are evaluated on the validation set, and the model parameters with the best performance are selected for solidification deployment. The deployed attention mechanism model can be used for real-time inference: the system inputs the user's skeleton key point time sequence features into the model, outputs the probability prediction of each frame in different cycle states, combines the key frame attention weight map, extracts the most sensitive group of dominant count joints and their corresponding time segments, and provides structured support for subsequent cycle recognition and action counting based on acceleration curve.

[0090] Further, the cross-entropy loss function is as follows:

[0091]

[0092] wherein, L is the cross-entropy loss value; N is the number of training samples; K is the total number of key joints; y ij is the true label of the i th sample on the j th class; is the predicted probability value of the attention mechanism network on the i th sample on the j th class.

[0093] In some embodiments, in a specific embodiment, the cross-entropy loss function is used as the training objective function of the attention mechanism neural network, to measure the difference between the action cycle classification probability distribution output by the model and the true label. The training data is composed of video samples collected by a rehabilitation therapist from multiple angles, and the system extracts the spatial trajectory and acceleration change curve of each skeletal key point in a complete action cycle from the video samples, and constructs a supervised learning data set by manually labeling the cycle stage labels of the action (such as start, proceed, and end).

[0094] In the model training stage, each sample is composed of a group of behavior feature vectors of key joints, which is input into the multi-head attention network, and the model outputs the corresponding probability distribution for each cycle stage. The system compares the predicted distribution with the actual label, and calculates the prediction error through the cross-entropy loss function. Specifically, if the model's predicted cycle stage probability is closer to the true label, the cross-entropy value is smaller, and vice versa. The system adjusts the response weight of each joint in the attention layer accordingly. This loss function optimization process prompts the model to automatically focus on the most discriminative joint in the action rhythm change, i.e., the dominant counting joint, thereby improving the generalization ability and accuracy of the model in the action cycle recognition task. Finally, the trained model will solidify its response mode to each cycle stage, and can be used to quickly recognize the action rhythm features in the user training data and support subsequent cycle division and action count statistics.

[0095] The present application also includes a rehabilitation training detection system based on artificial intelligence, which is applied to any one of the rehabilitation training detection methods based on artificial intelligence described above, comprising:

[0096] A standard library module is used to extract the skeletal key point time sequence features and joint angle change features of each complete action cycle based on the rehabilitation action videos recorded by a rehabilitation therapist from multiple angles, and to construct a standard action library.

[0097] A real-time training data module is used to extract the coordinates and confidence values of the skeletal key points in each frame of image based on the real-time video images collected by a camera during the user training process, and to generate a user skeletal key point time sequence using the MoveNet network.

[0098] The training correction module performs spatial and temporal matching between the user's skeleton key point time sequence and the standard action library, calculates the skeleton key point position difference, joint angle change rate difference and trajectory acceleration difference, generates a dynamic matching degree index, and generates a correction prompt in the user's video picture;

[0099] The frequency statistics module extracts the spatial trajectory of each joint in the continuous frames based on the user's skeleton key point time sequence, inputs an attention mechanism model to analyze the contribution of each joint to the action cycle recognition, selects the dominant counting joint and extracts its acceleration curve, divides the action cycle stage based on the peak value and turning feature of the acceleration change, and combines the amplitude and time length threshold to count the effective rehabilitation training action frequency

[0100] The above embodiments only describe the preferred embodiments of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by ordinary engineering technicians in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A rehabilitation training detection method based on artificial intelligence, characterized in that: The following steps are involved: S1. Based on rehabilitation movement videos recorded at several angles, the timing characteristics of skeletal key points and joint angle change characteristics of each complete movement cycle are extracted to build a standard movement library; S2. Based on the real-time video images captured by the camera during user training, the MoveNet network is used to extract the coordinates and confidence values ​​of the skeleton key points in each frame of the image to generate a time series of the user's skeleton key points; S3. Perform spatial temporal matching between the user's skeletal key point time series and the standard motion library, calculate the differences in skeletal key point positions, joint angle change rate differences, and trajectory acceleration differences, generate dynamic matching indicators, and generate correction prompts; S4. Based on the time series of the user's skeletal key points, the spatial trajectory of each joint in continuous frames is extracted and input into the attention mechanism model to analyze the contribution of each joint to the motion cycle recognition. The dominant counting joints are selected and their acceleration curves are extracted. The motion cycle stages are divided based on the peak and turning characteristics of the acceleration changes, and the number of effective rehabilitation training movements is counted in combination with the amplitude and duration thresholds.

2. The artificial intelligence-based rehabilitation training detection method according to claim 1, characterized in that: Said S1 comprises the following steps: S11, based on the rehabilitation movement videos recorded at several angles, performing image frame processing on each video, extracting the human skeleton area in each frame image, and performing skeleton point recognition on each frame image to obtain a skeleton key point coordinate sequence; S12, calculating the joint angle changes between consecutive frames in chronological order based on the skeleton key point coordinate sequence, and obtaining the joint angle change time series features corresponding to each video segment; S13. Based on the coordinate sequence of the skeleton key points and the temporal characteristics of the joint angle changes, the skeleton motion trajectory and angle change template of the standard action are constructed, and stored in the standard action library according to the action category.

3. The artificial intelligence-based rehabilitation training detection method according to claim 2, characterized in that: The process of performing skeleton point recognition on each frame of image to obtain a skeleton key point coordinate sequence comprises: Each frame of the image after image segmentation is input into the MoveNet skeleton recognition network to extract the initial two-dimensional coordinates and corresponding confidence scores of the 17 skeleton key points in each frame of the image; Based on the initial two-dimensional coordinates and confidence data, combined with the maximum response position of the heat map and the offset correction parameter, sub-pixel coordinate regression and confidence screening are performed on each skeleton key point to obtain a set of high-confidence skeleton key points; Based on the high-confidence skeleton key point set, the coordinate data of each frame is spliced ​​on the time axis in the frame order to generate a skeleton key point coordinate time series.

4. The artificial intelligence-based rehabilitation training detection method according to claim 2, characterized in that: The S12 includes the following steps: Based on the time series of the coordinates of the skeleton key points, the joint-related point pairs are selected, and the angle between the vectors formed by each pair of key points in adjacent frames is calculated to obtain the instantaneous joint angle sequence between consecutive frames; Based on the instantaneous joint angle sequence, a sliding window filtering method is used to smooth the angle curve, remove local jitter and abnormal fluctuations, and obtain the angle change trajectory during the continuous movement process; According to the angle change trajectory during the continuous action process, the joint angle change feature vector within each complete action cycle is extracted.

5. The artificial intelligence-based rehabilitation training detection method according to claim 1, characterized in that: The S3 includes the following steps: Based on the time series of the user's skeleton key points and the target action template in the standard action library, frame-level time alignment processing is performed to construct the key point correspondence between the user action and the standard action in the same behavior period; Based on the aligned skeleton key point sequence, the Euclidean position difference of the corresponding key points, the angle change rate difference of the joints, and the acceleration difference of the key point trajectory are calculated frame by frame to obtain the error feature matrix in the spatial and temporal dimensions. Based on the error feature matrix, a dynamic matching index is generated by weighted fusion of spatial differences, angular velocity differences, and trajectory acceleration differences, and a correction prompt is generated.

6. The artificial intelligence-based rehabilitation training detection method according to claim 5, characterized in that: Generating a correction prompt includes: Based on the error feature matrix in the dynamic matching index, the joint positions whose error amplitude exceeds the preset threshold are located and the coordinate positions in the image frame are visually marked to indicate the body parts with posture deviations.

7. The artificial intelligence-based rehabilitation training detection method according to claim 1, characterized in that: The S4 comprises the following steps: S41. Based on the time series of the user's skeletal key points, the spatial position trajectory of each joint in consecutive frames is extracted and input into the pre-built attention mechanism model to calculate the characteristic response degree of each joint to the change of the action cycle, and obtain the weight distribution of the key joints; S42. Based on the weight distribution of the key joints, a group of dominant joints with the highest weights are selected, their spatial trajectory data are extracted, and their first-order derivatives and second-order derivatives are calculated to obtain velocity curves and acceleration curves, respectively; S43. Determine the acceleration segment, peak segment, and deceleration segment corresponding to each dominant joint during the training process based on the velocity curve and the acceleration curve, and extract the start and end boundaries of the action cycle through the acceleration extreme point and the zero crossing point; S44. Calculate the number of effective rehabilitation training movements according to the start and end boundaries of the movement cycle in combination with a preset amplitude threshold and a preset duration threshold.

8. The artificial intelligence-based rehabilitation training detection method according to claim 7, characterized in that: The attention mechanism model is constructed through the following steps: Based on rehabilitation movement videos recorded by rehabilitation therapists from multiple angles, the spatial trajectory and acceleration characteristics of each skeletal key point in each sample during the complete movement cycle are extracted and labeled to construct a key joint behavior feature sequence. The key joint behavior feature sequence and the corresponding annotated action cycle label are input into a multi-head attention neural network, and the response weights of different key joints to the action rhythm boundary are learned through training; During the training process, the cross entropy loss function is used to optimize the attention weight distribution, so that the model can identify the dominant count joints that are sensitive to periodic changes; After training is completed, the model parameters are solidified to complete the construction of the attention mechanism model.

9. The artificial intelligence-based rehabilitation training detection method according to claim 8, characterized in that: The cross entropy loss function is as follows: Among them, L is the cross entropy loss value; N is the number of training samples; K is the total number of categories of key joints; y ij is the true label of the i-th sample in the j-th category; is the predicted probability value of the attention mechanism network for the i-th sample in the j-th category.

10. An artificial intelligence-based rehabilitation training detection system, applied to the artificial intelligence-based rehabilitation training detection method according to any one of claims 1 to 9, characterized in that: include: The standard library module is used to extract the timing characteristics of skeletal key points and joint angle change characteristics of each complete movement cycle based on rehabilitation movement videos recorded by rehabilitation therapists from multiple angles, and to build a standard movement library; The real-time training data module uses the MoveNet network to extract the coordinates and confidence values ​​of the skeleton key points in each frame of the real-time video images captured by the camera during the user training process, and generates a time series of the user's skeleton key points; The training correction module matches the user's skeletal key point time series with the standard action library in spatial and temporal order, calculates the differences in skeletal key point positions, joint angle change rates, and trajectory accelerations, generates dynamic matching indicators, and generates correction prompts in the user's video. The frequency statistics module extracts the spatial trajectory of each joint in continuous frames based on the time series of the user's skeletal key points, inputs the attention mechanism model to analyze the contribution of each joint to the motion cycle recognition, selects the dominant counting joint and extracts its acceleration curve, divides the motion cycle stage based on the peak and turning characteristics of the acceleration change, and combines the amplitude and duration thresholds to count the number of effective rehabilitation training movements.

Images