Rehabilitation action counting method

Abstract

The invention provides a rehabilitation action counting method. The method comprises the following steps: collecting standard rehabilitation action data; mapping high-dimensional data and constructing a standard action library; the method comprises the following steps: carrying out space homogenization processing on a high-dimensional standard action; real-time human body posture extraction and data stream processing are carried out, human body postures are extracted in real time through a posture extraction model, three-dimensional coordinates of skeleton joint points are output, and redundant frames are eliminated through a residual calculation method; performing high-dimensional mapping and dynamic matching on the attitude data, mapping the real-time skeleton point sequence to a high-dimensional space which is the same as a standard action library through the same feature extractor, and performing space homogenization operation to obtain a real-time patient attitude sequence Y; an optimal action matching point is obtained through a real-time action matching algorithm; performing real-time action counting and quality feedback, and performing action counting and progress tracking. According to the method, through high-dimensional space mapping, a dynamic matching algorithm and a real-time feedback mechanism, precise counting of medical-level rehabilitation actions is achieved, and counting of various complex rehabilitation actions is supported.

Description

Technical Field

[0001] The present invention relates to the technical field of medical rehabilitation action counting, in particular to a rehabilitation action counting method. Background Art

[0002] With advances in medical technology and the development of rehabilitation medicine, patients are increasingly demanding more precise and standardized rehabilitation training. Common rehabilitation exercises, such as range of motion training, balance and coordination training, and muscle strengthening training, must strictly adhere to the movement standards established by the therapist to ensure effective recovery. The degree of completion and effectiveness of rehabilitation exercises are crucial to the patient's recovery. Therefore, patient rehabilitation training is crucial, and the demand for rehabilitation assistive devices is growing.

[0003] Motion counting has been widely studied in various fields. However, few studies have applied it to the field of rehabilitation. Currently, the main methods for most computer vision-related motion counting are: first extract the skeleton points of the human body (BlazePose, MoveNet, etc.), then calculate the relevant angle changes, vectors and other mathematical quantities based on the coordinates of the skeleton points, and then compare them with preset thresholds, or use some deep learning-based models to count movements. However, the former motion counting method is generally only applicable to simple movements in specific fitness exercises. For uncommon complex movements or rehabilitation-related movements, its generalization is weak and the counting is inaccurate. The latter motion counting speed is slow and cannot provide good real-time feedback.

[0004] A Chinese invention patent application, publication number CN119360270A, discloses a method for real-time human action counting optimized in the spatiotemporal domain based on skeleton points. The method comprises: generating several action dictionary entries corresponding to standard demonstration videos of different actions, each entry containing an action index, a mapping function between skeleton points and action state values, an ideal time-action state waveform, and an expected keyframe sequence with different dense sampling densities; identifying each action video in the input video stream, identifying the action type in the action video, extracting keyframes from the video based on the keyframe sequence, dynamically adjusting the expected keyframe sequence used based on the state values ​​of the extracted keyframes, and finally determining whether the action is qualified based on the extracted keyframes; counting actions until the video stream is interrupted or terminated, and outputting the counting results. However, this method has significant limitations in scalability, complex action processing, background robustness, and hardware compatibility, particularly in fields such as medical rehabilitation that require high precision and multi-scenario support. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention proposes a rehabilitation movement counting method, which realizes the accurate counting of medical-grade rehabilitation movements through high-dimensional space mapping, dynamic matching algorithm and real-time feedback mechanism, and supports the counting of various complex rehabilitation movements.

[0006] The present invention aims to provide a method for counting rehabilitation movements, comprising collecting standard rehabilitation movement data and further comprising the following steps:

[0007] Step 1: High-dimensional data mapping and standard action library construction, map the standard action sequence to the high-dimensional feature space through the feature extractor, and construct the standard action sequence X = {x1, x2, ..., x i ,…,x M},

[0008] Among them, x i is the i-th processed standardized posture in the action cycle, and M is the total number of frames in the standard action sequence;

[0009] Step 2: By performing spatial homogenization on the high-dimensional standard action, the temporally non-uniformly sampled skeleton point sequence is converted into a spatially uniformly distributed path sequence through interpolation.

[0010] Step 3: Real-time human posture extraction and data stream processing: Use the posture extraction model to extract human posture in real time, output the three-dimensional coordinates of skeletal joints, and use the residual calculation method to eliminate redundant frames;

[0011] Step 4: High-dimensional mapping and dynamic matching of posture data: The real-time skeleton point sequence is mapped to the same high-dimensional space as the standard motion library through the same feature extractor, and the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y i ,…,y N},

[0012] Among them, y i The i-th processed patient motion posture in the motion cycle, N is the total number of patient posture frames currently processed;

[0013] Step 5: Real-time action matching algorithm to obtain the optimal action matching point;

[0014] Step 6: Real-time action counting and quality feedback, action counting and progress tracking. Each time a standard ring match is completed, the count increases by 1.

[0015] Preferably, the standard rehabilitation movement data is performed by a rehabilitation physician, and multi-angle, multi-cycle video movement sequences are collected through a depth camera; each movement is collected for at least 10 cycles, covering different body shapes, speeds and ranges of joint movement.

[0016] In any of the above solutions, preferably, step 1 includes the following sub-steps:

[0017] Step 11: Calculate joint angle features;

[0018] Step 12: Calculate joint distance features;

[0019] Step 13: Encode the joint angle features and the joint distance features.

[0020] In any of the above solutions, preferably, step 11 includes calculating the three-dimensional angle θ of the key joint based on the coordinates of the skeletal joint points. joint , the formula is:

[0021]

[0022] in, is the adjacent limb vector.

[0023] In any of the above solutions, preferably, step 12 includes calculating the Euclidean distance d between the key joint points. joint , the formula is:

[0024] d joint =‖G a -G b ‖

[0025] Among them, G a and G b are the joint point coordinates.

[0026] In any of the above solutions, preferably, step 13 includes inputting the joint angle feature and the joint distance feature into an encoder to generate a 128-dimensional feature vector H t .

[0027] 11. In any of the above solutions, preferably, the feature vector generated by the feature encoder is represented by H t ={G1,G2,...,G M}, each element G t The feature code representing the human body posture at the t-th moment, the formula for the adaptive compensation s in the spatially uniform distribution is:

[0028]

[0029] If the motion distance d between adjacent frames t >s, you need to insert between these two frames The specific insertion formula for posture data is:

[0030]

[0031] Among them, G t is the high-dimensional human posture feature vector at time t, G t+1 is the high-dimensional human posture feature vector at time t+1, G new,q It is in Gt and G t+1 The qth point newly inserted between them, q = 1, 2,…, f.

[0032] In any of the above solutions, it is preferred that the three-dimensional coordinates of the skeletal joint points are G t ={g1,g2,…,g 33}, where g w {x w ,y w ,z w} represents the position of the joint point w in the camera coordinate system.

[0033] In any of the above solutions, preferably, step 4 includes inputting the patient's bone point sequence {Gt′} into the feature extractor to calculate the joint angle θ joint Distance d from joint joint , generate a 128-dimensional feature vector H t ∈R 128 Then, the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y i ,…,y N}.

[0034] In any of the above solutions, preferably, step 5 includes extracting the patient's current frame y for each matching. t and its preceding K-1 frames constitute the comet tail trajectory segment {y t-K+1 ,y t-K+2 ,…,y t}, where K represents the length of the comet tail trajectory segment, and then calculates the Euclidean distance between it and each frame of the standard action sequence segment, and combines the trajectory direction angle constraint to select the optimal matching point;

[0035] If there are multiple optimal matching points or the action is a sub-cycle type action, the nearest point suppression mechanism algorithm is used to process it and finally obtain the optimal action matching point.

[0036] In any of the above schemes, preferably, step 5 also includes forming a comet tail trajectory segment from the current frame of the patient's action sequence and its historical continuous frames, and dynamically matching the correct matching position of the current action in the standard action trajectory by calculating the similarity between the historical trajectory segment and the standard ring sequence.

[0037] In any of the above solutions, preferably, the calculation formula of the dynamic Euclidean distance between the comet tail trajectory segment and each frame of the standard action ring sequence is:

[0038]

[0039] Among them, c takes the value of 0, 1, ..., M-K+1, fragment[j] is the jth frame of the patient's real-time comet tail trajectory fragment, and Standard_fragment[c][j] is the jth frame of the i-th comet tail trajectory fragment in the standard action sequence.

[0040] In any of the above solutions, preferably, step 5 further includes a direction angle constraint, calculating the average motion direction angle of the trajectory segment, and excluding the candidate matching point if the direction angle difference between the patient segment and the standard segment exceeds a preset threshold.

[0041] In any of the above schemes, preferably, step 5 also includes calculating the distance between all sequences in the standard action sequence and the patient's current action based on the position of the previous matching point as the starting point, obtaining multiple matching points, selecting the first point that meets the matching requirements as the current matching point, and then updating the starting point as the matching point, and so on, until the current action count is completed.

[0042] In any of the above solutions, it is preferred that the completion percentage is calculated based on the current optimal matching point position tid

[0043] The present invention proposes a rehabilitation movement counting method. The core algorithm of this method is lightweight and can be run on low-computing-power devices. It is suitable for real-time counting, scoring and corrective feedback in rehabilitation training, thereby improving the standardization of patient training and rehabilitation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of a preferred embodiment of the rehabilitation movement counting method according to the present invention.

[0045] Figure 2 This is a flowchart of an embodiment of obtaining a standard motion template according to the rehabilitation motion counting method of the present invention.

[0046] Figure 3 Flowchart of another preferred embodiment of the rehabilitation movement counting method according to the present invention.

[0047] Figure 4 Schematic diagram of the structure of an embodiment of a rehabilitation device according to the rehabilitation movement counting method of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] like Figure 1As shown, a rehabilitation movement counting method executes step 100 to collect standard rehabilitation movement data. The standard rehabilitation movement data is performed by a rehabilitation physician and multi-angle, multi-cycle video movement sequences are collected through a depth camera; each movement is collected for at least 10 cycles, covering different body shapes, speeds and joint ranges of motion.

[0051] Execute step 110, high-dimensional data mapping and standard action library construction, map the standard action sequence to the high-dimensional feature space through the feature extractor, and form the standard action sequence X = {x1, x2, ..., x i ,…,x M}, where x i is the i-th processed standardized posture in the action cycle, M is the total number of frames in the standard action sequence, and step 110 includes the following sub-steps:

[0052] Execute step 111 to calculate the joint angle features, including calculating the three-dimensional angle θ of the key joint based on the coordinates of the skeletal joint points joint , the formula is:

[0053]

[0054] in, is the adjacent limb vector.

[0055] Execute step 112 to calculate the joint distance feature, including calculating the Euclidean distance d between key joint points joint , the formula is:

[0056] d joint =‖G a -G b ‖

[0057] Among them, G a and G b are the joint point coordinates.

[0058] Execute step 113 to encode the joint angle feature and the joint distance feature, including inputting the joint angle feature and the joint distance feature into an encoder to generate a 128-dimensional feature vector H t .

[0059] Execute step 120, by performing spatial homogenization processing on the high-dimensional standard action, the skeleton point sequence sampled unevenly in time is converted into a spatially uniformly distributed path sequence through interpolation processing, and the feature vector generated by the feature encoder is represented as H t ={G1,G2,...,G M}, each element G t The feature code representing the human body posture at the t-th moment, the formula for the adaptive compensation s in the spatially uniform distribution is:

[0060]

[0061] If the motion distance d between adjacent frames t >s, you need to insert between these two frames The specific insertion formula for posture data is:

[0062]

[0063] Among them, G t is the high-dimensional human posture feature vector at time t, G t+1 is the high-dimensional human posture feature vector at time t+1, G new,q It is in G t and G t+1 The qth point newly inserted between them, q = 1, 2,…, f.

[0064] Execute step 130, real-time human posture extraction and data stream processing, use the posture extraction model to extract the human posture in real time, output the three-dimensional coordinates of the skeleton joints and use the residual calculation method to eliminate redundant frames. The coordinates of the three-dimensional coordinates of the skeleton joints are G t ={g1,g2,…,g 33}, where g w {x w ,y w ,z w} represents the position of the joint point w in the camera coordinate system.

[0065] Execute step 140, high-dimensional mapping and dynamic matching of posture data, map the patient's real-time bone point sequence {Gt′} to the same high-dimensional space as the standard motion library through the same feature extractor, and calculate the joint angle θ joint Distance d from joint joint , generate a 128-dimensional feature vector H t ∈R 128 , and also perform spatial homogenization operation to obtain the real-time patient posture sequence Y={y1,y2,…,y i ,…,y N}, where y i The i-th processed patient motion posture in the motion cycle, N is the total number of patient posture frames currently being processed.

[0066] Execute step 150, real-time motion matching algorithm to obtain the optimal motion matching point, including extracting the patient's current frame y for each matching t and its preceding K-1 frames constitute the comet tail trajectory segment {y t-K+1 ,y t-K+2 ,…,y t}, where K represents the length of the comet tail trajectory segment, and then calculates the Euclidean distance between it and each frame of the standard action sequence segment, and combines the trajectory direction angle constraint to select the optimal matching point;

[0067] If there are multiple optimal matching points or the action is a sub-cycle type action, the nearest point suppression mechanism algorithm is used to process it and finally obtain the optimal action matching point.

[0068] Step 150 includes forming a comet tail trajectory segment from the current frame of the patient's motion sequence and its historical continuous frames, and dynamically matching the correct matching position of the current motion in the standard motion trajectory by calculating the similarity between the historical trajectory segment and the standard circular sequence. The dynamic Euclidean distance between the comet tail trajectory segment and each frame of the standard motion circular sequence is calculated as follows:

[0069]

[0070] Among them, c takes the value of 0, 1, ..., M-K+1, fragment[j] is the jth frame of the patient's real-time comet tail trajectory fragment, and Standard_fragment[c][j] is the jth frame of the cth comet tail trajectory fragment in the standard action sequence.

[0071] Step 150 includes directional angle constraint, calculating the average motion directional angle of the trajectory segments, and excluding the candidate matching point if the directional angle difference between the patient segment and the standard segment exceeds a preset threshold.

[0072] Step 150 includes calculating the distance between all sequences in the standard action sequence and the patient's current action based on the position of the previous matching point as the starting point, obtaining multiple matching points, selecting the first point that meets the matching requirements as the current matching point, and then updating the starting point as the matching point, and so on, until the current action count is completed.

[0073] Execute step 160, real-time action counting and quality feedback, action counting and progress tracking, each time a standard ring matching is completed, the count is increased by 1.

[0074] Example 2

[0075] This paper proposes a method for counting rehabilitation movements. Through high-dimensional space mapping, dynamic matching algorithms, and real-time feedback mechanisms, it achieves accurate counting of medical-grade rehabilitation movements and supports counting of a variety of complex rehabilitation movements.

[0076] A method for counting rehabilitation movements, characterized by comprising the following steps:

[0077] Step 1: Standardized rehabilitation movement data collection. A rehabilitation physician performs standardized movements, and a depth camera captures multi-angle, multi-cycle video sequences of these movements. Each movement is captured for at least 10 cycles, covering different body types, speeds, and joint ranges of motion.

[0078] The repetitive action videos collected in various scenarios in step 1 are specifically repetitive rehabilitation actions such as knee flexion and extension, shoulder abduction, etc.

[0079] Step 2: High-dimensional data mapping and standard action library construction. The standard action sequence is mapped to the high-dimensional feature space through the feature extractor to form the standard action sequence X = {x1, x2, ..., x M}, each x i represents the i-th processed normalized pose in the action cycle.

[0080] The feature extractor of step 2 includes the following sub-steps:

[0081] a. Joint angle calculation: Calculate the three-dimensional angle of the key joint based on the coordinates of the skeletal joint points. The formula is:

[0082]

[0083] in is the adjacent limb vector;

[0084] b. Joint distance calculation: Calculate the Euclidean distance between key joint points. The formula is:

[0085] d joint =‖G a -G b ‖

[0086] Among them G a and G b is the joint point coordinate;

[0087] c. Feature encoding: Input the angle and distance features into the encoder to generate a 128-dimensional feature vector H t .

[0088] Step 3: By spatially homogenizing the high-dimensional standard motions, the temporally non-uniformly sampled skeletal point sequence (e.g., inconsistent point spacing due to motion speed) is converted into a spatially uniformly distributed path sequence through interpolation. This ensures that the physical distance between adjacent points in the motion sequence remains constant, eliminating the impact of sampling density on the algorithm.

[0089] The adaptive step size formula in step 3 spatial homogenization is:

[0090]

[0091] Among them, M represents the total number of frames of standard action, G i Represents the high-dimensional human posture data at the i-th moment. If the motion distance d between adjacent frames t >s, you need to insert between these two frames Posture data, specifically inserted into the formula:

[0092]

[0093] Among them, G t is the starting posture feature vector of the interpolation interval, G t+1 is the end pose feature vector of the interpolation interval, G new,q It is in G t and G t+1 The qth point newly inserted between them, q = 1, 2,…, f.

[0094] Step 4: Real-time human posture extraction and data stream processing: use the posture extraction model to extract human posture in real time, output the three-dimensional coordinates of skeletal joints, and use the residual calculation method to eliminate redundant frames to reduce the computational load.

[0095] In step 4, a lightweight version of the human posture model is used to optimize the inference speed through the TensorRT engine, supporting real-time operation at ≥30fps on edge devices. For each frame of RGB-D image input, the three-dimensional coordinates G of the bone joint points are output. t ={g1,g2,…,g 33}, g w {x w ,y w ,z w} represents the position of the joint point w in the camera coordinate system.

[0096] Step 5: High-dimensional mapping and dynamic matching of posture data. The real-time skeleton point sequence is mapped to the same high-dimensional space as the standard motion library through the same feature extractor, and the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y N}.

[0097] In step 5, the patient's bone point sequence {Gt′} is input into the feature extractor according to claim 2 to calculate the joint angle θ joint Distance d from joint joint , generate a 128-dimensional feature vector H t ∈R 128 Then, the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y N}.

[0098] Step 6: Real-time motion matching algorithm, each matching will extract the patient's current frame y t and its preceding K-1 frames constitute the comet tail trajectory segment {y t-K+1 ,y t-K+2 ,…,y t}, where K represents the length of the comet tail trajectory segment. The Euclidean distance between this segment and each frame of the standard motion sequence is then calculated, and the optimal matching point is selected in combination with the trajectory direction angle constraint. If multiple optimal matching points exist, or the action is a sub-cycle type action, the closest point suppression mechanism algorithm is used to process them and ultimately determine the optimal action matching point.

[0099] Step 6 forms a comet tail trajectory segment from the current frame of the patient's action sequence and its historical continuous frames, which is mainly used to eliminate the matching problem in the cyclic action scenario. By calculating the similarity between the historical trajectory segment and the standard circular sequence, the correct matching position of the current action in the standard action trajectory is dynamically matched.

[0100] The formula for calculating the dynamic Euclidean distance between the comet tail trajectory segment and each frame of the standard action sequence in step 6 is:

[0101]

[0102] Among them, c takes values ​​of 0, 1, ..., M-K+1, M represents the total number of frames of the standard action, K represents the length of the comet tail trajectory segment, which is 5 here, fragment[j] is the patient's real-time comet tail trajectory segment, and Standard_fragment represents the comet tail trajectory segment of the standard action at each moment. For example, Standard_fragment[c][j] corresponds to the jth frame of the cth fragment in the standard action sequence.

[0103] The dynamic matching in step 6 further includes a direction angle constraint: calculating the average motion direction angle of the trajectory segment, and if the direction angle difference between the patient segment and the standard segment exceeds a preset threshold, the candidate matching point is excluded.

[0104] The step 6 further includes a closest point suppression mechanism method, which is mainly used to eliminate matching problems in multi-sub-cycle action scenarios. Specifically, based on the position of the previous matching point as the starting point, the distance between all sequences in the standard action sequence and the patient's current action is calculated to obtain multiple matching points. The first point that meets the matching requirements is selected as the current matching point, and then the starting point is updated as the matching point, and so on, until the current action count is completed.

[0105] Step 7: Real-time action counting and quality feedback, action counting and progress tracking. Each time a standard ring matching is completed, the count is increased by 1. Based on the current optimal matching point position tid, the completion percentage is calculated. Where M is the total number of frames in the standard action sequence.

[0106] The method adopts a lightweight algorithm architecture, and the core matching algorithm is encapsulated as an independent module, which supports real-time operation in low-computing power edge devices.

[0107] Example 3

[0108] The present invention provides a method for counting rehabilitation movements, comprising the following steps:

[0109] Collect standard rehabilitation movement data in advance and obtain a standard movement template library;

[0110] Continuously collect patient rehabilitation movement data through depth cameras and process the data at the same time;

[0111] Perform high-dimensional mapping of the patient's motion sequence to a predefined standard motion ring sequence;

[0112] Based on the comet tail matching algorithm and the closest point suppression mechanism, the position of the patient's movements in the standard ring is matched frame by frame;

[0113] The number of cycles is counted based on the matching results, and the real-time action completion progress is generated.

[0114] The present invention proposes a method and system for counting rehabilitation movements based on a depth camera. Aiming at the problems of poor real-time performance, low motion matching accuracy, and multi-cycle motion misjudgment in the existing rehabilitation movement monitoring technology, a novel motion counting method is proposed. Specifically, the method collects the patient's motion sequence through a depth camera, maps the standard motion and the patient's motion into high-dimensional space motion sequences respectively, and performs spatial homogenization on the motion data. Then, the comet tail matching algorithm is used in combination with the motion trajectory direction information for dynamic matching, and the nearest point suppression mechanism is introduced to resolve the multi-sub-cycle positioning ambiguity. The similarity measurement of nonlinear motion trajectories is optimized through spatial homogenization processing to achieve accurate tracking of motion progress and number of cycles. The core algorithm of this solution is lightweight and can run on low-computing power devices. It is suitable for real-time counting, scoring, and correction feedback in rehabilitation training, improving the standardization of patient training and rehabilitation efficiency.

[0115] Example 4

[0116] This embodiment uses a mobile device to capture training video images of the trainee, uses a human posture model combined with a mobile phone camera to collect skeletal point location information, extracts features from the skeletal point information, identifies rehabilitation training movements, and uses a counting algorithm to calculate the number of completed training movements and the progress of the movement. This method can be run directly on depth cameras and other terminals, and can also be applied to servers. Other terminals can be smart phones, computers, tablets, smart TVs, smart watches, portable computers, or desktop computers, as well as other intelligent terminals with computing and data analysis capabilities.

[0117] When obtaining the standard template of the standard action, it is necessary to process the video of the standard action to extract the high-dimensional vector sequence of the standard action for subsequent matching calculation. This process can be implemented in many ways in practice. This application provides some embodiments, which are described below:

[0118] like Figure 2 As shown, the standard template can be obtained in the following manner, which includes:

[0119] 101. Collection of standard rehabilitation movement data.

[0120] This method requires obtaining template videos of standard movements. A rehabilitation physician can perform standard movements (such as knee flexion and extension, shoulder abduction, etc.) and use a depth camera to capture multi-angle, multi-cycle video sequences of these movements. Each movement should be captured for at least 10 cycles, covering different body types, speeds, and joint ranges of motion.

[0121] 102. Extract key skeleton points of the human body from standard action videos.

[0122] This application needs to first obtain the human skeleton point information. There are many ways to obtain the skeleton point information. It can be directly input skeleton point information or obtained from a standard video stream. For example, a standard action video stream is input, each frame is processed, and the skeleton points of all frames are obtained. An action is composed of multiple consecutive frames of images. After obtaining the multi-frame action skeleton point sequence, the data needs to be processed by the feature extractor. The feature extractor implementation process is as follows:

[0123] 103. Extract geometric features from data

[0124] 1031. Joint Angle Calculation: Calculate the three-dimensional angles of key joints (such as knees and elbows) to describe movement patterns. For example, taking the calculation of knee joint angle as an example:

[0125]

[0126] in Represents the thigh vector, which is calculated from the nodes at the hip and knee joints. The representative calf vector is calculated from the joint points at the knee and ankle joints.

[0127] 1032. Joint distance feature: Calculates the Euclidean distance between key joints to reflect the range of motion. Taking shoulder width as an example, the specific formula is:

[0128] d shoulder =‖G left -G right ‖

[0129] Among them G left With G right Represent the coordinate positions of the left and right shoulders respectively.

[0130] 1033. Feature coding: input the geometric features (angle, distance) of each frame into the encoder and output the 128-dimensional feature code H of the human body posture at each moment. t .

[0131] The feature extractor processes each standard action accordingly to obtain a high-dimensional standard action template sequence library, which can be used for subsequent action counting statistics.

[0132] Example 5

[0133] like Figure 3 The method for counting actions includes:

[0134] 201. Load standard action sequences corresponding to different action types.

[0135] Since different actions require different counting algorithms, the user's action type should be determined first, and the corresponding standard action sequence should be preloaded and processed according to the user action type, such as loop action and sub-loop action.

[0136] 202. Spatial homogenization of standard action sequences.

[0137] In rehabilitation movements, the influence of time factors is not particularly important, but the accuracy, effectiveness and completion of counting are factors that need special consideration. In order to reduce the problem of uneven point spacing in the original data due to changes in movement speed, which affects the counting accuracy, this method also creatively proposes the concept of spatial homogenization. It is converted into a spatially uniformly distributed path sequence through interpolation processing. The core is to ensure that the physical distance between adjacent points is constant and eliminate the influence of sampling density on the algorithm. Therefore, it is first necessary to perform spatial homogenization on the data of the standard action template sequence obtained above, and then perform sliding window extraction on the homogenized standard action path to generate a local fragment library. First, the algorithm input is the time-uniform standard template sequence G obtained by the feature extractor std =[G1,G2,…,G t ]∈R M*128 , G t The feature encoding represents the human posture at time t. In order to convert the time uniform path into a spatial uniform path and ensure that the distance between adjacent points is constant, the adaptive step size s needs to be calculated first. The calculation formula of s is as follows:

[0138]

[0139] Among them, M represents the total number of frames of standard action, G t is the high-dimensional human posture feature vector at time t;

[0140] If the motion distance d between adjacent frames t >s, you need to insert between these two frames The specific insertion formula for posture data is:

[0141]

[0142] Among them, G t is the high-dimensional human posture feature vector at time t, G t+1 is the high-dimensional human posture feature vector at time t+1, G new,q It is in G t and G t+1 The qth point newly inserted between them, q = 1, 2,…, f.

[0143] After processing, the final output space is uniform standard action sequence G std ={G1,G2,...,G M′}, satisfying ||G t+1 -G t ||≈s.

[0144] 203. The coordinates of the key skeletal points of the human body are extracted from the real-time collected human body image to obtain the patient's real-time high-dimensional motion sequence, so as to facilitate the use of the matching counting algorithm.

[0145] In the real-time acquisition stage, the angles of the key skeletal points of the human body in the human body image are collected by the camera. The key skeletal points here correspond to the key skeletal points of the standard action video collected in step 101, and the method of collecting the skeletal points can also be similar to the method used in step 101. The subsequent patient data processing and dimensionality conversion are also the same as step 101. The data also needs to be spatially homogenized, which is similar to step 202 and will not be repeated here. In actual applications, the standard action video is generally played through the display device on the rehabilitation equipment, and the user follows the video to move. The camera on the rehabilitation equipment captures the user's movements for extraction and analysis.

[0146] 204. Match the patient's real-time high-dimensional motion sequence with the standard motion template sequence using a comet tail matching and closest point suppression algorithm. This is specifically divided into the following steps:

[0147] 2041. Similarity Calculation

[0148] After real-time data processing, it will be divided into a fragment fragment∈R by the comet tail matching algorithm. K*128, this segment represents the action data of the most recent K frames, and each segment of action data will be compared frame by frame with the standard segment library. By comparing each frame, the similarity curve between each segment can be obtained. The specific implementation formula is as follows:

[0149]

[0150] For example, if the action has 120 frames and K = 5, then standard_fragment will be divided into 120 - 5 + 1 = 116 segments. Each segment is a sequence of 5 adjacent standard action frames. For example, standard_fragment[0] = {sequence 0, sequence 1, sequence 2, sequence 3, sequence 4}, standard_fragment[1] = {sequence 1, sequence 2, sequence 3, sequence 4, sequence 5}, and so on. The fragment of the real-time action fragment = {sequence 0, sequence 1, sequence 2, sequence 3, sequence 4} will use the form of a sliding window to calculate the Euclidean distance between segments. The window slides 1 sequence each time, so as to obtain the similarity score between the action fragment and all segments of the standard action.

[0151] 2042. Search for the best matching point

[0152] First, according to the designed threshold th, generate a binary array arr. If similarity_curve[i] < th, then arr[i] = 1; otherwise, arr[i] = 0. Then, the candidate action segment in the standard action that best matches the patient's movement can be obtained from this. At the same time, calculate the average velocity direction angle between the real-time segment and the candidate segment, and exclude the candidate segments with opposite directions. Due to the problem of cyclic substructure actions, multiple best matching segments may be obtained, so the method of nearest point suppression will also be used for processing. Assume that the previous matching point is pre_tid. Then, first take the first matching segment of the best matching segment as the segment for this match, and then select the point with the minimum distance tid within this segment, and update the matching point to the current matching point tid. Through the above steps, the best matching point tid between the patient's action and the standard action can be obtained.

[0153] 205. Action counting and progress tracking

[0154] When tid jumps back from the last frame M of the standard action template to the starting frame 0 of the standard action template, it is determined that a complete cycle is completed, and the count is incremented by 1.

[0155] The real-time action progress can be simply obtained by the formula: where tid is the real-time action matching point and M is the length of the action sequence.

[0156] Example 6

[0157] Such as Figure 4 The present application further provides a portable rehabilitation motion counting device, comprising a display, a depth camera, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the aforementioned motion counting method embodiments are implemented. When the processor executes the computer program, the functions of the various modules in the aforementioned device embodiments are implemented, such as the functions of the feature extraction, spatial homogenization processing, comet tail matching, and counting logic modules.

[0158] The computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete all the functions of the first and second embodiments of the present application. The modules include:

[0159] Data acquisition module: controls the depth camera to collect the patient's bone point data in real time, and performs de-jittering and normalization processing;

[0160] Feature encoding module: calculates joint angle and distance features and generates a 128-dimensional high-dimensional feature vector;

[0161] Spatial homogenization module: converts the action sequence into a spatially uniform path through an adaptive interpolation algorithm;

[0162] Dynamic matching module: performs sliding window comparison of comet tail trajectory segments with the standard motion library, and selects the optimal matching points based on directional angle constraints;

[0163] Counting and feedback module: Completes action counting based on the matching point jump logic, and provides real-time feedback on progress and abnormal prompts through the display screen and voice unit.

[0164] The terminal device is an integrated portable rehabilitation training terminal, and its specific hardware configuration is as follows:

[0165] 1. Memory:

[0166] Internal storage unit: used to store standard action library (JSON format) and patient training records;

[0167] External expansion interface: supports MicroSD card (up to 256GB) for offline data backup.

[0168] 2. Processor:

[0169] Main control chip: Rockchip RK3588S quad-core processor (NPU computing power 6TOPS), built-in 4GB LPDDR4 memory;

[0170] Coprocessor: FPGA accelerates feature encoding and interpolation calculations to ensure real-time processing frame rate ≥ 30fps.

[0171] 3. Depth camera module: Integrates the Intel RealSense L515 lidar camera, deployed on the top of the device, and supports high-precision skeleton point capture within a range of 0.2m-5m.

[0172] 4. Interaction module:

[0173] 7-inch touch screen (1280×800), real-time display of counts, progress bar and joint angle curves;

[0174] Voice feedback unit: supports multi-language prompts.

[0175] The software architecture of the terminal device is based on a lightweight design, and the specific process includes:

[0176] 1. Initialization phase: load the standard motion template from the memory, and the camera automatically calibrates the angle and focal length;

[0177] 2. Real-time processing stage:

[0178] The data acquisition module outputs the coordinates of bone points at 30fps;

[0179] The feature encoding module generates high-dimensional feature vectors, and the spatial homogenization module dynamically interpolates and expands the sequence;

[0180] The dynamic matching module calculates the dynamic Euclidean distance between the comet tail fragment and the standard library (see claim 4 for the formula), and selects matching points in combination with the direction angle constraint (threshold ±15°);

[0181] The counting module triggers counting according to the matching point jump logic (end point → start point) and uses the formula Update progress.

[0182] 3. Exception handling: If there is no valid match for 10 consecutive frames, the counting will be paused and a prompt "Please restart the action" will be given.

[0183] In order to better understand the present invention, the above is described in detail in conjunction with the specific embodiments of the present invention, but it is not intended to limit the present invention. Any simple modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention. Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A method for counting rehabilitation movements, comprising collecting standard rehabilitation movement data, characterized in that: The following steps are also included: Step 1: High-dimensional data mapping and standard action library construction, map the standard action sequence to the high-dimensional feature space through the feature extractor, and construct the standard action sequence X = {x1, x2, ..., x i ,…,x M }, Among them, x i is the i-th processed standardized posture in the action cycle, and M is the total number of frames in the standard action sequence; Step 2: By performing spatial homogenization on the high-dimensional standard action, the temporally non-uniformly sampled skeleton point sequence is converted into a spatially uniformly distributed path sequence through interpolation. Step 3: Real-time human posture extraction and data stream processing: Use the posture extraction model to extract human posture in real time, output the three-dimensional coordinates of skeletal joints, and use the residual calculation method to eliminate redundant frames; Step 4: High-dimensional mapping and dynamic matching of posture data: The real-time skeleton point sequence is mapped to the same high-dimensional space as the standard motion library through the same feature extractor, and the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y i ,…,y N }, where y i The i-th processed patient motion posture in the motion cycle, N is the total number of patient posture frames currently processed; Step 5: Real-time action matching algorithm to obtain the optimal action matching point; Step 6: Real-time action counting and quality feedback, action counting and progress tracking. Each time a standard ring match is completed, the count increases by 1.

2. The rehabilitation movement counting method according to claim 1, wherein: The step 1 includes the following sub-steps: Step 11: Calculate joint angle features; Step 12: Calculate joint distance features; Step 13: Encode the joint angle features and the joint distance features.

3. The rehabilitation movement counting method according to claim 2, wherein: The step 11 includes calculating the three-dimensional angle θ of the key joint based on the coordinates of the skeletal joint points joint , the formula is: in, is the adjacent limb vector.

4. The rehabilitation movement counting method according to claim 3, wherein: The step 12 includes calculating the Euclidean distance d between the key joint points joint , the formula is: d joint =‖G a -G b ‖ Among them, G a and G b are the joint point coordinates.

5. The rehabilitation movement counting method according to claim 4, wherein: The step 13 includes inputting the joint angle feature and the joint distance feature into an encoder to generate a 128-dimensional feature vector H t .

6. The rehabilitation movement counting method according to claim 5, wherein: The feature vector generated by the feature encoder is represented as H t ={G1,G2,...,G M }, the formula for the adaptive compensation s in the spatially uniform distribution is: If the motion distance d between adjacent frames t >s, you need to insert between these two frames The specific insertion formula for posture data is: Among them, G t is the high-dimensional human posture feature vector at time t, G t+1 is the high-dimensional human posture feature vector at time t+1, G new,q It is in G t and G t+1 The qth point newly inserted between them, q = 1, 2,…, f.

7. The rehabilitation movement counting method according to claim 6, wherein: The three-dimensional coordinates of the skeletal joint points are G t ={g1,g2,…,g 33 }, where g w {x w ,y w ,z w } represents the position of the joint point w in the camera coordinate system.

8. The rehabilitation movement counting method according to claim 7, wherein: Step 4 includes inputting the patient's bone point sequence {Gt′} into the feature extractor to calculate the joint angle θ joint Distance d from joint joint , generate a 128-dimensional feature vector H t ∈R 128 Then, the same spatial homogenization operation is performed to obtain the real-time patient posture sequence Y = {y1, y2, ..., y i ,…,y N }.

9. The rehabilitation movement counting method according to claim 8, wherein: The step 5 includes extracting the patient's current frame y for each matching t and its preceding K-1 frames constitute the comet tail trajectory segment {y t-K+1 ,y t-K+2 ,…,y t }, where K represents the length of the comet tail trajectory segment, and then calculates the Euclidean distance between it and each frame of the standard action sequence segment, and combines the trajectory direction angle constraint to select the optimal matching point; If there are multiple optimal matching points or the action is a sub-cycle type action, the nearest point suppression mechanism algorithm is used to process it and finally obtain the optimal action matching point.

10. The rehabilitation movement counting method according to claim 9, wherein: The step 5 also includes forming a comet tail trajectory segment from the current frame of the patient's action sequence and its historical continuous frames, and dynamically matching the correct matching position of the current action in the standard action trajectory by calculating the similarity between the historical trajectory segment and the standard ring sequence.

Images