A real-time monitoring method for human rehabilitation training of a rehabilitation device

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a real-time monitoring method for human rehabilitation training of a rehabilitation device, which comprises the following steps: calculating a local motion energy factor according to the distribution of instantaneous speed and instantaneous acceleration in a local trajectory segment at each time point, calculating a local ideal trajectory and a root mean square error of the local trajectory segment to obtain a local trajectory stability, combining the two with a local average speed to calculate a dynamic detail saliency score, adjusting a global basic threshold to obtain an adaptive distance threshold, applying a Douglas-Poke algorithm to process an original motion trajectory sequence to obtain an optimized trajectory sequence, calculating a training stability of a patient according to a root mean square error of an ideal motion trajectory sequence and the optimized trajectory sequence, and generating a visual rehabilitation report. The present application effectively filters random noise and smooths the motion trajectory while retaining pathological detail features that affect rehabilitation evaluation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing. More particularly, the present application relates to a real-time monitoring method for human rehabilitation training of a rehabilitation device. BACKGROUND

[0002] In modern rehabilitation medicine, using a rehabilitation device to assist patients in scientific and standardized rehabilitation training is an important way to restore physical function; in order to objectively evaluate the rehabilitation effect and guide the follow-up training plan, it is necessary to monitor and quantitatively analyze the performance of the patient during the training process in real time.

[0003] Among them, converting complex and continuous human dynamic data into quantifiable and traceable key performance indicators, such as action completion degree, stability, and fluency, is the core link to achieve accurate rehabilitation evaluation; at present, a mainstream technical solution is to capture the video stream of the patient when using the rehabilitation device for training through image acquisition devices such as cameras, and to extract the human key joint points using pose estimation algorithms in machine vision, and to analyze and calculate rehabilitation evaluation indicators.

[0004] However, in real environments such as homes or non-professional rehabilitation rooms, factors such as changes in lighting, background interference, and loose patient clothing will inevitably cause the joint point coordinate sequence output by the pose estimation algorithm to contain a large amount of noise, jitter, and abnormal jumps; if these original data containing noise are directly used to calculate rehabilitation indicators, it will seriously affect the reliability of the results, and the system may mistakenly interpret the algorithm's own technical errors as the patient's physiological performance.

[0005] In order to solve the above-mentioned data noise problem, introducing a trajectory simplification algorithm to preprocess the joint point coordinate sequence has become a common improvement idea; among them, the Douglas-Peucker (DP) algorithm is widely used due to its high efficiency and shape preservation, and the Douglas-Peucker algorithm removes points in the trajectory that deviate from the main path by a small distance by setting a distance threshold, thereby simplifying and smoothing the trajectory and achieving the purpose of filtering out part of the noise.

[0006] The traditional Douglas-Peucker algorithm uses a globally fixed distance threshold, which cannot distinguish the nature of the trajectory details, and will mistakenly remove the microscopic details of the trajectory that have diagnostic value when filtering visual noise, losing the core information necessary for fine and differentiated rehabilitation evaluation, and thus cannot accurately quantify the patient's true movement state. SUMMARY

[0007] To solve the technical problem that the existing Douglas-Peucker algorithm will mistakenly remove the microscopic details of the trajectory with diagnostic value when filtering visual noise, the present application provides a real-time monitoring method for human rehabilitation training of a rehabilitation device, comprising: collecting monitoring video when a hand disease patient imitates the action in a hand recovery test video; using a human pose estimation algorithm to extract a plurality of joint nodes in each frame of image, taking any joint node as a target joint node, combining the spatial coordinates of the target joint node in a plurality of frames of image into an original motion trajectory sequence, and calculating the instantaneous speed vector and the instantaneous acceleration vector of the target joint node at each time point; calculating the local motion energy factor according to the distribution of the instantaneous speed and the instantaneous acceleration in the local trajectory segment at each time point; performing local path fitting on the local trajectory segment by the least square method, calculating the root mean square error between the obtained local ideal trajectory and the local trajectory segment, and obtaining the local trajectory stability; calculating the dynamic detail saliency score according to the local motion energy factor, the local trajectory stability index and the local average speed, adjusting the global basic threshold, obtaining the adaptive distance threshold used when judging whether the coordinate point of the target joint node at each time point is retained, and applying the Douglas-Peucker algorithm to process the original motion trajectory sequence to obtain an optimized trajectory sequence; performing path fitting on the spatial coordinates of the target joint node in the hand recovery test video by the least square method, calculating the training stability of the patient according to the root mean square error between the obtained ideal motion trajectory sequence and the optimized trajectory sequence, and generating a visual rehabilitation report.

[0008] The present application constructs a dynamic detail saliency score index by comprehensively analyzing the energy, stability and speed of local motion, and dynamically adjusts the threshold of the filtering algorithm, so that the filtering algorithm can effectively filter out random noise and smooth the motion trajectory, while accurately retaining pathological detail features that are crucial for rehabilitation evaluation, thereby greatly improving the data quality on which the final generated rehabilitation evaluation report is based, and making the evaluation result more accurate and real reflecting the patient's neuromuscular control ability.

[0009] Preferably, the target joint node includes but is not limited to the right shoulder, the right elbow and the right wrist, and when the patient performs rehabilitation exercise with the left hand, the target joint node includes but is not limited to the left shoulder, the left elbow and the left wrist.

[0010] Preferably, the method for obtaining the local trajectory segment at each time point is: for the time point , a local time window is obtained, the local time window contains time points from the time point to the time point , wherein, is the half width of the local time window ; and the local motion energy factor is calculated according to the distribution of the instantaneous speed and the instantaneous acceleration in the local time window From the original motion trajectory sequence of the target joint Extract the corresponding local trajectory segment from the middle .

[0011] Preferably, the step of calculating the local motion energy factor based on the distribution of instantaneous velocity and instantaneous acceleration within the local trajectory segment at each time point includes: In the formula, For the target key point at time point The local motion energy factor of the local trajectory segment; , The target key points at time points are respectively The instantaneous velocity vector and instantaneous acceleration vector; Indicates the magnitude of the vector; For local trajectory segments All time points The standard deviation of the magnitude of the instantaneous acceleration vector; For local trajectory segments All time points The mean of the magnitude of the instantaneous acceleration vector.

[0012] This invention combines the amplitude of instantaneous velocity with the dispersion of instantaneous acceleration, which can effectively amplify signals with regular high-frequency oscillations, such as physiological tremors. From the perspective of physical energy, it can initially distinguish meaningful tremors from irregular and isolated noise points, providing a basis for subsequent significance judgment.

[0013] Preferably, the calculation of the root mean square error between the local ideal trajectory and the local trajectory segment to obtain the local trajectory stability includes: calculating the local trajectory segment... From each coordinate point in the local ideal trajectory The Euclidean distances of the corresponding projection points are calculated, and the root mean square of all Euclidean distances is taken as the root mean square error between the local ideal trajectory and the local trajectory segment. The root mean square error is then inversely normalized, and the result is used as the target joint point at time point. Local trajectory stability The inverse proportional normalization is achieved through... To achieve, among which, It is a natural exponential function.

[0014] This invention obtains trajectory stability by comparing the actual local trajectory with an ideal smooth fitted trajectory. It can identify the persistent small-scale trajectory roughness caused by muscle instability. Even if the energy of these fluctuations is not strong, they can be identified as unstable. This provides supplementary information for distinguishing trajectory details of different causes, namely, high-energy tremors and geometric instability.

[0015] Preferably, the calculating the dynamic detail saliency score according to the local motion energy factor, the local trajectory stability index and the local average speed comprises: ; wherein, is the dynamic detail saliency score of the target joint node at the time point ; is the local motion energy factor of the local trajectory segment of the target joint node at the time point ; is the local trajectory stability of the target joint node at the time point ; is the instantaneous speed vector of the target joint node at the time point ; denotes the vector module length; is the average of the module lengths of the instantaneous speed vectors of all the time points in the local trajectory segment , representing the local average speed at the time point .

[0016] The present application combines the energy factor representing the motion intensity with the stability representing the geometric smoothness, and takes the average speed as a context adjustment factor, which enables the scoring model to make complex decisions: tolerating greater fluctuations in high-speed motion, and being more sensitive to slight jitter in low-speed fine control, thereby realizing intelligent and contextualized judgment of the importance of trajectory details.

[0017] Preferably, the adjusting the global basic threshold to obtain an adaptive distance threshold used for judging whether the coordinate point of the target joint node at each time point is retained comprises: calculating the difference between the digital 1 and the dynamic detail saliency score of the target joint node at the time point in the corresponding frame image ; calculating the product of the difference and the global basic threshold to obtain the adaptive distance threshold used for judging whether the coordinate point is retained.

[0018] Preferably, the applying the Douglas-Peucker algorithm to process the original motion trajectory sequence to obtain the optimized trajectory sequence comprises: judging whether each coordinate point is retained through the adaptive distance threshold; for the coordinate point : calculating the perpendicular distance from the coordinate point to the line segment connecting the previous and next key points and when judging whether the coordinate point is retained. ​​​and the coordinate point corresponding adaptive distance threshold is compared, if , the coordinate point is retained ; otherwise, the coordinate point is removed; the judgment process is repeated until all coordinate points in the original motion trajectory sequence are processed, and an optimized trajectory sequence is obtained; wherein, initially, the first coordinate point and the last coordinate point in the original motion trajectory sequence are taken as key points ; and subsequent key points are retained coordinate points.

[0019] Preferably, the root mean square error between the obtained ideal motion trajectory sequence and the optimized trajectory sequence is calculated to obtain the training stability of the patient, including: for a target joint point: calculating the root mean square error between the optimized trajectory sequence and the ideal motion trajectory sequence, inversely proportionally normalizing the root mean square error, and taking the obtained result as the training stability of the target joint point; the inverse proportional normalization is realized by , wherein, is a natural exponential function; and calculating the mean value of the training stabilities of all joint points as the training stability of the patient.

[0020] The optimized trajectory sequence obtained by the present application not only filters out interference noise, but also retains true physiological tremor and the like, so that the error between the optimized trajectory sequence and the ideal trajectory can more truly reflect the actual motion control level of the patient. Compared with the evaluation results based on original noisy data or excessively simplified data, the stability index generated by the present application is not easily polluted by algorithm artifacts, thereby improving the reliability of the final rehabilitation evaluation index.

[0021] are respectively coordinate points of the target joint point in frame images corresponding to time points ; and is a time interval for collecting the frame images; and are respectively instantaneous velocity vectors of the target joint point at time points .

[0022] The present application has the following beneficial effects:

[0023] ​​​​​​​​The application comprehensively analyzes the energy, stability and speed of local motion, constructs a dynamic detail saliency score index, and dynamically adjusts the threshold value of the filtering algorithm, so that the filtering algorithm can effectively filter out random noise and smooth the motion trajectory, while accurately retaining pathological detail features that are crucial for rehabilitation evaluation, thereby greatly improving the data quality on which the final generated rehabilitation evaluation report is based, and the evaluation result more accurately and truly reflects the patient's neuromuscular control ability. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flowchart schematically showing a real-time monitoring method for human rehabilitation training of a rehabilitation device in the application;

[0025] Figure 2 is a flowchart schematically showing step S2;

[0026] Figure 3 is a schematic diagram schematically showing the curves of the original motion trajectory sequence and the ideal motion trajectory sequence;

[0027] Figure 4 is a result schematically showing that the original motion trajectory sequence in Figure 3 is processed by the standard DP algorithm based on the global basic threshold value;

[0028] Figure 5 is a result schematically showing that the original motion trajectory sequence in Figure 3 is processed by the adaptive P algorithm based on the adaptive distance threshold value. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0030] The specific embodiments of the application will be described in detail below with reference to the drawings.

[0031] The embodiments of the application disclose a real-time monitoring method for human rehabilitation training of a rehabilitation device, with reference to Figure 1 , comprising steps S1 to S4:

[0032] S1: Collecting monitoring videos when a hand disease patient imitates the action in the hand recovery test video, using a human pose estimation algorithm to extract target joint nodes in each frame of image, and combining the spatial coordinates of the target joint nodes in the continuous multiple frames of image into an original motion trajectory sequence of the target joint nodes.

[0033] It should be noted that in the rehabilitation process of hand disease patients, the hand needs to be continuously exercised through a special hand rehabilitation exercise device; in order to better perform hand rehabilitation exercise, the patient is required to watch a hand recovery test video after each exercise and try to imitate the actions therein; in this process, by monitoring the hand movements of the patient and analyzing the differences between the monitored video and the hand movements in the hand recovery test video, the hand rehabilitation condition of the patient can be better understood.

[0034] Specifically, the hand rehabilitation exercise device for medical care is used to perform hand rehabilitation exercise on the hand disease patient, and after each hand rehabilitation exercise, the hand disease patient is required to watch a hand recovery test video and try to imitate the actions therein, and in this process, the image acquisition unit matched with the rehabilitation exercise device captures the video of the patient performing rehabilitation training in real time at a preset frame rate.

[0035] Further, for the continuous image frames in the rehabilitation training video, a plurality of joint nodes and the spatial coordinates of the joint nodes in each frame image are extracted through an existing human pose estimation algorithm; for a target joint node among them: the coordinate point of the target joint node in the time point Corresponding frame image is recorded as , wherein , The horizontal coordinate and the vertical coordinate of the target joint node in the time point Corresponding frame image are respectively recorded as

[0036] Among them, the existing human pose estimation algorithm includes but is not limited to OpenPose and MediaPipe; when the hand of the patient performing rehabilitation exercise is the right hand, the joint nodes include but are not limited to the right shoulder, the right elbow and the right wrist, and when the hand of the patient performing rehabilitation exercise is the left hand, the joint nodes include but are not limited to the left shoulder, the left elbow and the left wrist.

[0037] Further, the spatial coordinates of the target joint node in all frame images are combined to form an original motion trajectory sequence of the target joint node containing a plurality of data points .

[0038] It should be noted that the original data input is provided for subsequent trajectory detail analysis and adaptive simplification, and the output Although it contains noise, it also completely records all original motion information, laying a foundation for subsequent differentiation between noise and effective details.

[0039] S2: According to the distribution of the instantaneous speed and the instantaneous acceleration in the local trajectory segment at each time point, the local motion energy factor is calculated, the root mean square error between the obtained local ideal trajectory and the local trajectory segment is obtained, and the local trajectory stability is obtained. The dynamic detail saliency score is calculated by combining the local average speed.

[0040] The flowchart of step S2 refers to Figure 2 , including steps S201 to S204, specifically:

[0041] S201: Calculate the instantaneous speed vector and the instantaneous acceleration vector of the target joint at each time point.

[0042] In order to calculate the local motion energy factor, the basic physical quantities describing the dynamic characteristics of the motion, the instantaneous speed vector and the instantaneous acceleration vector, need to be extracted from the discrete original motion trajectory sequence . Specifically, according to the difference approximation of the coordinate points of the target joint over time, the instantaneous speed vector and the instantaneous acceleration vector of the target joint at time point are calculated, and the instantaneous speed vector , , is the coordinate point of the target joint in the corresponding frame image at time point , ; and the instantaneous acceleration vector , , , is the instantaneous speed vector of the target joint at time point , ; and the time interval for collecting frame images is .

[0043] S202: According to the instantaneous speed and the instantaneous acceleration in the local trajectory segment at each time point, the local motion energy factor of the target joint at each time point is calculated.

[0044] It should be noted that although the real physiological tremor or the tremor caused by muscle fatigue has small displacement, its speed and acceleration will show a regular high-frequency oscillation with concentrated energy consumption; while the random noise generated by the algorithm tends to be irregular isolated spikes in speed and acceleration changes, therefore, by comprehensively considering the amplitude of speed change and the concentration degree of acceleration change, the two different nature of trajectory details can be distinguished.

[0045] Specifically, for time point , its local time window is obtained, and the local time window comprises time points to time points co time points, wherein, is a half width of the local time window , and is in the range of [2, 6], the present application sets to 3; according to the local time window , a corresponding local trajectory segment is intercepted from the original motion trajectory sequence of the target joint point .

[0046] Further, according to the instantaneous speed and the instantaneous acceleration within the local trajectory segment at time point , a local motion energy factor of the local trajectory segment of the target joint point at time point is calculated, and the specific calculation formula is:

[0047] ;

[0048] In the formula, is the local motion energy factor of the local trajectory segment of the target joint point at time point ; is a half width of the local time window ; , are respectively the instantaneous speed vector and the instantaneous acceleration vector of the target joint point at time point ; represents the vector length; represents the instantaneous kinetic energy of the target joint point at time point , and reflects the degree of motion; is the standard deviation of the lengths of the instantaneous acceleration vectors of all time points in the local trajectory segment , and is used to measure the discrete degree of acceleration change; is the average of the lengths of the instantaneous acceleration vectors of all time points in the local trajectory segment , and is used to measure the overall acceleration level.

[0049] In the formula, the calculation formula of the local motion energy factor multiplies the kinetic energy term with the modulation term composed of the statistical characteristics of the acceleration, when the trajectory segment has regular high-frequency tremor, such as physiological tremor, the acceleration will oscillate violently around a non-zero mean, resulting in being larger, and ​is kept at a certain level, at which the ratio of the two will increase significantly, thus amplifying the value of the local motion energy factor ; on the contrary, for isolated noise points in smooth motion, although they can generate a single larger acceleration, the statistical average effect in the local time window is not obvious, thus the and of the overall smooth motion are small, and the ratio of the two will not abnormally increase; therefore, the higher the value of the local motion energy factor , the more likely it is that the local trajectory segment contains meaningful physiological details rather than pure algorithm noise.

[0050] It should be noted that the calculated local motion energy factor provides a basis for subsequent adaptive algorithms and can identify potential important details in the trajectory based on kinematic principles.

[0051] S203: locally fitting the local trajectory segment by the least square method to obtain a local ideal trajectory of the target joint at each time point, calculating the root mean square of the Euclidean distance between the local trajectory segment and the local ideal trajectory to obtain the local trajectory stability of the target joint at each time point.

[0052] It should be noted that in addition to tremor, rehabilitation assessment also concerns the stability of the patient's movement, that is, the degree to which the actual motion trajectory deviates from the ideal smooth path, therefore, a local trajectory stability index independent of tremor intensity is constructed to quantify the local geometric smoothness of the trajectory.

[0053] Specifically, by the least square method, local path fitting is performed on all coordinate points in the local trajectory segment to obtain a smooth parabolic segment representing a local ideal path in the local time window , as a local ideal trajectory of the target joint at time point .

[0054] Further, the Euclidean distance of each coordinate point in the local trajectory segment to its corresponding projection point on the local ideal trajectory is calculated, and the root mean square of all Euclidean distances is taken as the root mean square error of the local ideal trajectory and the local trajectory segment; the root mean square error is inversely proportional to the normalization, and the obtained result is taken as the local trajectory stability of the target joint at time point ; the inverse proportional normalization is realized by , wherein, is a natural exponential function.

[0055] It should be noted that the obtained local trajectory stability quantifies the trajectory details from a geometric perspective, characterizing the local roughness of the trajectory. Lower local trajectory stability indicates that the local trajectory segment... This could stem from irregular and unstable muscle control, or from significant noise generated by the algorithm, providing information about the deviation of the path morphology for subsequent decisions.

[0056] S204: Calculate the dynamic detail significance score of the target joint at each time point based on the local motion energy factor, local trajectory stability index, and local average velocity.

[0057] It should be noted that whether the details in the trajectory are important and worth preserving depends not only on their own physical characteristics, such as tremor intensity, but also on the motion context in which they occur. Therefore, by integrating the local motion energy factor and local trajectory stability obtained in the preceding steps, and introducing motion speed as a context adjustment factor, a comprehensive detail saliency score is generated.

[0058] Specifically, based on the local motion energy factor, local trajectory stability index, and local average velocity, the target joint at time point is calculated. Dynamic detail saliency score The specific calculation formula is as follows:

[0059] ;

[0060] In the formula, For the target key point at time point Dynamic detail saliency score; For the target key point at time point The local motion energy factor of the local trajectory segment; For the target key point at time point Local trajectory stability; For the target key point at time point The instantaneous velocity vector; Indicates the magnitude of the vector; For local trajectory segments All time points The mean of the magnitude of the instantaneous velocity vector, representing the time point. The local average velocity at that location.

[0061] Among them, the local motion energy factor With terms characterizing path stability Multiplication combines the intrinsic physical properties of details with the macroscopic context of motion, and generates a local average velocity. Normalization is performed: the more unstable the path, The smaller the value, The closer the term is to 1, the smaller the influence of the energy factor is; when the path is more stable, The greater the value is, The closer the term is to 0, the score value is inhibited; meanwhile, the speed term in the denominator plays a role of context adjustment, and for low-speed motion, the denominator is smaller, so that the dynamic characteristics in the numerator are amplified.

[0062] S3: adjusting the global basic threshold according to the dynamic detail saliency score to obtain an adaptive distance threshold used when judging whether the coordinate point of the target joint node at each time point is reserved; through the adaptive distance threshold, Douglas-Peucker algorithm is applied to process the original motion trajectory sequence to obtain an optimized trajectory sequence.

[0063] It should be noted that in order to enable the Douglas-Peucker algorithm to treat trajectory points of different properties differently, the globally fixed comparison threshold is replaced by a locally adaptive threshold that dynamically changes according to the characteristics of each trajectory point.

[0064] Specifically, the global basic threshold is adjusted according to the dynamic detail saliency score to obtain an adaptive distance threshold used when judging whether the coordinate point of the target joint node at each time point is reserved, and the specific calculation formula is:

[0065] ;

[0066] In the formula, is the adaptive distance threshold used when judging whether the coordinate point is reserved; the coordinate point is the coordinate point of the target joint node in the frame image corresponding to the time point ; is the global basic threshold, which represents the expected maximum trajectory simplification degree or the basic noise level in the absence of any special details, so the global basic threshold is in the range of [2, 10], and the present application sets to 3; is the dynamic detail saliency score of the target joint node at the time point .

[0067] wherein, tends to 1, indicating that there is important detail here, tends to 0, and a very small distance threshold means that the coordinate point can only be removed when it almost accurately falls on the line connecting and , which makes it highly probable that the trajectory point containing slight oscillation is retained; on the contrary, when tends to 0, it indicates that this is a smooth motion, Approaching a global base threshold The algorithm is simplified in a conventional manner, effectively filtering out noise, but it is difficult to eliminate coordinate points containing details.

[0068] It should be noted that the core of the standard Douglas-Pok algorithm is improved by applying the dynamic detail saliency score calculated in the last step to the decision-making process of the Douglas-Pok algorithm, and an adaptive distance threshold is constructed. In each step of the algorithm, the expectation of preserving details is integrated, so that the simplification behavior of the algorithm is targeted; finally, the adaptive threshold is applied to the standard Douglas-Pok algorithm process, which can complete the adaptation of the original trajectory and obtain an optimized result with purity and information richness.

[0069] Specifically, the Douglas-Pok algorithm is applied to the original motion trajectory sequence of the target joint , and through the adaptive distance threshold, it is determined whether each coordinate point is retained or not. For coordinate points : When determining whether to retain coordinate points , the perpendicular distance from coordinate point to the line segment connecting the key points and before and after it is calculated , and compared with the adaptive distance threshold corresponding to coordinate point . If , coordinate point is retained; otherwise, the coordinate point is removed. Repeat the judgment process until all coordinate points in the original motion trajectory sequence are processed, and an optimized trajectory sequence that is both pure and retains key details is obtained.

[0070] Among them, the first coordinate point and the last coordinate point in the original motion trajectory sequence are taken as key points and at the beginning; the subsequent key points are the retained coordinate points.

[0071] It should be noted that the obtained optimized trajectory sequence provides a high-quality data basis for subsequent accurate rehabilitation evaluation, achieving smooth denoising in a macroscopic sense and protecting valuable motion details in a microscopic sense.

[0072] S4: Through the least square method, the spatial coordinates of the target joint in the hand recovery test video are path-fitted, and the root mean square error between the obtained ideal motion trajectory sequence and the optimized trajectory sequence is calculated as the training stability, which is used to generate a visual rehabilitation report.

[0073] It should be noted that this step is the final application link of the method of the present application, and the purpose is to use the optimized trajectory data obtained in the previous steps to generate a quantitative evaluation report with practical guiding significance for clinical, so as to realize the closed loop of the whole monitoring method.

[0074] Firstly, for the continuous test image frames in the hand recovery test video, the spatial coordinates of the target joint in each frame of test image are extracted by the existing human pose estimation algorithm; the spatial coordinates of the target joint in all frames of test image are path fitted by the least square method, and a smooth parabolic segment representing the ideal path is obtained as the ideal motion trajectory sequence of the target joint.

[0075] Then, for any target joint: calculate the root mean square error of the optimized trajectory sequence and the ideal motion trajectory sequence, inversely proportionally normalize the root mean square error, and obtain the result as the training stability of the target joint; calculate the mean value of the training stability of all joints as the training stability of the patient; the inversely proportional normalization is realized by , wherein, is a natural exponential function.

[0076] Wherein, since the optimized trajectory sequence retains the real physiological tremor, the training stability index obtained can more accurately reflect the patient's neuromuscular control ability without being contaminated by algorithm noise.

[0077] Finally, the training stability obtained in each rehabilitation training is stored in association with the patient information, training time, etc., and a visual rehabilitation report is generated.

[0078] Exemplarily, the schematic diagram of the curves of the original motion trajectory sequence and the ideal motion trajectory sequence is shown in Figure 3 ; for the original motion trajectory sequence Figure 3 in , the processing result based on the global basic threshold through the standard DP algorithm is shown in Figure 4 , all fluctuations are roughly filtered out, although the overall trajectory is clean, the pathological tremor with diagnostic value is completely removed, and at the same time, due to the large global basic threshold, some macroscopic path deviation characteristics of the trajectory, i.e. instability, are also oversimplified; for the original motion trajectory sequence Figure 3 in , the processing result based on the adaptive distance threshold through the adaptive DP algorithm is shown in Figure 5As shown, most random noise is also filtered out, making the smooth part of the trajectory clean, and most importantly, in the latter half of the trajectory, due to the lower adaptive distance threshold in this region, the pathological tremor details are accurately preserved, and the macro-pathway deviation characteristics of the trajectory are also better preserved.

[0079] In summary, compared with the standard DP algorithm, the adaptive DP algorithm of the present application has better effect in processing complex rehabilitation patient movement trajectories, and realizes filtering out technical artifacts, i.e., noise, while identifying and preserving pathological details (such as tremor) which are crucial for rehabilitation assessment, thereby providing a high-quality data basis for subsequent generation of accurate and reliable rehabilitation assessment indicators.

Claims

1. A real-time monitoring method for human rehabilitation training for a rehabilitation device, characterized in that, The method comprises the following steps: Collecting monitoring videos when a hand disease patient imitates actions in a hand recovery test video; Extracting a plurality of joint nodes in each frame of image by using a human posture estimation algorithm, taking any joint node as a target joint node, combining spatial coordinates of the target joint node in a plurality of frames of image into an original motion trajectory sequence, and calculating instantaneous speed vectors and instantaneous acceleration vectors of the target joint node at each time point; According to the distribution of the instantaneous speed and the instantaneous acceleration in the local trajectory segment at each time point, a local motion energy factor is calculated; a local path fitting is performed on the local trajectory segment by using a least square method, a root mean square error between a local ideal trajectory obtained by calculation and the local trajectory segment is calculated, and a local trajectory stability is obtained; a dynamic detail saliency score is calculated according to the local motion energy factor, the local trajectory stability index and the local average speed, a global basic threshold is adjusted, an adaptive distance threshold used when judging whether the coordinate point of the target joint node at each time point is retained or not is obtained, and a Douglas-Puik algorithm is applied to process the original motion trajectory sequence to obtain an optimized trajectory sequence; The spatial coordinates of the target joint node in the hand recovery test video are path fitted by using a least square method, a root mean square error between an ideal motion trajectory sequence obtained by calculation and the optimized trajectory sequence is calculated, and a training stability of the patient is calculated, which is used to generate a visual rehabilitation report; According to the local motion energy factor, the local trajectory stability index and the local average speed, a dynamic detail saliency score is calculated, including: ; wherein is the dynamic detail saliency score of the target joint at time point ; is the local motion energy factor of the local trajectory segment of the target joint at time point ; is the local trajectory stability of the target joint at time point ; is the instantaneous velocity vector of the target joint at time point ; denotes the vector norm; is the mean of the norm of the instantaneous velocity vector of all time points in the local trajectory segment , representing the local average velocity at time point ; Adjust the global base threshold to obtain the adaptive distance threshold used to determine whether the coordinates of the target joint at each time point are retained, including: the distance threshold for the target joint at each time point. Coordinates of the corresponding frame image : Calculate the relationship between the number 1 and the target key point at time points Dynamic detail saliency score The difference Calculate the product of the difference and the global baseline threshold to obtain the judgment coordinate point. The adaptive distance threshold used when deciding whether to retain; The Douglas-Puk algorithm is applied to the original motion trajectory sequence of the target joints. The process involves using an adaptive distance threshold to determine whether to retain each coordinate point. : Determining whether to retain coordinate points When calculating coordinate points To the key points before and after and perpendicular distance of the connecting line segments and coordinate points Corresponding adaptive distance threshold If a comparison is made, Then retain the coordinates. Otherwise, remove the coordinate point; repeat the judgment process until the original motion trajectory sequence is obtained. After all coordinate points in the data have been processed, the optimized trajectory sequence is obtained. .

2. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The target joint node includes but is not limited to a right shoulder, a right elbow and a right wrist, and when the hand of the patient performing rehabilitation exercise is a left hand, the target joint node includes but is not limited to a left shoulder, a left elbow and a left wrist.

3. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The method for obtaining the local trajectory segment at each time point comprises the following steps: For a time point , a local time window is obtained , the local time window contains time points from the time point to the time point , wherein, is the half-width of the local time window , according to the local time window , a corresponding local trajectory segment is intercepted from the original motion trajectory sequence of the target joint point.

4. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The method for calculating the local motion energy factor according to the distribution of the instantaneous speed and the instantaneous acceleration in the local trajectory segment at each time point comprises the following steps: ； wherein is the local motion energy factor of the local trajectory segment of the target joint at time point ; is the half-width of the local time window ; , are the instantaneous velocity vector and the instantaneous acceleration vector of the target joint at time point , respectively; denotes the vector norm; is the standard deviation of the norm of the instantaneous acceleration vector of all time points in the local trajectory segment ; is the mean of the norm of the instantaneous acceleration vector of all time points in the local trajectory segment .

5. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The method for calculating the root mean square error between the local ideal trajectory obtained by calculation and the local trajectory segment to obtain the local trajectory stability comprises the following steps: Calculate local trajectory segments From each coordinate point in the local ideal trajectory The Euclidean distance between the corresponding projection points is calculated, and the root mean square of all Euclidean distances is taken as the root mean square error between the local ideal trajectory and the local trajectory segment. This root mean square error is then inversely normalized, and the result is used as the target joint point at time point. Local trajectory stability The inverse proportional normalization is achieved through... To achieve, among which, It is a natural exponential function.

6. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, Initially, the original motion trajectory sequence The first coordinate point and the last coordinate point in the middle are taken as key points And The subsequent key points are the reserved coordinate points.

7. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The method for calculating the training stability of the patient according to the root mean square error between the ideal motion trajectory sequence obtained by calculation and the optimized trajectory sequence comprises the following steps: For the target joint node: calculate the optimized trajectory sequence The root mean square error of the ideal motion trajectory sequence, inversely proportional normalization is performed on the root mean square error, and the obtained result is taken as the training stability of the target joint node; the inverse proportional normalization is realized by Implementation, wherein, is a natural exponential function; The mean value of the training stabilities of all joint nodes is calculated as the training stability of the patient.

8. A real-time monitoring method for human rehabilitation training for rehabilitation equipment according to claim 1, characterized in that, The method for calculating the instantaneous speed vectors and the instantaneous acceleration vectors of the target joint node at each time point comprises the following steps: instantaneous velocity vector , , are coordinate points in the corresponding frame image of the target joint at the time points , ; is the time interval for collecting the frame images; instantaneous acceleration vector , , are the instantaneous velocity vectors of the target joint at the time points , , respectively.

Images