Rehabilitation training action real-time analysis and feedback system based on machine vision

Abstract

The invention relates to the technical field of rehabilitation training, in particular to a rehabilitation training action real-time analysis and feedback system based on machine vision, and the system comprises an action collection module which is used for collecting rehabilitation training action data, carrying out the recognition of the rehabilitation training action data, and generating a posture model at each time point according to the key point of a human body posture; the action recognition module is used for recognizing action duration and action amplitude during rehabilitation training according to the posture model and the rehabilitation training action data, obtaining an action difference curve according to the action duration and the action amplitude, and analyzing the action difference curve to obtain a rehabilitation training result; obtaining an action displacement evaluation value related to the action displacement and an action association value related to the action increase amplitude; the action difference comparison module is used for comparing the rehabilitation training action data with the posture model to obtain a relative overlapping area and an effective difference area of the rehabilitation training action data at each time point; and the rehabilitation training effect and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a real-time analysis and feedback system for rehabilitation training movements based on machine vision. Background Art

[0002] Traditional rehabilitation training usually relies on the subjective evaluation of physical therapists and the self-perception of patients. This method has the following limitations: Lack of real-time feedback: patients cannot get timely feedback on whether the movements are correct during training; Inconsistent evaluation: different therapists may have different evaluations of the same movement, resulting in unstable rehabilitation effects; Inefficiency: Manually recording and analyzing rehabilitation data is time-consuming and labor-intensive, affecting the efficiency of rehabilitation training; Lack of personalization: It is difficult to customize the most optimized rehabilitation plan according to the specific situation of each patient.

[0003] For example, Chinese patent publication number CN115410690A discloses a rehabilitation training information management system and method, the system includes: a training strategy receiving module, used to receive a patient rehabilitation training strategy formulated by a physician; an instruction module, used to display the patient rehabilitation training strategy to the patient; a training data monitoring module, used to monitor the patient's training process according to the patient's rehabilitation training strategy and obtain monitoring data; a state monitoring module, used to collect the patient's physiological state parameters during the training process; an analysis module, used to obtain the training completion status based on the monitoring data analysis, obtain the patient's physical state based on the physiological state parameters, and adjust the patient's rehabilitation training strategy in real time according to the training completion status and the physical state.

[0004] Existing technologies tend to focus on evaluating the number of completed movements, but ignore the specific circumstances of each incomplete movement and whether there is any correlation between these completed movements, which makes it difficult to detect any flaws in the patient's rehabilitation training during the final rehabilitation training, resulting in reduced feedback effects of the rehabilitation training. Summary of the invention

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a real-time analysis and feedback system for rehabilitation training movements based on machine vision, including: a motion acquisition module, used to collect rehabilitation training movement data, identify the rehabilitation training movement data, and generate a posture model at each time point according to the key points of human posture.

[0006] The motion recognition module is used to identify the motion duration and motion amplitude during rehabilitation training according to the posture model and rehabilitation training motion data, obtain the motion difference curve according to the motion duration and motion amplitude, analyze the motion difference curve, and obtain the motion displacement evaluation value of the motion displacement and the motion association value of the motion growth amplitude.

[0007] The motion difference comparison module is used to compare the rehabilitation training motion data and the posture model to obtain the relative overlap area and effective difference area of ​​the rehabilitation training motion data at each time point; according to the relative overlap area and effective difference area at each time point, the motion difference bias of the rehabilitation training motion is determined, and the motion difference score is set according to the distribution of the motion difference bias.

[0008] The action verification module is used to determine the action interval, quantity and action frequency corresponding to the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion.

[0009] The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain a feedback evaluation result.

[0010] The beneficial effects of the present invention are: 1. The present invention uses machine vision technology to capture the patient's posture in real time and generate a posture model at each time point; by comparing the posture models, it can accurately capture every movement detail of the patient and reduce human errors; ensure that each movement is performed in accordance with the specifications and improve the rehabilitation effect.

[0011] 2. The present invention adopts a motion recognition module: it automatically recognizes parameters such as the duration and amplitude of rehabilitation training movements, and performs real-time analysis; then the output results of multiple modules are comprehensively calculated to provide instant feedback evaluation results. Patients can get feedback immediately, quickly correct wrong movements, and avoid the formation of bad habits.

[0012] 3. The present invention evaluates the overlap and difference areas of the movements, calculates the movement association value based on the movement growth amplitude and the movement duration difference ratio, helps identify which parts need special attention, and ensures that the rehabilitation training conforms to the predetermined pattern; the system can recommend personalized rehabilitation plans to improve the rehabilitation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0014] Figure 1 It is a system framework diagram of a real-time analysis and feedback system for rehabilitation training movements based on machine vision.

[0015] Figure 2 The system diagram is a real-time analysis and feedback system for rehabilitation training movements based on machine vision.

[0016] Figure 3 The present invention is a flow chart of the motion displacement evaluation value of a real-time analysis and feedback system for rehabilitation training motion based on machine vision.

[0017] Figure 4 The present invention is a flow chart of action correlation values ​​of a real-time analysis and feedback system for rehabilitation training actions based on machine vision. DETAILED DESCRIPTION

[0018] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. If no specific techniques or conditions are specified in the embodiments, the techniques or conditions described in the literature in the art or the product specifications are used.

[0019] See also Figure 1 , Figure 2 A real-time analysis and feedback system for rehabilitation training movements based on machine vision includes: a movement acquisition module, a movement recognition module, a movement difference comparison module, a movement verification module, and a feedback evaluation module; wherein the output end of the movement acquisition module is connected to the movement recognition module, the output end of the movement recognition module is connected to the movement difference comparison module, the output end of the movement difference comparison module is connected to the movement verification module, and the output end of the movement verification module is connected to the feedback evaluation module.

[0020] The motion acquisition module is used to collect rehabilitation training motion data, identify the rehabilitation training motion data, and generate a posture model at each time point according to the key points of the human body posture.

[0021] The motion recognition module is used to identify the motion duration and motion amplitude during rehabilitation training according to the posture model and rehabilitation training motion data, obtain the motion difference curve according to the motion duration and motion amplitude, analyze the motion difference curve, and obtain the motion displacement evaluation value of the motion displacement and the motion association value of the motion growth amplitude.

[0022] The motion difference comparison module is used to compare the rehabilitation training motion data and the posture model to obtain the relative overlap area and effective difference area of ​​the rehabilitation training motion data at each time point; according to the relative overlap area and effective difference area at each time point, the motion difference bias of the rehabilitation training motion is determined, and the motion difference score is set according to the distribution of the motion difference bias.

[0023] The action verification module is used to determine the action interval, quantity and action frequency corresponding to the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion.

[0024] The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain a feedback evaluation result.

[0025] In one embodiment of the present invention, a motion acquisition module is used to collect rehabilitation training motion data, obtain multiple key points of human posture from the rehabilitation training motion data, and generate a posture model at each time point according to the key points of the human posture; the rehabilitation training motion data can use a camera or other sensors, such as a depth sensor, an inertial measurement unit IMU, etc. to capture the patient's motion video or motion data during rehabilitation training, and these motion videos are identified according to the human posture appearing in the video, and the key points of the human posture at this time are obtained for processing to generate a posture model. The key points of the human posture will use the joint positions on the human body, or points on the main training parts under the corresponding rehabilitation training; these points will represent the main movement parts in the rehabilitation training, and the generated posture model represents the specific positions of these parts; then the rehabilitation training of the current person is verified through the posture model, and errors in the rehabilitation training movements are corrected. The posture model will represent the standard movements of the rehabilitation training, and these standard movements will be adjusted according to the needs of the current user to determine the posture model required by the current user, so that the user's movements can be reminded and evaluated.

[0026] The processing method of generating the posture model at each time point according to the key points of the human posture also includes obtaining the three-dimensional posture information and the target rehabilitation training action of the current user, determining the key points of the human posture of the current user according to the three-dimensional posture information and the target rehabilitation training action, and selecting the posture model corresponding to the current target rehabilitation training action according to the moving position of the key points of the human posture at each time point. At this time, the specific content of the rehabilitation training action is essentially determined according to the three-dimensional posture, and the posture model is selected according to the moving position at each time. The selected posture model will compare the key points identified at this time, and perform real-time analysis and feedback on the rehabilitation training action based on the content of these key points.

[0027] In one embodiment of the present invention, the motion recognition module is used to identify the action duration and action amplitude required for the rehabilitation training action under the action corresponding to the posture model based on the posture model and the rehabilitation training action data, and the obtained action duration and action amplitude are combined into a motion difference curve in the form of time points, and finally the values ​​when the action displacement and action amplitude change are obtained.

[0028] In the motion recognition module, the rehabilitation training movements are mainly decomposed into each individual movement, and the time from the start to the completion of each movement, as well as the displacement of the key points on the body trunk when the movement is completed are used as the movement amplitude recognized at this time. The value of the movement amplitude when a single movement is ongoing is judged, as well as the change in the movement duration when the movement amplitude value changes.

[0029] For example, when performing leg rehabilitation, rehabilitation training will be carried out for leg movements according to a certain trajectory. If a movement with a large amplitude is recognized at this time, if the user hears the posture model's reminder of this movement and then makes a corrective movement, the time will be recorded to determine whether the current user can perform this movement and how long it will take the user to reach a more standard rehabilitation movement during adjustment. The movement amplitude and time obtained at this time can show the user's own rehabilitation situation. The movement amplitude here will be recorded as the movement growth amplitude because of the numerical change, and the corresponding time will also be used to compare with the standard time to obtain the difference ratio of the movement duration. The standard time can be set according to the standard value of the current user completing a rehabilitation training movement.

[0030] like Figure 3 As shown, the implementation method of the action displacement evaluation value includes: setting the marking points of the rehabilitation training action according to the key points of the human body posture; identifying the displacement value and angle value of the marking points in the rehabilitation training action data under the corresponding action amplitude; the marking points set at this time are used to mark the key points of the current human body posture, and these key points are regarded as the marking points used at this time after setting the labels, for example, according to the position on the human body posture and the direction of the currently identified coordinate axis, the horizontal direction of the current action collection is set as the X-axis, the horizontal direction perpendicular to the X-axis is regarded as the Y-axis, and the vertical direction perpendicular to the ground is regarded as the Z-axis to determine the coordinate axis corresponding to the current marking point; at the same time, the recognized displacement value represents the distance moved in this coordinate axis, and this distance will be decomposed into distance values ​​in three directions according to the current calculation needs during calculation. The same is true for the angle value. At this time, in addition to identifying the size of the angle value, it is also necessary to judge the speed of the angle value increase and decrease, as well as the time when the current rehabilitation action completes a standard action, to obtain the specific situation of the corresponding action amplitude.

[0031] According to the displacement value and angle value of the marked point under the corresponding movement amplitude, the movement difference curve under different movement durations is determined. The movement difference curve is used to compare the difference between the completed position and the preset position of these movements when the required movement duration of the current rehabilitation movement changes, and whether the difference in the length of rehabilitation training time will affect the smoothness of the change of the current displacement value and angle value, thereby reducing the overall effect of rehabilitation training.

[0032] The action difference curves were analyzed to calculate the displacement difference, angle overlap ratio, and angle overlap period of adjacent actions during the current rehabilitation training. The displacement difference represents the sum of the displacement values ​​between the markers on adjacent actions, which reflects the amount of change in a part of the body moving from one state to another when the patient performs continuous actions.

[0033] The angle overlap ratio measures the similarity between the angle ranges of the same joint in two adjacent movements. Specifically, it is the ratio of the intersection to the union of two angle intervals; that is, the overlap ratio between the angle ranges of the same joint when performing consecutive rehabilitation movements.

[0034] Angle overlap period refers to the period of time when the angle change curves of two adjacent actions overlap on the time axis. In other words, it refers to the length of time that two actions reach similar angles in the same time period.

[0035] Generally, when performing rehabilitation training, one or more movements will be performed multiple times along a fixed trajectory. At this time, the identified displacement difference, angle overlap ratio and angle overlap period will indicate the actual execution of the rehabilitation training, and whether the patient can accurately master the correct rehabilitation training movements in this case.

[0036] According to the displacement difference, the angle overlap ratio, and the angle overlap period, a motion displacement evaluation value regarding the motion displacement is set.

[0037] The action displacement evaluation value is expressed as: Among them, E VA represents the movement displacement evaluation value, n represents the number of rehabilitation training movements, and the value range of i is 1 to n; D i represents the displacement difference of the i-th rehabilitation training action, θ overlap,i represents the angle overlap ratio of the i-th rehabilitation training action, t A,i represents the length of the angle overlap period of the i-th rehabilitation training action, D′ represents the standard value of the displacement difference, θ′ overlap Indicates the standard value of the angle overlap ratio, t′ A,i represents the standard action duration of the i-th rehabilitation training action, and the standard action duration represents the average value set in the historical data corresponding to a single rehabilitation training action; t′ A represents the average value of standard movement duration, and the average value of standard movement duration represents the average value of standard movement duration of all rehabilitation training movements; C represents a constant term, which is used to ensure that there is at least one minimum score even when all other terms are zero.

[0038] After completing the setting of the action displacement evaluation value, it is also necessary to determine the action increase amplitude when the action amplitude increases, and the action duration difference ratio corresponding to the action increase amplitude; identify the relative relationship between the action increase amplitude and the action duration difference ratio, so as to complete the setting of the action basic score; the relative relationship between the action increase amplitude and the action duration difference ratio can be expressed as the part association frequency between different parts of the corresponding torso position, and set it according to this part association frequency and the combined probability of the corresponding values ​​appearing in these parts, so as to obtain the action association value.

[0039] like Figure 4 As shown in the figure, the action association value is calculated by obtaining the action growth amplitude and the action duration difference ratio corresponding to the action growth amplitude; according to the key points of the human body posture, the distribution position of the action growth amplitude and the action duration difference ratio during rehabilitation action training is determined. The distribution position here indicates where these values ​​appear on the human body during rehabilitation training, and the values ​​recorded at these positions are read to obtain the distribution position of the action growth amplitude and the action duration difference ratio.

[0040] According to the distribution position of the difference ratio of the action growth amplitude and the action duration during rehabilitation training, the part frequent association between the difference ratio of the action growth amplitude and the action duration and different parts of the trunk position is determined; and based on the part frequent association, the action association value is calculated. At this time, the part frequent association indicates which part of the rehabilitation exercise is mainly manifested when there is a large difference between the action growth amplitude and the action duration, so as to identify the current user's rehabilitation exercise in which places is prone to large errors, and the specific manifestation of the error value corresponding to these situations. According to these contents, it can be known that the current rehabilitation training needs to focus on the content of identification, so as to realize the feedback analysis of the user. For example, when performing rehabilitation training similar to the legs or hands, there are multiple parts in the rehabilitation training action at this time, and key points are set for these parts. Then, according to the values ​​of the difference ratio of the action growth amplitude and the action duration of these key points in the rehabilitation training action, the part frequent association between the body parts corresponding to the rehabilitation training action is set. This value will be comprehensively calculated according to the values ​​of the difference ratio of the action growth amplitude and the action duration on different parts, so as to obtain the part frequent association corresponding to a rehabilitation training action, and finally complete the setting of the action association value of the rehabilitation training action.

[0041] The frequent correlation degree of parts can be set to, Among them, F i represents the frequent association degree of the parts of the i-th rehabilitation training action, N represents the number of parts, j ranges from 1 to N, and the number of parts represents the number of joints that mainly move in a rehabilitation action; P(A ij) represents the probability value of the increase in the movement amplitude of the jth part in the i-th rehabilitation training action, P(T ij ) represents the probability value of the difference ratio of the action duration of the jth part in the i-th rehabilitation training action, I(P(A ij ),P(T ij )) indicates that the indicator function is an indicator function set on the corresponding part based on the difference ratio of movement increase amplitude and movement duration, which expresses the corresponding correlation of different parts during rehabilitation training.

[0042]

[0043] P′(T) represents the standard value of the probability value of the action duration difference ratio. The standard value set at this time is set according to the average value of the probability value of the action duration difference ratio in the historical data. P′(A) represents the standard value of the probability value of the action growth amplitude. This standard value is the average value of the probability value of the action growth amplitude in the historical data.

[0044] The action association value is comprehensively set according to the frequent association degree of the parts. For example, the action association value is combined according to the weight of each rehabilitation training action to reach the final action association value.

[0045] Among them, E R Indicates the action associated value, w i Represents the weight of the i-th rehabilitation training action. The final action association value is an association value of different rehabilitation training actions relative to the patient's trunk, indicating the correlation between the corresponding actions of the trunk. According to this value, the effect of the rehabilitation training action in the overall process can be judged, as well as the parts that need to be marked, so as to facilitate the subsequent adjustment of the patient's rehabilitation training by the doctor.

[0046] In one embodiment of the present invention, the action difference comparison module is used to compare the rehabilitation training action data and the posture model, determine the relative overlap area and the effective difference area at each time point; according to the distribution of the relative overlap area and the effective difference area at the corresponding time point, find the action difference bias related to the relative overlap area and the effective difference area, and set the action difference score according to the distribution of the action difference bias. The action difference score represents the difference in action between the specific action of the current rehabilitation training action data and the posture model, and the action score set for the difference, which is the specific situation of the current rehabilitation training action.

[0047] The above-mentioned relative overlapping area represents the overlapping area between the rehabilitation training action and the corresponding action under the posture model at the corresponding time point, and the effective difference area represents the difference in area value between the rehabilitation training action and the corresponding action under the posture model. This part is generally the area after the intersection of the area between the current rehabilitation training action and the action in the posture model minus the union; by determining the combination of these two areas, the main differences in the actions can be clearly found, and combined with the distribution of different actions under the corresponding marking points in the action recognition module, the direction of the current action difference can be clearly identified, and these directions are used as the main processing objects at this time to complete the analysis of the action differences.

[0048] When setting the action difference score, the action difference comparison module is specifically implemented as follows: compare the rehabilitation training action at each time point with the posture model, determine the overlapping area between the rehabilitation training action and the corresponding action in the posture model, and use it as the relative overlapping area at each time point; determine the difference area between the rehabilitation training action and the posture model at each time point according to the relative overlapping area at each time point, and use the area of ​​the difference area as the effective difference area at each time point; the effective difference area at this time is essentially the intersection of the area of ​​the rehabilitation training action and the corresponding action in the posture model minus the relative overlapping area; calculate the effective difference area according to the effective difference area and the relative overlap area. Area, identify the distribution between the rehabilitation training action and the relative overlap area and the effective difference area, regard the area deviation and direction deviation generated by the rehabilitation training action and the corresponding action in the posture model at different parts as the action difference bias, divide the action difference bias into multiple difference points, and set the index value of the action difference bias according to the probability of the action difference bias taking values ​​in the corresponding direction and area; normalize the relative overlap area and the effective difference area to obtain the index value of the relative overlap area and the index value of the effective difference area; calculate the action difference score according to the index value of the action difference bias, the index value of the relative overlap area and the index value of the effective difference area.

[0049] The above-mentioned action difference bias specifically describes the spatial deviation and directionality between the actual action and the posture model. It not only reflects the area difference of the action on different parts, that is, the relative overlap area and the effective difference area; it also points out which parts of the body these differences mainly occur in and their spatial distribution trends;

[0050] At this time, the action difference bias will determine the deviation area between the rehabilitation training action and the corresponding action in the posture model through the effective difference area and the relative overlap area, and calculate the distance of the deviation area as the area deviation of the action difference bias. This distance is set by calculating the Euclidean distance. At this time, there are multiple key points on the corresponding action in the rehabilitation training action and the posture model. These key points will represent the distance difference between the deviation areas in the corresponding space. At the same time, the sum of the distance differences in the deviation areas represented by these actions is used as the area deviation at this time. The deviation area is transformed into a vector, the direction of the transformed vector is identified, and the change value of the vector direction is used as the direction deviation of the action difference bias. At this time, the deviation area will be converted into a vector form to determine the direction in which the rehabilitation training action produces a deviation, and the difference of the vector represented by this direction is used as the direction deviation at this time. Finally, the probability value corresponding to the area deviation and direction deviation of the action difference bias is set as the index value of the action bias. This probability value represents the probability of the area deviation and direction deviation occurring together.

[0051] Therefore, the action difference score is expressed as follows: the index value of the action difference bias, the index value of the relative overlap area, and the index value of the effective difference area are obtained, and the action difference score is calculated: Among them, E d represents the action difference score, O t Indicates the index value of relative overlap area, D t The index value representing the effective difference area, D t,max The maximum value of the index value representing the effective difference area, B t The index value representing the action difference bias, w O Represents the weight of the relative overlap area, w D represents the weight of the effective difference area, w B Represents the weight of the action difference bias; the weights of the action difference bias, relative overlap area and effective difference area can be set to 0.4, 0.3 and 0.3 respectively; if it is necessary to count the differences in the relative overlap area and the effective difference area at multiple time points, the action difference scores obtained at this time are summed up and the average is taken to get the corresponding action difference score. At the same time, when counting the action difference scores at multiple time points, the weights of the corresponding action difference bias, relative overlap area and effective difference area are expressed as the ratio of the index values ​​of the action difference bias, relative overlap area and effective difference area to the corresponding total data, so as to get the action difference score.

[0052] In one embodiment of the present invention, the action verification module is used to obtain the action interval, quantity and action frequency corresponding to the rehabilitation training action in the rehabilitation training mode, and judge whether the current rehabilitation training action is completed based on these data to obtain the action completion score for the completion of the action. The action completion score represents the overall completion of the rehabilitation training action data, and the patient is evaluated according to this completion status.

[0053] The action completion score is expressed as, Among them, E S It represents the action completion score, It represents the action interval, It′ represents the expected value of the action interval, and the expected value represents the average value of the action interval in normal rehabilitation training. This value can be obtained from the historical data of the corresponding rehabilitation training action; It max Indicates the maximum value allowed for the action interval. This maximum value indicates the maximum time interval allowed when completing a rehabilitation training action. This value can be set from the maximum value of the corresponding action interval that appears in the historical data; n indicates the number of rehabilitation training actions, n tar represents the target number of rehabilitation training actions, which indicates how many actions are needed to normally complete a set of rehabilitation training; f represents the action frequency, f tar The target value of the action frequency indicates the frequency of normal completion of rehabilitation training, which can be set from the average value of normal completion of rehabilitation training in historical data; w I represents the weight of the action interval, w n represents the weight of the number of rehabilitation training actions, w f Represents the weight of action frequency. The weights of action interval, quantity, and action frequency are set to 0.3, 0.4, and 0.3 respectively.

[0054] In one embodiment of the present invention, the feedback evaluation module mainly comprehensively processes multiple scores calculated by the action recognition module, the action difference comparison module and the action association module to verify the patient's main movement labels and specific training levels during rehabilitation training. These can more comprehensively reflect whether each action of the current patient during rehabilitation training is completed accurately, and the form of the unfinished part, so as to identify what form of movement deviation the current patient has, and need to use the system for voice prompts or final evaluation prompts to assist the patient in rehabilitation training.

[0055] The feedback evaluation result is realized by obtaining the action completion score, action difference score, action association value and action displacement evaluation value, calculating the feedback evaluation score, and outputting the feedback evaluation score as the feedback evaluation result. According to the obtained feedback evaluation result, the specific performance of the current patient's rehabilitation training action is judged, and the corresponding output is transmitted to the external terminal.

[0056] Among them, E fin Denotes the feedback evaluation score, E S Indicates the action completion score, E d represents the action difference score, E R Indicates the action associated value, E VA represents the action displacement evaluation value; represents the weight of the action completion score, represents the weight of the action difference score, represents the weight of the action associated value, Representing the weight of the action displacement evaluation value, the weights of the action completion score, action difference score, action association value and action displacement evaluation value are expressed in sequence as 0.2, 0.3, 0.3, 0.2; or the average value of the weights of the action completion score, action difference score, action association value and action displacement evaluation value in historical data is used as the weight used at this time.

[0057] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention and they are still covered by the protection scope of the present invention.

Claims

1. A real-time analysis and feedback system for rehabilitation training based on machine vision, characterized in that: include: The motion collection module is used to collect rehabilitation training motion data, identify the rehabilitation training motion data, and generate a posture model at each time point according to the key points of human posture; The motion recognition module is used to identify the motion duration and motion amplitude during rehabilitation training according to the posture model and the rehabilitation training motion data, obtain the motion difference curve according to the motion duration and motion amplitude, analyze the motion difference curve, and obtain the motion displacement evaluation value of the motion displacement and the motion association value of the motion growth amplitude; The action difference comparison module is used to compare the rehabilitation training action data and the posture model, and obtain the relative overlap area and effective difference area of ​​the rehabilitation training action data at each time point; determine the action difference bias of the rehabilitation training action according to the relative overlap area and effective difference area at each time point, and set the action difference score according to the distribution of the action difference bias; An action verification module is used to determine the action interval, quantity and action frequency corresponding to the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion; The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain a feedback evaluation result.

2. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The processing method of generating a posture model at each time point according to the key points of human posture also includes: obtaining the three-dimensional posture information and target rehabilitation training movements of the current user, determining the key points of the human posture of the current user according to the three-dimensional posture information and the target rehabilitation training movements, and selecting the posture model corresponding to the current target rehabilitation training movement according to the moving positions of the key points of the human posture at each time point.

3. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The implementation methods of the motion displacement evaluation value include: According to the key points of human body posture, set the marking points of rehabilitation training action; identify the displacement value and angle value of the marking points in the rehabilitation training action data under the corresponding action amplitude; According to the displacement value and angle value of the marked point under the corresponding action amplitude, the action difference curve under different action durations is determined; Analyze the action difference curve and calculate the displacement difference, angle overlap ratio and angle overlap period of adjacent actions during the current rehabilitation training; According to the displacement difference, the angle overlap ratio, and the angle overlap period, a motion displacement evaluation value regarding the motion displacement is set.

4. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 3 is characterized in that: The action displacement evaluation value is expressed as: Among them, E VA represents the movement displacement evaluation value, n represents the number of rehabilitation training movements, and the value range of i is 1 to n; D i represents the displacement difference of the i-th rehabilitation training action, θ overlap,i represents the angle overlap ratio of the i-th rehabilitation training action, t A,i represents the length of the angle overlap period of the i-th rehabilitation training action, D′ represents the standard value of the displacement difference, θ′ overlap Indicates the standard value of the angle overlap ratio, t′ A,i represents the standard action duration of the i-th rehabilitation training action, t′ A represents the average value of the standard action duration, and C represents a constant term.

5. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 4, characterized in that: The action association value is calculated as follows: Obtain the movement increase amplitude and the movement duration difference ratio corresponding to the movement increase amplitude; determine the distribution position of the movement increase amplitude and the movement duration difference ratio during rehabilitation movement training according to the key points of human body posture; According to the distribution of the difference ratio of movement increase amplitude and movement duration during rehabilitation movement training, the part frequency correlation between the difference ratio of movement increase amplitude and movement duration and different parts of the trunk position is determined; and based on the part frequency correlation, the action correlation value is calculated.

6. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 5, characterized in that: The frequent correlation degree of parts is set as: Among them, F i represents the frequent association degree of the parts of the i-th rehabilitation training action, N represents the number of parts, j ranges from 1 to N, P(A ij ) represents the probability value of the increase in the movement amplitude of the jth part in the i-th rehabilitation training action, P(T ij ) represents the probability value of the difference ratio of the action duration of the jth part in the i-th rehabilitation training action, I(P(A ij ),P(T ij )) represents the indicator function; The action associated value is expressed as: Among them, E R Indicates the action associated value, w i represents the weight of the i-th rehabilitation training action.

7. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The implementation of the action difference comparison module is as follows: The rehabilitation training movements at each time point are compared with the posture model, and the overlapping area between the rehabilitation training movements and the corresponding movements in the posture model is determined as the relative overlapping area at each time point; according to the relative overlapping area at each time point, the difference area between the rehabilitation training movements and the posture model at each time point is determined, and the area of ​​the difference area is taken as the effective difference area at each time point; according to the effective difference area and the relative overlapping area, the distribution between the rehabilitation training movements and the relative overlapping area and the effective difference area is identified, and the area deviation and direction deviation generated by the rehabilitation training movements and the corresponding movements in the posture model at different parts are regarded as the movement difference bias, and the index value of the movement difference bias is set according to the probability of the movement difference bias taking values ​​in the corresponding direction and area; the relative overlapping area and the effective difference area are normalized to obtain the index value of the relative overlapping area and the index value of the effective difference area; according to the index value of the movement difference bias, the index value of the relative overlapping area and the index value of the effective difference area, the movement difference score is calculated.

8. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 7, characterized in that: The action difference score is expressed as: Among them, E d represents the action difference score, O t Indicates the index value of relative overlap area, D t The index value representing the effective difference area, D t,max The maximum value of the index value representing the effective difference area, B t The index value representing the action difference bias, w O Represents the weight of the relative overlap area, w D represents the weight of the effective difference area, w B The weight representing the action difference bias.

9. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The action completion score is expressed as: Among them, E S represents the action completion score, It represents the action interval, It′ represents the expected value of the action interval, and It max represents the maximum value allowed for the action interval, n represents the number of rehabilitation training actions, tar represents the target number of rehabilitation training actions, f represents the action frequency, and f tar represents the target value of the action frequency, w I represents the weight of the action interval, w n represents the weight of the number of rehabilitation training actions, w f Represents the weight of the action frequency.

10. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The feedback evaluation result is realized by obtaining the action completion score, the action difference score, the action association value and the action displacement evaluation value, calculating the feedback evaluation score, and outputting the feedback evaluation score as the feedback evaluation result; Among them, E fin Denotes the feedback evaluation score, E S Indicates the score of action completion, E d represents the action difference score, E R Indicates the action associated value, E VA represents the action displacement evaluation value; represents the weight of the action completion score, represents the weight of the action difference score, represents the weight of the action associated value, Represents the weight of the action displacement evaluation value.

Images