A real-time analysis and feedback system for rehabilitation training movements based on machine vision

Abstract

The present invention relates to the technical field of rehabilitation training, and specifically to a real-time analysis and feedback system for rehabilitation training movements based on machine vision. The system comprises: a movement acquisition module, for collecting rehabilitation training movement data, identifying the rehabilitation training movement data, and generating a posture model at each time point according to key points of human posture; a movement recognition module, for identifying movement duration and movement amplitude during rehabilitation training according to the posture model and rehabilitation training movement data, obtaining a movement difference curve according to the movement duration and movement amplitude, analyzing the movement difference curve to obtain a movement displacement evaluation value for movement displacement and a movement correlation value for movement growth amplitude; and a movement difference comparison module, for comparing the rehabilitation training movement data with the posture model, and obtaining a relative overlap area and an effective difference area of ​​the rehabilitation training movement data at each time point, thereby improving the effect and efficiency of rehabilitation training.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a real-time analysis and feedback system for rehabilitation training movements based on machine vision. Background Art

[0002] Traditional rehabilitation training usually relies on the subjective assessment of physical therapists and the patient's self-perception. This method has the following limitations: lack of real-time feedback: patients cannot get timely feedback on whether the movements are correct during training; inconsistent assessment: different therapists may have different evaluations of the same movement, resulting in unstable rehabilitation effects; inefficiency: manual recording and analysis of rehabilitation data is time-consuming and labor-intensive, affecting the efficiency of rehabilitation training; lack of personalization: it is difficult to customize the optimal rehabilitation plan according to the specific situation of each patient.

[0003] For example, Chinese patent publication number CN115410690A discloses a rehabilitation training information management system and method, which includes: a training strategy receiving module for receiving a patient rehabilitation training strategy formulated by a physician; an instruction module for displaying the patient rehabilitation training strategy to the patient; a training data monitoring module for monitoring the patient's training process according to the patient's rehabilitation training strategy and obtaining monitoring data; a state monitoring module for collecting the patient's physiological state parameters during the training process; an analysis module for obtaining the training completion status based on the monitoring data analysis, obtaining the patient's physical state based on the physiological state parameters, and adjusting the patient's rehabilitation training strategy in real time based on the training completion status and physical state.

[0004] Existing technologies tend to focus on evaluating the number of completed movements, but ignore the specific circumstances of each incomplete movement and whether there is any correlation between these completed movements. This makes it difficult to detect any flaws in the patient's rehabilitation training during the final rehabilitation training, resulting in reduced feedback effects from the rehabilitation training. Summary of the Invention

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a real-time analysis and feedback system for rehabilitation training movements based on machine vision, including: a motion acquisition module for collecting rehabilitation training movement data, identifying the rehabilitation training movement data, and generating a posture model at each time point according to the key points of human posture.

[0006] The motion recognition module is used to identify the duration and amplitude of movements during rehabilitation training based on the posture model and rehabilitation training motion data, obtain the motion difference curve based on the duration and amplitude of the movement, analyze the motion difference curve, and obtain the motion displacement evaluation value of the motion displacement and the motion association value of the motion growth amplitude.

[0007] The movement difference comparison module is used to compare the rehabilitation training movement data and the posture model to obtain the relative overlap area and effective difference area of ​​the rehabilitation training movement data at each time point; based on the relative overlap area and effective difference area at each time point, the movement difference bias of the rehabilitation training movement is determined, and the movement difference score is set according to the distribution of the movement difference bias.

[0008] The action verification module is used to determine the action interval, quantity and action frequency corresponding to the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion.

[0009] The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain feedback evaluation results.

[0010] The beneficial effects of the present invention are: 1. The present invention uses machine vision technology to capture the patient's posture in real time and generate a posture model at each time point; by comparing the posture models, it can accurately capture every detail of the patient's movement and reduce human errors; ensure that each movement is performed in accordance with the specifications, thereby improving the rehabilitation effect.

[0011] 2. The present invention adopts a motion recognition module: it automatically identifies parameters such as the duration and amplitude of rehabilitation training movements and performs real-time analysis; it then comprehensively calculates the output results of multiple modules to provide instant feedback evaluation results. Patients can get feedback immediately, quickly correct incorrect movements, and avoid the formation of bad habits.

[0012] 3. The present invention evaluates the overlap and difference areas of movements, calculates the movement association value based on the movement growth amplitude and movement duration difference ratio, helps identify which parts need special attention, and ensures that rehabilitation training conforms to the predetermined pattern; the system can recommend personalized rehabilitation plans to improve rehabilitation effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present invention will be further described below with reference to the accompanying drawings and examples.

[0014] Figure 1 This is a system framework diagram of a real-time analysis and feedback system for rehabilitation training movements based on machine vision.

[0015] Figure 2 The system diagram is a real-time analysis and feedback system for rehabilitation training movements based on machine vision.

[0016] Figure 3 The present invention is a flowchart of the motion displacement evaluation value of a real-time analysis and feedback system for rehabilitation training motion based on machine vision.

[0017] Figure 4 The present invention is a flow chart of the action correlation value of a real-time analysis and feedback system for rehabilitation training actions based on machine vision. DETAILED DESCRIPTION

[0018] The following embodiments of the present invention are described in detail. The embodiments described below are exemplary and are only used to explain the present invention, and are not to be construed as limiting the present invention. Where specific techniques or conditions are not specified in the embodiments, the techniques or conditions described in the literature in the art or in the product specifications shall be followed.

[0019] See Figure 1 、 Figure 2 A real-time analysis and feedback system for rehabilitation training movements based on machine vision includes: a movement acquisition module, a movement recognition module, a movement difference comparison module, a movement verification module, and a feedback evaluation module; wherein the output end of the movement acquisition module is connected to the movement recognition module, the output end of the movement recognition module is connected to the movement difference comparison module, the output end of the movement difference comparison module is connected to the movement verification module, and the output end of the movement verification module is connected to the feedback evaluation module.

[0020] The motion acquisition module is used to collect rehabilitation training motion data, identify the rehabilitation training motion data, and generate a posture model at each time point according to the key points of the human body posture.

[0021] The motion recognition module is used to identify the duration and amplitude of movements during rehabilitation training based on the posture model and rehabilitation training motion data, obtain the motion difference curve based on the duration and amplitude of the movement, analyze the motion difference curve, and obtain the motion displacement evaluation value of the motion displacement and the motion association value of the motion growth amplitude.

[0022] The movement difference comparison module is used to compare the rehabilitation training movement data and the posture model to obtain the relative overlap area and effective difference area of ​​the rehabilitation training movement data at each time point; based on the relative overlap area and effective difference area at each time point, the movement difference bias of the rehabilitation training movement is determined, and the movement difference score is set according to the distribution of the movement difference bias.

[0023] The action verification module is used to determine the action interval, quantity and action frequency corresponding to the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion.

[0024] The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain feedback evaluation results.

[0025] In one embodiment of the present invention, a motion acquisition module is used to collect rehabilitation training motion data, obtain multiple key points of human posture from the rehabilitation training motion data, and generate a posture model at each time point according to the key points of the human posture; the rehabilitation training motion data can use a camera or other sensors, such as a depth sensor, an inertial measurement unit (IMU), etc. to capture the patient's motion video or motion data during rehabilitation training, and these motion videos are identified according to the human posture appearing in the video, and the key points of the human posture at this time are obtained for processing to generate a posture model. The key points of the human posture will use the joint positions on the human body, or points on the main training parts corresponding to the rehabilitation training; these points will represent the main movement parts in the rehabilitation training, and the generated posture model represents the specific positions of these parts; then the rehabilitation training of the current person is verified through the posture model, and errors in the rehabilitation training movements are corrected. The posture model will represent the standard movements of the rehabilitation training, and these standard movements will be adjusted according to the needs of the current user to determine the posture model required by the current user, so that the user's movements can be reminded and evaluated.

[0026] The process of generating a posture model at each time point based on the key points of human posture also includes obtaining the current user's 3D posture information and the target rehabilitation training movement, determining the key points of the current user's posture based on the 3D posture information and the target rehabilitation training movement, and selecting the posture model corresponding to the current target rehabilitation training movement based on the movement position of the key points of the human posture at each time point. In essence, the specific content of the rehabilitation training movement is determined based on the 3D posture, and the posture model is selected based on the movement position at each time point. The selected posture model compares the key points identified at that time and provides real-time analysis and feedback on the rehabilitation training movement based on the content of these key points.

[0027] In one embodiment of the present invention, the motion recognition module is used to identify the action duration and action amplitude required for the rehabilitation training action under the action corresponding to the posture model based on the posture model and the rehabilitation training action data, and the obtained action duration and action amplitude are combined into a motion difference curve in the form of time points, and finally the values ​​when the action displacement and action amplitude change are obtained.

[0028] In the motion recognition module, the rehabilitation training movements are mainly decomposed into each individual movement, and the time from the start to the completion of each movement, as well as the displacement of the key points on the body trunk when the movement is completed, are used as the movement amplitude recognized at this time. The value of the movement amplitude when a single movement is in progress is judged, as well as how much the movement duration changes when the movement amplitude value changes.

[0029] For example, when conducting leg rehabilitation, rehabilitation training will be carried out for leg movements according to a certain trajectory. If a movement with a large amplitude is recognized at this time, if the user hears the posture model's reminder of this movement and then makes a corrective movement, the time will be recorded to determine whether the current user can perform this movement and how long it will take the user to reach a more standard rehabilitation movement during adjustment. The movement amplitude and time obtained at this time can show the user's own rehabilitation status. The movement amplitude here will be recorded as the movement growth amplitude due to the numerical change, and the corresponding time will also be used to compare with the standard time to obtain the difference ratio of the movement duration. The standard time can be set according to the standard value of the current user completing a rehabilitation training movement.

[0030] like Figure 3 As shown, the implementation method of the motion displacement evaluation value includes: setting the marking points of the rehabilitation training action according to the key points of the human body posture; identifying the displacement value and angle value of the marking point in the rehabilitation training action data under the corresponding action amplitude; the marking points set at this time are used to mark the key points of the current human body posture, and these key points are regarded as the marking points used at this time after setting the labels, for example, according to the position on the human body posture and the direction of the currently identified coordinate axis, the horizontal direction of the current action collection is set as the X-axis, the horizontal direction perpendicular to the X-axis is regarded as the Y-axis, and the vertical direction perpendicular to the ground is regarded as the Z-axis to determine the coordinate axis corresponding to the current marking point; at the same time, the identified displacement value represents the distance moved within this coordinate axis, and this distance will be decomposed into distance values ​​in three directions according to the current calculation needs during calculation. The same is true for the angle value. At this time, in addition to identifying the size of the angle value, it is also necessary to judge the speed of the angle value increase and decrease, as well as the time when the current rehabilitation action completes a standard action, to obtain the specific situation of the corresponding action amplitude.

[0031] According to the displacement and angle values ​​of the marked points under the corresponding movement amplitude, the movement difference curves under different movement durations are determined. The movement difference curves are used to compare the differences between the completed positions and the preset positions of the current rehabilitation movements when the required movement duration changes, and whether the different lengths of rehabilitation training time will affect the smoothness of the changes in the current displacement and angle values, thereby reducing the overall effect of rehabilitation training.

[0032] The movement difference curves were analyzed to calculate the displacement difference, angular overlap ratio, and angular overlap period for adjacent movements during the current rehabilitation training. The displacement difference represents the sum of the displacement values ​​between the markers on adjacent movements and reflects the change in the patient's body part from one state to another during the continuous movement.

[0033] The angular overlap ratio measures the similarity between the angular ranges of the same joint in two adjacent movements. Specifically, it is the ratio of the intersection to the union of two angular ranges; in other words, the overlap ratio between the angular ranges of the same joint during consecutive rehabilitation movements.

[0034] Angle overlap refers to the time period when the angle change curves of two adjacent actions overlap on the time axis. In other words, it refers to the length of time that two actions reach similar angles within the same time period.

[0035] Generally, when performing rehabilitation training, one or more movements will be practiced multiple times along a fixed trajectory. The identified displacement difference, angle overlap ratio, and angle overlap period will indicate the actual execution of the rehabilitation training and whether the patient can accurately master the correct rehabilitation training movements in this case.

[0036] According to the displacement difference, the angle overlap ratio, and the angle overlap period, a motion displacement evaluation value regarding the motion displacement is set.

[0037] The motion displacement evaluation value is expressed as: Among them, E VA represents the movement displacement evaluation value, n represents the number of rehabilitation training movements, and the value range of i is 1 to n; D i represents the displacement difference of the i-th rehabilitation training action, θ overlap,i represents the angle overlap ratio of the i-th rehabilitation training action, t A,i represents the length of the angle overlap period of the i-th rehabilitation training action, D′ represents the standard value of the displacement difference, θ′ overlap Indicates the standard value of the angle overlap ratio, t′ A,i represents the standard action duration of the i-th rehabilitation training action, and the standard action duration represents the average value set in the historical data of the corresponding single rehabilitation training action; t′ A represents the average value of the standard movement duration, and the average value of the standard movement duration represents the average value of the standard movement duration of all rehabilitation training movements; C represents a constant term, which is used to ensure that there is at least one minimum score even when all other terms are zero.

[0038] After completing the setting of the action displacement evaluation value, it is also necessary to determine the action growth amplitude when the action amplitude increases, and the action duration difference ratio corresponding to the action growth amplitude; identify the relative relationship between the action growth amplitude and the action duration difference ratio, so as to complete the setting of the action basic score; the relative relationship between the action growth amplitude and the action duration difference ratio can be expressed as the part association frequency between different parts of the corresponding torso position, and according to this part association frequency and the combined probability of the corresponding values ​​appearing on these parts, it is set to obtain the action association value.

[0039] like Figure 4 As shown in the figure, the calculation method for action association values ​​is to obtain the action increase amplitude and the action duration difference ratio corresponding to the action increase amplitude. According to the key points of human posture, the distribution of the action increase amplitude and the action duration difference ratio during rehabilitation movement training is determined. The distribution location here indicates where these values ​​appear on the human body during rehabilitation training. The values ​​recorded at these locations are read to obtain the distribution location of the action increase amplitude and action duration difference ratio.

[0040] Based on the distribution of the movement growth amplitude and movement duration difference ratios during rehabilitation training, the part frequency associations between the movement growth amplitude and movement duration difference ratios and different parts of the torso are determined; and based on the part frequency associations, the movement association value is calculated. The part frequency association indicates which part of the rehabilitation exercise is most likely to exhibit large differences in movement growth amplitude and movement duration, thereby identifying where the user's rehabilitation exercise is prone to large errors and the specific manifestations of the corresponding error values. Based on this information, it is possible to determine the content that needs to be focused on during the current rehabilitation training to implement user feedback analysis. For example, when performing rehabilitation training for the legs or hands, the rehabilitation training movement involves multiple parts, and key points are set for these parts. Then, based on the values ​​of the movement growth amplitude and movement duration difference ratios of these key points in the rehabilitation training movement, the part frequency association between the body parts corresponding to the rehabilitation training movement is set. This value is then comprehensively calculated based on the values ​​of the movement growth amplitude and movement duration difference ratios for different parts to obtain the part frequency association corresponding to the rehabilitation training movement, ultimately completing the setting of the movement association value for the rehabilitation training movement.

[0041] The frequent correlation degree of parts can be set to, Among them, F i represents the frequent association degree of the parts of the i-th rehabilitation training action, N represents the number of parts, j ranges from 1 to N, and the number of parts represents the number of joints that mainly move in a rehabilitation action; P(A ij) represents the probability value of the increase in the movement of the jth part in the i-th rehabilitation training movement, P(T ij ) represents the probability value of the difference ratio of the action duration of the jth part in the i-th rehabilitation training action, I(P(A ij ),P(T ij )) indicates that the indicator function is an indicator function set on the corresponding part based on the difference ratio between the movement growth amplitude and the movement duration, which expresses the corresponding correlation between different parts during rehabilitation training.

[0042]

[0043] P′(T) represents the standard value of the probability value of the action duration difference ratio. The standard value set at this time is set according to the average value of the probability value of the action duration difference ratio in the historical data. P′(A) represents the standard value of the probability value of the action growth amplitude. This standard value is the average value of the probability value of the action growth amplitude in the historical data.

[0044] The action association value is comprehensively set according to the frequent association degree of the parts. For example, the action association value is combined according to the weight of each rehabilitation training action to reach the final action association value.

[0045] Among them, E R Indicates the action associated value, w i Represents the weight of the i-th rehabilitation training action. The resulting action association value is a correlation value between different rehabilitation training actions and the patient's torso parts, indicating the correlation between the corresponding actions on the torso. Based on this value, the overall effect of the rehabilitation training action can be determined, as well as the parts that need to be marked, to facilitate subsequent adjustments to the patient's rehabilitation training by the doctor.

[0046] In one embodiment of the present invention, the motion difference comparison module is used to compare the rehabilitation training motion data with the posture model to determine the relative overlap area and effective difference area at each time point. Based on the distribution of the relative overlap area and effective difference area at the corresponding time point, the module identifies the motion difference bias associated with the relative overlap area and effective difference area, and sets a motion difference score based on the distribution of the motion difference bias. The motion difference score represents the difference between the specific motion in the current rehabilitation training motion data and the posture model, as well as the motion score assigned to this difference. This motion score specifically reflects the specific situation exhibited by the current rehabilitation training motion.

[0047] The above-mentioned relative overlapping area represents the overlapping area between the rehabilitation training action and the corresponding action under the posture model at the corresponding time point, and the effective difference area represents the difference in area value between the rehabilitation training action and the corresponding action under the posture model. This part is generally the area after subtracting the union of the intersection of the areas between the current rehabilitation training action and the action in the posture model; by determining the combination of these two areas, the main differences in the actions can be clearly found, and combined with the distribution of different actions under the corresponding marking points in the action recognition module, the direction of the current action difference can be clearly identified, and these directions are used as the main processing objects at this time to complete the analysis of the action differences.

[0048] When setting the action difference score, the action difference comparison module is specifically implemented as follows: compare the rehabilitation training action at each time point with the posture model, determine the overlapping area between the rehabilitation training action and the corresponding action in the posture model, and use it as the relative overlapping area at each time point; determine the difference area between the rehabilitation training action and the posture model at each time point based on the relative overlapping area, and use the area of ​​the difference area as the effective difference area at each time point; the effective difference area at this time is actually obtained by subtracting the relative overlapping area from the intersection of the area of ​​the rehabilitation training action and the corresponding action in the posture model; calculate the effective difference area according to the effective difference area and the relative overlap area. Area, identify the distribution between rehabilitation training movements and the relative overlap area and the effective difference area, regard the area deviation and direction deviation generated by the rehabilitation training movements and the corresponding movements in the posture model at different parts as movement difference bias, divide the movement difference bias into multiple difference points, and set the index value of the movement difference bias according to the probability of the movement difference bias taking values ​​in the corresponding direction and area; normalize the relative overlap area and the effective difference area to obtain the index value of the relative overlap area and the index value of the effective difference area; calculate the movement difference score based on the index value of the movement difference bias, the index value of the relative overlap area and the index value of the effective difference area.

[0049] The aforementioned motion difference bias specifically describes the spatial deviation and directionality between the actual motion and the posture model. It not only reflects the area differences between different parts of the body during the motion, namely the relative overlap area and the effective difference area, but also indicates the body parts where these differences primarily occur and their spatial distribution trends.

[0050] The action difference bias determines the deviation area between the rehabilitation training movement and the corresponding movement in the posture model using the effective difference area and the relative overlap area. The distance between the deviation areas is calculated as the area deviation of the action difference bias. This distance is set by calculating the Euclidean distance. Multiple key points exist on the rehabilitation training movement and the corresponding movement in the posture model. These key points represent the distance difference between the deviation areas in the corresponding space. The sum of the distance differences in the deviation areas represented by these movements is used as the area deviation. The deviation area is transformed into a vector, and the direction of the transformed vector is identified. The change in the vector direction is used as the directional deviation of the action difference bias. The deviation area is converted into a vector to determine the direction of the deviation of the rehabilitation training movement. The difference between the vectors representing this direction is used as the directional deviation. Finally, the probability values ​​corresponding to the area deviation and directional deviation of the action difference bias are set as the action bias index. This probability value represents the probability of both area deviation and directional deviation occurring.

[0051] Therefore, the action difference score is expressed as follows: the index value of the action difference bias, the index value of the relative overlap area, and the index value of the effective difference area are obtained, and the action difference score is calculated: Among them, E d represents the action difference score, O t Indicates the index value of relative overlap area, D t The index value representing the effective difference area, D t,max The maximum value of the index value representing the effective difference area, B t The index value representing the action difference bias, w O Represents the weight of the relative overlap area, w D Represents the weight of the effective difference area, w B Represents the weight of the action difference bias; the weights of the action difference bias, relative overlap area and effective difference area can be set to 0.4, 0.3 and 0.3 respectively; if it is necessary to count the differences in the relative overlap area and the effective difference area at multiple time points, the action difference scores obtained at this time are summed up and the average is taken to obtain the corresponding action difference score. At the same time, when counting the action difference scores at multiple time points, the weights of the corresponding action difference bias, relative overlap area and effective difference area are expressed as the ratio of the index values ​​of the action difference bias, relative overlap area and effective difference area to the corresponding total data, so as to obtain the action difference score.

[0052] In one embodiment of the present invention, the movement verification module is used to obtain the movement interval, number, and movement frequency corresponding to the rehabilitation training movements in the rehabilitation training mode. Based on this data, it determines whether the current rehabilitation training movement is completed and generates a movement completion score. The movement completion score represents the overall completion status of the rehabilitation training movement data and is used to evaluate the patient based on this completion status.

[0053] The action completion score is expressed as, Among them, E S It represents the action completion score, It represents the action interval, It′ represents the expected value of the action interval, and the expected value represents the average value of the action interval under normal circumstances of rehabilitation training. This value can be obtained from the historical data of the corresponding rehabilitation training action; It max Indicates the maximum value allowed for the action interval. This maximum value indicates the maximum time interval allowed when completing a rehabilitation training action. This value can be set from the maximum value of the corresponding action interval that appears in the historical data; n represents the number of rehabilitation training actions, n tar represents the target number of rehabilitation training actions, which indicates how many actions are needed to normally complete a set of rehabilitation training; f represents the action frequency, f tar The target value of the action frequency is the frequency of normal completion of rehabilitation training, which can be set from the average value of normal completion of rehabilitation training in historical data; I represents the weight of the action interval, w n The weight of the number of rehabilitation training actions, w f The weights of action frequency are set to 0.3, 0.4, and 0.3 respectively for action interval, number, and action frequency.

[0054] In one embodiment of the present invention, the feedback evaluation module mainly comprehensively processes multiple scores calculated by the action recognition module, the action difference comparison module and the action association module to verify the patient's main movement labels and specific training level during rehabilitation training. These can more comprehensively reflect whether each action of the current patient during rehabilitation training is completed accurately, and the form of the unfinished part, so as to identify what form of movement deviation the current patient has and need to use the system for voice prompts or final evaluation prompts to assist the patient in rehabilitation training.

[0055] The feedback evaluation results are generated by obtaining the movement completion score, movement difference score, movement association value, and movement displacement evaluation value, calculating the feedback evaluation score, and outputting the feedback evaluation score as the feedback evaluation result. The obtained feedback evaluation results are used to determine the specific performance of the patient's current rehabilitation training movements and transmit the corresponding output to an external terminal.

[0056] Among them, E fin Denotes the feedback evaluation score, E S Indicates the score of action completion, E d represents the action difference score, E R Indicates the action associated value, E VA Indicates the action displacement evaluation value; represents the weight of the action completion score, represents the weight of the action difference score, represents the weight of the action-associated value, The weight of the action displacement evaluation value is represented by the weights of the action completion score, action difference score, action association value and action displacement evaluation value, which are expressed as 0.2, 0.3, 0.3 and 0.2 in that order; or the average value of the weights of the action completion score, action difference score, action association value and action displacement evaluation value in the historical data is used as the weight used at this time.

[0057] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered by the scope of protection of the present invention.

Claims

1. A real-time analysis and feedback system for rehabilitation training based on machine vision, characterized in that: include: The motion acquisition module is used to collect rehabilitation training motion data, identify the rehabilitation training motion data, and generate a posture model at each time point according to the key points of the human body posture; The motion recognition module is used to identify the duration and amplitude of the motion during rehabilitation training based on the posture model and rehabilitation training motion data, obtain a motion difference curve based on the duration and amplitude, analyze the motion difference curve, and obtain a motion displacement evaluation value for the motion displacement and a motion association value for the motion growth amplitude; The action difference comparison module is used to compare the rehabilitation training action data and the posture model to obtain the relative overlap area and effective difference area of ​​the rehabilitation training action data at each time point; based on the relative overlap area and effective difference area at each time point, the action difference bias of the rehabilitation training action is determined, and the action difference score is set according to the distribution of the action difference bias; An action verification module is used to determine the action interval, number and action frequency of the rehabilitation training action in the corresponding rehabilitation training mode according to the rehabilitation training action data, and determine the action completion score of the rehabilitation training action regarding the action completion; The feedback evaluation module is used to comprehensively calculate the output results of the action recognition module, the action difference comparison module and the action verification module to obtain a feedback evaluation result; The feedback evaluation result is realized by obtaining the action completion score, the action difference score, the action correlation value and the action displacement evaluation value, calculating the feedback evaluation score, and outputting the feedback evaluation score as the feedback evaluation result; ； Among them, E fin Denotes the feedback evaluation score, E S Indicates the score of action completion, E d represents the action difference score, E R Indicates the action associated value, E VA Indicates the action displacement evaluation value; represents the weight of the action completion score, represents the weight of the action difference score, represents the weight of the action-associated value, Represents the weight of the action displacement evaluation value; The processing method of generating a posture model at each time point according to the key points of the human body posture also includes: obtaining the three-dimensional posture information and target rehabilitation training movements of the current user, and determining the key points of the current user's human body posture according to the three-dimensional posture information and the target rehabilitation training movements.

2. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The processing method of generating the posture model at each time point according to the key points of the human posture also includes: selecting the posture model corresponding to the current target rehabilitation training action according to the moving position of the key points of the human posture at each time point.

3. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The implementation methods of the motion displacement evaluation value include: According to the key points of human body posture, set the marker points of rehabilitation training action; identify the displacement value and angle value of the marker points in the rehabilitation training action data under the corresponding action amplitude; According to the displacement and angle values ​​of the marked points under the corresponding movement amplitude, the movement difference curves under different movement durations are determined; Analyze the movement difference curve and calculate the displacement difference, angle overlap ratio and angle overlap period of adjacent movements during the current rehabilitation training; According to the displacement difference, the angle overlap ratio, and the angle overlap period, a motion displacement evaluation value regarding the motion displacement is set.

4. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 3, characterized in that: The motion displacement evaluation value is expressed as: ； Among them, E VA represents the movement displacement evaluation value, n represents the number of rehabilitation training movements, and the value range of i is 1 to n; D i represents the displacement difference of the i-th rehabilitation training action, θ overlap,i represents the angle overlap ratio of the i-th rehabilitation training action, t A,i represents the length of the angle overlap period of the i-th rehabilitation training action, D′ represents the standard value of the displacement difference, θ′ overlap Indicates the standard value of the angle overlap ratio, t′ A,i represents the standard action duration of the i-th rehabilitation training action, t′ A represents the average value of the standard action duration, and C represents a constant term.

5. The real-time analysis and feedback system for rehabilitation training based on machine vision according to claim 4 is characterized in that: The action association value is calculated as follows: Obtaining the movement increase amplitude and the movement duration difference ratio corresponding to the movement increase amplitude; determining the distribution position of the movement increase amplitude and the movement duration difference ratio during rehabilitation movement training according to the key points of human posture; According to the distribution of the difference ratio of movement growth amplitude and movement duration during rehabilitation movement training, the position frequency correlation between the difference ratio of movement growth amplitude and movement duration and different parts of the trunk position is determined; and based on the position frequency correlation, the movement association value is calculated.

6. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 5, characterized in that: The frequent correlation degree of parts is set as: ； Among them, F i represents the frequent association degree of the parts of the i-th rehabilitation training action, N represents the number of parts, j ranges from 1 to N, P(A ij ) represents the probability value of the increase in the movement of the jth part in the i-th rehabilitation training movement, P(T ij ) represents the probability value of the difference ratio of the action duration of the jth part in the i-th rehabilitation training action, I(P(A ij ),P(T ij )) represents the indicator function; The action associated value is expressed as: ； Among them, E R Indicates the action associated value, w i represents the weight of the i-th rehabilitation training action.

7. The real-time analysis and feedback system for rehabilitation training based on machine vision according to claim 1, characterized in that: The implementation of the action difference comparison module is as follows: The rehabilitation training movements at each time point are compared with the posture model, and the overlapping area between the rehabilitation training movements and the corresponding movements in the posture model is determined as the relative overlapping area at each time point; based on the relative overlapping area at each time point, the difference area between the rehabilitation training movements and the posture model at each time point is determined, and the area of ​​the difference area is used as the effective difference area at each time point; according to the effective difference area and the relative overlapping area, the distribution between the rehabilitation training movements and the relative overlapping area and the effective difference area is identified, and the area deviation and direction deviation generated by the rehabilitation training movements and the corresponding movements in the posture model at different parts are regarded as movement difference bias, and the index value of the movement difference bias is set according to the probability that the movement difference bias takes values ​​in the corresponding direction and area; the relative overlapping area and the effective difference area are normalized to obtain the index value of the relative overlapping area and the index value of the effective difference area; according to the index value of the movement difference bias, the index value of the relative overlapping area and the index value of the effective difference area, the movement difference score is calculated.

8. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 7, characterized in that: The action difference score is expressed as: ； Among them, E d represents the action difference score, O t Indicates the index value of relative overlap area, D t The index value representing the effective difference area, D t,max The maximum value of the index value representing the effective difference area, B t The index value representing the action difference bias, w O Represents the weight of the relative overlap area, w D Represents the weight of the effective difference area, w B The weight representing the action difference bias.

9. The real-time analysis and feedback system for rehabilitation training movements based on machine vision according to claim 1, characterized in that: The action completion score is expressed as: ； Among them, E S It represents the action completion score, It represents the action interval, It′ represents the expected value of the action interval, It max Indicates the maximum value allowed for the action interval, n indicates the number of rehabilitation training actions, n tar represents the target number of rehabilitation training actions, f represents the action frequency, and f tar Indicates the target value of the action frequency, w I represents the weight of the action interval, w n The weight of the number of rehabilitation training actions, w f Represents the weight of the action frequency.

Images