Joint motion range detection method and device based on human body key points

Through the detection method based on key points of the human body, the joint rotation plane is constructed and the joint mobility is calculated, which solves the problem of time-consuming joint mobility test in the existing technology and the inability to record rotation speed and acceleration in real time, achieving high-precision and efficient joint mobility degree measurement.

CN119068545APending Publication Date: 2024-12-03SHANGHAI SHULI INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411042782.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The current joint motion test method requires the rehabilitator to manually operate the protractor, which is time-consuming and cannot record the rotation speed and acceleration in real time, and lacks a method to directly calculate the rotation motion of the joint.

Method used

Using joint mobility detection method based on human body key points, the joint rotation plane is constructed through the human posture estimation model, the BlazePose33 point model is used to obtain the human body key points, calculate the rotation angle, angular velocity and angular acceleration, package the message into json format and visualize it Plot and data export.

Benefits of technology

It realizes high-precision joint mobility measurement, reduces human factors, improves the objectivity and speed of measurement, and is suitable for mobility detection of multiple joints.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068545A_ABST
    Figure CN119068545A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a joint motion range detection method and device based on human body key points, and the method comprises the steps: obtaining a human body posture estimation model which comprises the human body key points; determining a rotary joint based on the human body key points and the to-be-evaluated limb part; the rotating joint is determined as a rotating key point, key points, corresponding to connecting joints on the two sides of the rotating joint, in the human body key points serve as a first connecting key point and a second connecting key point correspondingly, and therefore a joint rotating plane is constructed; the joint rotation plane comprises a first vector formed from the first connection key point to the rotation key point and a second vector formed from the second connection key point to the rotation key point; and based on the rotation key point, the first vector and the second vector, obtaining a joint motion range detection result. According to the method, high-precision measurement can be realized, the method is more accurate than a traditional visual inspection or simple measurement tool, the measurement result does not depend on subjective judgment, the influence of human factors on the result is reduced, and the measurement objectivity is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of machine vision posture estimation and the technical field of rehabilitation motion assessment, and in particular to a method and device for detecting joint range of motion based on key points of the human body. Background Art

[0002] In the evaluation of limbs and trunk, the most important and basic content is the examination of muscle strength and joint mobility. The range of motion test (ROMT) is the maximum range of motion that a joint can achieve in a specific body position. It is the basic element and prerequisite of human motor function.

[0003] The range of motion test refers to a method for quantifying and evaluating the arc of motion of a joint. The conventional range of motion test method is to use a protractor, ruler and other measuring tools to test the proximal and distal bone motion arcs of the joint. In addition to being able to directly obtain the patient's maximum range of motion, ROMT can also be used as a pre-assessment of muscle strength, spasticity and other assessment data to qualitatively assess the patient's muscle strength, spasticity and other conditions. Improving the speed and accuracy of ROMT is of considerable value in the assessment scenario of accelerating the test speed of rehabilitation therapists and improving the patient's rehabilitation experience.

[0004] Currently, ROMT tests are mostly completed by rehabilitation therapists using a protractor. The entire measurement process requires the therapist to manually fix the rotating arm and fixed arm of the protractor. The use of the protractor is relatively direct and the results are more reliable, but the entire process is relatively time-consuming and cannot accurately extract and record information such as the speed and acceleration of the rotation process in real time. At present, there are many studies related to limb movement in the academic and industrial fields in the field of vision, but there is a lack of methods to directly calculate the range of motion of the joint during joint rotation and solutions that can provide information about the movement process. Summary of the invention

[0005] The present application aims to provide a method and device for detecting the range of motion of joints based on key points of the human body, in order to address the technical problems that the current ROMT test is mostly completed by rehabilitation therapists using a protractor, which is time-consuming and laborious, and there is a lack of a method for directly calculating the range of motion of joint rotation during the joint rotation process.

[0006] In order to achieve the above technical objectives, this application adopts the following technical solutions.

[0007] In a first aspect, an embodiment of the present application provides a method for detecting joint range of motion based on key points of a human body, comprising:

[0008] Acquire a human body posture estimation model, wherein the human body posture estimation model includes human body key points;

[0009] Based on the human body key points and the limb parts to be evaluated, a rotation joint is determined; the rotation joint is determined as a rotation key point, and the key points of the human body key points corresponding to the connecting joints on both sides of the rotation joint are respectively used as the first connection key point and the second connection key point, so as to construct a joint rotation plane; the joint rotation plane includes a first vector formed by the first connection key point to the rotation key point, and a second vector formed by the second connection key point to the rotation key point;

[0010] A joint range of motion detection result is obtained based on the rotation key point, the first vector and the second vector.

[0011] In some implementations, a human pose estimation model is obtained based on a human image using a BlazePose 3 3-point model.

[0012] In some embodiments, the joint range of motion detection result includes a rotation angle of the limb part to be evaluated; obtaining the joint range of motion detection result based on the coordinates of the rotation key point, the first connection key point, and the second connection key point includes:

[0013] Determine the inner product of the first vector and the second vector, expressed as follows:

[0014] inner_product = p1·p2 = p 1x p 1y +p 2x p 2y ;

[0015] Where inner_product is the inner product of the first vector and the second vector, p1 is the first vector, p2 is the second vector, and p 1x is the horizontal coordinate of the first vector, p 1y is the ordinate of the first vector, p 2x is the horizontal coordinate of the second vector, p 2y is the ordinate of the second vector;

[0016] Dividing the inner product by the product of the modulus of the first vector and the modulus of the second vector to obtain a cosine value of an angle between the first vector and the second vector;

[0017] According to the cosine value, the angle between the first vector and the second vector is obtained, and the angle is used as the rotation angle of the limb part to be evaluated.

[0018] In some embodiments, the method further comprises:

[0019] According to the rotation angle at the initial moment of this test, the rotation angle at the end moment of this test, and the time used for this test, the angular velocity of the limb part to be evaluated in this test is determined, and the expression is as follows:

[0020]

[0021] Where velocity is the angular velocity, angle is the rotation angle at the end of this detection, angle last is the rotation angle at the initial moment of this detection, and time is the time used for this detection.

[0022] In some embodiments, the method further comprises:

[0023] According to the angular velocity at the beginning of the test, the angular velocity at the end of the test, and the time taken for the test, the angular acceleration of the limb part to be evaluated in the test is determined, and the expression is as follows:

[0024]

[0025] Acclerate is the angular acceleration, vekicity is the angular velocity at the end of this detection, and velocity last It is the acceleration at the initial moment of this detection.

[0026] In some embodiments, the method further comprises:

[0027] The joint range of motion detection result is packaged into a message in json format, the message is parsed to obtain a parsing result, and a drawing is drawn according to the parsing result.

[0028] In some embodiments, the method further includes exporting the joint range of motion detection result in a file in a preset format.

[0029] In a second aspect, an embodiment of the present application provides a joint range of motion detection device based on key points of a human body, comprising:

[0030] A human body model building module is configured to obtain a human body posture estimation model, wherein the human body posture estimation model includes human body key points;

[0031] The joint rotation plane construction module is configured to determine the rotation joint based on the human body key points and the limb part to be evaluated; determine the rotation joint as the rotation key point, and use the key points of the human body key points corresponding to the connecting joints on both sides of the rotation joint as the first connection key point and the second connection key point, so as to construct the joint rotation plane, wherein the joint rotation plane includes a first vector formed by the first connection key point to the rotation key point, and a second vector formed by the second connection key point to the rotation key point;

[0032] The joint activity analysis module is configured to obtain a joint activity detection result based on the rotation key point, the first vector and the second vector.

[0033] In some embodiments, the device further comprises a data transmission module and a visual mapping module;

[0034] The data transmission module is configured to package the joint activity detection result into a message in json format, and transmit it to the visual drawing module through a preset protocol and port;

[0035] The visual drawing module is configured to receive the message, parse the message and perform drawing.

[0036] In some embodiments, the device also includes a data export module and a data storage module, the data export module is configured to export the joint range of motion detection results in a file in a preset format; the data storage module is configured to store the joint range of motion detection results in a file in a preset format.

[0037] Compared with the prior art, the joint range of motion detection method based on human key points provided in the embodiment of the present application can achieve high-precision measurement by constructing a joint rotation plane and calculating the joint range of motion based on the rotation key points, the first connection key points and the second connection key points, which is more accurate than traditional visual inspection or simple measurement tools; the measurement results do not rely on subjective judgment, reduce the influence of human factors on the results, and improve the objectivity of the measurement. This method can be applied to the range of motion detection of multiple joints of the human body, including but not limited to shoulder joints, hip joints, knee joints, etc., which provides the possibility for comprehensive joint function evaluation.

[0038] The joint mobility detection device based on human key points provided in the embodiment of the present application has the same beneficial technical effects as the above-mentioned joint mobility detection method based on human key points, which will not be elaborated on. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present application in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes and are used to help understand the present application. They do not specifically limit the shapes and proportional dimensions of the components of the present application. Under the guidance of the present application, those skilled in the art can select various possible shapes and proportional dimensions to implement the present application according to specific circumstances. In the drawings:

[0040] Figure 1 This application is a schematic diagram of a method for detecting joint range of motion based on key points of the human body provided by an embodiment;

[0041] Figure 2 Schematic diagram of the BlazePose key point model used in the embodiments of the present application;

[0042] Figure 3 A schematic diagram of a joint rotation plane in a joint activity detection method based on human key points provided in an embodiment of the present application;

[0043] Figure 4 Several different shoulder posture extraction situations in the joint activity detection method based on human key points provided in the embodiment of the present application;

[0044] Figure 5 The joint range of motion detection method based on human key points provided in the embodiment of the present application is used to detect the joint range of motion Figure 1 ;

[0045] Figure 6 This is a schematic diagram of the angle analysis results based on joint points in an embodiment of the present application;

[0046] Figure 7 The joint range of motion detection method based on human key points provided in the embodiment of the present application is used to detect the joint range of motion Figure 2 ;

[0047] Figure 8 This is a schematic diagram of the angular velocity analysis results based on joint points according to an embodiment of the present application;

[0048] Fig. 9 The joint range of motion detection method based on human key points provided in the embodiment of the present application is used to detect the joint range of motion Figure 3 ;

[0049] Fig.10 This is a schematic diagram of the angular acceleration analysis results based on joint points according to an embodiment of the present application;

[0050] Fig.11 A method for detecting joint range of motion based on key points of the human body provided in another embodiment of the present application is provided to execute a process after obtaining the detection result;

[0051] Fig.12 A schematic diagram of the principle of data storage after obtaining the detection results of the joint activity detection method based on key points of the human body provided by another embodiment of the present application;

[0052] Fig.13 A schematic diagram of exporting angle data into an Excel file for a joint activity detection method based on key points of the human body provided in an embodiment of the present application;

[0053] Fig.14 A schematic diagram of exporting angle data as a PNG file in a joint range of motion detection method based on key points of the human body provided in an embodiment of the present application;

[0054] Fig.15 A schematic diagram of exporting angular velocity data into an Excel file for a joint activity detection method based on key points of the human body provided in an embodiment of the present application;

[0055] Fig.16 A schematic diagram of exporting angular velocity data as a PNG file in a joint activity detection method based on key points of the human body provided in an embodiment of the present application;

[0056] Fig.17 A schematic diagram of exporting angular acceleration data into an Excel file according to a method for detecting joint activity based on key points of a human body provided in another embodiment of the present application;

[0057] Fig.18 A schematic diagram of exporting angular acceleration data as a PNG file according to a method for detecting joint activity based on key points of a human body provided in another embodiment of the present application;

[0058] Fig.19 It is a schematic diagram for comparing the experimental results in the embodiments;

[0059] Fig. 20 A schematic diagram of the structure of a joint range of motion detection device based on key points of the human body provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to enable those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0061] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features.

[0062] The physiological movement forms of joints include sliding, angular movement, rotation and circumduction. The measurement of joint range of motion is the measurement of the range of angular movement and rotation. The protractor measurement method is easy to operate and is most commonly used in clinical practice. The measurement of joint range of motion is a very strict assessment technique with high reliability and validity requirements. Its high reliability is reflected in the consistency of the values ​​maintained by the same joint and the same examiner for multiple measurements, and the consistency of the examination results between different examiners of the same joint. Its validity is reflected in the fact that the measured joint range of motion is measured within the limits of the protractor to the maximum range of motion, and the difference with the results measured by the electronic angle meter is within the allowable range. The correctness of the operation method directly affects the reliability of the measurement results. Therefore, the examiner should strictly master the use of the measuring tool, the measurement method of different joints and the precautions in the operation. If the measured results show that the range of joint motion is abnormal (limited or excessive), the cause of the abnormality should be found to provide a basis for formulating a rehabilitation treatment plan.

[0063] The definition of the maximum and minimum angles measured by the ROMT method commonly used in clinical practice is as follows: the active range of motion (AROM) and passive range of motion (PROM) angles, that is, the angle at the beginning and end of an active or passive movement, such as 0° to 90°. The angle at the end is the position where the movement is restricted. The most commonly used method to determine the range of joint movement is the neutral position method (anatomical 0 position) recommended by the Joint Movement Committee of the American Orthopaedic Association, that is, the limb position at the anatomical 0 position is set as 0°. When recording the range between the starting position and the ending position of the measurement, if the person being measured cannot move from the anatomical 0° position, that is, from the starting position, the angle of the actual starting position should be accurately recorded. Combined with the normal range of motion of a specific joint, its movement restriction is judged. Taking the various measurement results of the knee flexion range of motion as an example, 0° to 150° indicates no joint movement restriction; 20° to 150° indicates knee extension restriction; 0° to 120° indicates knee flexion restriction; 20° to 120° indicates both knee flexion and extension restriction. When a joint of the person being measured shows abnormal hyperextension, a "-", or negative sign, can be used to indicate this. For example, "-20°" for the knee joint means that the knee joint is 20° hyperextended. When a joint that normally performs bidirectional motion can only perform unidirectional motion due to a lesion, the range of motion in the restricted direction is recorded as "none". For example, in patients with wrist flexion (15° to 80° flexion contracture), the wrist joint cannot reach the anatomical "0" position or cannot be extended. Therefore, the wrist extension column should be recorded as "none". The entire process requires the rehabilitation therapist to be proficient in the measurement method and to operate the protractor correctly, and the reading of the results is usually slow.

[0064] At present, ROMT tests are mostly completed by rehabilitation therapists using protractors, which is time-consuming and laborious, and there is a lack of technical problems in the method of directly calculating the joint rotation range of motion during the joint rotation process. The embodiments of the present application provide a joint range of motion detection method and device based on the key points of the human body. By constructing a joint rotation plane based on a human posture estimation model for analysis, it can flexibly adapt to the joint range of motion detection needs of different people and different body types. Using computer technology and image processing technology, human body images can be quickly acquired and bone models can be constructed, greatly shortening the detection time; the present application does not require complex equipment or site requirements, and only needs to acquire human body images through cameras and other equipment, and can perform detection based on analysis of human body images, which is convenient for promotion and application in multiple fields such as medical treatment, rehabilitation, and sports.

[0065] The following is a further description of the application in conjunction with the accompanying drawings and specific embodiments. Figure 1 As shown, a method for detecting joint activity based on key points of a human body includes:

[0066] Step S1: obtaining a human body posture estimation model based on a human body image, wherein the human body posture estimation model includes human body key points;

[0067] Step S2: Based on the key points of the human body and the limb part to be evaluated, determine the rotation joint; determine the rotation joint as the rotation key point, and use the key points of the human body key points corresponding to the joints connected on both sides of the rotation joint as the first connection key point and the second connection key point, respectively, to construct a joint rotation plane, the joint rotation plane includes a first vector formed from the first connection key point to the rotation key point, and a second vector formed from the second connection key point to the rotation key point;

[0068] Step S3: Based on the rotation key point, the first vector and the second vector, obtain the joint activity detection result.

[0069] In human motion analysis, the key point generation component is very important. It can obtain the position information of the key points of the human body through images or sensors, and provide data support for subsequent motion analysis.

[0070] In this field, OpenPose and DeepPose are two important human pose estimation models, and Blaze Pose and YOLOv7 are two other related human pose estimation model structures.

[0071] DeepPose is a pose estimation method proposed by Microsoft Research in 2014. DeepPose uses deep convolutional neural networks to estimate human poses and is one of the important milestones in the field of deep learning in pose estimation. DeepPose is a pose estimation system based on deep learning that uses convolutional neural networks to learn human pose features. Unlike OpenPose, DeepPose focuses on single-person pose estimation, with the emphasis on accurately estimating human poses from a single picture, including joint positions and pose angles.

[0072] OpenPose is a multi-person pose estimation system based on convolutional neural networks (CNNs), which uses real-time multi-person 2D pose estimation technology. Its uniqueness lies in the use of Part Affinity Fields (PAFs) technology to detect the association between key points of the human body, thereby achieving accurate estimation of multi-person poses. PAFs can help the system identify the connection relationship in human poses, improving the accuracy and robustness of pose estimation.

[0073] BlazePose is a model specifically designed for human posture detection. It uses a lightweight neural network structure to achieve real-time human posture estimation. BlazePose has highly accurate key point detection capabilities, can accurately identify the key points of the human body, and excels in real-time performance.

[0074] YOLOv7 is a multi-target detection model that uses the YOLO algorithm to achieve real-time target detection and positioning. Although YOLOv7 can identify the bounding box of the human body and provide the location information of key points, its main design purpose is to achieve target detection and is usually used for personnel detection. Compared with BlazePose, its performance in posture detection may be slightly insufficient.

[0075] In general, OpenPose and DeepPose differ in processing multi-person poses and single-person poses, while BlazePose and YOLOv7 also have their own characteristics in human pose detection. Choosing which model to use depends on the specific application scenario and requirements.

[0076] OpenPose is more focused on multi-person pose estimation, using PAFs to identify the correlation between key points, while DeepPose is more focused on single-person pose estimation. BlazePose is more professional and accurate in human pose detection, while YOLOv7 is more suitable for general target detection. YOLOv7 supports multi-target detection, while BlazePose cannot be used for multi-target pose detection, only single-target detection. For the same video, if it is run on the CPU, BlazePose will have an absolute performance and accuracy advantage, but when the GPU is allowed to be used, YOLOv7 will be faster than BlazePose in inference. Therefore, when implementing human pose detection, whether there is GPU support, whether you need to detect multiple people or a single person, you need to have a clear understanding of your hardware configuration.

[0077] In step S1 of the embodiment of the present application, a human body posture estimation model is obtained based on a human body image using the BlazePose3 3-point model.

[0078] In some embodiments, in order to maximize the rate of key point generation, possible GPU (graphics processing unit) interfaces are retained, and different human posture estimation models are added to the front end as a detection component, so that the detection scenarios and costs can be as close to actual needs as possible.

[0079] In a specific implementation, after detecting the human body posture estimation model (i.e., human body skeleton model), the angle analysis of the joints to be detected can be performed as needed. As an example, taking the BlazePose 33-point model as an example, the BlazePose 33-point model will generate the human body into a skeleton including limbs and head after detection, such as Figure 2 shown.

[0080] In some embodiments, the joint rotation plane constructed in step S2 is as follows: Figure 3As shown, the joint rotation plane is used to implement the analysis method based on the synthesis of two vectors from three key points of the plane, that is, the calculation of the joint rotation angle and angular velocity is realized based on the two vector information. In view of the limited depth information that a monocular camera can provide, the embodiment of the present application uses the key point information of the plane to simplify the calculation. Figure 3 In the joint rotation plane shown, the joint quadrant positions of 0° (360°), 90°, 180°, and 270° are calibrated. By obtaining the rotation key point P2 and the connection key points P1 and P3 of the corresponding joint, the rotation angle angle can be obtained.

[0081] As an example, several different shoulder pose extraction situations can be referred to Figure 4 shown.

[0082] In some embodiments, step S3 includes obtaining the coordinates of the rotation key point P2 and the connection key points P1 and P3, and determining the fixed edge P 12 and the rotation edge P 32 , solve for the first vector P 12 (denoted as p1) and the second vector P 32 (denoted as p2) inner product inner_product, where p 1x 、p 1y is the horizontal and vertical coordinates of vector p1, p 2x 、p 2y Similarly, divide the inner_product by the vector L 12 The modulus length p 1norm and vector L 32 The modulus length p 2norm The product of the two vectors gives the cosine value of the vector, and arccos is used to obtain the actual rotation angle angle.

[0083] inner_product = p1·p2 = p 1x p 1y +p 2x p 2y ;

[0084]

[0085] by Figure 2 For example, when the right shoulder joint rotation angle analysis is needed, only point 11 needs to be set as the connection key point P1, point 12 as the rotation key point P2, and point 14 as the connection key point P3 to obtain the minimum analysis module, that is, Figure 3 Joint analysis plane shown.

[0086] The joint activity detection method based on human key points provided in the embodiment of the present application realizes a minimum joint analysis system based on three plane key points, which can measure and analyze the positions of the joint points, calculate the angle changes of the joints, and thus evaluate the joint activity and stability. Figure 5 It can be shown in the figure that in the minimum analysis system with the left shoulder key point and the right elbow key point as the connection and the right shoulder key point as the rotation point, the measured degree is 173.078 degrees. The angle analysis results based on the joint points are as follows Figure 6 shown.

[0087] In addition to angle analysis, angular velocity analysis based on joint points is also very important. It can help to more comprehensively understand the speed changes of joint movements and provide more data support for motion assessment and rehabilitation training. In some embodiments, the method also includes determining the angular velocity velocity of the limb part to be evaluated based on the rotation angle at the initial moment of this detection, the rotation angle at the end moment of this detection, and the time used for this detection. The expression is as follows:

[0088]

[0089] As an example, Figure 7 and Figure 8 , you can see the real-time change of angular velocity from -137 degrees per second to 400 degrees per second.

[0090] In some embodiments, the method further includes determining the angular acceleration of the limb part to be evaluated in this detection according to the angular velocity at the beginning of this detection, the angular velocity at the end of this detection, and the time taken for this detection, and the expression is as follows:

[0091]

[0092] Where accelerate is the angular acceleration, velocity is the angular velocity at the end of this detection, velocity last is the acceleration at the initial moment of this detection. The angular acceleration analysis based on the joint points is as follows Fig. 9 and Fig.10 shown.

[0093] In some embodiments, based on the above embodiments, the provided joint activity detection method based on key points of the human body further includes packaging the joint activity detection results into a message in json format, parsing the message and drawing a graph. The message parsed in json format is subjected to high-performance synchronous drawing. It is measured that the message parsing can be performed at a speed of 1000hz, and a high-speed running thread is used to draw a graph. The process is as follows: Fig.11 shown.

[0094] In some embodiments, the method further includes exporting the joint range of motion detection result to a file in a preset format. Fig.12 A schematic diagram of the principle of data storage after obtaining the detection results of a joint mobility detection method based on key points of the human body provided in another embodiment of the present application, which can respectively use two data recording modes, image (PNG file) and table (Excel file), to meet the basic data quantitative recording.

[0095] Fig.13 A method for detecting joint range of motion based on key points of the human body provided in another embodiment of the present application exports angle data as a schematic diagram of an Excel file; Fig.13 As shown in the figure, in order to meet the quantitative analysis needs of hand motion data by data engineers and researchers, the data set can be saved in Excel format. And you can choose to save single-channel tabulation results or multi-channel tabulation results as needed.

[0096] For doctors, in order to meet the needs of qualitative analysis, only one global chart line of the rehabilitation process can be output to meet the corresponding analysis. It can be saved as a PNG format image suitable for network distribution. You can choose to save single-channel mapping results or multi-channel mapping results as needed, such as Fig.14 As shown, Fig.14 A schematic diagram of exporting angle data as a PNG file in a method for detecting joint range of motion based on key points of the human body provided in another embodiment of the present application;

[0097] like Fig.15 As shown, Fig.15 Another embodiment of the present application provides a method for detecting the range of motion of joints based on key points of the human body. The angular velocity data is exported as an Excel file. To meet the quantitative analysis needs of data engineers and researchers for hand motion data, the data set can be saved as an Excel format. And you can choose to save the single-channel tabulation results or multi-channel tabulation results as needed.

[0098] like Fig.16 As shown, Fig.16 Another embodiment of the present application provides a method for detecting the range of motion of joints based on key points of the human body. The angular velocity data is exported as a schematic diagram of a PNG file. For doctors, in order to meet the needs of qualitative analysis, only one global chart line of the rehabilitation process is output to meet the corresponding analysis. It can be saved as a PNG format image suitable for network distribution. You can choose to save single-channel mapping results or multi-channel mapping results as needed.

[0099] like Fig.17 , Fig.17Another embodiment of the present application provides a method for detecting the range of motion of joints based on key points of the human body. The angular acceleration data is exported as an Excel file diagram. To meet the quantitative analysis needs of data engineers and researchers for hand motion data, the data set can be saved as an Excel format. And you can choose to save the single-channel tabulation results or multi-channel tabulation results as needed.

[0100] like Fig.18 As shown, Fig.18 Another embodiment of the present application provides a method for detecting the range of motion of joints based on key points of the human body. The angular acceleration data is exported as a schematic diagram of a PNG file. For doctors, in order to meet the needs of qualitative analysis, only one global chart line of the rehabilitation process is output to meet the corresponding analysis. It can be saved as a PNG format image suitable for network distribution. You can choose to save single-channel mapping results or multi-channel mapping results as needed.

[0101] The joint mobility detection method based on human body key points provided in the embodiment of the present application can quickly and efficiently generate joint mobility data and provide data comparison at different time points, so that doctors and researchers can more quickly observe the changes in the motor function of patients or subjects, thereby adjusting treatment plans or research strategies more timely.

[0102] In one embodiment, the manual measurement time of the patient is compared. The single time consumption of manual measurement depends on the movement angle of the patient's joint, while the time consumption of machine vision to measure the joint angle once depends on the speed of generating key points by single reasoning. Fig.19 This result can be reflected in that, in 5 rounds of angle measurement, the method provided in the embodiment of the present application basically takes about 0.03 seconds to measure an angle once, and the entire time consumption is overwhelmingly faster than the manual method.

[0103] Corresponding to the joint activity detection method based on human key points provided in the above embodiments, the present application embodiment also provides a joint activity detection device based on human key points, such as Fig. 20 As shown, it includes a human body model building module, a joint rotation plane building module and a joint range of motion analysis module.

[0104] The human body model building module is configured to obtain a human body pose estimation model, wherein the human body pose estimation model includes human body key points;

[0105] The joint rotation plane construction module is configured to determine the rotation joint based on the human body key points and the limb part to be evaluated; determine the rotation joint as the rotation key point, and use the key points of the human body key points corresponding to the joints connected on both sides of the rotation joint as the first connection key point and the second connection key point, respectively, to construct the joint rotation plane, the joint rotation plane includes a first vector formed from the first connection key point to the rotation key point, and a second vector formed from the second connection key point to the rotation key point;

[0106] The joint activity analysis module is configured to obtain a joint activity detection result based on the rotation key point, the first vector and the second vector.

[0107] Human pose estimation models can identify and locate key points of the human body in images or videos, such as joints. These models can be based on deep learning techniques and have high accuracy and robustness.

[0108] By analyzing the key points of the human body, especially the key points related to the limbs to be evaluated, the rotation joints (i.e., rotation key points) can be determined. At the same time, the joints connected on both sides of the rotation joint are used as the first connection key points and the second connection key points, which together with the rotation key points constitute the joint rotation plane.

[0109] The joint range of motion analysis module can calculate the range of motion of the joint in different directions based on the rotation key point, the first vector and the second vector. This calculation method takes into account both the rotation center of the joint (rotation key point) and the connection conditions on both sides of the rotation joint (first vector, second vector), so it can more accurately reflect the actual range of motion of the joint.

[0110] The joint activity detection device based on the key points of the human body provided in the embodiment of the present application can reduce human errors and improve the accuracy of evaluation through accurate joint activity measurement. Compared with the traditional manual measurement method, the system can provide more reliable data and provide more reliable reference for doctors and researchers.

[0111] In some embodiments, the device also includes a data export module and a data storage module. The data export module is configured to export the joint range of motion detection results in a file in a preset format; the data storage module is configured to store the joint range of motion detection results in a file in a preset format.

[0112] The joint range of motion detection device based on the key points of the human body provided in the embodiment of the present application has the function of curve preservation and analysis, and can preserve and analyze the joint range of motion curve data of patients or subjects. This enables doctors and researchers to have a deeper understanding of the trend of changes in the motor function of patients or subjects, and provide stronger support for personalized treatment and customized rehabilitation programs. At the same time, this also helps to monitor the treatment effect or improvement of motor function more comprehensively and carefully.

[0113] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for detecting joint range of motion based on key points of the human body, characterized in that: include: Acquire a human body posture estimation model, wherein the human body posture estimation model includes human body key points; Determine the rotation joint based on the key points of the human body and the limb parts to be evaluated; The rotation joint is determined as a rotation key point, and the key points of the human body key points corresponding to the connection joints on both sides of the rotation joint are respectively used as the first connection key point and the second connection key point, so as to construct a joint rotation plane; The joint rotation plane includes a first vector formed by the first connection key point to the rotation key point, and a second vector formed by the second connection key point to the rotation key point; A joint range of motion detection result is obtained based on the rotation key point, the first vector and the second vector.

2. The joint activity detection method based on human key points according to claim 1 is characterized in that: Using the BlazePose33-point model, a human pose estimation model is obtained based on human images.

3. The joint activity detection method based on human key points according to claim 1 is characterized in that: The joint range of motion test result includes the rotation angle of the limb part to be evaluated; Based on the coordinates of the rotation key point, the first connection key point and the second connection key point, obtaining a joint range of motion detection result includes: Determine the inner product of the first vector and the second vector, expressed as follows: inner_product=p1·p2=p 1x p 1v +p 2x p 2y ; Where, inner_product is the inner product of the first vector and the second vector, p1 is the first vector, p2 is the second vector, and p 1x is the horizontal coordinate of the first vector, P 1y is the ordinate of the first vector, p 2x is the horizontal coordinate of the second vector, p 2y is the ordinate of the second vector; Dividing the inner product by the product of the modulus of the first vector and the modulus of the second vector to obtain a cosine value of an angle between the first vector and the second vector; According to the cosine value, the angle between the first vector and the second vector is obtained, and the angle is used as the rotation angle of the limb part to be evaluated.

4. The joint activity detection method based on human key points according to claim 1 is characterized in that: The method further comprises: According to the rotation angle at the initial moment of this test, the rotation angle at the end moment of this test, and the time used for this test, the angular velocity of the limb part to be evaluated in this test is determined, and the expression is as follows: Where velocity is the angular velocity, angle is the rotation angle at the end of this detection, angle last is the rotation angle at the initial moment of this detection, and time is the time used for this detection.

5. The joint activity detection method based on human key points according to claim 4 is characterized in that: The method further comprises: According to the angular velocity at the beginning of the test, the angular velocity at the end of the test, and the time taken for the test, the angular acceleration of the limb part to be evaluated in the test is determined, and the expression is as follows: Among them, accelerate is the angular acceleration, velocity is the angular velocity at the end of this detection, velocity last It is the acceleration at the initial moment of this detection.

6. The joint activity detection method based on human key points according to claim 1 is characterized in that: The method further comprises: The joint range of motion detection result is packaged into a message in json format, the message is parsed to obtain a parsing result, and a drawing is drawn according to the parsing result.

7. The joint activity detection method based on human key points according to claim 1 is characterized in that: The method further comprises exporting the joint range of motion detection result in a file in a preset format.

8. A joint activity detection device based on key points of the human body, characterized in that: include: A human body model building module is configured to obtain a human body posture estimation model, wherein the human body posture estimation model includes human body key points; A joint rotation plane construction module is configured to determine the rotation joint based on the human body key points and the limb parts to be evaluated; The rotation joint is determined as a rotation key point, and the key points of the human body key points corresponding to the connection joints on both sides of the rotation joint are respectively used as the first connection key point and the second connection key point, so as to construct a joint rotation plane, wherein the joint rotation plane includes a first vector formed by the first connection key point to the rotation key point, and a second vector formed by the second connection key point to the rotation key point; The joint activity analysis module is configured to obtain a joint activity detection result based on the rotation key point, the first vector and the second vector.

9. The joint activity detection device based on human key points according to claim 8, characterized in that: The device also includes a data transmission module and a visual drawing module; The data transmission module is configured to package the joint activity detection result into a message in json format, and transmit it to the visual drawing module through a preset protocol and port; The visual drawing module is configured to receive the message, parse it to obtain a parsing result, and draw a picture according to the parsing result.

10. The joint activity detection device based on human key points according to claim 8, characterized in that: The device further comprises a data export module and a data storage module, wherein the data export module is configured to export the joint range of motion detection result in a file of a preset format; The data storage module is configured to store the joint activity detection result in a file of a preset format.