Individual rehabilitation motion trajectory planning system based on non-contact human body data acquisition

Through depth camera and image registration technology, the problems of inconvenience and insufficient accuracy of human data collection in the existing technology are solved, accurate rehabilitation movement trajectory planning and remote online guidance are realized, and the user experience of rehabilitation equipment and data traceability are improved. sex.

CN114820713BActive Publication Date: 2024-11-22XIDIAN UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210541405.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-17
Publication Date
2024-11-22
Estimated Expiration
2042-05-17

AI Technical Summary

Technical Problem

The prior art has problems such as inconvenient wearable structure, high cost and no deep information in the collection of human body data, which leads to the inability to accurately calculate human body movement data, affecting the accuracy of rehabilitation trajectory planning.

Method used

Using a contactless human data acquisition system based on a depth camera, combining image registration and data resolution, the human joint motion data and limb length parameters are calculated, and color and depth images are captured through a depth camera, combined with Kalman filter and polynomial fitting, Generate accurate rehabilitation movement trajectory and provide online guidance via cloud server remotely sharing data.

Benefits of technology

It realizes accurate calculation of human movement data, optimizes rehabilitation movement trajectory, improves the user experience of rehabilitation equipment, reduces equipment costs, and provides remote online rehabilitation guidance, enhancing the traceability and intuitiveness of data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114820713B_ABST
    Figure CN114820713B_ABST
Patent Text Reader

Abstract

The present invention discloses an individual rehabilitation motion trajectory planning system based on non-contact human body data collection, which mainly solves the problems of inaccurate data and imperfect functions of the existing human body data collection system, and includes: a human body data collection module, a data calculation and storage module, and a data remote sharing module. The human body image acquisition module collects human body color and depth images and transmits them to the data calculation and storage module; the data storage module uses a local computer, which contains different functional sub-modules, and respectively completes image registration, human body individual data solution and transmission, rehabilitation motion trajectory planning and data preservation; the data remote sharing module remotely receives human body individual data and rehabilitation motion trajectory, and reproduces them on the remote computer in the form of numerical characters, waveforms, videos and animations. The human body data collected by the present invention is authentic and reliable, and can plan individual rehabilitation motion trajectories for patients, and can be used for remote online rehabilitation guidance for patients with movement disorders.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer graphics, and in particular relates to an individual rehabilitation motion trajectory planning system, which can be used for remote online rehabilitation guidance of patients with movement disorders. Technical Background

[0002] As the problem of population aging in my country worsens, the number of patients with movement disorders is gradually increasing, and human movement rehabilitation has gradually become a focus of social attention. In the field of movement rehabilitation, the use of rehabilitation robots to help people with rehabilitation training is very important for human data collection and rehabilitation trajectory planning. On the one hand, collecting human data can help rehabilitation therapists understand the patient's current motor function and state, making it easier to formulate rehabilitation plans; on the other hand, the collected individual data is the basis for adjusting rehabilitation trajectory parameters and connection retention when patients use rehabilitation equipment, so as to better help patients recover.

[0003] Existing methods for collecting human body data mainly include three categories: the first is to use wearable structures to fix sensors on various parts of the human body, and collect fixed parameters and motion states of the human body according to the motion state of the human body. The disadvantage of this method is that the wearable structure will affect the user's experience. For people with limb injuries, it is more inconvenient to fix the sensors; the second is based on optical motion capture, by sticking reflective marking points on the human body, using multiple infrared cameras to detect and track the movement of the marking points, and synthesizing the motion data of the marking points into the motion data of the human body. The disadvantage of this method is that each experiment requires complex recalibration of the camera group, and the whole set of equipment is expensive and has poor portability; the third method uses image detection methods to shoot the human body's motion process with a camera and perform motion analysis. The disadvantage of this method is that there is no safety reliance on two-dimensional image coordinates, lacks information in the depth direction, and cannot accurately calculate the real data of the human body. The results obtained may be invalid and cannot provide reasonable data support for trajectory planning. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and, on the basis of the existing third data acquisition method, propose an individual rehabilitation motion trajectory planning system based on non-contact human body data acquisition to accurately calculate the motion data and limb length parameters at the major joints of the human body, and plan the current stage of rehabilitation trajectory for the patient based on these individual data.

[0005] To achieve the above-mentioned purpose, the present invention provides an individual rehabilitation motion trajectory planning system based on non-contact human body data collection, comprising a human body image collection module, a data calculation and storage module, and a data remote sharing module; the human body image collection module and the data calculation and storage module are connected by wire; the data calculation and storage module and the data remote sharing module are connected by wireless, and are characterized in that:

[0006] The human body image acquisition module includes a depth camera and a camera bracket, which are used to capture color images and depth images of the human body;

[0007] The data calculation and storage module adopts a local computer and includes:

[0008] An image registration submodule, used for receiving the color image and depth image taken by the human body image acquisition module and performing image registration processing;

[0009] The data solver module is used to solve the human body parameters and joint motion data, and encapsulate the color image and the solved data into a specific data structure and send it in real time;

[0010] The trajectory planning submodule is used to plan the patient's rehabilitation movement trajectory according to the human personality data and rehabilitation needs calculated by the data solving submodule;

[0011] The storage submodule is used to save the trajectory data and human body data as a local file, and to animate the planned trajectory to intuitively display the rehabilitation movements of the human body;

[0012] The data remote sharing module includes a cloud server and a remote computer, which are wirelessly connected. The cloud server receives the calculation results of the data calculation and storage module and sends them to the remote computer. The remote computer reproduces the received data in the form of numerical characters, waveforms, videos and animations.

[0013] Compared with the prior art, the present invention has the following advantages:

[0014] 1) Since the present invention uses a depth camera in the human body image acquisition module to capture the image information of the human body, the depth data can be fused on the basis of the two-dimensional image, so that the calculated data is more real and reliable;

[0015] 2) The present invention can effectively avoid the discomfort caused to the patient during the rehabilitation exercise process by adjusting the parameters of the connection between the rehabilitation equipment and the human body according to the human personality data through the trajectory planning submodule in the data calculation and storage module. The generated rehabilitation exercise trajectory can be used to guide the patient's rehabilitation exercise;

[0016] 3) The present invention can realize the traceability of rehabilitation data because the human personality data and trajectory data are stored as local files through the storage submodule in the data calculation and storage module;

[0017] 4) The present invention can realize remote online rehabilitation guidance by receiving the calculation results sent by the data calculation storage module in the data remote sharing module and reproducing them in the form of numerical characters, waveforms and animations on the remote computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a structural schematic diagram of the present invention;

[0019] Figure 2 A schematic diagram of human joint points that need to be detected in the present invention;

[0020] Figure 3 The figure is a flowchart of the system of the present invention. DETAILED DESCRIPTION

[0021] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings:

[0022] Reference Figure 1 The present invention includes a human image acquisition module 1, a data calculation and storage module 2 and a data remote sharing module 3. The human image acquisition module 1 and the data calculation and storage module 2 are connected by wire, and the data calculation and storage module 2 and the data remote sharing module 3 are connected by wireless.

[0023] The human body image acquisition module 1 includes a depth camera and a camera bracket. The camera bracket can adjust the support height and adjust the placement of the depth camera according to the actual use environment.

[0024] The depth camera is supported by a camera bracket and is used to capture a color image and a depth image in front of the camera, wherein the data stored in each pixel of the depth image is the spatial straight-line distance between the target object represented by the pixel in the image and the optical center of the depth camera; since the color image and the depth image are captured in different camera coordinate systems, that is, the color coordinate system and the depth coordinate system do not overlap, the images need to be registered, that is, the depth image needs to be converted to a coordinate system, and the formula is:

[0025]

[0026] Among them, (u D ,v D ) represents the coordinate position of a pixel in the depth image, W represents the transformation matrix from the depth coordinate system to the color coordinate system, (u R ,v R ) indicates that the coordinates of a pixel point in the depth image are mapped to the pixel coordinate position represented on the color image.

[0027] The data calculation and storage module 2 uses a local computer, which includes: an image registration submodule 21, a data solution submodule 22, a trajectory planning submodule 23, and a storage submodule 24. The local computer is connected to the depth camera via a USB data transmission line, wherein each submodule is implemented as follows:

[0028] The image registration submodule 21 receives the captured color image and depth image information, and completes the registration process of the two images.

[0029] The data solver module 22 first uses a human key point detection program to identify the joint points of the target person from the color image, such as Figure 2 As shown in the figure, after successful recognition, the pixel coordinates of the joint points of the target person are filtered by the Kalman filter to reduce the interference of external factors on the detection results. Then, combined with the depth information d after registration, the three-dimensional coordinate values ​​(x, y, z) of the joint points are calculated. The formula is:

[0030] x=z(u R -c x ) / f x

[0031] y=z(v R -c y ) / f y

[0032] z=d / s

[0033] Among them, (u R ,v R ) represents the pixel coordinates of the joint point in the color image, (c x ,c y ) represents the camera optical center position, (f x ,f y ) represents the focal length of the camera, s represents the scale factor related to the depth value unit, and in this example, s=1000 is taken but not limited to;

[0034] According to the three-dimensional spatial coordinates of the joint points, the length l and vector of each segment of the human body are calculated respectively.

[0035]

[0036]

[0037] Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) represent the spatial coordinates of the ith key point and the spatial coordinates of the jth key point adjacent to it, that is, the spatial position coordinates of two adjacent joint points;

[0038] According to the vectors of each segment of the human body, calculate the angle θ of each joint:

[0039]

[0040] in, and They respectively represent the vector of the i-th limb segment and the vector of the j-th limb segment adjacent to it, that is, the vectors of two adjacent limbs.

[0041] After completing the data solution, the data solution submodule 22 encapsulates the color image and the solved data into a specific data structure and sends it in real time.

[0042] The trajectory planning submodule 23 adjusts the connection and fixing parameters of the rehabilitation robot when it is used according to the calculated human personality data and the patient's rehabilitation needs, and then optimizes the joint movement angle according to the connection and fixing parameters, the limb length of the rehabilitation part and the range of joint motion to avoid discomfort to the patient during the rehabilitation movement process, and performs polynomial fitting on the optimized joint angle to generate a continuous rehabilitation movement trajectory. The motion planning trajectory can serve as an important basis for rehabilitation therapists to plan rehabilitation training for patients.

[0043] The storage submodule 24 first stores the image information, human personality data and motion trajectory data in the system cache, and when the system work is completed, writes these contents to the computer hard disk.

[0044] On the local computer, the system software design work is completed based on the MFC software design tool. On the one hand, it can display the patient's current personality data in real time. On the other hand, after the data collection task is completed, the collection results and rehabilitation exercise planning trajectory are saved, and the rehabilitation exercise planning trajectory is animated to show the data more intuitively. The system software is designed with a network transmission function. When connected to the Internet, the human personality data can be sent to the data remote sharing module 3 in real time.

[0045] The data remote sharing module 3 includes a cloud server and a remote computer, which are wirelessly connected. The cloud server receives the calculation results of the data calculation storage module 2 and sends them to the remote computer. A data transmission program is written on the remote computer based on the Socket network, and the human personality data sent by the cloud server is received through the Internet. The user interface is designed using the MFC software development tool. The user interface is divided into four areas: a numerical display area, a waveform display area, a video display area, and an animation display area. The received data is reproduced in the form of numerical characters, angle change waveforms, videos, and animations, respectively, to achieve remote online rehabilitation guidance. Among them, the numerical characters are real-time numerical values ​​of limb length and joint angle, the angle change waveform is a change image of each joint angle within a set time interval, the video is a color image taken in real time by a depth camera, and the animation is an animation demonstration image of the current human posture.

[0046] Reference Figure 3 , the working principle of the present invention is as follows:

[0047] When the system starts running, it first uses the depth camera to capture the image of the current human body, and transmits the captured color image and depth image to the local computer via a USB data cable; secondly, the human body key point detection method is used to identify the human body's joints from the color image, and the spatial coordinates of each joint point are calculated; thirdly, the required human limb parameters and joint motion data are solved by the formula and sent in real time; then, when the collection task is completed, the collected data is saved, and the patient's individual data is used to plan the rehabilitation trajectory; finally, the human body's individual data and rehabilitation motion trajectory are sent to the remote computer, and the rehabilitation therapist provides online rehabilitation guidance.

Claims

1. An individual rehabilitation motion trajectory planning system based on non-contact human body data acquisition, comprising a human body image acquisition module (1), a data calculation and storage module (2), and a data remote sharing module (3); the human body image acquisition module (1) and the data calculation and storage module (2) are connected by wire; the data calculation and storage module (2) and the data remote sharing module (3) are connected by wireless, characterized in that: The human body image acquisition module (1) comprises a depth camera and a camera bracket, and is used to capture a color image and a depth image of a human body; The data calculation and storage module (2) adopts a local computer and comprises: An image registration submodule (21) is used to receive the color image and depth image captured by the human body image acquisition module (1) and perform image registration processing; A data solver module (22) is used to calculate the vectors of each segment of the human body through key point detection, obtain human body parameters and joint motion data, and encapsulate the color image and the solved data into a data structure and send it in real time; The trajectory planning submodule (23) is used to plan the patient's rehabilitation movement trajectory according to the human individual data and rehabilitation needs calculated by the data solving submodule (22); specifically, the connection fixing parameters between the rehabilitation device and the human body are first adjusted; then the movement angle of the joint is optimized according to the connection fixing parameters, the limb length of the rehabilitation part and the range of joint movement to avoid discomfort to the patient during the rehabilitation movement process; finally, a polynomial fitting is performed on the optimized joint angle to generate a continuous rehabilitation movement trajectory for guiding the patient's rehabilitation movement; A storage submodule (24) is used to save the trajectory data and the human body data as a local file, and to perform an animation demonstration on the planned trajectory to intuitively display the rehabilitation movements of the human body; The data remote sharing module (3) comprises a cloud server and a remote computer, which are connected wirelessly. The cloud server receives the calculation results of the data calculation storage module (2) and sends them to the remote computer, and the remote computer reproduces the received data in the form of numerical characters, waveforms, videos and animations.

2. The system according to claim 1, characterized in that When the human body image acquisition module (1) acquires human body images, the placement position and height of the depth camera are adjusted according to the height of the user.

3. The system according to claim 1, characterized in that The depth image acquired by the human body image acquisition module (1) is a collection of distances between human body joints and the optical center of the camera in space.

4. The system according to claim 1, characterized in that The image registration submodule (21) performs image registration processing on the color image and the depth image captured by the human body image acquisition module (1), using the coordinate system of the color image as a reference, and maps the depth image to the color image coordinate system through spatial transformation, so that the position of any point in the space corresponds one-to-one in the two images.

5. The system according to claim 1, characterized in that The data solver module (22) obtains human body parameters and joint motion data, and is implemented as follows: First, the pixel positions of human joints in the color image are detected using the existing human key point detection program. R ,v R ), find the depth value d of the corresponding point from the registered depth image, and calculate the three-dimensional spatial coordinates (x, y, z) of the human joint point: x=z(u R -c x ) / f x y=z(v R -c y ) / f y z=d / s Among them, (c x ,c y ) represents the camera optical center position, (f x ,f y ) represents the focal length of the camera, s represents the scale factor related to the depth value unit, and s=1000; Secondly, according to the three-dimensional spatial coordinates of the joint points, the length l and vector of each segment of the human body are calculated respectively. Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) represent the spatial position coordinates of two adjacent joint points respectively; Again, according to the vectors of each segment of the human body, calculate the angle θ of each joint: in, and Respectively represent the vectors of two adjacent limbs.

6. The system according to claim 1, characterized in that The remote computer in the data remote sharing module (3) reproduces the received data in the form of numerical characters, waveforms, videos and animations. A data transmission program is written on the remote computer based on the Socket network to receive the human personality data sent by the cloud server, and a user interface is designed using an MFC software development tool. The limb length values ​​and joint angle values ​​of the rehabilitation parts are displayed in the user interface, and an angle waveform image of the joint angle within a set time is drawn. A real-time color image is displayed and a simulated animation of the movement is displayed, so that the rehabilitation therapist can provide rehabilitation guidance online.

Citation Information

Patent Citations

  • Human body motion capture and gait analysis method based on RGBD depth camera

    CN110837751A

  • Automated system for workspace, range of motion and functional analysis

    US20130324857A1

  • An Error Modeling Method For End-Effector Space-Curve Trajectory Of Six Degree-of-Freedom Robots

    US20190176325A1