A digital human body modeling method based on action recognition

Through three-dimensional depth image processing and infrared technology, the identification of human body nodes is solved, and the problems of complex construction of traditional digital human body models and inaccurate motion capture are achieved, simple and low-cost digital human body modeling and high-precision motion capture are achieved, satisfying the medical care Research and rehabilitation training needs.

CN112184898BActive Publication Date: 2024-12-03ANHUI DYNAMIC INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202011131281.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-10-21
Publication Date
2024-12-03
Estimated Expiration
2040-10-21

AI Technical Summary

Technical Problem

The steps for building traditional digital mannequin are complicated and cannot accurately display human movements, which brings inconvenience to medical applications.

Method used

By obtaining three-dimensional depth images, identifying human body parts, integrating the joint nodes to form a skeleton model, and calculating human body movements using specific joint nodes, using infrared cameras and image processing algorithms to capture and build human body motion.

Benefits of technology

It realizes simple construction and high-precision motion capture of digital mannequins, reduces the cost of use and meets the needs of medical research and rehabilitation training.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112184898B_ABST
    Figure CN112184898B_ABST
Patent Text Reader

Abstract

The present invention discloses a digital human modeling method based on action recognition, which belongs to the field of digital modeling technology, and includes the following steps: obtaining a three-dimensional depth image of a person and a scene through a camera; separating the human body from the background and identifying the human body parts by performing image processing on the depth image; obtaining the coordinates of the human body joints according to each human body part, integrating all the joints to form a human body skeleton model; selecting specific joints in the above human body skeleton model as target features, and calculating the human body action through the target features. By collecting the image of the human body, separating the human body image part from the background, and then building a skeleton model according to the human body part, the motion state of the person is calculated according to the state of the joints in the skeleton model. Therefore, the digital modeling method can accurately simulate the human body, which is convenient for medical scholars to explore the mysteries of life and solve medical problems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of digital modeling, and in particular to a digital human body modeling method based on action recognition. Background Art

[0002] Digital human body is a world-leading scientific research topic combining life science and information science, and is an important field for cross-disciplinary research. Digital human body mainly uses modern high-tech to digitize the human body structure, explore the mysteries of life, and solve medical problems through digital human body modeling. The so-called digital human body mathematical model refers to the mathematical expression that describes the interaction between human elements, subsystems, levels, and the interaction between the human body system and the environment. The construction of digital human body mathematical model is a tool for quantitative and microscopic qualitative analysis of human body system, which points out the direction for the development of medicine. With the acquisition of CT, MRT, milling specimen images, remote sensing technology, supercomputers, and high-performance workstations, the foundation has been laid for the construction of digital human body mathematical model. Using computers to solve complex mathematical equations, such as high-order algebraic equations and nonlinear differential equations, the operation procedures are greatly simplified. The close combination of computational mathematics and high-performance computers is a significant feature of digital human body mathematical model. Based on advanced calculation methods, scientific graphics operations and high-speed computers, scientific visualization has become a highlight of digital human body mathematical model.

[0003] The construction of traditional digital human mathematical models generally involves complex steps and requires a large amount of basic data to build a complete human body model. In addition, it is difficult to capture the body's movements and postures, and it is difficult to accurately display the current body's movements through digital models, which brings inconvenience to related treatments using human body models. Summary of the invention

[0004] The purpose of the present invention is to provide a digital human body modeling method based on motion recognition in order to solve the problem that the steps of building the above-mentioned digital human body model are complicated and the constructed model cannot accurately display the corresponding human body movements. The method has the advantages of simple construction of the digital human skeleton model, low cost of use, and more accurate human body motion capture.

[0005] The present invention achieves the above-mentioned purpose through the following technical scheme, a digital human body modeling method based on action recognition, comprising the following steps:

[0006] Obtain three-dimensional depth images of people and scenes;

[0007] By performing image processing on the depth image, human body parts are identified;

[0008] Obtain the coordinates of human joints according to various human body parts, and integrate all joints to form a human skeleton model;

[0009] Specific joint points in the above human skeleton model are selected as target features, and human body movements are calculated through the target features.

[0010] Preferably, the three-dimensional depth image is collected by an infrared camera with an infrared transmitter, and the infrared transmitter is used to project a near-infrared spectrum so that the infrared light irradiates a rough object or penetrates frosted glass to form speckles which are then read by the infrared camera.

[0011] Preferably, the method for obtaining the three-dimensional depth image of the person and the scene is:

[0012] A. Take a reference surface at a certain distance and record its speckle pattern;

[0013] B. These reference surfaces are combined to form a speckle pattern for the entire space, completing the light source calibration;

[0014] C. Perform cross-correlation operations on the speckle pattern of each object and the speckle pattern of each reference surface, and determine the spatial position and shape of the object based on the operation results.

[0015] Preferably, the human body part recognition separates the human body from the background through an edge detection algorithm, a noise threshold processing algorithm and an algorithm for extracting target feature points, and recognizes the human body part by scanning the pixels of the human body part to determine the part of the human body to which the pixel belongs.

[0016] Preferably, the human body joints are skeletal joints at the connection of two human body parts, and the joints are divided into three categories including trunk joints, height joints and limb joints. The approximate shape of the human skeleton is determined by the trunk joints; the height joints judge the height of the human body by the vertical distance from the head node to the left and right foot nodes, and the height joints can also be used to judge the upright or sitting state of the human body; the limb joints are used to determine the movement posture characteristics of the human body.

[0017] Preferably, the target features include joint point angles, joint point vector direction angles and joint point position differences. The degree of joint bending is calculated by the joint point angles, the position of the joint is calculated by the joint point vector direction angles, and the coordinates of other joint points are calculated by selecting a joint point coordinate as a reference coordinate point using the joint point position differences.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] 1. By collecting images of the human body and separating the human body image part from the background, a skeleton model is built based on the human body part, and the motion state of the person is calculated according to the state of the joints in the skeleton model. Therefore, this digital modeling method can accurately simulate the human body, making it easier for medical scholars to explore the mysteries of life and solve medical problems.

[0020] 2. By classifying the skeletal joints and accurately calculating the human body's movements based on the joint point angles and joint point vector direction angles, the human skeleton model can visualize human body movements and meet the interactive needs of human rehabilitation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a flow chart of the method for constructing a digital human body model of the present invention.

[0022] Figure 2 The figure is a schematic diagram of a human skeleton model composed of joints of the present invention.

[0023] Figure 3 It is a schematic diagram showing the joint points of the present invention in the coordinate system. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1 As shown, a digital human body modeling method based on action recognition includes the following steps:

[0026] Step S101, a three-dimensional depth image of a person and a scene is obtained by a camera, wherein the camera includes an infrared camera with an infrared transmitter, and the infrared transmitter is used to project a near-infrared spectrum, so that after infrared light irradiates a rough object or penetrates frosted glass, speckles are formed and then read by the infrared camera, and the infrared camera transmits the read speckle signal to an image processing chip, and the image processing chip processes the three-dimensional depth image of the person and the scene in the following method:

[0027] A. Take a reference surface every 5mm and record its speckle pattern. The narrower the spacing, the denser the speckle pattern, and the finer the three-dimensional image formed. However, the narrower the spacing, the higher the difficulty of data processing. Therefore, it is best to choose a reference surface with a spacing of 3mm-8mm according to the image processing capability.

[0028] B. These reference surfaces are combined to form a speckle pattern for the entire space, completing the light source calibration;

[0029] C. Due to the randomness of laser speckle, the speckle pattern of each object in space is different. The speckle pattern of each object is cross-correlated with the speckle pattern of each reference surface. The spatial position and shape of the object can be determined based on the calculation results.

[0030] Step S102, by performing image processing on the depth image, the human body and the background are separated from each other, and the human body parts are identified. The human body and the background are separated from each other by an edge detection algorithm, a noise threshold processing algorithm and an algorithm for extracting target feature points. The human body parts are identified by scanning the pixels of the human body parts to determine the part of the human body part to which the pixel belongs. The pixels of each human body part are different, and the standard human body pixels are stored in a database. When scanning the pixel images of various parts of the human body in the depth image, they are compared with the standard pixels in the database to determine the probability of the corresponding body part. For example, a certain pixel has an 80% probability of being a foot, a 50% probability of being a head, and a 20% probability of being a hand. At this time, the human body part to which the pixel belongs will not be directly determined by the maximum probability, but all possibilities are input into the next processing step until all body parts are fully identified and then judged. The body part is determined by the probability with the highest and non-repetitive probability. If there is a repeated probability, the body part is determined according to the human body part pixels of the adjacent part of the pixel.

[0031] Step S103, obtaining the coordinates of the human body joints according to each human body part, and integrating all the joints to form a human body skeleton model;

[0032] Step S104, selecting specific joint points in the human skeleton model as target features, and calculating human body movements through the target features.

[0033] The human joints are the skeletal joints at the connection of two human parts. The joints are divided into three categories, including trunk joints, height joints and limb joints. The trunk joints are used to determine the approximate shape of the human skeleton; the height joints are used to determine the height of the human body by the vertical distance from the head node to the left and right foot nodes, and the height joints can also be used to determine whether the human body is standing upright or sitting; the limb joints are used to determine the movement posture characteristics of the human body, such as Figure 2As shown, the trunk joints mainly include eight joints, such as the head, left and right shoulders, spine, left and right hips, etc., which are respectively composed of joints 1.2.3.4.11.12.13.14. The height joints can be used to calculate the height of the human body through the distance between joints 1 to joints 19 and joints 20. The limb joints are composed of 5.6.7.8.9.10.15.16.17.18.19.20. The target features include joint angles, joint vector direction angles and joint position differences. The degree of joint bending is calculated by the joint angles, and the position of the joint is calculated by the joint vector direction angles. The joint position difference selects a joint coordinate as a reference coordinate point to calculate the coordinates of other joints. As shown in FIG. Figure 3 As shown, raise the left arm and select the elbow joint p 6 and wrist joint 8 Make the joint point vector p 8 p 6 , this vector will produce angles α, β, and γ with the X-axis, Y-axis, and Z-axis, and these angles can be used to determine the spatial position of this joint.

[0034] The joint point vector can be expressed as

[0035] p 8 p 6 =(x 6 -x 8 )i+(y 6 -y 8 )j+(z 6 -z 8 )

[0036] The vector direction angle can be expressed as

[0037] Angle with X axis:

[0038] Angle with Y axis:

[0039] Angle with Z axis:

[0040] In this way, the spatial position of each joint can be uniquely determined.

[0041] The joint point position difference is normalized by selecting a joint as the normalized joint reference, and the whole body joints are normalized. If the selected joint point p 11 As the reference joint point, the joint point p 2 to p 11 The length of this joint is used as the normalized reference, and the remaining joint points can be expressed as

[0042] x′ i =x i -x11 1≤i≤20

[0043] y′ j =y i -y 11 1≤j≤20

[0044] z′ k =z k -z 11 1≤k≤20

[0045] Thus, the new position coordinates (x′, y′, z′) of the 20 joint points can be obtained, and the spatial position difference of the joint points is D=(x′ 1 , y′ 1 , z′ 1 , ..., x′ 20 , y′ 20 , z′ 20 ), joint point p 2 to p 11 The length of this joint can be expressed as

[0046]

[0047] Then the normalized bone joint points can be expressed as

[0048]

[0049] This achieves the standardization of joint point coordinates.

[0050] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0051] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A digital human modeling method based on action recognition, It is characterized in that The following steps are involved: Obtain three-dimensional depth images of people and scenes; By performing image processing on the depth image, human body parts are identified; Obtain the coordinates of human joints according to various human body parts, and integrate all joints to form a human skeleton model; Select specific joint points in the human skeleton model as target features, and calculate human body movements through the target features; Among them, the human body part is identified by scanning the pixels of the human body part to determine which part of the human body the pixel belongs to. The pixels of each human body part are different, and the standard human body pixels are stored in a database. When scanning the pixel images of various parts of the human body in the depth image, they are compared with the standard pixels in the database to determine the probability of the corresponding body part. For pixels with multiple probabilities, the human body part to which the pixel belongs will not be directly determined by the maximum probability. Instead, all possibilities are input into the next processing step until all body parts are identified. The body part is determined by selecting the probability with the highest and non-repetitive probability. If there is a repeated probability, the body part is determined based on the human body part pixels adjacent to the pixel.

2. According to the method for digital human body modeling based on action recognition in claim 1, It is characterized in that The three-dimensional depth image is collected by an infrared camera with an infrared transmitter. The infrared transmitter is used to project a near-infrared spectrum so that the infrared light irradiates a rough object or penetrates frosted glass to form speckles which are then read by the infrared camera.

3. A digital human body modeling method based on action recognition according to claim 2, It is characterized in that The method for obtaining the three-dimensional depth image of the person and the scene is: A. Take a reference surface at a certain distance and record its speckle pattern; B. These reference surfaces are combined to form a speckle pattern for the entire space, completing the light source calibration; C. Perform cross-correlation operations on the speckle pattern of each object and the speckle pattern of each reference surface, and determine the spatial position and shape of the object based on the operation results.

4. The method for digital human body modeling based on action recognition according to claim 1, It is characterized in that The human body part recognition separates the human body from the background through an edge detection algorithm, a noise threshold processing algorithm and an algorithm for extracting target feature points, and recognizes the human body part by scanning the pixels of the human body part to determine the part of the human body to which the pixel belongs.

5. The method for digital human body modeling based on action recognition according to claim 1, It is characterized in that The human body joints are skeletal joints at the connection of two human body parts. The joints are divided into three categories, including trunk joints, height joints and limb joints. The trunk joints are used to determine the approximate shape of the human skeleton; the height joints are used to determine the height of the human body by the vertical distance from the head node to the left and right foot nodes, and the height joints can also be used to determine the upright or sitting state of the human body; the limb joints are used to determine the movement posture characteristics of the human body.

6. The method for digital human body modeling based on action recognition according to claim 1, It is characterized in that The target features include joint point angles, joint point vector direction angles and joint point position differences. The joint bending degree is calculated through the joint point angles, and the position of the joint is calculated through the joint point vector direction angles. The joint point position differences are used to calculate the coordinates of other joint points by selecting a joint point coordinate as a reference coordinate point.

Citation Information

Patent Citations

  • Method for recovering real-time three-dimensional body posture based on multimodal fusion

    CN102800126A

  • Man-computer interaction method for intelligent human skeleton tracking control robot on basis of kinect

    CN103399637A

  • Video capture contrasting system in industrial production

    CN104378605A

  • Posture detection system

    JP2012120647A

  • Method for implementing human skeleton tracking system based on depth data

    US20180047175A1