A human height detection method based on skeleton key point recognition

Abstract

The application discloses a human height detection method based on skeleton key point recognition, which comprises the following steps: firstly, obtaining an image containing a target to be detected by a depth camera in real time; obtaining pixel coordinates of skeleton key points of the target to be detected in the image by using a human skeleton detector; calculating a relative height from a nose to an ankle in the image based on the pixel coordinates of the skeleton key points; bringing the pixel coordinates of any skeleton key point into the depth camera to calculate a distance between the skeleton key point and the depth camera; then calculating a longitudinal distance visible to the depth camera; and finally calculating a real height of the target to be detected. The application accurately utilizes a depth camera to cooperate with skeleton detection to measure the height of a human body, and the height of the human body can be detected even when the human body is in a non-standing posture. The application only needs a depth image, and then outputs a reliable result, thereby saving a large amount of manpower and time.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of human height detection, and particularly relates to a human height detection method based on skeleton key point recognition. BACKGROUND

[0002] In daily life, height is of great concern. However, judging height by the naked eye is always non-standard. Judging height by a depth camera can not only improve accuracy but also automatically obtain human height information by a computer for further processing. In recent years, with the development of deep learning and image processing technology, height detection technology is also constantly developing and improving. At present, height detection technology is very lacking, and there are few technologies for height detection by neural networks. Existing height detection needs to use a reference for comparison, and accuracy and practicality are lacking. The development trend of height detection technology is constantly developing towards more accurate, diversified and personalized. At present, height detection needs to calibrate a reference, and the height of a person is obtained by converting the scale ratio of the person and the reference. This is not only difficult to determine the reference, but also the algorithm needs to be implemented in a fixed scene, which is very inconvenient.

[0003] Recently, researchers from the University of California, San Diego and Adobe have proposed a monocular vision-based measurement method that can recover the absolute scale of a scene and target by measuring the height of the target in the photo, the height of the camera, and the viewing angle orientation parameter. This technology can accurately achieve monocular vision in an unrestricted environment. Figure 1 For its detection result graph, it can be found that its detection principle is to compare the white stool to convert the scale, so when the size of the white stool changes, the height of the human body will change accordingly. When there is no stool in the environment, the height of the human body cannot be detected, and when the human body is in different postures, the detected height will also change accordingly, and the real-time performance is poor. In the method and device for estimating the absolute depth of an image of patent CN115797432 A, the target's Mahalanobis trunk index is calculated according to the skeleton key points, and then the absolute height of the target is obtained from the corresponding table of Mahalanobis trunk index and height according to the Mahalanobis trunk index through table lookup.

[0004] In the fields of three-dimensional reconstruction, medical treatment, and clothing size, human height data is indispensable. In most cases, we will ask the testee to stand straight, and then measure the height with an instrument or other tools, which will consume a lot of time and manpower. Especially in practical applications, if we do not have measuring tools, or the person being measured is a child, or is injured and cannot stand straight, it will be very difficult to measure the height. SUMMARY

[0005] The application aims to design a human height detection method based on skeleton key point recognition, solve the technical problem that the current height measurement technology needs a reference object and cannot measure the height of a human body in a non-standing posture. The height of the target to be measured is obtained according to the image of the target to be measured obtained by the depth camera, and the technical blank of height recognition by deep learning is filled.

[0006] In order to achieve the above-mentioned purpose, the application relates to a human height detection method based on skeleton key point recognition, which specifically comprises the following steps:

[0007] (1) obtaining an image containing a target to be measured by a depth camera in real time, the image size is M pixels x N pixels, when the target to be measured stands, step (2) is performed, and when the target to be measured does not stand, step (3) is performed;

[0008] (2) obtaining the pixel coordinates of the skeleton key points of the target to be measured in the image by using a human skeleton detector, including nose (X0, Y0) and right ankle (X4, Y4), or nose (X0, Y0) and left ankle (X7, Y7),

[0009] calculating the relative height Y of the nose to the ankle in the image through the nose (X0, Y0) and the right ankle (X4, Y4), or the nose (X0, Y0) and the left ankle (X7, Y7),

[0010] or

[0011] (3) obtaining the pixel coordinates of the skeleton key points of the target to be measured in the image by using a human skeleton detector, including nose (X0, Y0), neck (X1, Y1), right hip (X2, Y2) and right knee (X3, Y3), right ankle (X4, Y4), or nose (X0, Y0), neck (X1, Y1), left hip (X5, Y5), left knee (X6, Y6) and left ankle (X7, Y7),

[0012] calculating the relative distance A of the neck to the hip in the image,

[0013] or

[0014] calculating the relative distance B between the hip and the knee in the image,

[0015] or

[0016] calculating the relative distance C of the knee to the ankle in the image,

[0017] or

[0018] Y is the relative height of the nose to the ankle in the image:

[0019] Y=Y0-Y1+A+B+C;

[0020] (4) The pixel coordinates of any bone key point in step (2) or (3) are brought into the depth camera, the distance Z between the bone key point and the depth camera is calculated, and the longitudinal distance H visible to the depth camera is calculated according to the following formula,

[0021]

[0022] Wherein, Z is the distance between the target to be measured and the camera, and θ is the longitudinal angle of the depth camera shooting image;

[0023] (5) Finally, the real height S of the target to be measured is calculated according to the following formula:

[0024]

[0025] Wherein, L is the real height of the nose to the ankle, and μ is the proportional constant between the real height S of the target to be measured and the height L of the nose to the ankle.

[0026] Specifically, the depth camera is RealSense, and the human body skeleton detector is openpose.

[0027] Specifically, the actual distance Z between the target to be measured and the camera is calculated according to the function aligned_depth_frame.get_distance(x,y) in the depth camera Realsense.

[0028] Compared with the prior art, the application accurately uses the depth camera to cooperate with the skeleton detection for human height measurement, and the height detection can also be performed when the person is in a non-standing posture. The application only needs a depth image, and then outputs reliable results, saving a lot of manpower and time. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 It is a human height measurement method based on monocular vision in the prior art.

[0030] Figure 2 It is a human height detection method flow chart based on skeleton key point recognition of the application.

[0031] Figure 3 It is a RealSense camera photo involved in example 1.

[0032] Figure 4 It is a network structure diagram of Openpose involved in example 1.

[0033] Figure 5 Fig. 1 is a diagram of human skeleton key point coordinates in standing posture.

[0034] Figure 6 Fig. 2 is a diagram of human skeleton key point coordinates in non-standing posture.

[0035] Figure 7 Fig. 3 is a diagram of RealSense RGB image size in standing posture.

[0036] Figure 8 Fig. 4 is a diagram of size between target and camera in shooting. Figure 7 DETAILED DESCRIPTION

[0037] The application will be further described by examples.

[0038] Example 1

[0039] The human height detection method based on skeleton key point recognition in this embodiment uses Openpose as a human skeleton detector, and combines RealSense depth camera to obtain the distance between the depth camera and the human, and judges the human height in combination with the distance information, and the specific process is as shown in Fig. 5. Figure 2

[0040] The two-dimensional position information of each pixel point in the target image obtained by the depth camera in this embodiment can be expressed by pixel coordinates, and the depth camera can also obtain the distance between each pixel point and the depth camera through the pixel coordinates, and the target is a human. The depth camera used in this embodiment is RealSense, and the manufacturer is Intel. The implementation principle of the current depth camera mainly includes three kinds: structure light, Tof, and binocular imaging. RealSense uses the structure light scheme. Figure 3 Fig. 6 is a RealSense camera, and the four cameras on the front are, from left to right, a left infrared camera, an infrared dot array projector, a right infrared camera, and an RGB camera.

[0041] The joint points of the human body in the target image are detected by the human skeleton detector in this embodiment as skeleton key points, and the human skeleton information is described by the skeleton key points. OpenPose is an open source library based on convolutional neural network and supervised learning and written in caffe framework, which can realize tracking of human facial expression, torso, limbs and even fingers, and is suitable for single person and multiple persons, and has good robustness. It can be called the first real-time multi-person two-dimensional pose estimation based on deep learning in the world, and is a milestone in human-computer interaction, which provides a high-quality information dimension for machine understanding of human.

[0042] ​​The embodiment relates to a human height detection method based on skeleton key point recognition, and specifically comprises the following steps.

[0043] (1) Real-time image containing a to-be-detected target is acquired through a depth camera (RealSense), the image size is M pixels x N pixels, when the to-be-detected target stands, step (2) is performed, and when the to-be-detected target does not stand, step (3) is performed.

[0044] In the embodiment, the image is a RealSense RGB image, the RealSense RGB image size is M pixels x N pixels, the pixel coordinate value of the upper right corner of the image is defined as (0, 0), and the pixel coordinate value of the lower left corner of the image is defined as (M, N).

[0045] (2) A human skeleton detector (such as openpose) is used to acquire pixel coordinates of skeleton key points of the to-be-detected target in the image, including a nose (X0, Y0) and a right ankle (X4, Y4), or the nose (X0, Y0) and a left ankle (X7, Y7),

[0046] The relative height Y of the nose to the ankle in the image is calculated through the nose (X0, Y0) and the right ankle (X4, Y4), or the nose (X0, Y0) and the left ankle (X7, Y7).

[0047] Or

[0048] (3) A human skeleton detector (such as openpose) is used to acquire pixel coordinates of skeleton key points of the to-be-detected target in the image, including a nose (X0, Y0), a neck (X1, Y1), a right hip (X2, Y2) and a right knee (X3, Y3), a right ankle (X4, Y4), or the nose (X0, Y0), the neck (X1, Y1), a left hip (X5, Y5), a left knee (X6, Y6) and a left ankle (X7, Y7),

[0049] The relative distance A of the neck to the hip in the image is calculated,

[0050] Or

[0051] The relative distance B between the hip and the knee in the image is calculated,

[0052] Or

[0053] The relative distance C of the knee to the ankle in the image is calculated,

[0054] Or

[0055] Y = Y0-Y1+A+B+C;

[0056] Y = Y0-Y1+A+B+C;

[0057] (4)Bring the pixel coordinates of any of the skeletal key points in step (2) or (3) into the depth camera, calculate the distance Z between the skeletal key point and the depth camera, and then calculate the longitudinal distance H visible to the depth camera according to the following formula, the longitudinal distance H corresponds to N in the image of the target to be measured:

[0058]

[0059] Wherein, Z is the distance between the target to be measured and the camera, and θ is the longitudinal angle of the depth camera when shooting the image;

[0060] Since the target to be measured is basically parallel to the depth camera when shooting, the distance between any skeletal key point and the depth camera is the same.

[0061] In this embodiment, the actual distance Z between the target to be measured and the camera is calculated according to the function aligned_depth_frame.get_distance(x,y) in the depth camera Realsense, and then the longitudinal distance H visible to the RealSense camera is calculated according to the following formula:

[0062]

[0063] Wherein, Z is the distance between the target to be measured and the camera, and θ is the longitudinal angle of the RealSense camera when shooting the Realsense RGB image;

[0064] (5) Finally, the real height S of the target to be measured is calculated according to the following formula:

[0065]

[0066] Wherein, L is the real height from the nose to the ankle, L corresponds to Y, and μ is the proportional constant between the real height S of the target to be measured and the height L from the nose to the ankle, and based on a large amount of experimental data, it is verified that μ = 1.07.

Claims

1. A human height detection method based on skeleton key point recognition, characterized in that, Specifically comprising the following steps: (1) Real-time acquisition of images containing the target to be measured by a depth camera, the image size is M pixels x N pixels, when the target to be measured stands, step (2) is performed, when the target to be measured does not stand, step (3) is performed; (2) The pixel coordinates of the skeletal key points of the target to be measured in the image are obtained by using a human body skeleton detector, including nose (X0, Y0) and right ankle (X4, Y4), or nose (X0, Y0) and left ankle (X7, Y7), The relative height Y of the nose to the ankle in the image is calculated by the nose (X0, Y0) and the right ankle (X4, Y4), or the nose (X0, Y0) and the left ankle (X7, Y7): or (3) The pixel coordinates of the skeletal key points of the target to be measured in the image are obtained by using a human body skeleton detector, including nose (X0, Y0), neck (X1, Y1), right hip (X2, Y2) and right knee (X3, Y3), right ankle (X4, Y4), or nose (X0, Y0), neck (X1, Y1), left hip (X5, Y5), left knee (X6, Y6) and left ankle (X7, Y7), The relative distance A of the neck to the hip in the image is calculated, or The relative distance B between the hip and the knee in the image is calculated, or The relative distance C of the knee to the ankle in the image is calculated, or Then: the relative height Y of the nose to the ankle in the image: Y=Y0-Y1+A+B+C; (4) The pixel coordinates of any skeletal key point in step (2) or (3) are brought into the depth camera to calculate the distance Z between the skeletal key point and the depth camera, and then the longitudinal distance H visible by the depth camera is calculated according to the following formula, Wherein, Z is the distance between the target to be measured and the camera, θ is the longitudinal angle of the depth camera when shooting the image; (5) Finally, the real height S of the target to be measured is calculated according to the following formula: Wherein, L is the real height from the nose to the ankle, μ is the proportional constant between the real height S of the target to be measured and the height L from the nose to the ankle, μ=1.

07.

2. The human height detection method based on skeleton key point recognition according to claim 1, characterized in that, The depth camera is RealSense, and the human body skeleton detector is openpose. 3.The human height detection method based on skeleton key point recognition according to claim 1, characterized in that, According to the function aligned_depth_frame.get_distance(x,y) in the depth camera Realsense, the actual distance Z between the target to be measured and the camera is calculated.

Images