A method, device and storage medium for human body position recognition
By combining depth cameras and keypoint detection networks with camera intrinsic parameters to calculate the three-dimensional coordinates of the human body, and using statistical modeling to identify body position, the problems of accuracy and computation speed in human pose recognition in capsule endoscopy examination systems have been solved, achieving efficient body position detection and automated guidance.
Patent Information
- Application Number
- CN202610561009.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-27
- Publication Date
- 2026-07-31
AI Technical Summary
Existing technologies in capsule endoscopy systems suffer from low accuracy in recognizing human posture, high computational overhead, and slow processing speed, making it difficult to effectively support automated guidance of capsule endoscopy.
A depth camera is used to capture both color and depth images simultaneously. Two-dimensional pixel coordinates of human body key points are extracted through a key point detection network. Three-dimensional coordinates are calculated by combining the camera intrinsic parameter matrix. Body posture is identified through statistical modeling, and angle thresholds are set using statistical methods to distinguish body postures.
It improves the accuracy of human body position detection, reduces system computing power consumption, increases processing speed, and supports automated guidance of capsule endoscopy.
Smart Images
Figure CN122493488A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a human body position recognition method, specifically to a vision-based human body position recognition method, device, and storage medium. Background Technology
[0002] Traditional capsule endoscopy systems require an external magnetic guidance system to move the capsule endoscope inside the body, enabling a comprehensive examination of the gastrointestinal tract. In fully automated capsule endoscopy systems, an external computer with artificial intelligence guides the capsule endoscope to automatically navigate and photograph the gastrointestinal tract, identifying lesions. In this scenario, the patient's different postures significantly affect the capsule endoscope's path; therefore, accurate identification of the patient's postures is crucial for providing robust technical support for automated capsule endoscopy guidance.
[0003] Chinese Patent Application No. 2024109965441 discloses a method, device, computer equipment, and storage medium for detecting the position of an endoscopic examination subject. This technical solution uses an attitude angle measurement sensor to acquire real-time measurement data of the subject, and the subject needs to wear a smart terminal for further processing. However, the measurement accuracy of this technical solution is not accurate enough and the error is relatively large.
[0004] Chinese Patent 2024102641884 provides a method and system for recognizing body postures. This technical solution uses a pre-trained body posture recognition model to recognize the lying posture of the subject. The recognition scheme of this technical solution has high computing power requirements, large system overhead, and the computing speed needs to be further improved.
[0005] Therefore, it is necessary to develop a body position recognition method with high recognition accuracy, low system computing power overhead, and fast processing speed. Summary of the Invention
[0006] This invention provides a method for recognizing human body position, comprising the following steps: Step 101: Use a depth camera to simultaneously capture color and depth images in the populated area; Step 102: Feed the color image into the key point detection network to extract the two-dimensional pixel coordinates of the left shoulder point, right shoulder point, left hip joint point and right hip joint point of the human body; Step 103: Based on the depth values of the two-dimensional pixel coordinates in the depth map, obtain the three-dimensional coordinates of the left shoulder point, right shoulder point, left hip joint point, and right hip joint point in the camera coordinate system through the calibrated camera intrinsic parameter matrix and the two-dimensional pixel coordinates. Step 104: Identify the left lateral, right lateral, and supine positions of the human body using statistical modeling based on the three-dimensional coordinates.
[0007] Furthermore, the key point detection network further includes a downsampling layer and an upsampling layer. The downsampling layer further includes a convolutional layer, a pooling layer, a batch normalization layer, and an activation layer. The upsampling layer further includes an unpooling layer and a deconvolutional layer.
[0008] Furthermore, after receiving the color image, the key point detection network processes it through a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer. Then, the result processed by the fourth downsampling layer is processed through a first upsampling layer, a second upsampling layer, and a third upsampling layer. The output result is then concatenated with the second downsampling layer and processed by a fifth downsampling layer. The output result is then processed with the data from the third downsampling layer by a sixth downsampling layer. The output result is then processed with the fourth downsampling layer by a seventh downsampling layer. The output result from the seventh downsampling layer is then processed by an eighth downsampling layer and further processed by a fully connected layer. Finally, the two-dimensional pixel coordinates of the four key points are output.
[0009] Furthermore, the three-dimensional coordinates are obtained through the following method: Acquire camera intrinsic parameters and depth information from depth maps; Establish the mapping relationship between pixel coordinates and 3D coordinates. The camera model is represented by Equation 1: Formula 1; Where f x f y c x and c y All of these are camera intrinsic parameters, and their unit of measurement is pixels, where f x It is the product of the camera's focal length f and the image pixel width ratio α; f y It is the product of the camera's focal length f and the image pixel height ratio β; c x c is the distance along the x-axis from the camera's optical center to the origin of the pixel coordinate system; y This is the distance along the y-axis from the camera's optical center to the origin of the pixel coordinate system; Further converting Formula 1 into matrix form, it is represented by Formula 2: Formula 2; in and These are the row and column coordinates of pixels. X, Y, and Z are the 3D coordinates of a point on the image based on the camera coordinate system. Z is obtained directly from the depth map generated by the camera. The camera's intrinsic parameters are obtained through camera calibration using the OpenCV checkerboard calibration method. x f y c x and c y ; The 3D coordinates of the four identified key points are used to obtain three body postures through statistical modeling. The left shoulder point is designated p1, the right shoulder point p2, the left hip node p3, and the right hip node p4. The camera's z-axis direction vector is... Then the direction vectors of shoulder points p1 and p2 Direction vectors of hip joint points p3 and p4 and z-axis direction vector The included angle and It can be calculated using formula 3: ; , formula 3; Finally, statistical methods were used to experimentally test the three body positions—left lateral decubitus, right lateral decubitus, and supine—and the included angles were calculated for each position. and The angle thresholds thresh1 and thresh2, which distinguish these three body postures, are obtained, and the body posture is identified by formula 4: Lying on your back; Lying on your left side; Right side lying, Formula 4; in express or ,and and Both satisfy the above relationship requirements: thresh1 = 60° and thresh2 = 140°, with the unit being degrees.
[0010] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the human body position recognition method as described in any of the first aspects.
[0011] Thirdly, the present invention provides a body position recognition device, comprising: One or more processors; Memory; and One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, wherein the processors, when executing the computer programs, implement the steps of the human body position recognition method as described in any of the first aspects.
[0012] The human body position recognition method of the present invention can significantly improve the accuracy of human body position detection, with low system computing power and fast processing speed, providing technical support for the automated guidance of magnetically controlled capsule endoscopy detection system. Attached Figure Description
[0013] Figure 1 : Flowchart of the human body position recognition method of the present invention.
[0014] Figure 2 : Key point detection network architecture diagram of the human body position recognition method of the present invention.
[0015] Figure 3 : A schematic diagram of the transformation between the pixel coordinate system and the camera coordinate system of the present invention.
[0016] Figure 4 : A schematic diagram of human body position recognition according to the present invention.
[0017] Figure 5 : Schematic diagram of the body position recognition device of the present invention. Detailed Implementation
[0018] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.
[0019] refer to Figure 1 The flowchart of the human body position recognition method of the present invention specifically includes the following steps: Step 101: Use a depth camera to simultaneously capture color and depth images in the populated area; Step 102: Feed the color image into the key point detection network to extract the two-dimensional pixel coordinates of the left shoulder point, right shoulder point, left hip joint point, and right hip joint point of the human body; Step 103: Based on the depth values of the two-dimensional pixel coordinates in the depth map, obtain the three-dimensional coordinates of the left shoulder point, right shoulder point, left hip joint point, and right hip joint point in the camera coordinate system through the calibrated camera intrinsic parameter matrix and the two-dimensional pixel coordinates. Step 104: Identify the left lateral, right lateral, and supine positions of the human body using statistical modeling based on the three-dimensional coordinates.
[0020] refer to Figure 2The key point detection network architecture diagram of the human body position recognition method of this invention is shown. The downsampling layer includes basic structural units such as convolutional layers, pooling layers, batch normalization layers, and activation layers; the upsampling layer includes basic structural units such as unpooling layers and deconvolutional layers. First, a color image is input. This color image is processed through a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer. The result after processing by the fourth downsampling layer is then processed through a first upsampling layer, a second upsampling layer, and a third upsampling layer. The output result is then concatenated with the second downsampling layer and processed by a fifth downsampling layer. The output result is then processed with the data from the third downsampling layer in a sixth downsampling layer. The output result is then processed with the data from the fourth downsampling layer in a seventh downsampling layer. The output result from the seventh downsampling layer is then processed by an eighth downsampling layer and further processed by a fully connected layer. Finally, the output is the two-dimensional pixel coordinates of four joints, namely the left shoulder point, the right shoulder point, the left hip node, and the right hip node. The parameters of the network structure include data acquisition, data annotation, data cleaning, and model training. The data acquisition form is color images captured by a depth camera. Data annotation uses a keypoint detection model trained on the COCO dataset to automatically annotate the dataset. After data cleaning and adjustment of unannotated data, the network is further trained and iterated repeatedly until convergence. Finally, the feature learning from the COCO dataset is transferred to the dataset, enabling the network model to have the ability to generalize to inference custom datasets.
[0021] refer to Figure 3 The schematic diagram of the pixel coordinate system to camera coordinate system transformation of the present invention first acquires the camera's intrinsic parameters and depth information from the depth map. The camera's intrinsic parameters include f. x f y c x and c y , where f x It is the product of the camera's focal length f and the image pixel width ratio α, expressed in pixels; f y It is the product of the camera's focal length f and the image pixel height ratio β, in pixels; c x c is the distance along the x-axis from the camera's optical center to the origin of the pixel coordinate system, in pixels; y This is the distance along the y-axis from the camera's optical center to the origin of the pixel coordinate system, expressed in pixels.
[0022] Furthermore, since a fisheye camera is not used, and camera distortion parameters are not considered for the time being, a camera model needs to be established to create a mapping relationship between pixel coordinates and 3D coordinates. The origin of the pixel coordinate system is at the upper left corner of the image, while the camera coordinate system has its origin at the optical center of the camera, located at the center of the image. The camera model can be represented by Equation 1: Formula 1; Further converting it into matrix form, as shown in Formula 2: Formula 2; Where u and v are the row and column of pixel coordinates, X, Y, and Z are the 3D coordinates of a point in the image based on the camera coordinate system. Z can be directly obtained from the depth map generated by the camera. The camera's intrinsic parameters are obtained through camera calibration using the OpenCV checkerboard calibration method. x f y c x and c y .
[0023] Further reference Figure 4 As shown, based on the three-dimensional coordinate information of the four key points identified above, three body postures of the human body are obtained through statistical modeling. First, assume the left shoulder point is p1, the right shoulder point is p2, the left hip node is p3, the right hip node is p4, and the camera's z-axis direction vector is... Then the direction vectors of shoulder points p1 and p2 Direction vectors of hip joint points p3 and p4 and z-axis direction vector The included angle and It can be calculated using formula 3: ; , formula 3; Finally, a statistical experiment was conducted on 5000 participants, who performed three body positions: left lateral decubitus, right lateral decubitus, and supine. The included angles were calculated for each position. and We obtain the angle thresholds thresh1 and thresh2 that distinguish these three body postures, and thus the body posture determination can be concluded from Formula 4: Lying on your back; Lying on your left side; Right side lying, Formula 4; in express or ,and and Both satisfy the above relationship requirements: thresh1 = 60° and thresh2 = 140°.
[0024] refer to Figure 5A schematic diagram of the body position recognition device of the present invention is shown. The body position recognition device 10 further includes one or more memories 20 and one or more processors 30, wherein the one or more computer programs are stored in the memories 20 and configured to be executed by the one or more processors 30, and the processors 30 implement the steps of the body position recognition method when executing the computer programs.
[0025] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.
[0026] The human body position recognition method of the present invention can significantly improve the accuracy of human body position detection and provide technical support for the automated guidance of the magnetically controlled capsule endoscopy detection system.
[0027] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for recognizing human body position, characterized in that, Includes the following steps: Step 101: Use a depth camera to simultaneously capture color and depth images in the populated area; Step 102: Feed the color image into the key point detection network to extract the two-dimensional pixel coordinates of the left shoulder point, right shoulder point, left hip joint point and right hip joint point of the human body; Step 103: Based on the depth values of the two-dimensional pixel coordinates in the depth map, obtain the three-dimensional coordinates of the left shoulder point, right shoulder point, left hip joint point, and right hip joint point in the camera coordinate system through the calibrated camera intrinsic parameter matrix and the two-dimensional pixel coordinates. Step 104: Identify the left lateral, right lateral, and supine positions of the human body using statistical modeling based on the three-dimensional coordinates.
2. The human body position recognition method as described in claim 1, characterized in that, The keypoint detection network further includes a downsampling layer and an upsampling layer. The downsampling layer further includes a convolutional layer, a pooling layer, a batch normalization layer, and an activation layer. The upsampling layer further includes an unpooling layer and a deconvolutional layer.
3. The human body position recognition method as described in claim 2, characterized in that, After receiving the color image, the key point detection network processes it through a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth downsampling layer. The result after the fourth downsampling layer is then processed through a first upsampling layer, a second upsampling layer, and a third upsampling layer. The output result is then concatenated with the second downsampling layer and processed by a fifth downsampling layer. The output result is then processed with the data from the third downsampling layer by a sixth downsampling layer. The output result is then processed with the data from the fourth downsampling layer by a seventh downsampling layer. The output result from the seventh downsampling layer is then processed by an eighth downsampling layer and further processed by a fully connected layer. Finally, the two-dimensional pixel coordinates of the four key points are output.
4. The human body position recognition method as described in claim 3, characterized in that, The three-dimensional coordinates are obtained through the following method: Acquire camera intrinsic parameters and depth information from depth maps; Establish the mapping relationship between pixel coordinates and 3D coordinates. The camera model is represented by Equation 1: Official 1; Where f x f y c x and c y All of these are camera intrinsic parameters, and their unit of measurement is pixels, where f x It is the product of the camera's focal length f and the image pixel width ratio α; f y It is the product of the camera's focal length f and the image pixel height ratio β; c x c is the distance along the x-axis from the camera's optical center to the origin of the pixel coordinate system; y This is the distance along the y-axis from the camera's optical center to the origin of the pixel coordinate system; Further converting Formula 1 into matrix form, it is represented by Formula 2: Official 2; in and These are the row and column coordinates of pixels. X, Y, and Z are the 3D coordinates of a point in the image based on the camera coordinate system. Z is obtained directly from the depth map generated by the camera. The camera's intrinsic parameters are obtained through camera calibration using the OpenCV checkerboard calibration method. x f y c x and c y ; The 3D coordinates of the four identified key points are used to obtain three body postures through statistical modeling. The left shoulder point is designated p1, the right shoulder point p2, the left hip node p3, and the right hip node p4. The camera's z-axis direction vector is... Then the direction vectors of shoulder points p1 and p2 Direction vectors of hip joint points p3 and p4 and z-axis direction vector The included angle and It can be calculated using formula 3: ； , Formula 3; Finally, statistical methods were used to experimentally test the three body positions—left lateral decubitus, right lateral decubitus, and supine—and the included angles were calculated for each position. and The angle thresholds thresh1 and thresh2, which distinguish these three body postures, are obtained, and the body posture is identified by formula 4: Lying on your back; Lying on your left side; Right side lying, Formula 4; in express or ,and and Both satisfy the above relationship requirements: thresh1 = 60°, thresh2 = 140°, and the units are all in degrees.
5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the human body position recognition method as described in any one of claims 1 to 4.
6. A body position recognition device, comprising: One or more processors; Memory; as well as One or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, characterized in that the processors, when executing the computer programs, implement the steps of the human body position recognition method as described in any one of claims 1 to 4.