A method for calculating key angles of human posture based on binocular depth estimation

Abstract

This invention discloses a method for calculating key angles of human pose based on binocular flow depth estimation, belonging to the field of computer vision technology, including S1, acquiring the binocular left image I L and binocular right image I R S2, transfer the left image from both eyes to I. L and binocular right image I R In the BFD system, depth information of spatial points is obtained through binocular optical flow depth estimation; S3, simultaneously, the binocular left image I L and binocular right image I R Keypoint detection is performed using the YOLOv8 model, and the acquired keypoint images are fused with depth information; S4, human body key angles are calculated; The human body posture key angle calculation method based on binocular flow depth estimation provided by this invention can effectively acquire the depth information and key angles of human movement, which not only improves the accuracy and practicality of motion analysis, but also provides a new perspective and tool for research and application in related fields.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for calculating key angles of human pose based on binocular flow depth estimation. Background Technology

[0002] Human motion analysis, as a core area of ​​interdisciplinary research in computer vision and biomechanics, aims to deeply explore the dynamic characteristics and underlying physiological mechanisms of human posture through advanced algorithm modeling and precise data analysis. This interdisciplinary direction has not only promoted the development of human motion science at the theoretical level, but also provided important technical support and scientific basis for multiple fields such as sports training, rehabilitation medicine, and interactive entertainment in practice, possessing extremely broad application areas.

[0003] Within the realm of sports science, whether for monitoring the training of professional athletes or guiding exercise for the general public, this method can accurately acquire depth information and key angles of human movement. This allows coaches and participants to intuitively understand movement details, adjust training strategies promptly, and significantly improve training effectiveness. In the field of rehabilitation medicine, this method revolutionizes the assessment of patients' limb rehabilitation progress. By meticulously capturing changes in the patient's posture during rehabilitation training, doctors can accurately assess the rehabilitation status, develop personalized rehabilitation plans, and help patients recover their physical functions faster and better. In intelligent security monitoring, by rapidly analyzing key angles and depth information of human posture in monitoring footage, the system can keenly detect abnormal behavior and issue timely alarms, strengthening the defense for public safety. In the emerging field of gaming and entertainment, especially in motion capture, it shines brightly. Through real-time and precise capture of players' body postures, every subtle movement of the player can be quickly translated into corresponding behaviors of the in-game character, greatly enhancing the immersion and interactivity of the game, allowing players to control the character as if they were actually there, opening up a completely new gaming experience.

[0004] Despite significant progress in pose analysis technology in practical applications, mainstream monocular vision systems still face some technical bottlenecks. Existing technology can be referenced in Chinese Patent CN111753747B, which discloses a method for detecting violent motion using a monocular camera and 3D pose estimation. It uses RGB images from a monocular camera as input, trains multiple models based on deep convolutional neural networks to acquire human position, 2D and 3D keypoint information, performs identity matching, extracts motion features, and uses a random forest model to detect the presence of violent motion. While monocular cameras are widely used due to their relatively low cost and simple equipment, they have inherent limitations in depth information acquisition and 3D spatial understanding. First, lacking direct perception of scene depth, the motion analysis results of monocular systems heavily rely on feature extraction and inference from 2D images. This method often leads to insufficient accuracy in pose description when handling complex movements. Second, monocular systems are poorly robust to changes in ambient lighting and background complexity, easily affected by noise and occlusion, especially in dynamic scenes, making it difficult to meet the requirements of real-time performance and high accuracy. Furthermore, this method is particularly ineffective at analyzing rapid human movement or multi-dimensional dynamic changes, limiting its applicability in demanding scenarios.

[0005] To overcome the aforementioned limitations, binocular vision systems, by simulating the parallax principle of human binoculars, provide three-dimensional depth perception capabilities for motion analysis, significantly improving spatial positioning accuracy and system robustness. Combined with the dynamic tracking capabilities of optical flow technology, binocular vision can achieve high-precision capture of human motion trajectories through the calculation of motion vector fields between pixels in adjacent frames. Compared to monocular systems, binocular optical flow technology exhibits stronger adaptability in handling fast-moving or high-frequency movements, especially in dynamic scenes, where it has significant advantages in angle estimation of key human motion points and continuous modeling of motion trajectories. More importantly, this technology can acquire and process dynamic data in real time at a high frame rate, providing strong support for high-frequency motion analysis in sports training and medical rehabilitation. Existing technology can be referenced in Chinese Patent Publication No. CN119131835A, which discloses a method, device, equipment, and product for human joint point localization based on binocular vision. It acquires and preprocesses human images using a binocular camera, obtains depth information using feature extraction and stereo matching algorithms, and locates the three-dimensional coordinates of joint points using a joint point model. The product is manufactured and tested through hardware integration and software optimization for application in multiple fields. Compared to monocular systems, its advantage lies in its ability to acquire 3D depth perception using the principle of parallax, improving spatial positioning accuracy and system robustness. This allows for accurate localization of human key points, and it performs better in handling fast-moving or high-frequency actions and complex scenes. However, due to a lack of innovative optimization of the optical flow model and unique vector angle calculation, it is prone to depth deviation in complex environments. Furthermore, its key point detection model may have weaker adaptability and background processing capabilities in terms of key point detection and localization accuracy, and the fusion of key points and depth information is not tight or effective enough. This limits its application in fields with high requirements for accuracy and scene adaptability.

[0006] Despite the immense potential of binocular optical flow technology, it still faces a series of technical challenges in practical applications. First, the calibration process for binocular vision systems is complex, requiring precise alignment of the relative positions and angles of the two cameras. Even minute calibration errors can lead to significant deviations in the final depth estimation. Second, optical flow algorithms are prone to drift when processing high-speed moving objects or objects with little texture due to edge blurring or texture loss, thus affecting the overall accuracy of motion analysis. Therefore, the robustness of existing methods remains insufficient, and these problems severely restrict the practical promotion and application of binocular optical flow technology.

[0007] Based on the above, this invention achieves a breakthrough at the algorithm level. Specifically, in the optical flow depth estimation process, it innovatively introduces vector representation and angle calculation methods. This method redefines and optimizes the spatial smoothness constraint parameters in the traditional optical flow model, fully considering the impact of complex real-world factors on optical flow calculation, and accurately locates the optical flow by calculating the angle between these vectors. This keeps the calculation error of key angles within a very small range, greatly improving the accuracy and stability of the entire system. Summary of the Invention

[0008] The purpose of this invention is to provide a method for calculating key angles of human posture based on binocular flow depth estimation, so as to solve the problems existing in the above-mentioned background art.

[0009] To achieve the above objectives, this invention provides a method for calculating key angles of human posture based on binocular flow depth estimation, comprising the following steps:

[0010] S1. Obtain the left image from both eyes. L and binocular right image I R ;

[0011] S2, Transfer the left image of the binoculars I L and binocular right image I R In the BFD system, depth information of spatial points is obtained through binocular depth estimation.

[0012] S3, Simultaneously, the left image of the binoculars I L and binocular right image I R Keypoint detection is performed using the YOLOv8 model, and the acquired keypoint images are fused with depth information.

[0013] S4. Perform calculations on key human body angles.

[0014] Preferably, step S2 specifically includes:

[0015] S21, Transfer the left image of the binoculars I L Binocular right image I R Optical flow analysis is input into the BFD system. Optical flow analysis is used to capture the motion information of each pixel over time and generate a motion vector field V(x,y)=(u(x,y),v(x,y)). Optical flow analysis can provide detailed data on the direction and speed of movement of points on the object surface between consecutive frames, which is crucial for subsequent parallax calculation.

[0016] S22. The depth value of a spatial point is derived through parallax calculation. The calculation formula is as follows:

[0017]

[0018] Where f represents the camera focal length, B is the baseline distance between the two cameras, and parallax is defined as the difference in horizontal displacement of a point in a uniform scene between the left and right images, i.e., Δx = x l -x r This parameter is directly related to the depth value D of the spatial point;

[0019] S23. When mapping depth information to a 3D coordinate system, the camera's internal parameters must be considered, including the focal length f and the principal point position (c). x ,c y ),as follows:

[0020] Z = D;

[0021] S24. Improve the depth estimation of binocular flow by using vector representation and angle calculation.

[0022] Preferably, step S24 specifically includes:

[0023] Considering the complexity of practical applications, relying solely on a basic binocular vision model may be insufficient to handle all situations. Therefore, introducing vector representation and angle calculation methods can further enhance the flexibility and accuracy of depth estimation. Vectors are constructed for the projections P1 and P2 of a spatial point P onto two different planes and their corresponding camera optical centers O1 and O2. and The position of P is precisely located by calculating the angle between these two vectors. Specifically, using the law of cosines and the law of sines, we obtain:

[0024]

[0025] in, Let D be the depth value of spatial point P relative to optical center O1. By combining optical flow analysis with binocular vision technology, not only can the accuracy of depth estimation be effectively improved, but the overall robustness of the system can also be enhanced.

[0026] Preferably, step S3 specifically includes:

[0027] S31, Regarding the left image of the binoculars I L and binocular right image I R Preprocessing is performed to ensure that the data input to the YOLOv8 model meets the requirements. This includes operations such as image cropping, scaling, and normalization. The aim is to remove irrelevant background information, reduce computation, and preserve the original features of the image.

[0028] S32. The preprocessed image is input into the YLOLv8 model. The size of the preprocessed image is usually adjusted to 640*640 pixels to fit the model's input requirements. Then, keypoint detection is performed, outputting 17 keypoints and their coordinates. The detection of these keypoints is based on the feature representations learned by the model during training, which can accurately locate the position of human joints, represented as:

[0029] K={(x i ,y i )|i=1,2,…,n};

[0030] Where K represents the set of all detected keypoints, n is the number of keypoints, and n = 17, each keypoint (x i ,y i () represents the two-dimensional coordinates of the i-th joint in the image;

[0031] S33. Calculate the two-dimensional coordinates (x, y) of each key point. i ,y i ) and its corresponding depth value D i Combined, we obtain the three-dimensional coordinates (X). i ,Y i Z i ), represented as:

[0032]

[0033] Z i =D i ;

[0034] Among them, (c x ,c y ) is the coordinate of the camera center in the image coordinate system.

[0035] Preferably, combining steps S2 and S3, the depth value D of each key point i Represented as:

[0036]

[0037] Through the above steps, each key point has not only planar position information but also depth information, forming a complete three-dimensional human body model.

[0038] Preferably, step S4 specifically includes:

[0039] Based on three key points a, b, and c and their corresponding coordinates A(x) a ,y a ,z a B(x) b ,y b ,z b ), C(x)c ,y c ,z c ), calculate vector and vector The coordinates are:

[0040]

[0041] vector The coordinates are:

[0042]

[0043] Calculate the cosine value of the included angle. B :

[0044]

[0045] Calculate the included angle B, and finally, use the inverse cosine function to obtain the included angle B:

[0046] B = arccos(cos B ).

[0047] Therefore, this invention employs the aforementioned method for calculating key angles of human posture based on binocular flow depth estimation, optimizing the traditional optical flow algorithm. Traditional methods are prone to depth estimation errors when faced with noise interference. This invention, however, constructs spatial point projections and camera optical center vectors, utilizing cosine and sine laws for precise positioning, reducing depth calculation errors caused by noise and effectively improving the system's robustness to noise. This allows the system to dynamically adapt to lighting changes and occlusion effects in complex scenes. Simultaneously, YOLOv8, with its superior detection speed and high-precision keypoint recognition capabilities, complements the optimized optical flow algorithm. Traditional methods may suffer from inaccurate keypoint localization due to human posture changes and background interference. However, the YOLOv8 model, trained on a large amount of data, exhibits stronger adaptability to human images in different scenes and postures. By removing irrelevant background information during preprocessing, the model focuses more on key human body parts. Even with partial occlusion, it can accurately reconstruct human motion posture through multi-dimensional information fusion, utilizing unoccluded keypoints and depth information, effectively mitigating the impact of occlusion on motion analysis. This method not only significantly improves the detection accuracy of key points in human motion but also effectively alleviates the drift problem of optical flow algorithms when handling high-speed motion and sparsely textured scenes. Furthermore, while ensuring real-time performance, this method achieves efficient integration and analysis of multi-dimensional dynamic data, thus providing a novel solution for human motion analysis in complex scenes.

[0048] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method for calculating key angles of human posture based on binocular flow depth estimation according to the present invention;

[0050] Figure 2 This is a schematic diagram of the vector representation and angle calculation of the present invention. Figure 1 ;

[0051] Figure 3 This is a schematic diagram of the vector representation and angle calculation of the present invention. Figure 2 ;

[0052] Figure 4 This is a structural diagram of the YOLOv8 model of the present invention;

[0053] Figure 5 This is a schematic diagram illustrating the calculation of the motion angle of the right elbow joint in an embodiment of the present invention. Detailed Implementation

[0054] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0055] Please see Figure 1 A method for calculating key angles of human posture based on binocular depth estimation includes the following steps:

[0056] S1. Obtain the left image from both eyes. L and binocular right image I R ;

[0057] In the field of stereo vision, the core challenge lies in identifying the corresponding positions of points in the same scene between the left and right views. This process relies on calculating the difference in horizontal displacement between these points, i.e., the disparity d = |x l -x r |, where x l ,x r These represent the column coordinates of the point in the left and right images, respectively. Based on this disparity information, the depth of the corresponding point in space can be calculated. However, traditional stereo matching algorithms typically simplify the search process based on the parallel vision assumption, but this idealized assumption is often not fully satisfied in the real world, especially when corresponding points are not in the same row, which can easily lead to matching errors.

[0058] According to geometric principles, the depth D of a point P in space can be accurately calculated using the following formula:

[0059]

[0060] Where f is the camera focal length, and B represents the baseline distance between the optical centers of the two cameras. This formula holds true only if the parallel vision model is strictly followed, requiring the line connecting the matching points to remain parallel to the baseline. If actual imaging conditions deviate from this ideal state, such as deviations caused by component errors or non-ideal calibration, it will directly lead to an increase in the uncertainty of depth estimation.

[0061] Perturbation analysis in practical applications and its impact on depth estimation include:

[0062] 1. The effect of focal length disturbance

[0063] A focal length perturbation occurs when there is a slight difference in the distance from the optical center of the two imaging planes to the imaging plane itself. This is usually a result of mismatch between imaging elements. This perturbation can cause the same spatial point to appear with different mapping sizes in the left and right images, increasing the risk of mismatches and potentially leading to complex matching relationships between one or more pairs of corresponding points, thus affecting the consistency and accuracy of depth estimation.

[0064] 2. Challenges posed by non-coplanar imaging

[0065] In non-coplanar imaging conditions, the light source and the reference plane remain parallel but are not on the same plane. This situation is common due to systematic errors introduced when using standard patterns for external parameter correction. To accommodate this configuration, the parallax-depth conversion formula needs to be adjusted to consider angular factors:

[0066]

[0067] Here, α represents the angle between the baseline and the imaging plane. As α changes, the final measured depth value will deviate from the value under ideal conditions by a certain proportion, which is expressed as a multiple relationship in the (0, 1) interval.

[0068] 3. Handling cases of non-perfect parallelism

[0069] When the rotation angle is not precisely calibrated, imperfect parallelism can occur. This not only disrupts the original parallel vision model but may also cause matching points that were originally in the same row to be scattered into different rows. Besides increasing the difficulty of stereo matching, this rotation also affects the column coordinates x used for disparity-depth transformation. l and x r It cannot accurately reflect the relative positional relationship between points P1 and P2, even though the triangle similarity relationship still exists, the key distance variable... It can no longer be obtained directly through simple coordinate transformations.

[0070] S2, Transfer the left image of the binoculars I L and binocular right image IR In the BFD system, depth information of spatial points is obtained through binocular depth estimation.

[0071] S21, Transfer the left image of the binoculars I L Binocular right image I R Optical flow analysis is input into the BFD system. Optical flow analysis is used to capture the motion information of each pixel over time and generate a motion vector field V(x,y)=(u(x,y),v(x,y)). Optical flow analysis can provide detailed data on the direction and speed of movement of points on the object surface between consecutive frames, which is crucial for subsequent parallax calculation.

[0072] S22. The depth value of a spatial point is derived through parallax calculation. The calculation formula is as follows:

[0073]

[0074] Where f represents the camera focal length, B is the baseline distance between the two cameras, and parallax is defined as the difference in horizontal displacement of a point in a uniform scene between the left and right images, i.e., Δx = x l -x r This parameter is directly related to the depth value D of a spatial point; through the above transformation, the depth distribution of three-dimensional space can be recovered from two-dimensional image information.

[0075] S23. To map depth information to a 3D coordinate system, the camera's internal parameters need to be considered, including the focal length f and the principal point position (c). x ,c y ),as follows:

[0076] Z = D;

[0077] S24. Improve the depth estimation of binocular streams by using vector representation and angle calculation, such as Figure 2-3 As shown, it includes:

[0078] Considering the complexity of practical applications, relying solely on a basic binocular vision model may be insufficient to handle all situations. Therefore, introducing vector representation and angle calculation methods can further enhance the flexibility and accuracy of depth estimation. Vectors are constructed for the projections P1 and P2 of a spatial point P onto two different planes and their corresponding camera optical centers O1 and O2. and The position of P is precisely located by calculating the angle between these two vectors. Specifically, using the law of cosines and the law of sines, we obtain:

[0079]

[0080]

[0081] in, Let D be the depth value of spatial point P relative to optical center O1. By combining optical flow analysis with binocular vision technology, not only can the accuracy of depth estimation be effectively improved, but the overall robustness of the system can also be enhanced.

[0082] S3, Simultaneously, the left image of the binoculars I L and binocular right image I R Keypoint detection is performed using the YOLOv8 model, and the acquired keypoint images are fused with depth information; specifically:

[0083] S31, Regarding the left image of the binoculars I L and binocular right image I R Preprocessing is performed to ensure that the data input to the YOLOv8 model meets the requirements. This includes operations such as image cropping, scaling, and normalization. The aim is to remove irrelevant background information, reduce computation, and preserve the original features of the image.

[0084] S32. The preprocessed image is input into the YLOLv8 model, such as... Figure 4 As shown, the YOLOv8 model detects key points of various joints in the human body through multi-scale feature extraction and attention mechanisms. The preprocessed image size is usually adjusted to 640*640 pixels to fit the model's input requirements, and then key point detection is performed, outputting the key point coordinates x. i ,y i This includes 17 key points on the human body, such as the shoulder, elbow, wrist, hip, knee, and ankle. The detection of these key points is based on the feature representations learned by the model during training, which can accurately locate the position of human joints. These key points are represented as follows:

[0085] K={(x i ,y i )|i=1,2,…,n};

[0086] Where K represents the set of all detected keypoints, n is the number of keypoints, and n = 17, each keypoint (x i ,y i () represents the two-dimensional coordinates of the i-th joint in the image;

[0087] S33. Calculate the two-dimensional coordinates (x, y) of each key point. i ,y i ) and its corresponding depth value D i Combined, we obtain the three-dimensional coordinates (X). i ,Y i Z i ), represented as:

[0088]

[0089] Z i =D i ;

[0090] Among them, (c x ,c y ) is the coordinate of the camera center in the image coordinate system.

[0091] Combining steps S2 and S3, the depth value D of each key point i Represented as:

[0092]

[0093] Through the above steps, each key point has not only planar position information but also depth information, forming a complete three-dimensional human body model.

[0094] S4. Perform calculations of key human body angles. In this embodiment, the right elbow joint is used as an example for illustration. Figure 5 As shown, it includes:

[0095] Consider three key points: shoulder (a), elbow (b), and wrist (c). Assume the coordinates of these three key points are A(x) and A(b). a ,y a ,z a B(x) b ,y b ,z b ), C(x) c ,y c ,z c );

[0096] Calculate vectors and vector The coordinates are:

[0097]

[0098] vector The coordinates are:

[0099]

[0100] Calculate the cosine value of the included angle. B :

[0101]

[0102] Calculate the included angle B, and finally, use the inverse cosine function to obtain the included angle B:

[0103] B = arccos(cos B ).

[0104] The pseudocode for the specific algorithm flow of the key steps in this embodiment is as follows:

[0105]

[0106]

[0107] Therefore, this invention employs the aforementioned method for calculating key angles of human posture based on binocular flow depth estimation. Combining the advantages of binocular vision and optical flow technology, it can effectively acquire depth information and key angles of human movement, making it applicable to fields such as sports training and rehabilitation medicine. This method not only improves the accuracy and practicality of motion analysis but also provides new perspectives and tools for research and applications in related fields.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for calculating key angles of human posture based on binocular depth estimation, characterized in that, Includes the following steps: S1. Obtain the left image from both eyes. and binocular right image ; S2, Transfer the left image from both eyes and binocular right image In the BFD system, depth information of spatial points is obtained through binocular depth estimation. S3, Simultaneously, the left image of both eyes and binocular right image Keypoint detection is performed using the YOLOv8 model, and the acquired keypoint images are fused with depth information. S4. Perform calculations of key human body angles; Step S2 specifically includes: S21, Transfer the left image of both eyes Binocular right image Optical flow analysis is input into the BFD system, where it is used to capture the motion information of each pixel over time and generate a motion vector field. ; S22. The depth value of a spatial point is derived through parallax calculation. The calculation formula is as follows: ； in, Represents the camera's focal length. It is the baseline distance between the two cameras. The difference in horizontal displacement between the left and right images is... ; S23. When mapping depth information to a 3D coordinate system, the camera's internal parameters, including focal length, must be considered. and the main point position ,as follows: ； S24. Improve the depth estimation of binocular streams by using vector representation and angle calculation, specifically: For spatial points Projections on two different planes and and its corresponding camera optical center and Constructing vectors and And by calculating the angle between these two vectors, precise positioning is achieved. Specifically, using the law of cosines and the law of sines, we obtain the following position: ； ； in, For spatial points Relative to the optical center depth value .

2. The method for calculating key angles of human posture based on binocular flow depth estimation according to claim 1, characterized in that, Step S3 specifically includes: S31, Regarding the left image of the binoculars and binocular right image Preprocessing is required; S32. The preprocessed image is input into the YOLOv8 model for keypoint detection, outputting 17 keypoints and their coordinates, as follows: ； in, This represents the set of all detected keypoints. It is the number of key points, and Each key point Indicates the first The two-dimensional coordinates of each joint in the image; S33. Convert the two-dimensional coordinates of each key point. Its corresponding depth value Combined, we obtain three-dimensional coordinates. , is represented as: ； ； ； in, It is the coordinate of the camera center in the image coordinate system.

3. The method for calculating key angles of human posture based on binocular flow depth estimation according to claim 2, characterized in that, Combining steps S2 and S3, the depth value of each key point Represented as: 。 4. The method for calculating key angles of human posture based on binocular flow depth estimation according to claim 3, characterized in that, Step S4 is as follows: Based on three key points and their corresponding coordinates , , Calculate vector and ,vector The coordinates are: ； vector The coordinates are: ； Calculate the cosine of the included angle : ； Calculate the included angle B, and finally, use the inverse cosine function to obtain the included angle B: 。

Images