Human body parameter measurement system based on binocular vision and deformable model

Abstract

The invention relates to the technical field of computer vision and image processing, and discloses a human body parameter measurement system based on binocular vision and a deformable model, and the system comprises a binocular vision collection module which is used for capturing human body image data through a binocular camera; and the image preprocessing and analyzing module comprises a dual-target calibration and correction unit. The human body parameter measurement system based on the binocular vision and the deformable model aims to acquire human body images through the binocular camera, and perform binocular calibration and binocular correction by using a Zhang Zhengyou camera calibration method, so as to ensure the coplanarity and row alignment of imaging planes of the left and right cameras. Secondly, segmenting the image by adopting a human body analysis self-correction (SCHP) method, and extracting contours of different parts of a human body; thirdly, fitting the extracted contour by using an ellipsoid deformation model and an egg-shaped curved surface model, and solving parameters such as skeleton length, volume and relative centroid of each part of the human body;

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and particularly to a human body parameter measurement system based on binocular vision and deformable models. Background Art

[0002] Human body parameter measurement technology has shown extensive application value in many fields such as medical diagnosis, sports analysis, virtual reality, and augmented reality. In medical diagnosis, accurate human body parameter measurement helps doctors evaluate the health status of patients and provides important basis for disease diagnosis and treatment plan formulation. In the field of sports analysis, by measuring the motion parameters of athletes, it helps them optimize their technical movements, improve sports performance, and prevent sports injuries at the same time. In virtual reality and augmented reality technologies, human body parameter measurement is a key link to achieve human-computer interaction and enhance user experience. It enables objects in the virtual scene to interact more naturally with the user's body, improving the immersion and interactivity. With the continuous progress of technology, human body parameter measurement technology will play an important role in more fields, bringing more convenience and possibilities to people's lives and work.

[0003] Traditional human body parameter measurement methods, such as using physical measurement tools like tape measures and weighing scales, although can meet the basic measurement needs to a certain extent, their limitations are also very obvious. First of all, these methods are usually time-consuming and laborious, requiring manual operation, which is not only inefficient but also prone to measurement errors due to human factors. Secondly, the accuracy of physical measurement tools is often limited, especially for complex human body shapes and dynamic changes, it is difficult to perform accurate measurements. With the rapid development of computer vision technology, human body parameter measurement based on images and videos has gradually become a new solution. However, the application of existing technologies in this field still faces many challenges. On the one hand, due to the diversity and complexity of human body shapes, as well as the influence of environmental factors such as light and occlusion, it is difficult to guarantee the measurement accuracy. On the other hand, existing human body parameter measurement models often have poor adaptability and are difficult to meet the needs of different individuals and scenarios, restricting the wide application of the technology. Therefore, how to improve the adaptability and robustness of the model while ensuring the measurement accuracy has become an urgent problem to be solved in the current human body parameter measurement technology. Summary of the Invention

[0004] The object of the present invention is to collect human body images through a binocular camera, perform binocular calibration and binocular correction using the Zhang Zhengyou camera calibration method to ensure the coplanarity and row alignment of the imaging planes of the left and right cameras. Then, the Self-Calibrated Human Parsing (SCHP) method is used to segment the images and extract the contours of different parts of the human body. Next, an ellipsoidal deformation model and an egg-shaped surface model are used to fit the extracted contours to solve parameters such as the bone lengths, volumes, and relative centroids of various parts of the human body, and a human parameter measurement system based on binocular vision and deformable models is proposed.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] A human parameter measurement system based on binocular vision and deformable models, the system includes:

[0007] A binocular vision acquisition module for capturing human body image data through a binocular camera;

[0008] An image preprocessing and analysis module, which includes:

[0009] A binocular calibration and correction unit for accurately calibrating the binocular camera using the camera calibration method and performing binocular correction through the Stereo Camera Calibrator toolbox of Matlab to ensure the coplanarity and row alignment of the imaging planes of the left and right cameras;

[0010] A human parsing and contour extraction unit for segmenting the preprocessed human body images using the Self-Calibrated Human Parsing (SCHP) method to accurately extract the contours of various parts of the human body;

[0011] A parameter solving and modeling module, which includes:

[0012] A deformable model fitting unit for fitting the extracted contours of various parts of the human body using an ellipsoidal deformation model and an egg-shaped surface model to solve parameters such as the bone lengths, volumes, and relative centroids of various parts of the human body;

[0013] A three-dimensional position calculation unit for combining the binocular disparity principle and using the fitted model parameters to solve the three-dimensional position coordinates of the joint points in the real space;

[0014] Wherein, the binocular calibration and correction unit, the human parsing and contour extraction unit, the deformable model fitting unit, and the three-dimensional position calculation unit work together to jointly achieve the accurate measurement of human parameters.

[0015] Based on the above technical solution, the present invention can also be improved as follows.

[0016] Furthermore, the binocular vision acquisition module includes:

[0017] High-resolution camera unit: This unit consists of a pair of high-precision and high-resolution cameras, which can capture delicate details of the human body image, providing a high-quality image data source for subsequent image preprocessing and analysis. The cameras are equipped with wide-angle lenses to ensure that the entire human body can be completely captured within a reasonable shooting distance, avoiding image truncation or distortion;

[0018] Synchronous trigger control unit: To ensure that the images captured by the left and right cameras are strictly synchronized in time, this module is built with a synchronous trigger control unit. By precisely controlling the shutter opening time of the cameras, it achieves image capture synchronization at the millisecond level, effectively eliminating parallax errors caused by time differences and improving the accuracy and stability of 3D reconstruction;

[0019] Ambient light compensation unit: Considering that changes in ambient light conditions in different environments may have a significant impact on image quality, the binocular vision acquisition module also includes an ambient light compensation unit. This unit automatically adjusts the exposure parameters of the cameras or uses external light sources for auxiliary lighting to ensure that images with uniform brightness and moderate contrast can be obtained under different light conditions, laying a good foundation for subsequent image processing steps;

[0020] Image preprocessing acceleration unit: To improve the processing efficiency of the entire system, the binocular vision acquisition module also integrates an image preprocessing acceleration unit. This unit uses hardware acceleration technologies such as GPU acceleration or dedicated image processing chips to quickly preprocess the captured raw images, including denoising, enhancing contrast, edge detection, etc., providing preprocessed image data for subsequent human body parsing and contour extraction, thereby shortening the overall processing time and improving the system response speed.

[0021] Furthermore, the binocular calibration and correction unit includes the following components:

[0022] Precision calibration board and calibration algorithm: This unit uses a high-precision and high-resolution calibration board as a calibration reference object, combined with an advanced calibration algorithm, to accurately calibrate the internal and external parameters of the binocular cameras. The calibration algorithm can fully consider factors such as lens distortion and non-linear errors of the cameras. Through an iterative optimization process, it ensures the accuracy and stability of the calibration results. In addition, the design of the calibration board considers various sizes and patterns to meet the requirements of different camera configurations and shooting environments;

[0023] Stereo correction matrix calculation module: After completing the camera calibration, the stereo correction matrix calculation module calculates and generates the stereo correction matrix of the binocular cameras according to the calibration results. This matrix can eliminate image distortion and parallax errors caused by the perspective difference of the binocular cameras, making the imaging planes of the left and right cameras strictly coplanar and row-aligned. By applying the stereo correction matrix, the 3D reconstruction accuracy and real-time performance of the binocular vision system can be significantly improved;

[0024] Calibration effect evaluation and feedback mechanism: To ensure the accuracy and effectiveness of binocular calibration and correction, this unit also includes a calibration effect evaluation and feedback mechanism. This mechanism evaluates the calibration effect by comparing the image data before and after calibration. At the same time, according to the evaluation results, the calibration parameters and correction algorithms are fine-tuned to form a closed-loop feedback system. This iterative optimization process enables the binocular calibration and correction unit to continuously improve the calibration accuracy and meet the requirements of different application scenarios.

[0025] Furthermore, the human body parsing and contour extraction unit includes the following processing steps:

[0026] Multi-scale image pyramid processing: To improve the recognition accuracy of the human body at different scales, this unit adopts multi-scale image pyramid processing technology. This technology constructs image pyramids with different resolutions to represent the original image at multiple scales, thereby capturing the feature information of the human body at multiple scales. This step helps to accurately segment the human body area in a complex background and provides high-quality input data for subsequent contour extraction.

[0027] Deep learning-based human body segmentation algorithm: To improve the accuracy of human body segmentation, this unit introduces a deep learning-based human body segmentation algorithm. This algorithm trains a large amount of human body image data to learn the feature representations of various parts of the human body and can accurately identify and segment the human body area in a complex environment. The application of the deep learning algorithm significantly improves the robustness and automation of human body segmentation and reduces the dependence on manual intervention.

[0028] Contour refinement and optimization technology: After initially segmenting the human body area, this unit adopts contour refinement and optimization technology to process the human body contour. Through morphological operations, edge detection, and curve fitting, etc., the smoothness and continuity of the contour are optimized, reducing contour jitter and discontinuity caused by image noise or segmentation errors. This step ensures that the extracted human body contour is more accurate and clear and provides reliable input data for subsequent human body parameter measurement.

[0029] Furthermore, the deformable model fitting unit includes the following steps:

[0030] Multi-parameter deformable model construction: This unit adopts a multi-parameter deformable model. This model not only includes the basic morphological parameters of the human body but also introduces additional parameters that can describe the subtle changes in the human body morphology. By constructing such a model, the morphological characteristics of the human body can be more comprehensively captured and described, improving the fitting accuracy and flexibility.

[0031] Model Initialization and Iterative Optimization: During the model fitting process, the model is first initialized based on the preliminary human contour extraction results. Subsequently, iterative optimization algorithms such as gradient descent, Newton's method, or genetic algorithms are used to continuously adjust and optimize the model parameters to minimize the error between the model and the real human contour. Through multiple iterations, it is ensured that the model can accurately fit the human form;

[0032] Model Adaptive Adjustment Mechanism: To cope with the changes in different human forms and postures, this unit also includes a model adaptive adjustment mechanism. This mechanism can dynamically adjust the shape parameters and constraint conditions of the model according to the complexity and differences of the human contour, ensuring that the model can maintain good fitting effects and generalization capabilities in different application scenarios.

[0033] Furthermore, the three-dimensional position calculation unit includes the following processing procedures:

[0034] Stereo Matching Algorithm: This unit adopts an advanced stereo matching algorithm. By comparing the corresponding feature points in the images captured by the left and right cameras, the corresponding relationship between them is established. This algorithm can fully consider problems such as illumination changes, noise interference, and occlusion in the images, ensuring the accuracy and stability of the matching results. Stereo matching is the basis for three-dimensional position calculation and provides reliable data support for subsequent three-dimensional reconstruction;

[0035] Three-dimensional Coordinate Calculation and Reconstruction: After establishing the stereo matching relationship, this unit uses the triangulation principle to calculate the three-dimensional coordinates of each matching feature point. By accumulating a sufficient number of three-dimensional coordinate points, the three-dimensional point cloud data of the human body can be constructed. Subsequently, three-dimensional reconstruction algorithms such as surface reconstruction and volume reconstruction are used to convert the point cloud data into a continuous three-dimensional model, thereby realizing the three-dimensional visualization of the human form;

[0036] Error Correction and Optimization: To improve the accuracy of three-dimensional position calculation, this unit also includes error correction and optimization steps. By considering the calibration error of the camera, the error of image processing, and various uncertainty factors in the three-dimensional reconstruction process, the calculated three-dimensional coordinates are corrected and optimized. This step ensures that the finally obtained three-dimensional model can accurately reflect the real form and position information of the human body.

[0037] Furthermore, the stereo matching algorithm in the three-dimensional position calculation unit adopts a semi-global matching strategy. This strategy not only considers the matching cost in the local area but also introduces the idea of global optimization. By using dynamic programming algorithms to search for the optimal matching path in multiple directions, the false matching is effectively reduced, and the matching accuracy and robustness are improved. In addition, the SGM algorithm can handle the matching problems within different disparity ranges, adapt to the diversity of human forms and postures, and ensure accurate three-dimensional position calculation in complex scenarios.

[0038] Furthermore, the model adaptive adjustment mechanism in the deformable model fitting unit combines human kinematic constraints. When adjusting the model parameters, this mechanism not only considers the contour information in the image data, but also introduces kinematic constraint conditions such as the movement range and relative position of human joints. By integrating these constraint conditions into the iterative optimization process, it can limit the search space of model parameters, avoid unreasonable fitting results, and improve the model's ability to accurately describe human body shapes and postures. At the same time, the model adaptive adjustment mechanism combined with human kinematic constraints can also enhance the system's adaptability to human motion changes and improve the measurement accuracy and stability in dynamic scenarios.

[0039] Compared with the prior art, the technical solution of this application has the following beneficial technical effects:

[0040] In view of the problems of time-consuming, laborious, low efficiency and easy generation of human errors in traditional physical measurement tools, this system introduces a binocular vision acquisition module. By using binocular cameras to capture human image data, non-contact automated measurement is achieved, greatly improving the measurement efficiency and reducing the errors caused by human factors. The binocular vision technology utilizes the parallax principle of two cameras to obtain richer spatial information, providing a basis for subsequent precise measurement. Secondly, in view of the problem that it is difficult to ensure the measurement accuracy due to the diversity and complexity of human body shapes in the existing technologies, this system designs an image preprocessing and analysis module. Among them, the binocular calibration and correction unit uses the camera calibration method and the Stereo CameraCalibrator toolbox of Matlab to ensure the coplanarity and row alignment of the imaging planes of the left and right cameras, thereby improving the accuracy of image matching and measurement accuracy. The human body parsing and contour extraction unit adopts the self-calibrating human parsing (SCHP) method, which can accurately extract the contours of various parts of the human body, providing accurate data input for subsequent model fitting. Furthermore, in view of the problem of poor adaptability of the existing measurement models, this system adopts the deformable model fitting unit in the parameter solving and modeling module. This unit uses the ellipsoidal deformation model and the egg-shaped surface model to fit the contours of various parts of the human body extracted. These models not only have good adaptability and can fit human bodies of different shapes, but also can solve parameters such as the bone lengths, volumes and relative centroids of various parts of the human body, providing more comprehensive and accurate human parameter measurement. Finally, this system also designs a three-dimensional position calculation unit. Combining the binocular parallax principle, it uses the fitted model parameters to solve the three-dimensional position coordinates of the joint points in the real space. This function not only improves the spatial resolution of the measurement, but also provides important data support for subsequent applications such as virtual reality and augmented reality, effectively solving the defects in traditional measurement methods and existing technologies, improving the measurement accuracy, efficiency and adaptability, and providing strong technical support for multiple fields such as medical diagnosis, motion analysis, virtual reality and augmented reality. Brief Description of the Drawings

[0041] Figure 1 is the experimental flow chart of the human body parameters of the present invention;

[0042] Figure 2 is the schematic structural diagram of extracting the right upper arm contour in the image of the present invention;

[0043] Figure 3 is the schematic diagram of coordinate transformation of the present invention;

[0044] Figure 4 is the fitting result of the right upper arm contour in the left image of the present invention;

[0045] Figure 5 is the fitting result of the right upper arm contour in the right image of the present invention. Detailed implementation mode

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0047] Combined with Figures 1 to 5 As shown, a human body parameter measurement system based on binocular vision and deformable model of the present invention includes:

[0048] A binocular vision acquisition module for capturing human body image data through a binocular camera;

[0049] An image preprocessing and analysis module, which includes:

[0050] A binocular calibration and correction unit that accurately calibrates the binocular camera using the camera calibration method and performs binocular correction through the Stereo Camera Calibrator toolbox of Matlab to ensure the coplanarity and row alignment of the imaging planes of the left and right cameras;

[0051] A human body parsing and contour extraction unit that segments the preprocessed human body image using the self-calibrating human body parsing (SCHP) method to accurately extract the contours of each part of the human body;

[0052] A parameter solving and modeling module, which includes:

[0053] A deformable model fitting unit that fits the extracted contours of each part of the human body using an ellipsoidal deformation model and an egg-shaped surface model to solve parameters such as the bone length, volume, and relative centroid of each part of the human body;

[0054] A three-dimensional position calculation unit that combines the binocular disparity principle and uses the fitted model parameters to solve the three-dimensional position coordinates of the joint points in the real space;

[0055] Among them, the binocular calibration and correction unit, the human body parsing and contour extraction unit, the deformable model fitting unit, and the three-dimensional position calculation unit work together to jointly achieve the accurate measurement of human body parameters.

[0056] In a preferred embodiment of the present invention, it can be further configured that the binocular vision acquisition module includes:

[0057] High-resolution camera unit: This unit consists of a pair of high-precision and high-resolution cameras, which can capture delicate details of the human body image, providing a high-quality image data source for subsequent image preprocessing and analysis. The cameras are equipped with wide-angle lenses to ensure that the entire human body can be completely captured within a reasonable shooting distance, avoiding image truncation or distortion;

[0058] Synchronous trigger control unit: To ensure that the images captured by the left and right cameras are strictly synchronized in time, this module is built with a synchronous trigger control unit. By precisely controlling the shutter opening time of the cameras, it achieves millisecond-level image capture synchronization, effectively eliminating parallax errors caused by time differences and improving the accuracy and stability of 3D reconstruction;

[0059] Ambient light compensation unit: Considering that changes in lighting conditions in different environments may have a significant impact on image quality, the binocular vision acquisition module also includes an ambient light compensation unit. This unit automatically adjusts the exposure parameters of the cameras or uses external light sources for auxiliary lighting to ensure that images with uniform brightness and moderate contrast can be obtained under different lighting conditions, laying a good foundation for subsequent image processing steps;

[0060] Image preprocessing acceleration unit: To improve the processing efficiency of the entire system, the binocular vision acquisition module also integrates an image preprocessing acceleration unit. This unit uses hardware acceleration technologies such as GPU acceleration or dedicated image processing chips to quickly preprocess the captured raw images, including denoising, enhancing contrast, edge detection, etc., providing preprocessed image data for subsequent human body parsing and contour extraction, thereby shortening the overall processing time and improving the system response speed.

[0061] In a preferred embodiment of the present invention, it can be further configured as follows: The binocular calibration and correction unit includes the following components:

[0062] Precision calibration board and calibration algorithm: This unit uses a high-precision and high-resolution calibration board as a calibration reference object, combined with an advanced calibration algorithm, to accurately calibrate the internal and external parameters of the binocular cameras. The calibration algorithm can fully consider factors such as lens distortion and non-linear errors of the cameras, and through an iterative optimization process, ensure the accuracy and stability of the calibration results. In addition, the design of the calibration board considers various sizes and patterns to meet the requirements of different camera configurations and shooting environments;

[0063] Stereo correction matrix calculation module: After completing the camera calibration, the stereo correction matrix calculation module calculates and generates the stereo correction matrix of the binocular cameras according to the calibration results. This matrix can eliminate image distortion and parallax errors caused by the perspective difference of the binocular cameras, making the imaging planes of the left and right cameras strictly coplanar and row-aligned. By applying the stereo correction matrix, the 3D reconstruction accuracy and real-time performance of the binocular vision system can be significantly improved;

[0064] Calibration Effect Evaluation and Feedback Mechanism: To ensure the accuracy and effectiveness of binocular calibration and correction, this unit also includes a calibration effect evaluation and feedback mechanism. This mechanism evaluates the calibration effect by comparing the image data before and after calibration. At the same time, according to the evaluation results, the calibration parameters and correction algorithms are fine-tuned to form a closed-loop feedback system. This iterative optimization process enables the binocular calibration and correction unit to continuously improve the calibration accuracy and meet the requirements of different application scenarios.

[0065] After setting up the binocular stereo vision system, 30 pairs of left and right views of the checkerboard calibration board at different positions are taken as calibration pictures. The Stereo Camera Calibrator toolbox in Matlab is used for calibration. Cameral is the image captured by the left camera, and Camera2 is the picture captured by the right camera. As can be seen from the figure, each interior angle point of the checkerboard calibration board can be accurately extracted.

[0066] The internal parameter matrix M of the left camera calibrated by the Stereo Camera Calibrator toolbox based on 30 groups of images l is:

[0067]

[0068] The distortion coefficient D of the left camera l is:

[0069] D l = [0.2791 1.7577 -0.0133 0.0786 -141.2774]

[0070] The internal parameter matrix M of the right camera r is:

[0071]

[0072] The distortion coefficient D of the right camera r is:

[0073] D r = [0.0775 5.0234 -0.0280 0.0726 -60.8008]

[0074] The rotation matrix R in the camera extrinsic parameter matrix is:

[0075]

[0076] The translation vector T is:

[0077] T = [-253.3836 -30.3778 -128.5292]

[0078] In the OpenCV library of Python, cv2.stereoRectify() is a function for stereo vision processing. Its main function is to calculate the rectification transformation of two cameras so that the images of the left and right views are coplanar and aligned in each row, that is, to "straighten" the images of the left and right views so that their pixel points are on the same horizontal line. The input parameters of the cv2.stereoRectify() function are the internal parameter matrices M l 、M r 、distortion coefficients D l 、D r 、image size, rotation matrix R, and translation vector T. Rectification using the cv2.stereoRectify() function can obtain a matrix as follows:

[0079]

[0080] where u0 = c x = -1482.9241, v0 = -c y = 859.4649, B = T x = 285.7143,

[0081] In a preferred embodiment of the present invention, it can be further configured that the human body parsing and contour extraction unit includes the following processing steps:

[0082] Multi-scale image pyramid processing: To improve the recognition accuracy of the human body at different scales, this unit uses multi-scale image pyramid processing technology. This technology constructs image pyramids with different resolutions to represent the original image at multiple scales, thereby capturing the feature information of the human body at multiple scales. This step helps to accurately segment the human body region in a complex background and provides high-quality input data for subsequent contour extraction;

[0083] Deep learning-based human body segmentation algorithm: To improve the accuracy of human body segmentation, this unit introduces a deep learning-based human body segmentation algorithm. This algorithm trains a large amount of human body image data to learn the feature representations of various parts of the human body and can accurately identify and segment the human body region in a complex environment. The application of the deep learning algorithm significantly improves the robustness and automation of human body segmentation and reduces the dependence on manual intervention;

[0084] Contour Refinement and Optimization Technology: After initially segmenting the human body region, this unit uses contour refinement and optimization technology to process the human body contour. Through morphological operations, edge detection, curve fitting, and other means, it optimizes the smoothness and continuity of the contour, reducing contour jitter and discontinuity caused by image noise or segmentation errors. This step ensures that the extracted human body contour is more accurate and clear, providing reliable input data for subsequent human body parameter measurement.

[0085] When the muscles of the human body's limbs move, they will deform, resulting in slight changes in the external contour of the limbs. Taking the movement of the arm as an example, when the arm exerts force to straighten, bend, lift heavy objects, etc., the muscles of the upper arm and lower arm will bulge slightly. The contour of the human arm has an axisymmetric characteristic with the bone as the axis, and the sizes at both ends are different, approximating an ellipsoidal deformation model. The expression of the ellipsoidal deformation model is as follows:

[0086]

[0087] In Equation (2.12), the midpoint of the ellipsoidal deformation model is at the origin of the coordinate axis, the model is symmetric about the x-axis, and r x , r y , r z are the radii of the model in each direction. The function f(x) is used to control the deformation of the model. When using the ellipsoidal deformation model to approximately fit the shape of the arm, the function f(x) is expressed as Equation (2.13).

[0088]

[0089] Among them, the parameter d determines the deformation of the model and is also known as the deformation factor. The larger the value of the parameter d, the greater the degree of model deformation, and vice versa. Half of the length of the arm bone is represented by Hb, and the general thickness at any part of the arm is represented by Hp. Substituting Hb and Hp as parameters into the ellipsoidal deformation model of Equation (2.12), the shape model of the human arm is obtained, as shown in Equation (2.14).

[0090]

[0091] The x-axis is parallel to the skeleton, the joint points at both ends of the skeleton are on the x-axis, and the midpoint of the skeleton is at the origin of the coordinate axis.

[0092] According to the description of Equation (2.14) and the binocular vision principle and the results of human body parsing mentioned above, this paper proposes a method for fitting an ellipsoidal deformation model of the human arm from binocular images and solving the bone length, volume, and three-dimensional coordinates of joint points. Taking the right upper arm of the human body as an example, first, the two-dimensional coordinate points of the right upper arm contour are extracted from the image processed by the SCHP method; then, a deformation factor is introduced, and the two-dimensional points are fitted with an ellipse model to obtain the two-dimensional ellipsoidal deformation model of the right upper arm. The endpoint coordinates at both ends of the long axis of the model are calculated as the joint points at both ends of the bone; combined with the parallax principle, the three-dimensional coordinates of the joint points at both ends of the bone in the real space and the bone length are obtained according to the joint coordinates at both ends of the right upper arm in the two images from the left and right perspectives; finally, the two-dimensional ellipsoidal deformation model is scaled proportionally and rotated around the long axis to obtain the ellipsoidal deformation model of the right upper arm in the three-dimensional space, and the volume of the model is calculated. The specific implementation steps of this method are as follows:

[0093] Ellipsoidal deformation fitting

[0094] Projecting the three-dimensional ellipsoidal deformation model of Equation (2.14) onto the xy plane can obtain the equation of the two-dimensional ellipsoidal deformation model, as shown in Equation (2.15). r x is half of the length of the long axis of the ellipsoidal deformation model, r y is half of the length of the short axis, the long axis is on the x-axis, and the midpoint of the long axis is at the coordinate origin.

[0095]

[0096] Since Figure 2 the two-dimensional contour in (c) is the point on the image pixel coordinate system, and the coordinate origin is at the upper right corner of the image. To facilitate fitting the contour into the ellipsoidal deformation model, the contour coordinates need to be transformed into the local coordinate system with the midpoint o' of the ellipse long axis as the origin. As Figure 3 shown, the original coordinate origin o is translated u1 unit lengths along the u-axis direction and v1 unit lengths along the v-axis direction, and then the rotation angle around the coordinate origin is θ. If the coordinate of a point Q on the image in the pixel coordinate system is (u2, v2), then the coordinate of point Q in the local coordinate system with point O' as the origin is:

[0097] x = -(u2 - u1)sinθ + (v2 - v1)cosθ (2.16)

[0098] y = (u2 - u1)cosθ + (v2 - v1)sinθ (2.17)

[0099] After converting the coordinates of all points on the right upper arm contour and substituting them into Equation (2.15), the iterative least squares method is used to fit the ellipsoidal deformation equation, and the translation parameters u1, v1 of the coordinate axis transformation, the rotation parameter θ, and the parameters d, r of the ellipsoidal deformation equation can be obtainedx 、r y 。

[0100] The elliptical deformation equation obtained by fitting the upper right arm contour of the left image is:

[0101]

[0102] The translation parameters u1 = 1415.4243 and v1 = 616.8064, and the rotation parameter θ = 54.3808 for the coordinate axis transformation on the left image.

[0103] The elliptical deformation equation obtained by fitting the upper right arm contour of the right image is:

[0104]

[0105] The translation parameters u1 = 536.1577 and v1 = 635.6233, and the rotation parameter θ = 54.4035 for the coordinate axis transformation on the right image.

[0106] Figure 4 and Figure 5 are the result graphs of fitting the elliptical deformation model to the upper right arm contour in the images from the left and right perspectives respectively. In the figures, the blue ones are the extracted contour data points, and the red ones are the deformed ellipses obtained by fitting. The fitting results basically coincide with the contour points. Points a and b in the figures are the two endpoints of the major axis of the ellipse, and also correspond to the joint points at both ends of the upper right arm bone, namely the shoulder joint point and the elbow joint point. According to Equation (2.18), Equation (2.19), and the corresponding translation and rotation parameters of the coordinate axis transformation, the pixel coordinates of points a and b in the left and right images can be deduced inversely. The image pixel coordinates (u la , v la ) of point a in the left image are (1731, 402), and the image pixel coordinates (u lb , v lb ) of point b are (1100, 831). The image pixel coordinates (u ra , v ra ) of point a in the right image are (863, 424), and the image pixel coordinates (u rb , v rb ) of point b are (209, 847).

[0107] In a preferred embodiment of the present invention, it can be further configured as follows: The deformable model fitting unit includes the following steps:

[0108] Construction of a multi-parameter deformable model: This unit uses a multi-parameter deformable model, which not only includes the basic morphological parameters of the human body but also introduces additional parameters that can describe the subtle changes in the human body's morphology. By constructing such a model, the morphological characteristics of the human body can be captured and described more comprehensively, improving the accuracy and flexibility of fitting;

[0109] Model Initialization and Iterative Optimization: During the model fitting process, the model is first initialized based on the preliminary human contour extraction results. Subsequently, iterative optimization algorithms such as gradient descent, Newton's method, or genetic algorithms are used to continuously adjust and optimize the model parameters to minimize the error between the model and the true human contour. Through multiple iterations, it is ensured that the model can accurately fit the human body shape;

[0110] Model Adaptive Adjustment Mechanism: To cope with the changes in different human body shapes and postures, this unit also includes a model adaptive adjustment mechanism. This mechanism can dynamically adjust the shape parameters and constraint conditions of the model according to the complexity and differences of the human contour, ensuring that the model can maintain good fitting effects and generalization abilities in different application scenarios.

[0111] Calculation of Bone Length and Volume

[0112] The difference in the horizontal coordinates of points a and b in the left and right images, Δu a = u la - u ra = 868, Δu b = u lb - u rb = 891. Substitute Δu a , Δu b and the parameters in the matrix Q obtained by binocular calibration into Equation (2.11), and the three-dimensional coordinates of points a and b in the real world can be obtained as (X a , Y a , Z a ) = (1057.89, -150.58, 2095.62), (X b , Y b , Z b ) = (828.24, -9.123, 2041.52). The unit length of the coordinates is mm. The distance between points a and b is 274.29 mm. Then Figure 2 (a) The length of the right upper arm of this person is 27.429 cm, and half of the length of the right upper arm Before the experiment, we manually measured the length of the right upper arm of this person with a tape measure to be 28.5 cm, with a difference of 1.071 cm from the experimental result, and the error rate is

[0113] According to the camera imaging principle, the process of an object in the real world projecting onto the image is a process of equal-proportion reduction of the object size. Therefore, according to the parameters r x = 381.6035, r y = 164.1128 in the elliptical deformation model formula (2.18) fitted on the image, the following can be obtained:

[0114]

[0115] Substituting Hb = 137.15, Hp = 58.98, and the parameter d = 0.0018 in the fitting result of Equation (2.18) into Equation (2.14), the ellipsoidal deformation model of the actual size of the right upper arm in three-dimensional space can be obtained as follows:

[0116]

[0117] In a preferred embodiment of the present invention, it can be further configured that the three-dimensional position calculation unit includes the following processing procedures:

[0118] Binocular stereo matching algorithm: This unit adopts an advanced binocular stereo matching algorithm. By comparing the corresponding feature points in the images captured by the left and right cameras, the corresponding relationship between them is established. This algorithm can fully consider problems such as illumination changes, noise interference, and occlusion in the images, ensuring the accuracy and stability of the matching results. Binocular stereo matching is the basis for realizing three-dimensional position calculation and provides reliable data support for subsequent three-dimensional reconstruction;

[0119] Three-dimensional coordinate calculation and reconstruction: After establishing the binocular stereo matching relationship, this unit uses the principle of triangulation to calculate the three-dimensional coordinates of each matching feature point. By accumulating a sufficient number of three-dimensional coordinate points, the three-dimensional point cloud data of the human body can be constructed. Subsequently, three-dimensional reconstruction algorithms such as surface reconstruction and volume reconstruction are used to convert the point cloud data into a continuous three-dimensional model, thereby realizing the three-dimensional visualization of the human body shape;

[0120] Error correction and optimization: In order to improve the accuracy of three-dimensional position calculation, this unit also includes error correction and optimization steps. By considering the calibration error of the camera, the error of image processing, and various uncertainty factors in the three-dimensional reconstruction process, the calculated three-dimensional coordinates are corrected and optimized. This step ensures that the finally obtained three-dimensional model can accurately reflect the true shape and position information of the human body.

[0121] In a preferred embodiment of the present invention, it can be further configured that the binocular stereo matching algorithm in the three-dimensional position calculation unit adopts a semi-global matching strategy. This strategy not only considers the matching cost in the local area but also introduces the idea of global optimization. By using the dynamic programming algorithm to search for the optimal matching path in multiple directions, the false matching can be effectively reduced, and the matching accuracy and robustness can be improved. In addition, the SGM algorithm can handle the matching problems in different parallax ranges, adapt to the diversity of human body shapes and postures, and ensure accurate three-dimensional position calculation in complex scenarios.

[0122] In a preferred embodiment of the present invention, it can be further configured that the model adaptive adjustment mechanism in the deformable model fitting unit combines human kinematic constraints. When adjusting the model parameters, this mechanism not only considers the contour information in the image data, but also introduces kinematic constraint conditions such as the range of motion and relative positions of human joints. By incorporating these constraint conditions into the iterative optimization process, it is possible to limit the search space of the model parameters, avoid unreasonable fitting results, and improve the ability of the model to accurately describe the human body shape and posture. At the same time, the model adaptive adjustment mechanism combined with human kinematic constraints can also enhance the adaptability of the system to human motion changes and improve the measurement accuracy and stability in dynamic scenarios.

[0123] The human parameter measurement system based on binocular vision and deformable models realizes precise and automated measurement of human parameters by integrating multiple highly specialized modules. The system starts from the binocular vision acquisition module, using a pair of high-precision and high-resolution cameras (configured with wide-angle lenses) to capture human image data. To ensure strict temporal synchronization of the left and right camera images, a synchronous trigger control unit is built into the system, achieving image capture synchronization at the millisecond level and effectively eliminating parallax errors caused by time differences. At the same time, the environmental light compensation unit automatically adjusts the camera exposure parameters according to the light conditions or uses an external light source to ensure stable image quality, laying a good foundation for subsequent processing.

[0124] The collected raw image data then enters the image preprocessing and analysis module. This module first uses a high-precision calibration board and advanced calibration algorithms to accurately calibrate the cameras through the binocular calibration and correction unit, and generates a correction matrix through the stereo correction matrix calculation module to eliminate image distortion and parallax errors. The correction effect evaluation and feedback mechanism ensures the continuous optimization of the calibration and correction process. Then, the human body parsing and contour extraction unit uses multi-scale image pyramids, deep learning-based human body segmentation algorithms, and contour refinement and optimization techniques to accurately segment and extract the contours of various parts of the human body.

[0125] The extracted human contour data is sent to the parameter solving and modeling module. In this module, the deformable model fitting unit uses an ellipsoidal deformation model and an egg-shaped surface model to fit the human contour, and accurately solves parameters such as the bone lengths, volumes, and relative centroids of various parts of the human body through multi-parameter deformable model construction, model initialization and iterative optimization, and a model adaptive adjustment mechanism (combined with human kinematic constraints). This step not only improves the fitting accuracy and flexibility, but also enhances the adaptability of the system to human body shape and posture changes.

[0126] Finally, the three-dimensional position calculation unit uses the binocular stereo matching algorithm (adopting the semi-global matching strategy to improve the matching accuracy and robustness) to establish the matching relationship of corresponding feature points in the left and right camera images, and calculates the three-dimensional coordinates of each matching feature point through the principle of triangulation. After accumulating a sufficient number of three-dimensional coordinate points, the system uses a three-dimensional reconstruction algorithm to construct the three-dimensional point cloud data of the human body and converts it into a continuous three-dimensional model. To improve the accuracy of three-dimensional position calculation, the system also includes an error correction and optimization step to correct and optimize the calculated three-dimensional coordinates.

[0127] In summary, through the collaborative work of steps such as binocular vision acquisition, image preprocessing and analysis, deformable model fitting, and three-dimensional position calculation, this system realizes the accurate measurement of human parameters. The entire system not only features high precision and high efficiency, but also has good adaptability and robustness, and can meet the application requirements in multiple fields such as medical diagnosis, motion analysis, virtual reality, and augmented reality.

[0128] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0129] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A human body parameter measurement system based on binocular vision and deformable model, characterized in that: The system includes: Binocular vision acquisition module, used to capture human image data through a binocular camera; Image preprocessing and analysis module, which includes: The binocular calibration and correction unit uses the camera calibration method to accurately calibrate the binocular camera, and uses the Matlab Stereo Camera Calibrator toolbox to perform binocular calibration to ensure the coplanarity and row alignment of the left and right camera imaging planes; The human body analysis and contour extraction unit uses the self-correction of human body analysis (SCHP) method to segment the pre-processed human body image to accurately extract the contours of various parts of the human body; Parameter solving and modeling module, which includes: The deformable model fitting unit uses the ellipsoid deformation model and the egg-shaped surface model to fit the extracted contours of various parts of the human body to solve the parameters such as the bone length, volume and relative center of mass of various parts of the human body; The three-dimensional position calculation unit, combined with the binocular parallax principle, uses the fitted model parameters to solve the three-dimensional position coordinates of the joint point in the real space; The dual-target positioning and correction unit, the human body analysis and contour extraction unit, the deformable model fitting unit and the three-dimensional position calculation unit work together to achieve accurate measurement of human body parameters.

2. A human body parameter measurement system based on binocular vision and deformable model according to claim 1, characterized in that: The binocular vision acquisition module includes: High-resolution camera unit: This unit consists of a pair of high-precision, high-resolution cameras that can capture delicate human image details and provide a high-quality image data source for subsequent image preprocessing and analysis. The camera is equipped with a wide-angle lens to ensure that the entire human body can be fully captured within a reasonable shooting distance to avoid image truncation or distortion; Synchronous trigger control unit: To ensure that the images captured by the left and right cameras are strictly synchronized in time, the module has a built-in synchronous trigger control unit. By precisely controlling the shutter opening time of the camera, it can achieve millisecond-level image capture synchronization, effectively eliminate the parallax error caused by time difference, and improve the accuracy and stability of 3D reconstruction. Ambient light compensation unit: Considering that changes in lighting conditions in different environments may have a significant impact on image quality, the binocular vision acquisition module also includes an ambient light compensation unit, which automatically adjusts the camera's exposure parameters or uses external light sources for auxiliary lighting to ensure that images with uniform brightness and moderate contrast can be obtained under different lighting conditions, laying a good foundation for subsequent image processing steps; Image preprocessing acceleration unit: To improve the processing efficiency of the entire system, the binocular vision acquisition module also integrates an image preprocessing acceleration unit. This unit uses hardware acceleration technology, such as GPU acceleration or dedicated image processing chips, to quickly preprocess the collected original images, including denoising, contrast enhancement, edge detection, etc., to provide preprocessed image data for subsequent human body analysis and contour extraction, thereby shortening the overall processing time and improving the system response speed.

3. The human body parameter measurement system based on binocular vision and deformable model according to claim 1, characterized in that: The dual-target alignment and calibration unit comprises the following components: Precision calibration plate and calibration algorithm: This unit uses a high-precision, high-resolution calibration plate as a calibration reference, combined with advanced calibration algorithms, to accurately calibrate the internal and external parameters of the binocular camera. The calibration algorithm can fully consider factors such as the camera's lens distortion and nonlinear error, and ensure the accuracy and stability of the calibration results through an iterative optimization process. In addition, the design of the calibration plate takes into account a variety of sizes and patterns to meet the needs of different camera configurations and shooting environments; Stereo correction matrix calculation module: After completing the camera calibration, the stereo correction matrix calculation module calculates and generates the stereo correction matrix of the binocular camera according to the calibration results. This matrix can eliminate the image distortion and parallax error caused by the difference in viewing angle of the binocular camera, so that the imaging planes of the left and right cameras are strictly coplanar and aligned. By applying the stereo correction matrix, the 3D reconstruction accuracy and real-time performance of the binocular vision system can be significantly improved; Correction effect evaluation and feedback mechanism: To ensure the accuracy and effectiveness of dual-target positioning and correction, this unit also includes a correction effect evaluation and feedback mechanism, which evaluates the correction effect by comparing the image data before and after correction; at the same time, the calibration parameters and correction algorithm are fine-tuned according to the evaluation results to form a closed-loop feedback system. This iterative optimization process enables the dual-target positioning and correction unit to continuously improve the correction accuracy and adapt to the needs of different application scenarios.

4. The human body parameter measurement system based on binocular vision and deformable model according to claim 1, characterized in that: The human body analysis and contour extraction unit comprises the following processing steps: Multi-scale image pyramid processing: In order to improve the recognition accuracy of human body at different scales, this unit adopts multi-scale image pyramid processing technology. This technology constructs image pyramids of different resolutions to represent the original image at multiple scales, thereby capturing the characteristic information of human body at multiple scales. This step helps to accurately segment the human body area in a complex background and provide high-quality input data for subsequent contour extraction; Human segmentation algorithm based on deep learning: In order to improve the accuracy of human segmentation, this unit introduces a human segmentation algorithm based on deep learning. The algorithm learns the feature representation of various parts of the human body by training a large amount of human image data, and can accurately identify and segment human areas in complex environments. The application of deep learning algorithms has significantly improved the robustness and automation of human segmentation and reduced dependence on manual intervention; Contour refinement and optimization technology: After the initial segmentation of the human body area, this unit uses contour refinement and optimization technology to process the human body contour. Through morphological operations, edge detection, curve fitting and other means, it optimizes the smoothness and continuity of the contour and reduces the contour jitter and discontinuity caused by image noise or segmentation errors. This step ensures that the extracted human body contour is more accurate and clear, providing reliable input data for subsequent human body parameter measurements.

5. The human body parameter measurement system based on binocular vision and deformable model according to claim 1, characterized in that: The deformable model fitting unit comprises the following steps: Multi-parameter deformable model construction: This unit uses a multi-parameter deformable model, which not only contains the basic morphological parameters of the human body, but also introduces additional parameters that can describe subtle changes in human morphology. By building such a model, the morphological characteristics of the human body can be more comprehensively captured and described, and the accuracy and flexibility of fitting can be improved; Model initialization and iterative optimization: In the process of model fitting, the model is first initialized according to the preliminary human body contour extraction results. Then, iterative optimization algorithms such as gradient descent method, Newton method or genetic algorithm are used to continuously adjust and optimize the model parameters to minimize the error between the model and the real human body contour. Through multiple iterations, it is ensured that the model can accurately fit the human body shape. Model adaptive adjustment mechanism: In order to cope with the changes in different human body shapes and postures, this unit also includes a model adaptive adjustment mechanism, which can dynamically adjust the shape parameters and constraints of the model according to the complexity and diversity of the human body contour, ensuring that the model can maintain good fitting effect and generalization ability in different application scenarios.

6. The human body parameter measurement system based on binocular vision and deformable model according to claim 1, characterized in that: The three-dimensional position calculation unit includes the following processing flow: Binocular stereo matching algorithm: This unit uses an advanced binocular stereo matching algorithm to establish a corresponding relationship between the corresponding feature points in the images captured by the left and right cameras. The algorithm can fully consider the problems of illumination changes, noise interference and occlusion in the image to ensure the accuracy and stability of the matching results. Binocular stereo matching is the basis for realizing three-dimensional position calculation and provides reliable data support for subsequent three-dimensional reconstruction. 3D coordinate calculation and reconstruction: After establishing the binocular stereo matching relationship, this unit uses the triangulation principle to calculate the 3D coordinates of each matching feature point. By accumulating a sufficient number of 3D coordinate points, the 3D point cloud data of the human body can be constructed. Subsequently, 3D reconstruction algorithms such as surface reconstruction and volume reconstruction are used to convert the point cloud data into a continuous 3D model, thereby realizing 3D visualization of the human body shape. Error correction and optimization: In order to improve the accuracy of three-dimensional position calculation, this unit also includes error correction and optimization steps. By considering the calibration error of the camera, the error of image processing and various uncertain factors in the three-dimensional reconstruction process, the calculated three-dimensional coordinates are corrected and optimized. This step ensures that the final three-dimensional model can accurately reflect the real shape and position information of the human body.

7. The human body parameter measurement system based on binocular vision and deformable model according to claim 6, characterized in that: The binocular stereo matching algorithm in the three-dimensional position calculation unit adopts a semi-global matching strategy, which not only considers the matching cost of the local area, but also introduces the idea of ​​global optimization. It searches for the optimal matching path in multiple directions through a dynamic programming algorithm, thereby effectively reducing mismatching and improving matching accuracy and robustness. In addition, the SGM algorithm can handle matching problems within different parallax ranges, adapt to the diversity of human body shapes and postures, and ensure accurate three-dimensional position calculation in complex scenes.

8. The human body parameter measurement system based on binocular vision and deformable model according to claim 5, characterized in that: The model adaptive adjustment mechanism in the deformable model fitting unit is combined with human kinematic constraints. When adjusting the model parameters, the mechanism not only considers the contour information in the image data, but also introduces kinematic constraints such as the motion range and relative position of the human joints. By incorporating these constraints into the iterative optimization process, the search space of the model parameters can be limited, unreasonable fitting results can be avoided, and the model's ability to accurately describe the human body's morphology and posture can be improved. At the same time, the model adaptive adjustment mechanism combined with human kinematic constraints can also enhance the system's adaptability to changes in human motion and improve measurement accuracy and stability in dynamic scenes.

Images