Image-based body component modeling and analysis method
Through image-based deep learning models, the 3D human body model is generated from 2D images, and the human body size and volume are measured, which solves the problems of low body fat rate measurement accuracy and complex equipment in traditional methods, and achieves fast and accurate body fat rate measurement.
Patent Information
- Application Number
- CN202410953197.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art has problems such as low accuracy, complex and cumbersome equipment, and the need for specific equipment when measuring the body fat rate of the human body. The traditional method has a large error and is inconvenient to be widely applicable.
Image-based body component modeling and analysis methods are used to generate a 3D human body model from 2D images through deep learning models, measure the size value of the designated parts of the human body, and use a linear regression model to calculate the body volume and density, thereby predicting the body fat rate.
Fast, accurate and convenient measurement of body fat rate in humans is achieved, and equipment dependence and error problems in traditional methods are avoided, and common equipment such as smartphones can be used in home or clinic environments.
Smart Images

Figure CN118967970A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of bioinformatics, and in particular to an image-based body component modeling and analysis method. Background Art
[0002] Human body volume can be used to calculate body composition to perform obesity assessment. Body fat percentage is the proportion of total body fat weight in human body weight. It is of great significance whether you want to lose weight or pay attention to your physical health. Different body fat percentages correspond to different exercise plans. Only in scientific exercise can you better get the weight loss results you want. In medicine, accurate measurement of body fat percentage also requires understanding of two parameters: body weight and body volume.
[0003] Most of the body composition analysis equipment currently on the market uses multi-frequency bioelectrical impedance analysis. With the help of electrodes placed on the body surface, single-frequency or multi-frequency tiny currents are input into the subject, and the corresponding electrical impedance and its changes are detected to obtain relevant body composition information. Its high-precision scientific technology greatly shortens the measurement time and can measure the body fat percentage of the human body. In this process, because there is no data on the body volume, the measurement indicators are not very accurate.
[0004] At the same time, traditional volume measurement methods usually use underwater weighing or air displacement methods, including: Underwater weighing method: measure underwater body weight and water body weight separately, and then substitute these values into a specific formula for conversion. The principle is that the relative densities of muscle and fat are different; Air displacement method: put the test object into a specific device and measure the body volume by displacing the air; The measurement errors of the air displacement method and the underwater weighing method are both 2% to 3%. At present, these methods are too complicated and cumbersome, and the errors are relatively large. In addition, the traditional method of measuring volume requires special equipment and devices, and the equipment integration makes it too large and cannot be generally applied.
[0005] In addition, traditional methods for measuring human body fat percentage mainly include skinfold thickness measurement, bioimpedance analysis, dual-energy X-ray absorptiometry, etc. These methods have their own characteristics. The slightly more convenient skinfold thickness measurement often requires the assistance of others for invasive measurement, and will be affected by the operator's technical level and measurement position, which is prone to measurement errors. Among them, bioimpedance analysis (BIA) and dual-energy X-ray absorptiometry require corresponding equipment to obtain more accurate human body composition values.
[0006] Therefore, it has become a very important issue to have a method for predicting human body composition that is quick and convenient to calculate and has a certain accuracy.
[0007] To this end, the present application specifically proposes an image-based body component modeling and analysis method to solve the above-mentioned technical problems, and the above-mentioned content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above-mentioned content is the closest prior art. Summary of the invention
[0008] The main purpose of the present invention is to provide an image-based body component modeling and analysis method to solve the technical problems raised in the background technology.
[0009] The present invention adopts the following technical solutions to solve the above technical problems:
[0010] A body component modeling and analysis method based on an image comprises the following specific steps:
[0011] S1. Input human 2D image and human size information;
[0012] S2. The input image and human body information are converted into mesh information through a pixel2mesh model based on deep learning, and a 3D human body model is further obtained;
[0013] S3. Optimize the generated 3D human body model and measure the size of the specified part of the human body;
[0014] S4. training a linear regression model based on the obtained size values of the specified parts to calculate the volume of the human body;
[0015] S5. After obtaining the human body volume, the human body density is obtained according to the principle of the two-compartment model of the human body. The proportional relationship between the body fat rate and weight of the person being measured is calculated through the density, and the proportion of human body fat composition is predicted.
[0016] Preferably, the specific operation steps of generating mesh information and obtaining a 3D human body model in step S2 include:
[0017] S21. Use a fully convolutional neural network to extract features of the input human body 2D image, and obtain a feature map of the human body 2D image by inputting the human body 2D image through multiple convolution pooling operations;
[0018] S22. Generate an ellipsoid mesh and extract its three-dimensional model features, and continuously deform the three-dimensional model to stretch and deform the ellipsoid mesh into a desired 3D model;
[0019] S23. Set a permanent feature pooling layer to associate the 2D image information and the 3D model information with each other, adjust the node state of the graph convolutional network in the 3D model through the feature size information, and finally output the 3D human body model information.
[0020] Preferably, the specific operation method of extracting the features of the three-dimensional model and continuously deforming the three-dimensional model in step S22 includes:
[0021] Input a standard two-dimensional human body image and use an implicit surface algorithm to generate an initial ellipsoid model with an ellipsoid mesh;
[0022] Uniformly initialize and select vertices on the ellipsoid model, set the initial shape feature as the three-dimensional coordinates of each vertex, and fuse the vertex position information and 2D image information;
[0023] The feature map is obtained based on the 2D human body image, and a new feature map is obtained after multiple convolutions through the VGG-16 network. For any vertex in the model, the three-dimensional coordinate C i-1 , calculate the specific position of its two-dimensional projection on the input 2D human body image plane T;
[0024] Use bilinear interpolation to concentrate the features of the four pixels in the adjacent position of the feature map obtained by convolution of the original feature map in the VGG-16 network, and let the feature information of the vertex be the perceptual feature information P of the vertex;
[0025] The perceptual feature P is combined with the 3D shape feature F attached to the vertices of the input graph. i-1 The comprehensive features of the vertices are obtained by cascading and input into the graph residual neural network for calculation to generate new coordinates C for each vertex i and 3D shape feature F i As the output of the mesh deformation module.
[0026] Preferably, when uniformly initializing the selected vertices in step S22, a de-pooling layer is established to increase the number of vertices of the model. The de-pooling layer is used to increase the number of vertices of the model by connecting the midpoints of each edge of the triangle formed by the vertices in the model and dividing the original triangle into four. After multiple operations, the de-pooling layer is used to output the vertex coordinates of a more refined 3D model. A multi-vertex set can obtain a 3D human body point cloud model with three-dimensional scattered point cloud data Z.
[0027] Preferably, the specific operation steps of optimizing the generated 3D human body model in step S3 include:
[0028] Predefine a threshold M and quantity value K. If the distance between a point and other points exceeds the threshold M, the point is defined as a noise point of the model.
[0029] Read the three-dimensional scattered point cloud data Z in the 3D human body point cloud model, for each point P in P i Calculate the distances between the K nearest neighboring points to the point and calculate P i The average distance M(Pi );
[0030] The average value of the distance M(P i ) is compared with the preset threshold M. If M(P i )≤M, then keep the point P i , continue to calculate the next point P, if M(P i )>M, then delete the point;
[0031] After all points have been searched and calculated, a 3D human body point cloud model with a smoother surface can be obtained.
[0032] Preferably, in step S3, the size of the designated part of the human body is set as a characteristic size of closure, and the specific operation steps of measuring the size of the designated part of the human body include:
[0033] Partially connected straight line segments can be obtained from the triangles located on the surface of the human body model and connected by the point cloud. By fitting the points of these straight line segments with B-spline curves, a smooth cross-section of the characteristic parts of the human body can be obtained. At this time, the length of the calculated curve is the size measurement information of this part.
[0034] Preferably, in step S3, the size of the designated part of the human body is set as a non-closed characteristic size, and the specific operation steps of measuring the size of the designated part of the human body include:
[0035] The position of the dimension feature point is determined by the geometric features of the characteristic dimension, and the human body model is intercepted using the feature points to obtain the human body cross-section. The path points on the cross-section are searched, the distances between these path points are calculated, and data fitting is performed to obtain the dimension measurement information of this part.
[0036] Preferably, the specific operation steps of training the linear regression model according to the obtained size values of the designated parts in step S4 include:
[0037] S41. Initialize a multi-layer neural network, randomly initialize the weights between nodes, and define a cross entropy function between the calculated volume value and the human body volume of the training data as a loss function;
[0038] S42. Obtain sample data externally, and use the size values of multiple specified parts of the human body of the sample data as input, and the volume values in the training set as true values to train the neural network, and use the gradient descent algorithm to modify the weight values between nodes during the back propagation process in each round of training to iteratively update the weights;
[0039] S43. The loss function is recorded in each round of training, and the minimum value R of the loss function is preset. When the loss function is less than R, the training is stopped. The weight information of this network is saved as the network for calculating body composition, which can be used as a neural network model for calculating human body volume.
[0040] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0041] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0042] It can be seen from the above technical solution that the present invention provides a body component modeling and analysis method based on images. Compared with the prior art, the present invention has the following advantages:
[0043] 1. The present invention generates an ellipsoid model by acquiring a 2D human body image and a computer built-in system, and can quickly obtain a human body 3D model corresponding to the image, and use measurement to obtain the size information and volume information of the human body 3D model to further quickly obtain the size information and volume information of the actual human body. It is convenient and fast, and a personalized model can be generated in a short time using a common, portable and accessible device such as a smart phone.
[0044] 2. The present invention can solve the problems of human body volume measurement, complicated collection process, complex scene requirements, long time, and the need for specific collection equipment. It has no requirements for the surrounding environment. In addition, the cost of the equipment that can be used for collection has also been reduced. Only the simplest consumer-grade digital camera or even a cheap, common, portable and accessible device such as a smart phone can be used to generate results in a short time. No expensive 3D scanning equipment is required. Consumer-grade digital cameras can be used to visualize the body and accurately estimate BV for women and men without obesity (BMI<30kg / m2), which is convenient for obesity management and monitoring in home or clinic environments.
[0045] 3. The present invention utilizes an end-to-end neural network architecture to generate a 3D mesh model from a single RGB image, calculates 3D geometry in a coarse-to-fine manner, is reliable and easy to learn, and calculates human body size and volume after obtaining the human body model.
[0046] 4. The present invention uses a single 2D picture of a human body in a natural posture with frontal stretch to reconstruct a 3D model more accurately. Compared with the traditional method, the accuracy deviation of calculating the 3D human body volume value from a 2D image is not large, and the human body volume can be measured more accurately.
[0047] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become easy to understand through the following description. Of course, it is not necessary to achieve all of the advantages described above simultaneously for any product implementing the present invention.
[0048] 5. Compared with the traditional human body composition measurement method in the prior art, the present invention can obtain relatively effective human body composition values using only one picture, which greatly facilitates the measurement of body composition such as body fat percentage, and avoids the traditional process of measuring body composition such as body fat percentage, which either requires corresponding equipment to obtain relatively accurate human body composition, or requires auxiliary measurement, thereby causing complex measurement conditions and cumbersome measurement procedures. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0050] Figure 1 is a flow chart of the method for calculating human body volume of the present invention;
[0051] Figure 2 A flow chart of the operation method of generating mesh information and obtaining a 3D human body model for the present invention;
[0052] Figure 3 This is a schematic diagram of the pixel2mesh network of the present invention;
[0053] Figure 4 It is a convolution operation demonstration diagram of the present invention;
[0054] Figure 5 It is a demonstration diagram of the pooling operation of the present invention;
[0055] Figure 6 It is a schematic diagram of a grid deformation module of the present invention;
[0056] Figure 7 Schematic diagram of the residual convolutional layer of the residual network of the present invention;
[0057] Figure 8 It is a schematic diagram of the reverse pooling operation of the present invention;
[0058] Fig. 9 This is a schematic diagram of the process of measuring the dimensions of characteristic parts of a human body according to the present invention;
[0059] Fig.10 A schematic diagram for generating a structure of a 3D human body model of the present invention;
[0060] Fig.11 A schematic diagram of the process of predicting human body volume by the model of the present invention;
[0061] Fig.12 Schematic diagram of linear regression analysis of body fat mass prediction and measurement values of the present invention. DETAILED DESCRIPTION
[0062] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. In the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0063] In the embodiment, see Figures 1 to 11 .
[0064] like Figure 1 The image-based body composition modeling and analysis method proposed in the embodiment of the present invention measures the real volume value of the human body by the body density method and measures the body composition value including body fat percentage by the bioelectrical impedance method. At the same time, the single-view 2D human body image is used as the starting point for prediction, and the obtained results are used as the sample set and test set in the subsequent training model process.
[0065] Specifically, the method for calculating human body volume and human body composition performs the following specific steps through a computer device:
[0066] S1. Input a 2D image of a human body in a normal posture under a single viewing angle and human body size information (especially size information of characteristic parts of the human body).
[0067] The human body 2D image is preferably a single 2D picture of a human body in a natural posture with a front view. By using a single 2D picture of a human body in a natural posture with a front view, a relatively accurate reconstruction of a 3D model can be ensured.
[0068] S2. The input image and human body information generates mesh information through the pixel2mesh model based on deep learning, and further obtains a 3D human body model.
[0069] The pixel2mesh model generates mesh information and obtains the entire network of the 3D human body model, which consists of an upper-layer image feature network and a lower-layer cascaded mesh deformation network. Figure 2 and Figure 3 , used to combine the permanent feature pooling layer to perform the following steps:
[0070] S21. In the upper layer image feature network, a fully convolutional neural network (CNN) is used to extract the features of the input human 2D image, and the human 2D image is input through multiple convolution pooling operations to obtain the feature map of the human 2D image:
[0071] Input a standard human 2D image, and obtain a feature map after convolution (using a 3*3 convolution kernel) and pooling.
[0072] At this time, multiple consecutive convolution operations are regarded as a convolution block. For the upper-layer image feature network, this network undergoes five convolution block operations, and a pooling layer is used in between to modify the size of the feature map obtained by the convolution. The vertex position and shape features of the feature map and the initial model are used as the input of the mesh deformation module to deform and obtain new vertices and shape features of the model.
[0073] It should be noted that the convolution here uses a 3*3 convolution kernel, and the pooling here selects the largest one of the 2*2 image points of the original image and outputs it to the new image, which is called maximum pooling; the specific operations of convolution and pooling are demonstrated as follows: Figure 4 and Figure 5 shown.
[0074] It is also specifically stated that the feature map results obtained in the first, third, and fifth convolution blocks need to pass through the permanent feature pooling layer as auxiliary information to enter the mesh deformation module to generate the final human body 3D model.
[0075] S22. Generate an ellipsoid mesh in the lower layer cascade mesh deformation network and extract its three-dimensional model features, and continuously deform the three-dimensional model, gradually stretching and deforming the ellipsoid mesh into the required 3D model, refer to Figure 6 :
[0076] In the first step, a standard human body two-dimensional image is input, and an initial ellipsoid model with an ellipsoid mesh is generated using an implicit surface algorithm for the mesh deformation module to perform deformation operations;
[0077] Here, the implicit surface algorithm is set as the mathematical function expression algebraic equation of the given surface: f(x, y, z) = 0, and since this function corresponds to a surface in three-dimensional space, the expression corresponding to the ellipsoid is found here for generation;
[0078] The second step is to uniformly initialize and select the vertices of the ellipsoid model. The initial shape feature is set as the three-dimensional coordinates of each vertex, and the position information of the vertex and the 2D image information are fused.
[0079] The third step is to obtain a feature map from the 2D human body image and obtain a new feature map after multiple convolutions through the VGG-16 network. For any vertex in the model, the three-dimensional coordinates Ci-1 , calculate the specific position of its two-dimensional projection on the input 2D human body image plane T;
[0080] The structure of the VGG-16 network here is almost completely similar to the image feature network in the upper layer;
[0081] The fourth step is to use bilinear interpolation to concentrate the features of the four pixels at adjacent positions of the feature map obtained by convolution of the original feature map in the VGG-16 network (the features extracted from multiple feature maps obtained after the convolution operation of the third, fourth, and fifth convolution blocks are connected, and the total dimension is 1280), and let the feature information of the vertex be the perceptual feature information P of the vertex;
[0082] Here, bilinear interpolation is used to obtain the values on the feature map of the surrounding position for mean calculation, and the feature map obtained by the VGG-16 network convolution should also include multiple feature maps obtained after the convolution operation of the third, fourth, and fifth convolution blocks;
[0083] The fifth step is to combine the perceptual feature P with the 3D shape feature F attached to the vertices of the input graph. i-1 The connection relationship of the points in the model represents the connection relationship of the points on the surface of the initial ellipsoid model with the ellipsoid mesh. The comprehensive features of the vertices are cascaded and input into the graph residual neural network based on the graph for calculation to generate a new coordinate C for each vertex. i and 3D shape feature F i As the output of the mesh deformation module.
[0084] Among them, because the shape features are not learned at the beginning, the perceptual features P are connected with the 3D features (vertex coordinates) in the first mesh deformation block.
[0085] In addition, reference Figure 7 The graph residual neural network calculates the new position of each vertex and the shape features of the model by inputting the fused vertex features and the shape feature information of the model. It consists of 14 graph residual convolutional layers and 128 channels, which are used to input the point coordinates and the connection relationship between points in the form of a matrix as the network input. Each layer operates on the graph information in the form of matrix transformation, and finally outputs the new point coordinates and the connection relationship between points.
[0086] Specifically, each layer of the graph residual neural network also convolutionally calculates the feature map and considers whether to retain the changes brought by the previous layer, so as to finally obtain the new 3D model shape features and vertex position information.
[0087] Furthermore, when uniformly initializing the selected vertices in step S22, an anti-pooling layer can be established to increase the number of vertices of the model, such as Figure 8As shown, by connecting the midpoints of each side of the triangle formed by the vertices in the model and dividing the original triangle into four, the number of vertices of the model is increased, and after multiple such operations, a more refined 3D model vertex coordinates are output. A multi-vertex set can obtain a 3D human body point cloud model with three-dimensional scattered point cloud data Z.
[0088] S23. The 2D image information above and the 3D model information below are linked together through a permanent feature pooling layer. The node status of the graph convolutional network in the 3D model is adjusted by referring to the feature size information of the 2D image of the human body. At this time, the process of the two network connection information modules is completed using a mesh deformation module, and the output result of this module is the model information.
[0089] In summary, the ellipsoid model generated by the implicit surface algorithm at the beginning can be outputted as the final 3D human body model after three mesh deformation modules and two anti-pooling modules.
[0090] S3. Optimizing the generated 3D human body model can obtain a smoother 3D human body point cloud model for measuring the size of a specified part of the human body.
[0091] (1) Optimize the human body point cloud model:
[0092] First, you need to predefine a threshold M and a predefined quantity value K. If the distance between a point and other points exceeds the threshold M, the point is defined as a noise point of the model.
[0093] Read the three-dimensional scattered point cloud data Z in the 3D human body point cloud model, for each point P in P i Calculate the distances between the K nearest neighboring points to the point and calculate P i The average distance M(P i ), the average value of the distance M(P i ) is compared with the preset threshold M. If M(P i )≤M, then keep the point P i , continue to calculate the next point P, if M(P i )>M, then delete the point.
[0094] After all points are searched and calculated and processed through the above operation flow, a 3D human body point cloud model with a smoother surface can be obtained. After obtaining the smooth 3D human body point cloud model, the characteristic parts in the processed human body point cloud model can be measured to calculate the corresponding human body size.
[0095] (2) Measuring the characteristic dimensions of characteristic parts of the human body:
[0096] like Fig. 9As shown, the characteristic dimensions of the human body can be divided into two categories based on their geometric characteristics. One category is closed dimensions, which represent closed curves, such as waist circumference; the other category is non-closed dimensions, such as shoulder width.
[0097] Extract the characteristic dimension of closure, mainly determine the height or width position of the human body where the characteristic dimension is located through the geometric characteristics of the characteristic dimension, and intersect the cross section at the height or width position with the human body model to obtain the intersection line of the human body model surface and the cross section;
[0098] Specifically, on the human body model, since the surface of the human body model is composed of triangles connected by point clouds, partially connected straight line segments can be obtained;
[0099] By fitting the points of these straight line segments (or the intersection lines on the cross section) with B-spline curves (connecting the discrete points smoothly by mathematical means), a smooth cross section of the characteristic part of the human body can be obtained. At this time, the calculated length of the curve is the size measurement information of this part.
[0100] To extract non-closed characteristic dimensions, the position of the dimension feature points is mainly determined through the geometric features of the characteristic dimensions. The human body model is intercepted using the characteristic points to obtain the human body cross-section. The path points on the cross-section are searched, the distances between these path points are calculated, and data fitting is performed to complete the extraction of the characteristic dimensions and obtain the dimension measurement information of this part. The characteristic dimensions of these characteristic parts can be collected and used as input for calculating the human body volume value.
[0101] S4. A linear regression model is obtained by training according to the obtained size values of the designated parts, and is used to calculate the volume of the human body.
[0102] The characteristic parts of the human body model are measured and the corresponding human body dimensions are calculated. The deep neural network is trained through the implicit relationship between the characteristic dimension values and the human body volume. The human body volume can be calculated based on the characteristic dimension values through the network.
[0103] Specifically, after obtaining the human body size information, the human body volume data obtained previously and the human body feature size obtained from the corresponding human body image are used for training to obtain the neural network model.
[0104] At this time, the dimensions of multiple specified parts of the human body are used as input, and the calculated value of the human body volume is used as output:
[0105] First, a multi-layer neural network is initialized, the weights between nodes are randomly initialized, and the cross entropy function between the calculated volume value and the human body volume of the training data is defined as the loss function;
[0106] Obtain sample data externally, and use the size values of multiple specified parts of the human body of the sample data as input. The volume values in the training set are used as true values to train the neural network. In each round of training, the gradient descent algorithm is used to modify the weight values between nodes during the back propagation process to iteratively update the weights.
[0107] The loss function is recorded in each round of training, and the minimum value R of the loss function is preset. When the loss function is less than R, it means that the difference between the human body volume value calculated on the training set and the actual human body volume value is relatively small. At this time, the training is stopped, and the weight and other information of this network are saved as the network for calculating body composition, which can be used as a neural network model.
[0108] Finally, this neural network model can be used to obtain the calculated human body volume value.
[0109] The calculated human body volume data is compared with the actual human body volume value measured by the body density method. The average difference between the calculated results is 2.03dm 3 , the mean relative deviation is 3.05%.
[0110] Therefore, compared with traditional methods, this measurement method has little accuracy deviation in calculating 3D human body volume values from 2D images, and can measure human body volume more accurately.
[0111] Furthermore, in a specific embodiment, the test method for the user during the specific implementation process is as follows:
[0112] Combination Fig.10 and Fig.11 It can be seen that by using an end-to-end neural network architecture, a 3D mesh model can be generated from a single RGB image, and the 3D geometry can be calculated in a coarse-to-fine manner, which is reliable and easy to learn. After obtaining the human body model, the human body size and volume can be calculated.
[0113] Specifically, firstly, a 2D image of a human body under a single perspective is prepared, and information such as the height and width of the human body is input as a calculation basis. A single 2D perspective human body image is used as the input of a pixel2mesh network to perform multiple network structures under multiple deep learning to generate a 3D human body model. The 3D human body model is processed and optimized, and the feature points of the 3D model are read to obtain the feature size of the feature part, which is the corresponding human body size. According to a neural network model between the human body size and the human body volume value obtained by previous training in the training data, the human body volume is calculated through this model.
[0114] At this time, since the ellipsoid model is generated by the acquired 2D human body image and the computer built-in system, the human body 3D model corresponding to the image can be quickly obtained, and the size information and volume information of the human body 3D model can be obtained by measurement to further quickly obtain the size information and volume information of the actual human body, which is convenient and fast. Therefore, a personalized model can be generated in a short time using common, portable and accessible devices such as smart phones, thereby solving the problems of human body volume measurement, cumbersome collection process, complex scene requirements, long time, and the need for specific collection equipment. There are no requirements for the surrounding environment. In addition, the cost of the equipment that can be used for collection has also been reduced. Only the simplest consumer-grade digital camera or even cheap, common, portable and accessible devices such as smart phones can be used to generate results in a short time. No expensive 3D scanning equipment is required. Consumer-grade digital cameras can be used to visualize the body and accurately estimate BV of women and men without obesity (BMI<30kg / m2), which is convenient for obesity management and monitoring in home or clinic environments.
[0115] S5. After obtaining the human body volume, according to the two-compartment model of the human body, through the density difference between the human body fat density and the lean body mass, the proportional relationship between the body fat rate and weight of the person being tested can be calculated through 3D scanning and the whole body density obtained after certain processing, and then the proportion of human body fat composition can be predicted.
[0116] After obtaining the human body volume, the human body can be divided into two parts: body fat and lean body mass according to the principle of the two-compartment model of the human body;
[0117] It should be noted at this point that the density of fat and lean body mass are relatively constant. Early research on body composition has shown that under normal physiological conditions, the fat-free body mass in the human body is mainly composed of bones, muscles and organs. The density of these tissues is relatively stable. Therefore, the density of adipose tissue is considered to be an approximately fixed constant, generally set to 1.1 grams / cubic centimeter, while the approximate value of human fat density is usually about 0.9 grams / cubic centimeter.
[0118] Combined with the above steps S1-S4, the human body volume can be easily obtained through the 3D scanned human body model. After deducting the residual gas volume and the gastrointestinal gas volume from the human body volume, the human body density can be obtained by dividing the human body weight by the obtained volume. The proportional relationship between the body fat rate and weight of the person being tested is calculated through the density, and the proportion of human body fat composition is predicted.
[0119] In summary, the predicted human body volume data is compared with the actual volume value of the human body measured by the body density method. The average difference between the predicted results is 2.03dm3, and the average relative deviation is 3.05%. The following table shows the specific situation of some actual values and calculated values on the test set.
[0120] <![CDATA[True volume / dm 3 > <![CDATA[Calculated volume / dm 3 > <![CDATA[Absolute error / dm 3 > Relative error User 1 65.71 67.21 1.50 2.28% User 2 73.84 75.97 2.13 2.80% User 3 85.55 88.17 2.62 3.07% User 4 60.18 62.07 1.89 3.14%
[0121] After the above processing, the volume information of the 3D reconstructed human body model can be obtained. At this time, the user is also required to provide accurate human body weight information.
[0122] refer to Fig.12 At this time, the volume needs to be further processed, and the gas volume inside the human body must be excluded to obtain the volume occupied by the main tissues of the human body.
[0123] TGV is the volume of gas in the human chest cavity:
[0124] TGV = FRC + 0.5 * VT;
[0125] Men: FRC = 0.0472*height+0.0090*age-5.290; VT = 1.2;
[0126] Women: FRC=0.0360*height+0.0031*age-3.182; VT=0.7.
[0127] The accurate human body volume is obtained by subtracting TGV from the volume of the 3D reconstructed model. The whole body density can be obtained by dividing the input weight by this volume. The Brozek formula or Siri formula is used to calculate body composition information such as body fat.
[0128] Brozek formula is expressed as: %BF = (4.57 / total body density – 4.142) * 100%;
[0129] The Siri formula is expressed as: %BF = (4.95 / total body density - 4.50) * 100%.
[0130] By verifying the results of the population in the data set, the determination coefficient of the human body fat information predicted by this model is R 2 =0.94, and the root mean square error (RMSE) of body fat prediction is 2.91 kg.
[0131] On the other hand, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0132] On the other hand, the present invention further discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0133] In another embodiment provided by the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the image-based body composition modeling and analysis methods in the above-mentioned embodiments.
[0134] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above method.
[0135] The embodiment of the present application also provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus.
[0136] Memory, used to store computer programs;
[0137] The processor is used to implement the above-mentioned image-based body component modeling and analysis method when executing the program stored in the memory.
[0138] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.
[0139] The communication interface is used for communication between the above electronic device and other devices.
[0140] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0141] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.
[0142] It should also be noted that electronic devices also include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablet computers (Pad), computers with wireless transceiver functions, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the terminal devices.
[0143] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.
[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
[0145] In addition, it should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.
[0146] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or schemes that A and B meet at the same time. In addition, in the embodiments of the present invention, "multiple" refers to more than two. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
Claims
1. A body composition modeling and analysis method based on an image, characterized in that: The specific steps include: S1. Input human 2D image and human size information; S2. The input image and human body information are converted into mesh information through a pixel 2mesh model based on deep learning, and a 3D human body model is further obtained; S3. Optimize the generated 3D human body model and measure the size of the specified part of the human body; S4. training a linear regression model based on the obtained size values of the specified parts to calculate the volume of the human body; S5. After obtaining the human body volume, the human body density is obtained according to the principle of the two-compartment model of the human body. The proportional relationship between the body fat rate and weight of the person being measured is calculated through the density, and the proportion of human body fat composition is predicted.
2. The image-based body composition modeling and analysis method according to claim 1, characterized in that: The specific operation steps of generating mesh information and obtaining a 3D human body model in step S2 include: S21. Use a fully convolutional neural network to extract features of the input human body 2D image, and obtain a feature map of the human body 2D image by inputting the human body 2D image through multiple convolution pooling operations; S22. Generate an ellipsoid mesh and extract its three-dimensional model features, and continuously deform the three-dimensional model to stretch and deform the ellipsoid mesh into a desired 3D model; S23. Set a permanent feature pooling layer to associate the 2D image information and the 3D model information with each other, adjust the node state of the graph convolutional network in the 3D model through the feature size information, and finally output the 3D human body model information.
3. The image-based body composition modeling and analysis method according to claim 2, characterized in that: The specific operation method of extracting the features of the three-dimensional model and continuously deforming the three-dimensional model in step S22 includes: Input a standard two-dimensional human body image and use an implicit surface algorithm to generate an initial ellipsoid model with an ellipsoid mesh; Uniformly initialize and select vertices on the ellipsoid model, set the initial shape feature as the three-dimensional coordinates of each vertex, and fuse the vertex position information and 2D image information; The feature map is obtained based on the 2D human body image, and a new feature map is obtained after multiple convolutions through the VGG-16 network. For any vertex in the model, the three-dimensional coordinate C i-1 , calculate the specific position of its two-dimensional projection on the input 2D human body image plane T; Use bilinear interpolation to concentrate the features of the four pixels in the adjacent position of the feature map obtained by convolution of the original feature map in the VGG-16 network, and let the feature information of the vertex be the perceptual feature information P of the vertex; The perceptual feature P is combined with the 3D shape feature F attached to the vertices of the input graph. i-1 The comprehensive features of the vertices are obtained by cascading and input into the graph residual neural network for calculation to generate new coordinates C for each vertex i and 3D shape feature F i As the output of the mesh deformation module.
4. The image-based body composition modeling and analysis method according to claim 3, characterized in that: In step S22, when uniformly initializing the selected vertices, an anti-pooling layer is established to increase the number of vertices of the model. The anti-pooling layer is used to increase the number of vertices of the model by connecting the midpoints of each edge of the triangle formed by the vertices in the model and dividing the original triangle into four. After multiple operations, it is used to output the vertex coordinates of a more refined 3D model. A multi-vertex set can obtain a 3D human body point cloud model with three-dimensional scattered point cloud data Z.
5. The image-based body composition modeling and analysis method according to claim 4, characterized in that: The specific operation steps of optimizing the generated 3D human body model in step S3 include: Predefine a threshold M and quantity value K. If the distance between a point and other points exceeds the threshold M, the point is defined as a noise point of the model. Read the three-dimensional scattered point cloud data Z in the 3D human body point cloud model, for each point P in P i Calculate the distances between the K nearest neighboring points to the point and calculate P i The average distance M(P i ); The average value of the distance M(P i ) is compared with the preset threshold M. If M(P i )≤M, then keep the point P i , continue to calculate the next point P, if M(P i )>M, then delete the point; After all points have been searched and calculated, a 3D human body point cloud model with a smoother surface can be obtained.
6. The image-based body composition modeling and analysis method according to claim 5, characterized in that: In the step S3, the size of the designated part of the human body is set as a closed characteristic size, and the specific operation steps for measuring the size of the designated part of the human body include: Partially connected straight line segments can be obtained from the triangles located on the surface of the human body model and connected by the point cloud. By fitting the points of these straight line segments with B-spline curves, a smooth cross-section of the characteristic parts of the human body can be obtained. At this time, the length of the calculated curve is the size measurement information of this part.
7. The image-based body composition modeling and analysis method according to claim 5, characterized in that: In the step S3, the size of the designated part of the human body is set as a non-closed characteristic size, and the specific operation steps of measuring the size of the designated part of the human body include: The position of the dimension feature point is determined by the geometric features of the characteristic dimension, and the human body model is intercepted using the feature points to obtain the human body cross-section. The path points on the cross-section are searched, the distances between these path points are calculated, and data fitting is performed to obtain the dimension measurement information of this part.
8. The image-based body composition modeling and analysis method according to claim 1, characterized in that: The specific operation steps of training the linear regression model according to the obtained size values of the designated parts in step S4 include: S41. Initialize a multi-layer neural network, randomly initialize the weights between nodes, and define a cross entropy function between the calculated volume value and the human body volume of the training data as a loss function; S42. Obtain sample data externally, and use the size values of multiple specified parts of the human body of the sample data as input, and the volume values in the training set as true values to train the neural network, and use the gradient descent algorithm to modify the weight values between nodes during the back propagation process in each round of training to iteratively update the weights; S43. The loss function is recorded in each round of training, and the minimum value R of the loss function is preset. When the loss function is less than R, the training is stopped. The weight information of this network is saved as the network for calculating body composition, which can be used as a neural network model for calculating human body volume.
Citation Information
Patent Citations
A method and system for measuring a body based on a 2D human body image
CN109559373A
Method and device for measuring body composition through posture density method
CN114271796A
3D model rapid reconstruction method based on 2D image
CN118015192A
Body fat measurements from a two-dimensional image
US11903730B1
Method for constructing three-dimensional human body model, and electronic device
US20230073340A1
Cited By
Method for body fat detection
TWI939330B
