Metabolic disease prediction method and device based on 2D image, and medium

Abstract

The invention discloses a metabolic disease prediction method based on a 2D image, and the method carries out the following steps through computer equipment: S1, obtaining a human body image, and separating the human body image from a background image; s2, acquiring key point information of a human body in the human body image; s3, fitting a human body model through the key point information; s4, noise is removed from the human body model obtained through fitting, and surface reconstruction is carried out; s5, training the human body model, and analyzing body components through the human body model; s6, judging the risk coefficient of the metabolic disease of the user according to the identity component data; in the step S1, the human body image is separated from the background image through model training or a neural network. According to the method, the 3D human body model is generated by fitting the 2D human body images obtained by separation, and the risk coefficient of the metabolic disease of the user can be judged by training the human body model and analyzing the human body components, so that the metabolic disease is quickly predicted, the working efficiency is improved, and the labor cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of bioinformatics, specifically to a method for predicting metabolic diseases based on 2D images. Background Technology

[0002] Metabolic diseases refer to a series of diseases caused by metabolic abnormalities. Common metabolic abnormalities include abnormal glucose metabolism, abnormal lipid metabolism, and abnormal calcium and phosphorus metabolism. Early prevention and testing of body composition are essential for preventing metabolic diseases.

[0003] Traditional methods for predicting the risk of a disease in a person require determining changes in the body's composition as indicated by the disease. This is typically done through 3D scanning. However, the process of collecting data from a three-dimensional human body model using 3D scanning equipment is cumbersome. The body must remain still during the scanning process, and even slight movement can cause significant data deviations, which is very inconvenient. The detection process during screening is also cumbersome and time-consuming, resulting in low work efficiency and increased workload for staff.

[0004] Therefore, this application proposes a method for predicting metabolic diseases based on 2D images to solve the above-mentioned technical problems. The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is the closest prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method for predicting metabolic diseases based on 2D images, so as to solve the technical problems mentioned in the background art.

[0006] The present invention solves the above-mentioned technical problems by adopting the following technical solutions:

[0007] A method for predicting metabolic diseases based on 2D images, comprising the following steps performed via computer equipment:

[0008] S1. Obtain the human body image and separate the human body image from the background image;

[0009] S2. Obtain key point information of the human body in the human body image;

[0010] S3. Fit a human body model using key point information;

[0011] S4. Remove noise from the fitted human body model and perform surface reconstruction;

[0012] S5. Train the human body model and analyze its body composition.

[0013] S6. Use identity component data to determine the risk factor of a user having metabolic diseases.

[0014] Preferably, in step S1, the human body image is a full-body photograph, and the human body image is separated from the background image by training a semantic segmentation model or extracting the human body contour through a neural network.

[0015] Preferably, the key point information in step S2 includes hand key point information and face key point information of the human image obtained by the human key point detection algorithm. The key point information uses the entire image as network input, and the specific operation steps include:

[0016] S21 predicts a two-dimensional confidence map of a set of body parts and a two-dimensional vector field of a set of body part affinity domains;

[0017] S22. Analyze the confidence map and part affinity domain through greedy reasoning, and output the two-dimensional key points of all figures in the image;

[0018] S23 extracts features through a feature extraction network, and inputs the obtained feature layer into the next network. The subsequent network is divided into two branches: the top branch predicts the confidence map and the bottom branch predicts the affinity field.

[0019] Preferably, the key point information in step S2 is subject to intermediate monitoring throughout the process of using the entire image as network input, in order to supplement the gradient. The specific execution steps of the intermediate monitoring are as follows:

[0020] The first step is for the network to receive the output of the feature extraction network and generate a set of detection confidence maps;

[0021] The second step is to connect the feature layer output by the feature extraction network to the input of the previous stage for refined prediction.

[0022] The third step involves using two loss functions to optimize the two-dimensional confidence map of the body parts and the two-dimensional vector field of the body part affinity domain.

[0023] Preferably, the specific steps for fitting the key point information to the human body model in step S3 include:

[0024] The detected keypoint data is used to fit a 3D human body model. The 3D human body model has a total of N corner points on its surface, controlled by K keypoints. The 3D human body model can be expressed mathematically as follows:

[0025] M(θ, β, ψ) = W(T) p (β, θ, ψ), J(β), θ, w);

[0026] in:

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] θ represents the action parameters. Each parameter represents the rotation relationship relative to the parent node, θ∈R. 3(K+1) θ can be divided into the following three categories: θ f It is the key point of the face, θ h It is the joint of the finger, θ b These are key points on the limbs. The β parameter in the formula represents the human body shape, and the ψ parameter represents facial expression. By inputting these three parameters, the final output is the coordinates of each vertex on the surface of the human model, i.e., M(θ, β, ψ) ∈ R. 3N Each vertex is a three-dimensional coordinate, i.e., 3N;

[0033] T p B represents the human body model, based on the average model (T), after adjustments for body shape, movement, and facial expression parameters. s (β;S), B e (ψ;ε) and B p (θ; P) represent the correction data calculated on the average model for body shape, action, and expression parameters, respectively; where S, e, and P are the orthogonal components of the vertex displacements on the model surface, respectively.

[0034] M(θ, β, ψ) = W(T) p (β, θ, ψ), J(β), θ, w) represents rotating the corrected shape at key points and then applying deformation weight w to obtain a human model with motion.

[0035] Preferably, the human model with motion can undergo gender training before training in step S5. The specific steps for gender training include:

[0036] Step 1: Build a training gender detector. The gender detector is built using a convolutional neural network. It uses the model's gender features to determine the gender. It reads data from the data directory and creates two folders to store female images and male images respectively.

[0037] Step 2: Shuffle the data and create a training set and a test set with a ratio of 8:2. Perform parameter training on the training set for at least 3000 rounds, using randomized training data each time. The auxiliary functions involved must be weighted, including biased initialization and fully connected matrix initialization. During training, the feature map is divided into several sub-regions, and statistical summaries are performed on each sub-region.

[0038] Step 3: Start the session to begin training, input the collected images, and verify the final judgment result using a test set. Iterate at least 3000 times. The model is divided into a male model, a female model, and a neutral model. The neutral model is called when the gender cannot be accurately determined.

[0039] Preferably, the specific steps for training the human body model and analyzing body composition through the human body model in step S5 include:

[0040] S51. Training set: The human model corresponds to a column vector with a dimension of 3 * number of point clouds, denoted as t. The column vectors of N training models are concatenated to form a t*N matrix.

[0041] S52. Center the column vectors with zero mean to obtain an m*N sample matrix M, and calculate the sample covariance matrix M*M. T Perform matrix decomposition on the covariance matrix to extract the eigenvectors {w1, w2, ..., w3} corresponding to the d largest eigenvalues. d The feature vectors are standardized to form a feature vector matrix W, and the original sample column vectors are combined with W. T Multiplying them yields a d-dimensional vector, which is the dimensionality-reduced human body parameter shape data;

[0042] S53. The human body composition data of the training set human body model are concatenated into a d*N matrix. The matrix concatenated from the human body parameter shape data of N users is used as input and the composition matrix is ​​used as output for learning to predict the composition value of the human body.

[0043] Preferably, the specific steps in step S6 for determining the risk factor of a user having a metabolic disease using identity component data include:

[0044] S61. Principal component analysis is applied to the human body model to reduce dimensionality and orthogonalize the human body model mesh data. The resulting principal components are used as predictors of body composition, and the principal components are mapped as parameters of human body shape to body composition statistics.

[0045] S62. The generated principal components are used as predictors of body composition. The principal components are mapped to body composition statistics as parameters of human body shape. The user's measurement results are used as a sample set through body composition statistics. The results of whether the sample set has metabolic diseases are used as the output value of the model. The result of having diseases is 1, and the result of not having diseases is 0.

[0046] S63. Train a linear regression neural network model, which outputs a prediction result of 0-1. The closer the prediction result is to 1, the greater the risk coefficient of having a metabolic disease; the closer the value is to 0, the smaller the risk coefficient of having the disease.

[0047] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0048] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0049] As can be seen from the above technical solution, the present invention provides a method for predicting metabolic diseases based on 2D images. Compared with the prior art, the present invention has the following advantages:

[0050] 1. This invention uses 2D human images obtained separately to fit and generate 3D human models. By training the human models and analyzing human body composition, it is possible to determine the risk factor of users having metabolic diseases, thereby quickly predicting metabolic diseases, improving work efficiency, and reducing labor costs.

[0051] 2. This invention can acquire human images using ordinary imaging devices, such as cameras, mobile phones, and smartwatches, and use these images to predict metabolic diseases, increasing the portability of image acquisition and reducing equipment costs.

[0052] 3. This invention obtains key information about the hands and face in a human image, uses the entire image as network input, and fits a three-dimensional human model. It can obtain a high-precision 3D human model from only a 2D human image, and has a high level of user experience and intelligence. This can improve the accuracy of identity component data analysis and further improve the accuracy of predicting metabolic diseases.

[0053] 4. By performing intermediate monitoring throughout the entire process of using the human image as network input, this invention can supplement gradients, solve the problem of gradient vanishing during the process, and ensure that the results develop according to the preset path.

[0054] 5. This invention, by prioritizing gender training of the constructed human body model before training and analyzing body composition using the human body model, can help improve the accuracy of identity component data analysis and further improve the accuracy of predicting metabolic diseases.

[0055] 6. This invention trains the human body model iteratively multiple times, using a matrix of the user's human body shape data as input and a composition matrix as output for learning. It can predict the composition values ​​of the human body with good accuracy, which facilitates further assessment of the user's risk factor for metabolic diseases.

[0056] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description

[0057] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0058] Figure 1 This is a flowchart of the metabolic disease prediction method of the present invention;

[0059] Figure 2 This is a flowchart illustrating the overall operation of the method in this invention that uses key point information as network input.

[0060] Figure 3 This is a flowchart illustrating the specific execution steps of the intermediate monitoring in this invention;

[0061] Figure 4 This is a schematic diagram of the overall process of training the gender model of the present invention;

[0062] Figure 5 This is a flowchart illustrating the specific method for training and analyzing human body models to obtain body composition in this invention.

[0063] Figure 6 This is a schematic diagram of the overall process for determining the risk factor of a user having a metabolic disease in this invention;

[0064] Figure 7 Examples of 15 isosurface cases according to the present invention;

[0065] Figure 8 This is a graph showing the test and evaluation results of the metabolic disease model sample of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] For details in the embodiments, please refer to Figures 1 to 8 .

[0068] like Figure 1 As shown. The metabolic disease prediction method based on 2D images proposed in this embodiment of the invention performs the following steps via a computer device:

[0069] S1. Obtain the human body image and separate the human body image from the background image.

[0070] In one specific embodiment, human images can be acquired by taking pictures with ordinary imaging devices. When taking pictures, try to choose a position with good lighting and aim at the shooting angle so that the pictures are clearer. Imaging devices used can include mobile phones and digital cameras. Tablets, smartwatches and 3D imaging devices can also be used to take pictures of the human body or through video. The user image is obtained by taking screenshots from the video. When taking pictures of the user, priority is given to taking full-body photos of the user. Metabolic diseases can be predicted by using the images taken by these devices, which increases the portability of image acquisition and reduces equipment costs.

[0071] It should be noted that when taking a full-body photo of a user, multiple angles can be taken of the same user to capture various details of the body parts;

[0072] After the user's photo is taken, the human body contour is extracted by training a semantic segmentation model or a neural network to separate the human body image from the background image.

[0073] In one specific embodiment, the human body image is obtained through semantic segmentation. The semantic segmentation model is trained by assigning each pixel in the image to a different semantic category, thus classifying the human body region and the background region into different categories. The model can classify each pixel to determine whether the pixel belongs to the human body background. The background part is filled with white. The end-to-end open-source image classification model is trained using the human body image data portion of the image detection dataset. The trained open-source image classification model can completely separate the human body from the background in the image data.

[0074] In one specific embodiment, the human body image is extracted through a neural network. The neural network extracts the human body contour in three parts: the first part is to segment and extract the parts of the human body image; the second part is to use a multi-level edge detection algorithm, such as the Canny (a human body contour algorithm) operator, to perform a coarse calculation of the human body contour; the third part is to use morphology and algorithms to modify the contour image, remove noise, and connect discontinuous edges.

[0075] The first part involves directly extracting human body contours from simple background target images using digital images. This primarily involves various image processing techniques to improve visual effects, highlight useful information, and lay the foundation for automatic recognition. Alternatively, encoding can be used to reduce the required storage space, transmission time, or bandwidth. When extracting complex background images, segmentation algorithms, such as Mask-RCNN (a segmentation algorithm), can be used for pixel-level segmentation. This requires at least 1000 training data points, and the images need to be high-resolution (at least 1024*1024) and properly labeled. The third part uses a flooding algorithm to fill the holes generated during edge detection.

[0076] S2. Obtain key point information of the human body in the human body image.

[0077] In one specific embodiment, reference is made to Figure 2 The key point information of the human body in the image is obtained through a human key point detection algorithm. The key point information includes hand key point information and facial key point information. The entire image is used as network input, and the overall process is as follows:

[0078] S21. Predict a two-dimensional confidence map of a set of body parts and a two-dimensional vector field of a set of body part affinity domains;

[0079] S22. Analyze the confidence map and part affinity domain through greedy reasoning, and output the two-dimensional key points of all figures in the image;

[0080] S23. Feature extraction is performed through a feature extraction network, and the obtained feature layer is input into the next network. The subsequent network is divided into two branches: the top branch predicts the confidence map and the bottom branch predicts the affinity field.

[0081] Furthermore, in one specific embodiment, reference is made to... Figure 3 The key point information is monitored during the process of using the entire image as network input. This monitoring is used to supplement gradients, solve the gradient vanishing problem, and ensure that the result develops according to the preset path. The specific execution steps of the intermediate monitoring are as follows:

[0082] The first step is for the network to receive the output of the feature extraction network and generate a set of detection confidence maps;

[0083] The second step is to connect the feature layer output by the feature extraction network to the input of the previous stage to refine the prediction.

[0084] The third step involves using two loss functions to optimize the two-dimensional confidence map of the body parts and the two-dimensional vector field of the body part affinity domain.

[0085] S3. Fit a human body model using key point information.

[0086] In one specific embodiment, the detected keypoint data is used to fit a three-dimensional human body model, wherein the surface of the three-dimensional human body model has a total of N corner points, controlled by K keypoints, and expressed by a mathematical formula, the three-dimensional human body model can be expressed as:

[0087] M(θ, β, ψ) = W(T) p (β, θ, ψ), J(β), θ, w);

[0088] Where θ represents the action parameter, and each parameter represents the rotation relationship relative to the parent node, θ∈R 3(K+1) K+1 is because, in addition to the K key points mentioned above, there is another point used to control the rotation of the entire body. θ can be divided into the following three categories: θ f It is the key point of the face, θ h It is the joint of the finger, θ b These are key points on the limbs. The three parameters in the formula represent the human body shape, and the ψ parameter represents facial expression. By inputting these three parameters, the final output is the coordinates of each vertex on the surface of the human model, i.e., M(θ, β, ψ)∈R. 3N Each vertex is a three-dimensional coordinate, i.e., 3N;

[0089] The meaning of each part in W:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] T p B represents the human body model, based on the average model (T), after adjustments for body shape, movement, and facial expression parameters. s (β;S), Be (ψ;ε) and B p (θ; P) represent the correction data calculated on the average model for body shape, action, and expression parameters, respectively; where S, e, and P are the orthogonal components of the vertex displacements on the model surface, respectively.

[0096] M(θ, β, ψ) = W(T) p (β, θ, ψ), J(β), θ, w);

[0097] This means rotating the corrected shape at key points and then applying deformation weights w to obtain a human model with animation.

[0098] At this point, it can be further explained that, to improve the user experience and intelligence of the model fitting, the human body model with motion can also be used for gender training of the constructed model, as shown in the reference. Figure 4 The specific steps for conducting gender training include:

[0099] Step 1: Build a training gender detector. The gender detector is constructed using a convolutional neural network. It uses the model's gender features to determine the gender. It reads data from the data directory and creates two folders to store female images and male images respectively.

[0100] Step 2: Shuffle the data and create training and test sets with a ratio of 8:2. Train the parameters on the training set for at least 3000 epochs, using randomized training data each time. The auxiliary functions involved must be weighted, including biased initialization and fully connected matrix initialization.

[0101] At this point, training the data includes: the first layer of convolution and pooling, the second layer of convolution and pooling, the third layer of convolution and pooling, the first fully connected layer, the second fully connected layer, and the fully connected output layer;

[0102] Pooling involves dividing the feature map into several sub-regions and summarizing the statistics for each sub-region.

[0103] In addition, to prevent the network from fitting, regularization is applied to the fully connected hidden layers; finally, the accuracy is calculated.

[0104] Step 3: Start the session to begin training, input the collected images, and verify the final judgment result on the test set. After at least 3000 iterations, the accuracy of the model can be guaranteed to reach 95%. The model is divided into a male model, a female model, and a neutral model. The neutral model is called when the gender cannot be accurately determined.

[0105] S4. Remove noise from the fitted human body model and perform surface reconstruction.

[0106] In one specific embodiment, the specific steps for eliminating high-frequency noise are as follows:

[0107] S41. Prioritize the use of Gaussian filters to eliminate high-frequency noise in the model, and select a fixed-size neighborhood (a two- or three-dimensional cube window) as the local filtering region;

[0108] S42. For each vertex in the neighborhood, calculate a weight that is determined by a Gaussian function based on distance. Note that vertices closer to the center vertex usually receive a higher weight, while vertices farther from the center vertex receive a lower weight.

[0109] S43. Use these weights to perform a weighted average of the vertex coordinates in the neighborhood to obtain smoothed vertex coordinates, which represent the sum of depth information in the neighborhood.

[0110] S44. Repeat steps S41 to S43 to move the filter window to each vertex of the grid data to generate the entire smoothed grid data or point cloud data.

[0111] In one specific embodiment, the specific operation steps for human body model surface reconstruction are as follows: a continuous and smooth human body surface model is reconstructed by a point cloud reconstruction algorithm. The point cloud reconstruction method obtains point cloud data by performing stereo matching on the obtained images or videos through an algorithm, including binocular stereo cameras, motion recovery structures, and machine learning.

[0112] Among them, the structure of the binocular stereo camera resembles that of a human eye. It uses two camera modules at a fixed distance to perceive parallax information and calculate point cloud data with depth information.

[0113] Furthermore, the point cloud reconstruction algorithm includes the following steps:

[0114] Step 1: Integrate the obtained human body model spatial point cloud data and color intensity data;

[0115] Step 2: Use modular, cross-platform, open-source programming technology for 3D point cloud processing to preprocess the point cloud data, including denoising, segmentation, filtering, registration, and sampling operations, to output point clouds with clear features and accurate data.

[0116] Step 3: Point cloud data meshing. The point cloud is approximated by meshing. Triangular meshes and quadrilateral meshes are used. This step is the transformation of the point cloud from a 3D representation to a mesh.

[0117] Step 4: Panoramic mapping, which maps the color and texture information collected by the imaging device onto the network model to output a realistic 3D model;

[0118] In the specific operation of point cloud reconstruction algorithms, for models with voids, rough surfaces, and uneven textures, point cloud reconstruction algorithms are used to reconstruct continuous and smooth human body surfaces. Specifically, for a large amount of point cloud data of a 3D model, for each point, the normal vector pointing inwards towards the model is estimated; for the set of neighboring points, their center positions are calculated to obtain the center point; the formula for calculating the deviation matrix for each point is as follows:

[0119] D=[[x1-x, y1-y, z1-z], [x2-x, y2-y, z2-z]...];

[0120] Each row represents the deviation of an adjacent point relative to the center point. The formula for calculating the covariance matrix is:

[0121]

[0122] For the covariance matrix, perform eigenvalue decomposition to find the main eigenvectors and eigenvalues. Select the eigenvector corresponding to the largest eigenvalue as the normal vector. Normalize this normal vector to obtain the unit normal vector of this point. Repeat the above operation for each point in the point cloud to obtain the normal vector of each point.

[0123] Define the spatial region of the model surface and interior as M, and define the model surface region as... A pointer function λM is predefined in space. The value inside the human body model is defined as greater than 0, and the value outside is less than 0. The part with a value of 0 is the isosurface.

[0124] First, the indicator function is convolved using a smoothing filter. Then, the gradient field of the smoothing function is calculated to obtain the gradient formula of the indicator function. F(q) represents a smoothing filter (generally a Gaussian filter is used for filtering), and Fp(q) = F(qp) is the translation of F along the p direction. Here, the smoothed λM□F (convolution makes the indicator function smoother) is used to approximate λM. Later, we will use... To represent the approximate function, Let p be the surface normal vector pointing inwards; according to Gaussian divergence theory, the gradient of the indicator function is equal to the vector field calculated using the surface normal field. The gradient formula for λM can be obtained using the normal field.

[0125]

[0126] To reduce computational cost, points representing the surrounding region are observed as P = (pi, ni). Consider a discrete point set Ω. The model is divided into non-overlapping regions □s, s∈Ω. The normal vector of the entire region can be replaced by a point within that region. The formula for expressing the 3D human body model is transformed into an integral summation, and each small integral is approximated as a constant function. The sum of the function values ​​corresponding to points sp is used to represent these points. Substitute the integral of the area:

[0127]

[0128] The relationship between the indicator function and the vector field can be obtained from the above:

[0129]

[0130] Differentiating both sides of the above equation, we obtain the Poisson equation:

[0131]

[0132] The indicator function can be found by iteratively solving the Poisson equation using the Laplace matrix. After solving, an approximate indicator function λM can be obtained. Then, the 3D isosurface extraction method (the values ​​of the indicator function on the model surface form an isosurface) can be used to extract the isosurface as the human body surface, i.e., the optimized 3D human body model.

[0133] In specific implementation, refer to Figure 7 The basic idea of ​​the 3D isosurface algorithm is as follows: Divide a 3D geometry into several small cubes (voxelization); then process the cubes (voxels) in the data field one by one, separate the cubes that intersect with the isosurface, and use interpolation to calculate the intersection points of the isosurface and the cube edges; according to the relative position of each vertex of the cube and the isosurface, connect the intersection points of the isosurface and the cube edges in a certain way to generate the isosurface, which serves as an approximate representation of the isosurface within the cube.

[0134] It should be noted that for each vertex of the small cube, which can be inside or outside the model, the value of the indicator function corresponding to each vertex has two possibilities (greater than or less than the value of the current isosurface). With 8 vertices, there are 256 possibilities. Considering the rotational symmetry of the cube, there will be 15 ways for the intersection points to exist inside the small cube. These 15 possibilities define the specific shapes for generating the isosurfaces. All cubes intersecting the indicator function are used to generate isosurfaces according to these 15 forms. Connecting all the generated isosurfaces yields the surface model (e.g., ...). Figure 7 (As shown).

[0135] In summary, many small cubes are used to divide the space. The plane inside the small cube that intersects with the point cloud model is used to approximate the current isosurface. The more small cubes there are, the better the approximation effect, but the computation time will increase. Therefore, an appropriate scale is selected for isosurface extraction. The finally extracted isosurface is the reconstructed 3D human body model.

[0136] Finally, it should be noted that gender training for the constructed model can also be performed after the human model has undergone noise removal and surface reconstruction.

[0137] S5. Train the human body model and analyze its body composition.

[0138] In one specific embodiment, reference is made to Figure 5 The specific steps for training a human body model and analyzing its composition include:

[0139] S51. Training set: The human model corresponds to a column vector with a dimension of 3 * number of point clouds, denoted as t. The column vectors of N training models are concatenated to form a t*N matrix.

[0140] S52. Center the column vectors with zero mean to obtain an m*N sample matrix M, and calculate the sample covariance matrix M*M. T Perform matrix decomposition on the covariance matrix to extract the eigenvectors {w1, w2, ..., w3} corresponding to the d largest eigenvalues. d The eigenvectors are standardized to form an eigenvector matrix W, and the original sample column vectors are then compared with W. T Multiplying them yields a d-dimensional vector, which is the dimensionality-reduced human body parameter shape data.

[0141] S53. The human body composition data of the training set human body model are concatenated into a d*N matrix. The matrix concatenated from the human body parameter shape data of N users is used as input and the composition matrix is ​​used as output for learning to predict the composition value of the human body.

[0142] Specifically, each column of M represents a zero-mean centered vector of a human body model, for an M*M... T After performing eigenvalue decomposition, the eigenvectors corresponding to the first d eigenvalues ​​are standardized to form the following eigenvector matrix W. F represents the component matrix, where each column of F represents a participant's body component information vector, and each item represents a specific value of a certain body component. The augmented matrix of W is defined as follows. And the linear relationship between body composition and W, F=M w->f W. M is solved by finding the inverse matrix. w->fThis allows us to learn the expressive relationship between W and F, that is, to learn predicted values ​​of body composition from the human body model, where:

[0143]

[0144]

[0145]

[0146] S6. Use identity component data to determine the risk factor of a user having metabolic diseases.

[0147] In one specific embodiment, for each user to find their corresponding column vector from the final component matrix, the values ​​in the column vector should be used as input parameters for the following neural network:

[0148] First, a multi-layer neural network is initialized, the weights between nodes are randomly initialized, and the cross-entropy function between the predicted result and the true value is defined as the loss function.

[0149] The neural network is trained using the component values ​​of the sample data as input and the values ​​for the presence or absence of metabolic diseases as the true values. In each training round, the gradient descent algorithm is used to modify the weight values ​​between nodes during backpropagation to iteratively update the weights. The loss function is recorded in each training round. When the loss function is small enough, training stops and the weights and other information of this network are saved as a network for predicting body composition. Finally, this model is used to obtain a normalized risk index.

[0150] In one specific embodiment, reference is made to Figure 6 The specific steps for determining a user's risk factor for metabolic diseases using body composition data obtained through training a 3D model include:

[0151] S61. Principal component analysis is applied to the 3D human body model obtained above to reduce the dimensionality and orthogonalize the human body model mesh data. The generated principal components are used as predictors of body composition. The principal components are the body composition statistics mapped to the parameters of human body shape.

[0152] S62. Using body composition statistics, the user's measurement results are used as a sample set. The result of whether the sample set has a metabolic disease is used as the output value of the model. The result of having a disease is 1, and the result of not having a disease is 0.

[0153] S63. Train a linear regression neural network model. The model will output a prediction result of 0-1. The closer the prediction result is to 1, the greater the risk of having a metabolic disease. The closer it is to 0, the lower the risk of having the disease.

[0154] It should be noted here that a predicted risk value greater than 0.75 is generally considered to indicate a high probability of having a metabolic disease, and further detailed examination is recommended. Combining these two parts, we obtain a predictive model for the risk of having metabolic diseases directly from a 3D human body model.

[0155] refer to Figure 8 The model predicts a test set. Using a test dataset of 100 people, the trained model was fed with the results: the model correctly classified 85 out of 90 people; it misclassified 5 people with metabolic diseases as not having them; and it misclassified 10 people without metabolic diseases as having them. The model's accuracy was calculated to be 85%. Further improvements in accuracy can be made by increasing the sample size. This model can quickly detect a user's metabolic status.

[0156] In one specific embodiment, waist circumference, waist-to-hip ratio, and other three-dimensional dimensions of the human body are stronger indicators of metabolic diseases and mortality risk than total body fat. Therefore, by using three-dimensional data and generating a human body model, and training it based on deep learning, the correlation between three-dimensional data and metabolic diseases can be obtained, and diseases can be predicted.

[0157] In summary, this method for predicting metabolic diseases can serve as an effective first step in cost-effective early screening for both systemic and localized obesity-related metabolic diseases. This early screening can also be conducted by non-professionals outside of medical clinics. By generating 3D human body models from 2D images, human body data can be collected without the use of fixed 3D equipment. By analyzing human body composition through the model, metabolic diseases can be predicted quickly, improving work efficiency and reducing labor costs. At the same time, the analysis of identity composition data can be improved during the establishment and analysis of the human body model, further enhancing the accuracy of the prediction results for metabolic diseases.

[0158] On the other hand, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0159] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0160] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the 2D image-based metabolic disease prediction methods described in the above embodiments.

[0161] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.

[0162] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0163] Memory, used to store computer programs;

[0164] The processor, when executing the program stored in memory, implements the above-described method for predicting metabolic diseases based on 2D images.

[0165] The communication bus mentioned in the aforementioned electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0166] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0167] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0168] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0169] It should also be noted that electronic devices include terminal devices, which can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Terminal devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the terminal devices.

[0170] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0171] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0172] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, in the embodiments of this invention, "multiple" refers to two or more. Moreover, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

Claims

1. A method for predicting metabolic diseases based on 2D images, characterized in that, The specific steps include the following: S1. Obtain the human body image and separate the human body image from the background image; S2. Obtain key point information of the human body in the human body image; S3. Fit a human body model using key point information; S4. Remove noise from the fitted human body model and perform surface reconstruction; S5. Train the human body model and analyze its body composition. S6. Use identity component data to determine the risk factor of a user having metabolic diseases.

2. The method for predicting metabolic diseases based on 2D images as described in claim 1, characterized in that, In step S1, the human body image is a full-body photograph, and the human body image is separated from the background image by training a semantic segmentation model or extracting the human body contour through a neural network.

3. The method for predicting metabolic diseases based on 2D images as described in claim 1, characterized in that, The key point information in step S2 includes hand key point information and face key point information of the human image obtained by the human key point detection algorithm. The key point information uses the entire image as network input, and the specific operation steps include: S21 predicts a two-dimensional confidence map of a set of body parts and a two-dimensional vector field of a set of body part affinity domains; S22. Analyze the confidence map and part affinity domain through greedy reasoning, and output the two-dimensional key points of all figures in the image; S23 extracts features through a feature extraction network, and inputs the obtained feature layer into the next network. The subsequent network is divided into two branches: the top branch predicts the confidence map and the bottom branch predicts the affinity field.

4. The method for predicting metabolic diseases based on 2D images as described in claim 3, characterized in that, The key point information in step S2 is used for intermediate monitoring throughout the process of using the entire image as network input, in order to supplement the gradient. The specific execution steps of intermediate monitoring are as follows: The first step is for the network to receive the output of the feature extraction network and generate a set of detection confidence maps; The second step is to connect the feature layer output by the feature extraction network to the input of the previous stage for refined prediction. The third step involves using two loss functions to optimize the two-dimensional confidence map of the body parts and the two-dimensional vector field of the body part affinity domain.

5. The method for predicting metabolic diseases based on 2D images as described in claim 1, characterized in that, The specific steps for fitting the human body model with key point information in step S3 include: The detected keypoint data is used to fit a 3D human body model. The 3D human body model has a total of N corner points on its surface, controlled by K keypoints. The 3D human body model can be expressed mathematically as follows: M(θ, β, ψ)=W(T p (β,θ,ψ),J(β),θ,w); in: θ represents the action parameters. Each parameter represents the rotation relationship relative to the parent node, θ∈R. 3(K+1) θ can be divided into the following three categories: θ f It is the key point of the face, θ h It is the joint of the finger, θ b These are key points on the limbs. The β parameter in the formula represents the human body shape, and the ψ parameter represents facial expression. By inputting these three parameters, the final output is the coordinates of each vertex on the surface of the human model, i.e., M(θ, β, ψ) ∈ R. 3N Each vertex is a three-dimensional coordinate, i.e., 3N; T p B represents the human body model, based on the average model (T), after adjustments for body shape, movement, and facial expression parameters. s (β;S), B e (ψ;ε) and B p (θ; P) represent the correction data calculated on the average model for body shape, action, and expression parameters, respectively; where S, e, and P are the orthogonal components of the vertex displacements on the model surface, respectively. M(θ, β, ψ) = W(T) p (β, θ, ψ), J(β), θ, w) represents rotating the corrected shape at key points and then applying deformation weight w to obtain a human model with motion.

6. The method for predicting metabolic diseases based on 2D images as described in claim 5, characterized in that, The animated human model can undergo gender training before training in step S5. The specific steps for gender training include: Step 1: Build a training gender detector. The gender detector is built using a convolutional neural network. It uses the model's gender features to determine the gender. It reads data from the data directory and creates two folders to store female images and male images respectively. Step 2: Shuffle the data and create a training set and a test set with a ratio of 8:

2. Perform parameter training on the training set for at least 3000 rounds, using randomized training data each time. The auxiliary functions involved must be weighted, including biased initialization and fully connected matrix initialization. During training, the feature map is divided into several sub-regions, and statistical summaries are performed on each sub-region. Step 3: Start the session to begin training, input the collected images, and verify the final judgment result using a test set. Iterate at least 3000 times. The model is divided into a male model, a female model, and a neutral model. The neutral model is called when the gender cannot be accurately determined.

7. The method for predicting metabolic diseases based on 2D images as described in claim 1, characterized in that, The specific steps for training the human body model and analyzing body composition using the human body model in step S5 include: S51. Training set: The human model corresponds to a column vector with a dimension of 3 * number of point clouds, denoted as t. The column vectors of N training models are concatenated to form a t*N matrix. S52. Center the column vectors with zero mean to obtain an m*N sample matrix M, and calculate the sample covariance matrix M*M. T， Perform matrix decomposition on the covariance matrix to extract the eigenvectors {w1, w2, ..., w} corresponding to the d largest eigenvalues. d The feature vectors are standardized to form a feature vector matrix W, and the original sample column vectors are combined with W. T Multiplying them yields a d-dimensional vector, which is the dimensionality-reduced human body parameter shape data; S53. The human body composition data of the training set human body model are concatenated into a d*N matrix. The matrix concatenated from the human body parameter shape data of N users is used as input and the composition matrix is ​​used as output for learning to predict the composition value of the human body.

8. The method for predicting metabolic diseases based on 2D images as described in claim 1, characterized in that, The specific steps in step S6 for determining the risk factor of a user having a metabolic disease using identity component data include: S61. Principal component analysis is applied to the human body model to reduce dimensionality and orthogonalize the human body model mesh data. The resulting principal components are used as predictors of body composition, and the principal components are mapped as parameters of human body shape to body composition statistics. S62. Using body composition statistics, the user's measurement results are used as a sample set. The result of whether the sample set has a metabolic disease is used as the output value of the model. The result of having a disease is 1, and the result of not having a disease is 0. S63. Train a linear regression neural network model, which outputs a prediction result of 0-1. The closer the prediction result is to 1, the greater the risk coefficient of having a metabolic disease; the closer the value is to 0, the smaller the risk coefficient of having the disease.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 8.

Images