A three-dimensional foot shape generation method based on conditional diffusion model

Abstract

The present invention discloses a three-dimensional foot shape generation method based on a conditional diffusion model. The method comprises the following steps: on the basis of constructing a coarse-grained foot shape point cloud, encoding the coarse-grained foot shape point cloud using a point cloud encoder to obtain shape features; using the shape features as guiding conditions, adopting a diffusion model to perform reverse diffusion based on the guiding conditions to generate a fine-grained foot shape point cloud; and reconstructing a smooth geometric surface based on the fine-grained foot shape point cloud. In this way, the three-dimensional foot shape can be reconstructed simply and quickly.

Description

Technical Field

[0001] The invention belongs to the technical field of three-dimensional reconstruction, and in particular relates to a three-dimensional foot shape generation method based on a conditional diffusion model. Background Art

[0002] The shoemaking industry is a traditional manufacturing industry. To improve production efficiency, industrial shoemaking methods no longer measure specific foot parameters during the production process. Instead, they produce shoes of universal sizes based on standard shoe lasts. However, even for feet of the same size, there are still differences in parameters such as toe circumference and tarsal circumference. To a certain extent, universal shoe sizes can reduce the wearing experience for users with different foot shapes and even affect foot health.

[0003] In recent years, with the development of internet e-commerce, e-commerce platforms have attracted a large number of users with their convenient and efficient supply-demand matching methods, and more and more transactions have shifted from offline to online. The open and transparent price comparison mechanisms of online platforms have also intensified market competition in the traditional manufacturing industry. As manufacturing processes mature, the proportion of manufacturing itself in the added value of industrial products has gradually decreased, prompting a shift in traditional manufacturing towards service-oriented manufacturing. For example, in the footwear industry, customized shoes have become popular with consumers due to their more comfortable wearing experience. By offering customized shoe services, shoe manufacturers can attract new users and increase customer stickiness.

[0004] The traditional process of handmade shoe customization primarily involves three steps: foot shape measurement, shoe last conversion, and shoe body fabrication. The shoe body's fabrication primarily relies on the shoe last, which is derived from foot shape measurement parameters. Therefore, the accuracy of foot shape parameter measurements is crucial for the fit of customized shoes to the user's foot. In addition to basic foot length and width measurements, foot shape measurement also involves measuring key parameters such as the user's plantar circumference height and instep circumference height, which reflect the morphology of the human foot. Measuring these parameters relies heavily on specialized knowledge and often requires manual intervention, making automation and mass production difficult. Therefore, improving the automation level of the shoe customization process, while ensuring precision and reducing labor costs, is crucial for promoting the customization business and enhancing the service competitiveness of the footwear industry.

[0005] With the advancement of computer technology, 3D reconstruction technology has made automated foot shape measurement possible. Traditional 3D reconstruction techniques are typically based on geometric methods. Depending on how information is collected, these methods can be divided into active and passive vision methods. Active methods first transmit a preset signal to the surface of the object being reconstructed. Sensors then capture the corresponding signal to obtain depth information about the object's surface, and then use this depth information to reconstruct a 3D model. 3D scanning devices for human feet built based on active 3D reconstruction principles can produce relatively accurate foot models, but these devices are typically expensive and require specialized personnel to operate. Passive methods use a camera to capture images of the object being reconstructed from multiple perspectives. Image feature matching is used to match identical pixels across the different perspectives, and finally triangulation is used to obtain 3D information about the reconstructed object's surface. Compared to active methods, passive methods require more common equipment, but image feature matching requires the reconstructed object to have richer texture on its surface. For 3D reconstruction of the human foot, due to its limited texture, complex methods are often required to artificially add texture, otherwise significant reconstruction errors will result. In order to promote the three-dimensional reconstruction technology of the foot for commercial applications, it is necessary to find a simpler and more efficient method to reconstruct a more accurate three-dimensional model of the human foot by using data collected by common devices (such as mobile phone cameras). Summary of the Invention

[0006] In view of the above, an object of the present invention is to provide a three-dimensional foot shape generation method based on a conditional diffusion model to achieve simple and efficient modeling of the three-dimensional foot shape.

[0007] To achieve the above-mentioned object of the invention, an embodiment provides a method for generating a three-dimensional foot shape based on a conditional diffusion model, comprising the following steps:

[0008] Obtain multi-view images of the entity foot and the corresponding camera parameters;

[0009] Extracting foot contours from multi-view images, generating a foot visual shell model based on the foot contours and camera parameters, and performing surface point cloud extraction from the foot visual shell model to obtain a coarse-grained foot point cloud;

[0010] Use the point cloud encoder to encode the coarse-grained foot point cloud to obtain shape features;

[0011] Taking shape features as guiding conditions, a diffusion model is used to perform reverse diffusion based on the guiding conditions to generate fine-grained foot point clouds.

[0012] The smooth geometric surface is reconstructed based on the fine-grained foot point cloud to obtain the three-dimensional foot shape.

[0013] Preferably, the BlendMask model is used to extract the foot contour from the multi-view image, including:

[0014] The BlendMask model includes a target detection branch and an instance mask segmentation branch. The target detection branch is used to detect targets from multi-view images, and the mask segmentation branch is used to calculate the target mask of the multi-view image. The foot type target is determined based on the target detection result output by the target detection peak, and the mask corresponding to the foot type target is extracted from the target mask output by the mask segmentation branch to generate a foot contour map.

[0015] Preferably, the point cloud encoder includes a plurality of feature extraction modules connected by a fully connected layer, each feature extraction module includes a spatial diffusion submodule, a spatial gradient feature calculation submodule and a multi-layer perceptron, the spatial diffusion submodule is used to fuse the local features of multiple independent points within a certain diffusion step range as spatial diffusion features, the spatial gradient feature calculation submodule is used to calculate the derivative of spatial gradient information as spatial gradient features, and the multi-layer perceptron is used to map the splicing results of the local features, spatial diffusion features and spatial gradient features of the independent points input by the feature extraction module to obtain surface continuous features, and output the surface continuous features after adding the surface continuous features to the local features of each independent point;

[0016] The fully connected layer performs dimension mapping on the output of the previous feature extraction module and uses it as the local features of the independent points input to the next feature extraction module;

[0017] The local features of the independent points input by the first feature extraction module are the features of the coarse-grained foot point cloud, and the output of the last feature extraction module is used as the shape feature.

[0018] Preferably, the diffusion model includes a noise prediction network with optimized parameters, and the normal distribution of each diffusion step point cloud is generated through the inverse process of the noise prediction network. In the t-1 diffusion step, the shape features, the normal distribution of the t diffusion step point cloud and the diffusion step t-1 are used as inputs. The noise prediction network predicts the normal distribution of the t-1 diffusion step based on the input, and the normal distribution of the point cloud generated by the initial 0 diffusion step is used as the fine-grained foot point cloud.

[0019] Preferably, the point cloud encoder and the diffusion model need to undergo parameter optimization before being applied, and a loss function is calculated using the fine-grained foot point cloud generated by the diffusion model and the real foot point cloud collected from the physical foot surface to optimize the parameters of the point cloud encoder and the diffusion model;

[0020] The loss function Lrec used is expressed as:

[0021]

[0022] Among them, i represents the index of the point, and its value is 1-N, where N is a natural number greater than 2; t represents the diffusion step, and its value is 2-T, where T is a natural number greater than 3. Represents the real foot point cloud, and Represent the point clouds of t diffusion step and t-1 diffusion step respectively, z represents the shape feature, Indicates that the noise distribution is added to the real foot point cloud during the forward diffusion process. Denotes the noise distribution predicted by the noise prediction network during the reverse diffusion process, D KL () represents KL divergence, and Eq() represents expectation.

[0023] Preferably, the reconstructing a smooth geometric surface based on the fine-grained foot point cloud comprises:

[0024] For each individual point in the fine-grained foot point cloud, the geometric features of the individual point are extracted. A first neural network is then used to calculate and learn a set of offsets based on the individual points and their geometric features. All these offsets are then added to the spatial coordinates of the individual points to obtain a set of offset points. A second neural network is then used to predict the normals of the offset points based on the offset points and trilinear interpolation results of the offset points on the feature volume. The offset points and their normals are then superimposed on the original foot point cloud to obtain a dense point cloud.

[0025] Then the smooth geometric surface is reconstructed based on the dense point cloud.

[0026] Preferably, the first neural network and the second neural network need to be parameterized before being applied, including:

[0027] Given a set of foot mesh models in a training set and an undirected point cloud obtained by random sampling of their surfaces, a first neural network and a second neural network are used to predict offset points and their normals based on the undirected point cloud, and a prediction rule network is reconstructed based on the offset points and their normals.

[0028] The true regular grid is calculated on the dense directed point cloud of the foot mesh model surface, and the parameters of the first neural network and the second neural network are updated by calculating the mean square error between the predicted regular network and the true regular grid.

[0029] Preferably, generating a foot visual shell model according to the foot outline image and camera parameters includes:

[0030] According to the camera parameters, a viewing cone emanating from the camera center and passing only through the object outline is constructed from the camera perspective corresponding to each foot outline. The outline cone is calculated at each perspective and projected into the same three-dimensional space. Then, the visual shell model of the foot is generated by taking the intersection of different cones.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] By constructing a coarse-grained foot point cloud, the point cloud encoder is used to encode the coarse-grained foot point cloud to obtain shape features; using the shape features as the guiding conditions, a diffusion model is used to perform reverse diffusion based on the guiding conditions to generate a fine-grained foot point cloud, and a smooth geometric surface is reconstructed based on the fine-grained foot point cloud. This can easily and quickly achieve three-dimensional foot reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 is a flow chart of a method for generating a three-dimensional foot shape based on a conditional diffusion model provided in an embodiment;

[0035] Figure 2 is a segmentation flow chart of the BlendMask model provided in the embodiment;

[0036] Figure 3 is a schematic diagram of generating a visual shell provided by an embodiment;

[0037] Figure 4 1 is a schematic diagram of the structure of a feature extraction module in a point cloud encoder provided by an embodiment;

[0038] Figure 5 Schematic diagram of diffusion of the conditional diffusion model provided in the embodiment;

[0039] Figure 6 1 is a training diagram of the conditional diffusion model provided in the embodiment;

[0040] Figure 7 Schematic diagram of the derivation of the conditional diffusion model provided in the embodiment;

[0041] Figure 8 This is a schematic diagram of surface reconstruction based on DPSR and deep learning provided by an embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0043] Figure 1 Flowchart of the method for generating three-dimensional foot shape based on conditional diffusion model provided by the embodiment. Figure 1 As shown, the three-dimensional foot shape generation method based on the conditional diffusion model provided in the embodiment includes the following steps:

[0044] Step 1: Obtain multi-view images of the entity foot and the camera parameters corresponding to the images.

[0045] In the embodiment, RGB images of different viewing angles of the entity foot are captured. Where ω and h are the width and height of the image, N is the number of multi-view images, and the camera parameters for taking RGB images are also recorded.

[0046] Step 2: Extract foot contours from multi-view images.

[0047] In the embodiment, the corresponding foot contour map is extracted from the multi-view RGB image by the instance segmentation algorithm Among them, the instance segmentation algorithm used is the BlendMask model, that is, the BlendMask model is used to extract the foot contour from the multi-view image, such as Figure 2 As shown in the figure, the BlendMask model includes a target detection branch and an instance mask segmentation branch. The target detection branch is used to detect the target from the multi-view image, and the mask segmentation branch is used to calculate the target mask of the multi-view image. The foot type target is determined based on the target detection result output by the target detection peak, and the mask corresponding to the foot type target is extracted from the target mask output by the mask segmentation branch to generate a foot contour map.

[0048] The mask segmentation branch includes the bottom module, the top module (Boxes Attns), and the blender. For the bottom module, the backbone network or feature pyramid (FPN) is first used to extract low-level features representing local information and high-level features representing global information from the image, and generate a score map for each pixel in the image belonging to each object category, such as Figure 2 For the top-level module, the top-level features extracted by the feature map pyramid (FPN) are mainly used to extract the bounding box P of each detected object and the attention block A within the bounding box through FCOS post-processing. The attention area is shown as follows: Figure 2 As shown in . For the mixer, the object bounding box P output by the top module, the attention block A and the Bases output by the bottom module are fused to obtain the final target mask.

[0049] Step 3: Generate a foot visual shell model based on the foot contour map and camera parameters, and perform surface point cloud extraction from the foot visual shell model to obtain a coarse-grained foot point cloud.

[0050] In the embodiment, a visual shell algorithm is used to generate a foot-shaped visual shell model, such as Figure 3 As shown in the figure, the input of the visual shell algorithm is a set of calibrated foot contour maps. From the camera perspective corresponding to each foot contour map, a viewing cone is constructed, which is emitted from the camera center and only passes through the object contour. The contour cone at each perspective is calculated and projected into the same three-dimensional space, and then the foot visual shell model is generated by taking the intersection of different viewing cones.

[0051] The overall processing flow of an exemplary visual shell algorithm is shown in Algorithm 1:

[0052]

[0053] In step 1, the algorithm generates N voxel grids of size lv×wv×hv based on the number of input foot contour images. In step 2, each voxel grid is stretched to one dimension, with the coordinates of each position in the original voxel grid recorded. Steps 4 through 9 form a structure repeated N times, corresponding to each of the N input contour images. Step 4 transforms the voxel grid coordinates to the camera coordinate system corresponding to the i-th contour image. Steps 5 through 8 map the points in the 3D camera coordinate system to the 2D image coordinate system based on the camera intrinsic parameters, filtering out points outside the image range. Step 9 extracts the corresponding position value from the contour image based on the mapped voxel grid coordinates. It is worth noting that steps 4 through 9 are performed independently in the algorithm, allowing for acceleration through GPU parallel computing. Step 11 restores the stretched voxel grid to its original shape, with each position storing the value of the corresponding point in the contour image. Finally, step 12 counts the number of identical positions with a value of 1 across the N voxel grids. If the number is the same as the number N of the input contour map, it means that the point is located inside the object contour at all angles. By screening out all voxel points that meet the above conditions, a visual shell model of the foot shape can be obtained.

[0054] After obtaining the foot's visible shell model, an undirected point cloud consisting of 2048 points is randomly sampled from the surface of the visible shell. This serves as the input for the subsequent reconstruction process, replacing the voxel model. The point cloud contains the 3D coordinates of the 2048 points sampled from the visible shell, with each point represented as a 3-bit floating point number (4 bytes). Ultimately, each model only requires 24KB of memory, significantly reducing memory usage and unnecessary training overhead.

[0055] Step 4: Use the point cloud encoder to encode the coarse-grained foot point cloud to obtain shape features.

[0056] In the embodiment, a point cloud encoder is used to encode the coarse-grained foot point cloud. The point cloud encoder realizes feature transfer by diffusing local features on the surface points, models the geometric surface information through the feature diffusion step t and the spatial gradient scale matrix A, and finally fuses the local information of the point cloud and the surface continuous information through a multi-layer perceptron.

[0057] The point cloud encoder includes multiple (for example, 4) feature extraction modules connected by a fully connected layer. The fully connected layer is connected between the head and tail of two feature extraction modules for dimensionality conversion. During training, the batch size used by the point cloud encoder is 128. Therefore, the overall input of the encoder is 128×2048×3 undirected point cloud coordinates and corresponding preprocessing information, and the output is 128×256 shape features.

[0058] like Figure 4 As shown, each feature extraction module includes a spatial diffusion submodule, a spatial gradient feature calculation submodule and a multi-layer perceptron. The spatial diffusion submodule mainly controls the propagation range of the point-by-point features of the point cloud in the point cloud surface space. Different from the previous methods that rely on a fixed sampling method to determine the feature transfer direction between points, the present invention transfers features between independent points by gradually diffusing point-by-point features in the point cloud surface space. The diffusion of point-by-point features in the surface space is controlled by a learnable continuous variable (i.e., feature diffusion step) t. When the feature diffusion step approaches 0, it tends to extract features on independent points. As the feature diffusion step increases, it tends to extract average features on a larger surface. Based on this, the local features of multiple independent points within a certain diffusion step range are fused as spatial diffusion features.

[0059] The spatial gradient feature calculation submodule expands the feature receptive field by constructing additional spatial gradient information using point-by-point features. Specifically, the derivative of the spatial gradient information is calculated as the spatial gradient feature. For N×D point cloud feature input, before officially starting training, the local spatial gradient on each feature channel of N points and D feature channels is first calculated. Then, the local spatial gradient on all feature channels of each point in the point cloud is stacked into w x After that, the spatial gradient dot product of the same feature channel is used as an additional spatial gradient feature gv, where Among them, A is a D×D matrix used to control the proportional coefficient of the additional information of different feature channels, ⊙ represents the dot product, represents w x Re() represents the real part of the dot product of the two, and tanh() represents the hyperbolic tangent function.

[0060] The multilayer perceptron (MLP) maps the concatenation of the local features, spatial diffusion features, and spatial gradient features of the individual points input into the feature extraction module to obtain surface continuous features. These surface continuous features are then added to the local features of each individual point and output. Specifically, the MLP input consists of the local features, spatial diffusion features, and spatial gradient features of the original individual points. These three features are concatenated and input to the MLP, which extracts surface continuous features containing surface continuous spatial information. The surface continuous features extracted by the MLP are then added point by point to the local features of the original individual points and output.

[0061] The fully connected layer performs dimension mapping on the output of the previous feature extraction module and uses it as the local features of the individual points as input to the next feature extraction module. The local features of the individual points input to the first feature extraction module are the features of the coarse-grained foot point cloud, and the output of the last feature extraction module is used as the shape features.

[0062] Step 5: Using shape features as guiding conditions, a diffusion model is used to perform reverse diffusion based on the guiding conditions to generate a fine-grained foot point cloud.

[0063] In the embodiment, the basic principle of the diffusion model: the diffusion probability model (DPM) is a latent variable model designed with reference to the principle of non-equilibrium thermodynamics. The purpose of the model is to learn how to reversely generate a feature after multi-step noise processing. The standard diffusion probability model is divided into two parts: the forward noise addition process and the reverse denoising process. The purpose of the forward process of the diffusion model is to destroy the structure of the input feature. Before the forward process starts, a maximum diffusion step T is first sampled as the maximum number of times the forward process adds noise, and then in each diffusion step t, the original feature x of the input is added. 0 When Gaussian noise is added to , the original feature will be transformed into pure noise x when the maximum diffusion step T is reached. T The entire process can be represented by a Markov chain as follows:

[0064]

[0065]

[0066] The noise added in each step of the forward process is sampled from the known noise distribution. The diffusion kernel function q(x t |x t-1 ) describes the noise sampling strategy, β is the hyperparameter that controls the sampling variance adjustment strategy, N represents the normal distribution, and I is the unit matrix. Formula (2) shows that the diffusion model calculates the output x of the previous diffusion step in each diffusion step t. t-1 The noise added is from xt-1 and β are sampled from the normal distribution defined by .

[0067] The purpose of the reverse process of the diffusion model is to reconstruct the characteristic structure destroyed in the forward process. Specifically, the reverse process mainly learns a denoising function μ θ , using μ θ From pure noise x T Backward prediction diffusion step t adds the distribution of noise, thereby gradually reconstructing the original feature x 0 The inverse noise addition process of the diffusion model can be expressed as a Markov chain as shown in formulas (3) and (4):

[0068]

[0069]

[0070] Among them, p(x T ) represents random noise x sampled from the standard normal distribution N(0,1) T The probability of z is the reverse reconstruction feature x 0 The guiding condition in the process. θ It is a function defined by parameter θ, which can be calculated based on the diffusion step t, the output x of diffusion step t t And the guidance condition z is used to predict the mean of the Gaussian noise added by the forward process in the diffusion step t-1. The purpose of training the diffusion model is to learn μ θ The parameter θ is thus derived through μ θ to gradually predict and eliminate noise.

[0071] The basic principle of conditional diffusion model: Figure 5 As shown, for point cloud In terms of x 0 Every point x in i Can be regarded as the distribution of points satisfying the point cloud shape feature z The diffusion process of a single point is consistent with the process defined by formula (1), and the generation probability of the entire point cloud is the product of the generation probability of each point. As the number of noise additions increases, the original point cloud X 0 Gradually spread into a meaningless chaotic point cloud X T :

[0072]

[0073] When only considering the point cloud generation task, the inverse process of the diffusion model is unsupervised, which allows the diffusion model to generate more diverse point clouds based on randomly sampled noise. For the foot reconstruction task to be solved in this chapter, the point cloud generated by the diffusion model must meet specific shape characteristics while ensuring that it conforms to the foot shape distribution. Specifically, for a randomly sampled noise point cloud X T , in its reverse denoising process, the foot shape feature z output by the point cloud encoder is introduced as a guiding condition, and X is calculated through the process described in formula (3). T Each point in is denoised, and the denoising process of the entire point cloud is shown as follows:

[0074]

[0075] The above conditional diffusion model and point cloud encoder need to be trained before being applied. The loss function construction process used in training includes: φ Represents the point cloud encoder, expressed in μ θ The noise prediction network represents the diffusion model. The training goal of the foot point cloud reconstruction part is to learn E φ The parameters φ and μ θ The parameter θ makes the coarse-grained foot point cloud X vh The fine-grained foot point cloud X obtained after reconstruction rec Compared with point cloud sampled from the real foot surface real The distribution of the two probability distributions is as close as possible. The similarity evaluation of the two probability distributions is usually achieved by calculating the KL divergence. The conditional diffusion model is forwarded to the real foot point cloud X in the tth step of the forward process. real The distribution of added noise is expressed as q(X t-1 |X t ,X 0 ), the reverse process is carried out by the noise prediction network μ θ The predicted noise distribution is denoted as p θ (X t-1 |X t ,z), by calculating the KL divergence of the above two distributions at each step of the Markov chain described in formula (6), the following loss function definition can be obtained:

[0076]

[0077] In formula (7), X refers to the actual foot surface point cloud X real , the guiding condition z is obtained by the point cloud encoder, that is, z = E φ (X vh ). In addition, since the first step of the forward process directly processes the original data X 0Noise is added, so it is represented by a separate term in the formula. The direct purpose of training based on the above loss function is to learn the noise prediction network μ θ Parameters of the encoder E φ The learning of parameters is implicitly represented by z. As shown in formula (5), the distribution of each point in the point cloud is independently sampled. By extending the loss function defined in formula (7) to individual points, the final loss function can be obtained:

[0078]

[0079] Among them, i represents the index of the point, and its value is 1-N, where N is a natural number greater than 2; t represents the diffusion step, and its value is 2-T, where T is a natural number greater than 3. Represents the real foot point cloud, and Represent the point clouds of t diffusion step and t-1 diffusion step respectively, z represents the shape feature, Indicates that the noise distribution is added to the real foot point cloud during the forward diffusion process. Denotes the noise distribution predicted by the noise prediction network during the reverse diffusion process, D KL () represents KL divergence, and Eq() represents expectation.

[0080] In the training phase, we first randomly sample a pair of coarse-grained visual shell point clouds X from the training set. vh and real foot point cloud X real , using the point cloud encoder E φ To X vh Encode to obtain the latent variable z representing the foot shape feature. Then, in the forward and reverse process of diffusion, z is used to guide the noise prediction network μ θ The prediction of the noise distribution at each step. Finally, the loss described by formula (8) is calculated and back-propagated to supervise μ θ And the parameters of the point cloud encoder are updated until the loss converges. The training diagram of the conditionally guided point cloud generation diffusion model is as follows Figure 6 shown.

[0081] After the training is completed, the derivation prediction is performed. In the derivation stage, given the coarse-grained foot point cloud X vh , first pass the trained point cloud encoder E φ Extract the shape feature z from it, and then sample a random noise point cloud X from the standard normal distribution N(0,1) T In the t-1 diffusion step, the shape features, the normal distribution of the point cloud of the t diffusion step, and the diffusion step t-1 are taken as inputs. The noise prediction network predicts the normal distribution of the t-1 diffusion step based on the input, and guides the noise prediction network μ through z θ To XT Perform reverse denoising and the normal distribution of the point cloud generated by the initial 0 diffusion step to obtain a fine-grained foot point cloud X that conforms to the foot shape and satisfies the shape characteristics. rec , the schematic diagram of the derivation of the condition-guided point cloud generation diffusion model is as follows Figure 7 shown.

[0082] Step 6: Reconstruct a smooth geometric surface based on the fine-grained foot point cloud to obtain a three-dimensional foot shape.

[0083] In the embodiment, a smooth geometric surface is reconstructed from a fine-grained foot point cloud based on a super-resolution algorithm DPSR and deep learning. Based on DPSR, the idea of ​​the surface reconstruction method combined with deep learning is to use two independently weighted neural networks (such as MLP) f θ and g θ To achieve point cloud enhancement and normal prediction respectively.

[0084] like Figure 8 As shown, for a fine-grained foot point cloud, the foot point cloud is an undirected point cloud. First, extract each point x through the point cloud encoder i The geometric characteristics of φ θ (x i ). For the point cloud enhancement task, the first neural network f θ From each independent point x in the point cloud i and its characteristic φ θ (x i ) to learn a set of offsets Δx = f θ (x,Φ θ (x)), all offsets are compared with the original point x i A set of offset points can be obtained by adding the spatial coordinates of As the formula As shown, the second neural network g θ Mainly used to predict f θ The obtained undirected point Normal direction.

[0085] In the embodiment, the obtained offset points are superimposed on the original foot-shaped point cloud to increase the density of the point cloud, thereby obtaining a dense point cloud. The dense point cloud is reconstructed using DPSR to obtain a smooth geometric surface.

[0086] In the embodiment, the first neural network and the second neural network need to be parameterized before being applied, including: given a set of foot mesh models in the training set and an undirected point cloud obtained by random sampling of the surface thereof, the first neural network and the second neural network are used to predict the offset points and their normals based on the undirected point cloud, and the prediction rule network is obtained by DPSR reconstruction based on the offset points and their normals.

[0087] PSR is used to calculate the true regular grid χ on the dense directed point cloud of the foot mesh model surface, and the regular network is predicted by calculation. and the mean square error between the true regular grid χ To update the parameters of the first and second neural networks, during the training process, 7 offset points are predicted for each point in the foot point cloud and optimized using the Adam optimizer.

[0088] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A three-dimensional foot shape generation method based on a conditional diffusion model, characterized in that: The following steps are involved: Obtain multi-view images of the entity foot and the corresponding camera parameters; Extracting foot contours from multi-view images, generating a foot visual shell model based on the foot contours and camera parameters, and performing surface point cloud extraction from the foot visual shell model to obtain a coarse-grained foot point cloud; Use the point cloud encoder to encode the coarse-grained foot point cloud to obtain shape features; Taking shape features as guiding conditions, a diffusion model is used to perform reverse diffusion based on the guiding conditions to generate fine-grained foot point clouds. The diffusion model includes a noise prediction network with optimized parameters. The normal distribution of each diffusion step point cloud is generated through the reverse process of the noise prediction network. In the t-1 diffusion step, the shape features, the normal distribution of the t diffusion step point cloud, and the diffusion step t-1 are used as inputs. The noise prediction network predicts the normal distribution of the t-1 diffusion step based on the inputs. The normal distribution of the point cloud generated by the initial 0 diffusion step is used as the fine-grained foot point cloud. The smooth geometric surface is reconstructed based on the fine-grained foot point cloud to obtain the three-dimensional foot shape.

2. The three-dimensional foot shape generation method based on the conditional diffusion model according to claim 1, characterized in that: The BlendMask model is used to extract foot contours from multi-view images, including: The BlendMask model includes a target detection branch and an instance mask segmentation branch. The target detection branch is used to detect targets from multi-view images, and the mask segmentation branch is used to calculate the target mask of the multi-view image. The foot type target is determined based on the target detection result output by the target detection peak, and the mask corresponding to the foot type target is extracted from the target mask output by the mask segmentation branch to generate a foot contour map.

3. The method for generating a three-dimensional foot shape based on a conditional diffusion model according to claim 1, wherein: The point cloud encoder includes multiple feature extraction modules connected by a fully connected layer, each feature extraction module includes a spatial diffusion submodule, a spatial gradient feature calculation submodule and a multi-layer perceptron, the spatial diffusion submodule is used to fuse the local features of multiple independent points within a certain diffusion step range as spatial diffusion features, the spatial gradient feature calculation submodule is used to calculate the derivative of spatial gradient information as spatial gradient features, and the multi-layer perceptron is used to map the splicing results of the local features, spatial diffusion features and spatial gradient features of the independent points input by the feature extraction module to obtain surface continuous features, and output the surface continuous features after adding the surface continuous features to the local features of each independent point; The fully connected layer performs dimension mapping on the output of the previous feature extraction module and uses it as the local features of the independent points input to the next feature extraction module; The local features of the independent points input by the first feature extraction module are the features of the coarse-grained foot point cloud, and the output of the last feature extraction module is used as the shape feature.

4. The three-dimensional foot shape generation method based on the conditional diffusion model according to claim 1, characterized in that: The point cloud encoder and diffusion model need to undergo parameter optimization before being applied. The fine-grained foot point cloud generated by the diffusion model and the real foot point cloud collected from the physical foot surface are used to calculate the loss function to optimize the parameters of the point cloud encoder and the diffusion model. The loss function Lrec used is expressed as: Among them, i represents the index of the point, and its value is 1-N, where N is a natural number greater than 2; t represents the diffusion step, and its value is 2-T, where T is a natural number greater than 3. Represents the real foot point cloud, and Represent the point clouds of t diffusion step and t-1 diffusion step respectively, z represents the shape feature, Indicates that the noise distribution is added to the real foot point cloud during the forward diffusion process. Denotes the noise distribution predicted by the noise prediction network during the reverse diffusion process, D KL () represents KL divergence, Express expectations.

5. The three-dimensional foot shape generation method based on the conditional diffusion model according to claim 1, characterized in that: The method of reconstructing a smooth geometric surface based on a fine-grained foot point cloud includes: For each individual point in the fine-grained foot point cloud, the geometric features of the individual point are extracted. A first neural network is then used to calculate and learn a set of offsets based on the individual points and their geometric features. All these offsets are then added to the spatial coordinates of the individual points to obtain a set of offset points. A second neural network is then used to predict the normals of the offset points based on the offset points and trilinear interpolation results of the offset points on the feature volume. The offset points and their normals are then superimposed on the original foot point cloud to obtain a dense point cloud. Then the smooth geometric surface is reconstructed based on the dense point cloud.

6. The three-dimensional foot shape generation method based on the conditional diffusion model according to claim 5, characterized in that: The first neural network and the second neural network need to be parameterized before being applied, including: Given a set of foot mesh models in a training set and an undirected point cloud obtained by random sampling of their surfaces, a first neural network and a second neural network are used to predict offset points and their normals based on the undirected point cloud, and a prediction rule network is reconstructed based on the offset points and their normals. The true regular grid is calculated on the dense directed point cloud of the foot mesh model surface, and the parameters of the first neural network and the second neural network are updated by calculating the mean square error between the predicted regular network and the true regular grid.

7. The method for generating three-dimensional foot shape based on conditional diffusion model according to claim 1, characterized in that: The method of generating a foot visual shell model according to the foot outline and camera parameters includes: According to the camera parameters, a viewing cone emanating from the camera center and passing only through the object outline is constructed from the camera perspective corresponding to each foot outline. The outline cone is calculated at each perspective and projected into the same three-dimensional space. Then, the visual shell model of the foot is generated by taking the intersection of different cones.

Images