Foot three-dimensional reconstruction method and system based on multi-view image

Abstract

The invention relates to the technical field of three-dimensional reconstruction, and provides a foot three-dimensional reconstruction method and system based on a multi-view image, and the method comprises the steps: carrying out the three-dimensional reconstruction through employing a multi-view image corresponding to a foot standing site, so as to obtain an initial three-dimensional model A; performing three-dimensional reconstruction by adopting a multi-view image corresponding to the footprint mold site to obtain an initial three-dimensional model B; separating a foot dorsum surface sub-model in the initial three-dimensional model A from other sub-models; separating a sole mark sub-model in the initial three-dimensional model B from other sub-models; and splicing the separated foot back surface sub-model and the sole mark sub-model to form a full-foot three-dimensional model. According to the method, the defect that an existing method cannot achieve integrated three-dimensional representation of the foot dorsum and the foot sole when the foot is in the standing posture is overcome, the full-foot three-dimensional data has integrity, lightweight performance and authenticity, and the requirements for supporting data storage, visual presentation, doctor diagnosis and the like are met.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and more specifically, to a method and system for three-dimensional reconstruction of a foot based on multi-view images. Background Art

[0002] As many diseases spread widely among many people and seriously threaten human health, people are constantly raising their awareness of physical examinations. For example, many podiatric diseases include inversion / valgus of the foot, hallux valgus, high arches and flat feet, etc., and many groups include teenagers, adults, athletes and overweight people.

[0003] With the widespread application of Internet+, providing online services for patients and doctors has become a trend. The user terminal collects image data (images or videos) of the foot, and the collected image data is pushed to the doctor after processing through the server. After the doctor diagnoses, the server sends a diagnosis report to the patient. This online model can greatly provide convenience for patients. However, if the camera data is blurred or there are many missing angles, it will not be possible to restore the current status of the foot in all directions and clearly, which will seriously affect the accuracy and convenience of the doctor's diagnosis, and even the doctor cannot make a diagnosis based on the image data.

[0004] Therefore, how to produce three-dimensional data suitable for representing the overall picture of the foot to meet the needs of data storage, visualization and doctor diagnosis has become an urgent problem to be solved. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a method and system for three-dimensional reconstruction of the foot based on multi-view images, so as to solve the problem that the prior methods cannot achieve an integrated three-dimensional representation of the dorsum and sole of the foot in a standing posture.

[0006] According to a first aspect of the present invention, a method for three-dimensional reconstruction of a foot based on multi-view images is provided, comprising:

[0007] Performing three-dimensional reconstruction using multi-view images corresponding to a foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet;

[0008] A three-dimensional reconstruction is performed using multi-view images corresponding to the scene of the footprint mold to obtain an initial three-dimensional model B, wherein the multi-view images corresponding to the scene of the footprint mold are obtained by taking multi-view photos of the double footprints in the footprint box, and the double footprints in the footprint box are left by a single person standing with both feet in the footprint box;

[0009] Separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model;

[0010] Separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model;

[0011] The dorsal surface sub-model and the plantar imprint sub-model are spliced ​​to form a three-dimensional model of the entire foot.

[0012] Based on the above technical solution, the present invention can also make the following improvements.

[0013] Optionally, the three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A includes:

[0014] Using a camera device to shoot at multiple angles around the two feet or one foot of a person standing on the floor, to obtain multi-view dorsal image data;

[0015] Based on the multi-view dorsal imaging data, an initial three-dimensional model A is constructed.

[0016] Optionally, the three-dimensional reconstruction using multi-view images corresponding to the footprint mold scene to obtain an initial three-dimensional model B includes:

[0017] Based on the camera device, multi-angle shooting is performed around a footprint box with double-foot prints to obtain multi-angle sole image data, wherein the double-foot prints are left by a single person standing in the footprint box with two feet;

[0018] Based on the multi-view plantar imaging data, an initial three-dimensional model B is constructed.

[0019] According to a second aspect of the present invention, there is provided a method for three-dimensional reconstruction of a foot based on multi-view images, comprising:

[0020] Generate an initial three-dimensional point cloud A using multi-view images corresponding to the foot standing scene;

[0021] An initial three-dimensional point cloud B is generated using multi-view images corresponding to the footprint mold site;

[0022] Separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud;

[0023] Separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud;

[0024] The dorsal surface sub-point cloud and the plantar imprint sub-point cloud are spliced ​​to form a three-dimensional point cloud of the entire foot.

[0025] Optionally, the step of separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud includes:

[0026] Separating the dorsal surface sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud A by using a pre-trained point cloud segmentation model point cloud segmentation model A, extracting the dorsal surface sub-point cloud and / or deleting the other sub-point clouds;

[0027] The step of separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud comprises:

[0028] The plantar imprint sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud B are separated by a pre-trained point cloud segmentation model B, and the plantar imprint sub-point cloud is extracted and / or other sub-point clouds are deleted.

[0029] According to a third aspect of the present invention, a method for three-dimensional reconstruction of a foot based on multi-view images is provided, comprising:

[0030] The multi-view images corresponding to the foot standing scene are used for three-dimensional reconstruction to obtain an initial three-dimensional model A;

[0031] An initial three-dimensional point cloud B is generated using multi-view images corresponding to the footprint mold site;

[0032] Separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model;

[0033] Separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud;

[0034] The dorsal surface sub-model is spliced ​​with the plantar imprint sub-point cloud to form full-foot three-dimensional data.

[0035] Optionally, the step of splicing the dorsal surface sub-model with the plantar imprint sub-point cloud to form full-foot three-dimensional data includes:

[0036] generating a corresponding plantar imprint sub-model based on the plantar imprint sub-point cloud;

[0037] The plantar imprint sub-model and the dorsal surface sub-model are spliced ​​together to form a three-dimensional model of the entire foot.

[0038] According to a fourth aspect of the present invention, a method for three-dimensional reconstruction of a foot based on multi-view images is provided, comprising:

[0039] Generate an initial three-dimensional point cloud A using multi-view images corresponding to the foot standing scene;

[0040] The multi-view images corresponding to the footprint mold scene are used for three-dimensional reconstruction to obtain an initial three-dimensional model B;

[0041] Separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud;

[0042] Separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model;

[0043] The dorsal surface sub-point cloud is spliced ​​with the plantar imprint sub-model to form full-foot three-dimensional data.

[0044] Optionally, the step of splicing the dorsal surface sub-point cloud with the plantar imprint sub-model to form full-foot three-dimensional data includes:

[0045] generating a corresponding dorsal surface sub-model based on the dorsal surface sub-point cloud;

[0046] The dorsal surface sub-model is spliced ​​with the plantar imprint sub-model to form a three-dimensional model of the entire foot.

[0047] According to a fifth aspect of the present invention, there is provided a foot three-dimensional reconstruction system based on multi-view images, comprising:

[0048] A reconstruction module, for performing three-dimensional reconstruction using multi-view images corresponding to a foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet; and for performing three-dimensional reconstruction using multi-view images corresponding to a footprint mold scene to obtain an initial three-dimensional model B;

[0049] A separation module, used to separate the dorsal surface sub-model in the initial three-dimensional model A from other sub-models; and also used to separate the plantar imprint sub-model in the initial three-dimensional model B from other sub-models;

[0050] The splicing module is used to splice the dorsal surface sub-model and the plantar imprint sub-model to form a three-dimensional model of the entire foot.

[0051] The present invention provides a method and system for reconstructing a foot based on multi-view images, which obtains a data portion representing the dorsal surface of the foot from the three-dimensional data representing the scene of the footprint mold, and obtains a data portion representing the plantar imprint from the three-dimensional data representing the scene of the foot standing, so that other redundant data portions are separated, and the data portion representing the plantar imprint and the data portion representing the dorsal surface of the foot are integrated and spliced. The three-dimensional data of the entire foot has integrity, lightness and authenticity, which overcomes the problem that the existing methods cannot achieve an integrated three-dimensional representation of the dorsal and sole of the foot in a standing posture, and helps to support the needs of data storage, visualization and doctor diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to the first embodiment of the present invention;

[0053] Figure 2 Schematic diagram of an initial three-dimensional model A according to the first embodiment of the present invention;

[0054] Figure 3 Schematic diagram of an initial three-dimensional model B according to the first embodiment of the present invention;

[0055] Figure 4 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to a second embodiment of the present invention;

[0056] Figure 5 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to Embodiment 3 of the present invention;

[0057] Figure 6 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to a fourth embodiment of the present invention;

[0058] Figure 7 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to a fifth embodiment of the present invention;

[0059] Figure 8 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to a sixth embodiment of the present invention;

[0060] Fig. 9 This is a flow chart of a method for three-dimensional reconstruction of a foot based on multi-view images according to Embodiment 7 of the present invention;

[0061] Fig.10 A schematic structural diagram of a foot three-dimensional reconstruction system based on multi-view images provided by the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not subject to the constraints of the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0063] In the prior art, a person stands on the ground, and a camera takes a picture of the person's feet, and then performs a three-dimensional reconstruction of the feet. However, due to the fact that the feet stand on the ground (or on a piece of paper placed on the ground), the instep and other surrounding objects (including the ground, paper, and reference objects, etc.) can be photographed, but the sole cannot be photographed. Therefore, the sole cannot be integrated with the instep surface in a three-dimensional representation, and the data representing other objects are partially redundant data, which consumes more storage and reduces the visualization effect.

[0064] See also Figure 1 The present invention provides a method for three-dimensional reconstruction of a foot based on multi-view images according to Embodiment 1, comprising:

[0065] S101, performing three-dimensional reconstruction using multi-view images corresponding to a foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet.

[0066] For example, a smartphone (shooting device) is used to shoot around the two feet (or one foot) of a single person standing on the floor to obtain image data, which can be 20 images with different perspectives or a 60-second video, wherein the feet standing on the floor is the foot standing scene. The smartphone uploads the shot image data to the server. If the image data is a 60-second video, the server extracts frames from the video to obtain multi-perspective images. If the image data is 20 images, the 20 images are stored as multi-perspective images.

[0067] For example, Figure 2 The initial three-dimensional model A shown reflects, in addition to the surfaces of the two feet in a standing position, a floor area, a reference object, and slippers, wherein the reference object may be a bank card, a bus card, or a campus card, etc.

[0068] S102, three-dimensional reconstruction is performed using multi-view images corresponding to the footprint mold scene to obtain an initial three-dimensional model B, wherein the multi-view images corresponding to the footprint mold scene are obtained by taking multi-view photos of the double footprints in the footprint box, where the double footprints in the footprint box are left by a single person standing with both feet in the footprint box.

[0069] For example, a footprint box is placed on the floor, and a person steps into the footprint box with both feet in a standing position, so that double footprints are left in the footprint box. After the two feet leave the footprint box, the footprint box and the floor on which it is located are the footprint mold scene. Similar to the above-mentioned foot standing scene, a mobile phone is used to shoot around the footprint box with double footprints, so that the server can obtain corresponding multi-view images.

[0070] For example, Figure 3The initial three-dimensional model B shown mainly reflects the floor area and the footprint box in the floor area, and there are two sole prints in the footprint box in a state of two feet being separated.

[0071] It should be noted that the 3D reconstruction in step 101 and step 102 adopts the existing technical route: using multi-view images, a sparse 3D point cloud is constructed through the motion recovery structure (SFM) algorithm, and the camera pose of each view image is estimated at the same time; however, a fine surface cannot be formed through the sparse 3D point cloud, so it is necessary to further construct a dense 3D point cloud through the multi-view stereo vision (MVS) algorithm. The dense 3D point cloud can already express the scene in a refined manner, but it still expresses the form of points rather than surfaces; through methods such as Poisson reconstruction, a 3D mesh white film is generated based on the dense 3D point cloud.

[0072] S103, separating the dorsal surface sub-model of the foot from other sub-models in the initial three-dimensional model A to obtain the dorsal surface sub-model of the foot.

[0073] S104, separating the plantar imprint sub-model from other sub-models in the initial three-dimensional model B to obtain the plantar imprint sub-model.

[0074] Exemplarily, the two initial three-dimensional models A and B can be set as mesh models respectively, so the separated plantar imprint sub-model and dorsal surface sub-model can be regarded as two mesh models respectively. A variety of existing separation technologies can be adaptively applied to separate the two initial three-dimensional models A and B, such as: a mesh segmentation algorithm based on curvature features, a mesh segmentation algorithm based on region growing, and a mesh segmentation algorithm based on a neural network (e.g., MV-RNN or Mesh-CNN).

[0075] S105, splicing the dorsal surface sub-model and the plantar imprint sub-model to form a three-dimensional model of the entire foot.

[0076] Exemplarily, the separated plantar imprint sub-model and dorsal surface sub-model are respectively regarded as two Mesh models. The splicing of Mesh models refers to aligning the overlapping parts of the two Mesh models to form a complete Mesh model. The existing Mesh model alignment scheme can be adaptively applied to the alignment of the plantar imprint sub-model and the dorsal surface sub-model so that the two can be synthesized into a full foot three-dimensional model. For example, at least one of the Mesh model alignment scheme based on curvature features, the Mesh model alignment scheme based on the closest point iteration (ICP) and the non-rigid Mesh model alignment scheme can be adopted.

[0077] Exemplarily, the principle of the curvature feature-based Mesh model registration scheme is as follows: the dorsal surface area (dorsal surface sub-model) separated from the initial three-dimensional model A is set as the source model M, and the plantar imprint area (plantar imprint sub-model) separated from the initial three-dimensional model B is set as the target model N, where M and N belong to three-dimensional mesh models. The local curvature features of each mesh vertex in the two models are calculated, the corresponding feature vertices are searched based on the feature distance, and the registration operation of sampling consistency is performed using the feature point pairs to achieve the registration of the two Mesh models and obtain the three-dimensional Mesh model of the whole foot.

[0078] The curvature-based mesh model registration scheme mainly includes the following steps:

[0079] 1. Calculation of curvature characteristics:

[0080] The curvature characteristics of the mesh vertices in the model are calculated. For any vertex v in the model, it has t related neighborhood triangles. Using discrete differential geometry, the Gaussian curvature K(v) of the triangular mesh surface at the vertex can be obtained.

[0081]

[0082] Among them, ∑ t θ i is the angle sum of the neighborhood triangles of vertex v, θ i represents the angle corresponding to the i-th neighborhood triangle, ∑ t θ i After subtracting it from 2π and dividing it by the area of ​​the region A(v), we can get the Gaussian curvature of v. We traverse all the vertices of the model and calculate them one by one to obtain the curvature characteristics of the model.

[0083] 2. Corresponding feature search:

[0084] For any vertex v in the source model M a , using the feature KD tree, search for the target model N that matches v a The vertex v whose characteristic distance D is less than a given threshold b , forming corresponding feature point pairs.

[0085] D=K(v a )-K(v b )

[0086] 3. Sampling consistency registration:

[0087] Each vertex v in the source model M a , find 1 to 3 candidate corresponding feature vertices in the target model N. Each time a match is made, each vertex randomly selects a unique feature vertex v from its corresponding feature vertex b, get n groups of corresponding point pairs consisting of the same number of source vertices and target vertices, and solve the rigid body transformation matrix. Calculate the error function of the model vertices after transformation to determine whether the current registration transformation meets the registration requirements. The error function here uses the Huber penalty function, which is recorded as in:

[0088]

[0089] m j For a given threshold, d j is the distance difference after transformation of the jth group of corresponding vertices. When the value of the error function is the smallest, it is considered optimal. The original model is transformed to obtain the registration result.

[0090] Exemplarily, the principle of the Mesh model registration scheme based on the closest point iteration (ICP) is: the closest point iteration can be used to effectively align the models, which mainly includes two steps: searching for the nearest corresponding point pair between models and calculating the rotation and translation matrix. The iteration is terminated when the average error of the model's closest point pair is less than a given threshold or the error change between two iterations is small, and the accumulated matrix is ​​used for the original Mesh model to obtain a full-foot three-dimensional Mesh model.

[0091] The Mesh model registration scheme based on the closest point iteration (ICP) mainly includes the following steps:

[0092] 1. Corresponding point pair search:

[0093] The KD tree is used to speed up the nearest point search. The KD tree is constructed with the vertex set of the target model N. For any vertex p in the source model M, h , taking the vertex as the search point, using the K nearest neighbor search method, obtain the vertex q with the closest spatial Euclidean distance to the vertex on the target model N h , construct a corresponding point pair set SF.

[0094] 2. Transformation matrix calculation:

[0095] After searching for a set SF consisting of n corresponding point pairs, the singular value decomposition (SVD) method is used to solve the model transformation, and the obtained rotation matrix R and translation matrix T can minimize the objective function F.

[0096]

[0097] Calculate the center of mass p c and q c , corresponding point minus the centroid, p h ′ =p h -p c and q h ′ =qh -q c , update the objective function.

[0098]

[0099] To solve the objective function, let Singular value decomposition can be obtained H = UWV T .

[0100] According to the orthogonal matrices U and V, the rotation matrix R = VU can be calculated respectively. T and the target matrix T = q c -R×p c , and further calculate the average error E between models M and N.

[0101]

[0102] 3. Multiple Iterations Registration:

[0103] The source model M is transformed according to the calculated rotation matrix R and translation matrix T, and its mesh vertices are used to re-search the nearest vertices in the target model N, and the new corresponding points are used to search for the nearest vertices in the SF ′ , recalculate the rotation matrix R and translation matrix T, and iterate this process repeatedly. The termination condition is that the average error E of the corresponding point pairs of the model is less than the given threshold or the error change between two iterations is small. Apply the rotation matrix R and translation matrix T used in each iteration to the original source model to complete the alignment of the source model and the target model, and obtain the complete foot Mesh model registration result.

[0104] Exemplarily, the principle of the non-rigid Mesh model registration scheme is as follows: the model consists of a vertex set V and an edge set L, allowing the model to undergo complex deformations in local shape, setting a series of gradually decreasing rigid weights to control the smoothness of the deformation, and performing non-rigid adjustments by iteratively searching for the nearest vertex pairs between the source model M and the target model N to obtain a full-foot three-dimensional Mesh model.

[0105] The non-rigid Mesh model registration scheme mainly includes the following steps:

[0106] 1. Determination of initial conditions:

[0107] Using the nearest neighbor search algorithm, each vertex in the source model M finds the vertex with the closest spatial distance in the target model N to form a corresponding point pair combination. For each vertex V:[V1..V n ] T Assign an affine transformation matrix X: [X1..X n ] T, which is used to define the local deformation behavior of vertices during the registration process and maintain the surface coherence and geometric characteristics by minimizing the difference in the transformation of adjacent vertices. The matrix X can be initialized as the identity matrix, indicating that the initial model vertices have no transformation, and a set of rigid weights α that gradually decrease uniformly from high to low are defined g , which is used to control the smoothness of the deformation.

[0108] 2. Optimal transformation solution:

[0109] The non-rigid registration is performed by i Perform different affine transformations X i To align to the target model N, the objective function of this process is: E = E d +αE s .

[0110]

[0111] Among them, E d represents the distance term between the deformed model and the target, d 2 (τ,X i V i ) represents the distance from the vertex in the source model M to the nearest corresponding point in the target model N after the transformation. The model vertex V is represented by the homogeneous coordinates V = [x, y, z, 1] T Indicates that ω i To match the reliability weights, the weights of vertices for which no corresponding points can be found are set to 0, and the weights of other vertices are set to 1.

[0112]

[0113] E s represents the additional rigid term for the canonical deformation, where the matrix G is used to penalize The weighted difference of the transformations of adjacent vertices under the norm, G:=diag(1,1,1,1).

[0114] 3. Iteration recent convergence:

[0115] Using the given initial corresponding points and rigid weights, the algorithm starts to iterate, minimizing the objective function in each iteration, finding the optimal transformation, ensuring that the source model M can effectively approach the target model N in each iteration, gradually aligning the two models, restoring the global transformation and local deformation, and stopping the iteration when the difference in parameter vectors of consecutive iterations is less than a certain threshold, aligning the source model and the target model to obtain the registration result.

[0116] After the dorsal surface sub-model and the plantar imprint sub-model are aligned and spliced, a three-dimensional model of the entire foot is formed. Based on the three-dimensional model of the entire foot, the foot shape characteristics and / or the functional area characteristics of the footprint can be analyzed from multiple dimensions through machine learning methods, so as to provide data support for further generating a foot health report containing multiple indicators, wherein the multiple indicators may include: foot length, foot width, hallux valgus angle, claw toe, medial arch height, hindfoot varus angle, hindfoot valgus angle, plantar adduction / abduction angle, forefoot imprint area, medial arch imprint area, lateral arch imprint area, transverse arch imprint area, heel imprint area, etc.

[0117] See also Figure 4 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a second embodiment of the present invention, the method comprising:

[0118] S201, generating an initial three-dimensional point cloud A using a multi-view image corresponding to the foot standing scene;

[0119] S202, generating an initial three-dimensional point cloud B using multi-view images corresponding to the footprint mold scene.

[0120] Exemplarily, an initial 3D point cloud A is generated based on a multi-view image corresponding to the standing scene of the foot, and an initial 3D point cloud B is generated based on a multi-view image corresponding to the scene of the footprint mold. The initial 3D point cloud A and the initial 3D point cloud B may belong to a sparse 3D point cloud or a dense 3D point cloud, and the other sub-point clouds at least include a point set that can represent the floor.

[0121] S203, separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud.

[0122] S204, separating the plantar imprint sub-point cloud from other sub-point clouds in the initial three-dimensional point cloud B to obtain the plantar imprint sub-point cloud.

[0123] Exemplarily, the dorsal surface sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud A are separated by the point cloud segmentation model A, and the dorsal surface sub-point cloud can be extracted or / and other sub-point clouds can be removed. The plantar imprint sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud B can be separated by the point cloud segmentation model B, and the plantar imprint sub-point cloud can be extracted or / and other sub-point clouds can be removed. Either of the point cloud segmentation models A and B can be pre-trained using a deep learning model such as Pointnet++ or RandLA-Net or OneFormer3D.

[0124] S205, splicing the dorsal surface sub-point cloud and the plantar imprint sub-point cloud to form a three-dimensional point cloud of the entire foot.

[0125] It should be understood that by aligning the overlapping parts of the two point clouds so that the two form a complete point cloud, the existing point cloud alignment scheme can be adaptively applied to the alignment of the plantar imprint sub-point cloud and the dorsal surface sub-point cloud so that the two can be synthesized into a three-dimensional point cloud of the entire foot. For example, at least one of a point cloud alignment scheme based on curvature features, a point cloud alignment scheme based on the closest point iteration (ICP), and a non-rigid point cloud alignment scheme can be adopted.

[0126] For example, the principle of the point cloud registration scheme based on curvature features is: the dorsal surface sub-point cloud and the plantar imprint sub-point cloud with overlapping parts are set as the source point cloud and the target point cloud respectively. The farthest point sampling is used to obtain a uniformly distributed downsampled point cloud from the source point cloud and the target point cloud, the local curvature feature of each sampling point is calculated, the corresponding feature point pair is searched based on the feature distance, and the registration operation of sampling consistency is performed to achieve the registration of the two point clouds and obtain the three-dimensional point cloud of the whole foot.

[0127] The point cloud registration scheme based on curvature features mainly includes the following steps:

[0128] 1. Farthest point sampling:

[0129] Randomly select a point P0 from the point cloud data as the starting point, obtain the sampling set S = {P0}, and calculate all points P i Distance L to P0 i , select the farthest point as P1 and add it to S, calculate the distance from all points to P1, if it is less than the distance to P0, update L i , ensure that L i It is always the shortest distance from each point to S, and the farthest point is repeatedly selected until the number meets N.

[0130] 2. Curvature feature calculation:

[0131] For each sampling point P i , use the KD tree to find a spherical neighborhood point set U with a radius of r from the original point cloud, and traverse the points P in U j , the total number is n i , calculate the center of mass P c , construct the covariance matrix cov(P i ).

[0132]

[0133] Solve cov(P i ) to obtain three eigenvalues, λ1, λ2, λ3 (and λ1>λ2>λ3), and calculate each point P in the point cloud i The curvature eigenvalue C i .

[0134]

[0135] 3. Feature point pair search:

[0136] Construct a KD tree based on the curvature feature, and search for one or more points with similar curvature features as the sampling points in the source point cloud S from the sampling points of the target point cloud G. If the feature distance between the two is less than C d , point P in the target point cloud G gj Add source point P si The feature point set SP.

[0137] SP={P tj |C i -C j <C d}

[0138] 4. Sampling consistency registration:

[0139] For each point P si , randomly select a point from its similar feature point set SP as the only corresponding point of the point in the target point cloud, and obtain n groups of corresponding point pairs consisting of the same number of source points and target points, one by one, calculate the rigid body transformation matrix between the corresponding points, solve the error function, and judge whether the current registration transformation meets the registration requirements. The error function here uses the Huber penalty function, which is recorded as

[0140]

[0141] Among them, m i For a given threshold, d i is the distance difference after the transformation of the i-th group of corresponding points. When the value of the error function is the smallest, the transformation is considered to be optimal, which is the final registration matrix. By transforming the original point cloud, the complete foot point cloud registration result can be obtained.

[0142] Exemplarily, the principle of the point cloud registration scheme based on iterative closest point (ICP) is as follows: use the KD tree to obtain multiple groups of corresponding point pairs from the source point cloud and the target point cloud, with the smallest spatial Euclidean distance, calculate the rotation matrix R and translation vector T to transform the source point cloud, re-obtain and update the corresponding point pairs, and iterate multiple times until the sum of the squares of the distances between the corresponding point pairs of the source point cloud and the target point cloud is minimized, that is, the registration of the two point clouds is achieved, and a full three-dimensional point cloud is obtained.

[0143] The point cloud registration scheme based on the closest point iteration (ICP) mainly includes the following steps:

[0144] 1. Corresponding point pair search:

[0145] Taking the minimum spatial Euclidean distance as the criterion for measuring whether two points correspond, a KD tree is established for the target point cloud, and a subset p0 is selected from the source point cloud P. For each point p in p0 i For the search point, use the K nearest neighbor search method to quickly find the nearest target point q i , so that ‖q i -p i ‖=min, and we get n pairs of corresponding points.

[0146] 2. Transformation matrix calculation:

[0147] On the basis of least squares, the rotation matrix R and the translation vector T are calculated with the constraint that the sum of squares of the Euclidean distances F(RT) between the transformed corresponding points is minimized.

[0148]

[0149] According to the calculated transformation matrices R and T, the subset p0 is rotated and translated, and then the closest point in space is obtained from the target again based on the transformed point coordinates, and the corresponding point pairs are updated.

[0150] 3. Multiple Iterations Registration:

[0151] After iteratively obtaining corresponding point pairs and calculating the rotation and translation matrices based on the point pairs, the distance d is used to determine whether the iteration should be terminated. When d is less than the set threshold, or the difference between the distances d of two iterations is less than the threshold, the algorithm stops iterating, otherwise it will continue iterating.

[0152]

[0153] By applying the rotation and translation matrix used in each iteration to the initial source point cloud, the source point cloud and the target point cloud can be aligned to obtain a complete foot point cloud registration result.

[0154] For example, the principle of the non-rigid point cloud registration scheme is: the dorsal surface sub-point cloud and the plantar imprint sub-point cloud in the overlapping area are set as the source point cloud and the target point cloud respectively. M ) can be expressed as the centroid of the Gaussian mixture model, M is the number of model centroids, and the centroid is transformed with the target point cloud X = (x1..x N ) overlap, and the point cloud registration is transformed into a probability density estimation problem. A set of parameters are solved by the EM method to redefine the center of mass position so that it fits the target point cloud. This can achieve non-rigid registration of the two point clouds and obtain an aligned full-foot 3D point cloud.

[0155] The non-rigid point cloud registration scheme mainly includes the following steps:

[0156] 1. Gaussian mixture model:

[0157] The point cloud registration formula is transformed into a probability density estimation problem. Based on the probability density estimation function, the probability density function p(x) of the joint Gaussian mixture model is obtained. At the same time, T(y m ) represents y m The non-rigid transformation result is based on the affine transformation and adds a non-rigid deformation vector v m .

[0158] T(y m )=sR(y m +v m )+T

[0159]

[0160] Among them, M, N are the number of points in the target point cloud and the source point cloud respectively, n is the point subscript in the target point cloud, m is the point subscript in the source point cloud, D represents the dimension, which is equal to 3 here, d(x n ,T(y m )) is the Euclidean distance between two points, σ 2 For x n With y m The variance of .

[0161] 2. Model EM solution:

[0162] First simplify the Gaussian mixture model p(x) and establish the maximum likelihood function E(θ).

[0163]

[0164] The expectation maximization (EM) algorithm is used to solve θ, which is divided into the E step and the M step. According to the initial value of θ or the estimated value after iteration, the two steps are iterated until the error is less than the threshold. The solution is completed and the source point set Y is non-rigidly transformed using the solution result to match Y with X to obtain a complete foot point cloud.

[0165] It should be pointed out that since the sole of the foot cannot be photographed when the foot stands on the floor, the dorsal surface sub-point cloud cannot represent the sole of the foot, but can represent the surface of the foot and calf. Therefore, the dorsal surface sub-point cloud and the plantar imprint sub-point cloud complement each other.

[0166] See also Figure 5 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a third embodiment of the present invention, comprising:

[0167] S201, generating an initial three-dimensional point cloud A using a multi-view image corresponding to the foot standing scene;

[0168] S202, generating an initial three-dimensional point cloud B using a multi-view image corresponding to the footprint mold site;

[0169] S203, separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain a dorsal surface sub-point cloud;

[0170] S204, separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud;

[0171] S205, splicing the dorsal surface sub-point cloud and the plantar imprint sub-point cloud to form a three-dimensional point cloud of the entire foot;

[0172] S206, generating a full-foot 3D model based on the spliced ​​full-foot 3D point cloud.

[0173] S206 is specifically as follows: the three-dimensional point cloud of the entire foot can be meshed using a Poisson algorithm to obtain a three-dimensional mesh white film that can represent the complete geometric details of the foot but does not yet represent the texture details. Furthermore, the texture of the three-dimensional mesh white film can be reconstructed based on the multi-view images corresponding to the standing scene of the foot and the multi-view images corresponding to the footprint mold scene.

[0174] It should be noted that the full-foot three-dimensional point cloud should be a dense three-dimensional point cloud. If the full-foot three-dimensional point cloud is a sparse three-dimensional point cloud, the full-foot three-dimensional point cloud needs to be encrypted through the MVS algorithm and then meshed.

[0175] See also Figure 6 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a fourth embodiment of the present invention, the method comprising:

[0176] S301, performing three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A;

[0177] S302, generating an initial three-dimensional point cloud B using a multi-view image corresponding to the footprint mold scene;

[0178] S303, separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model;

[0179] S304, separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud;

[0180] S305, splicing the dorsal surface sub-model with the plantar imprint sub-point cloud to form full foot three-dimensional data.

[0181] It should be understood that the 3D reconstruction is performed based on the multi-view images corresponding to the standing scene of the foot to obtain the initial 3D model A, and the multi-view images corresponding to the footprint mold scene are used to generate the initial 3D point cloud B. Then the dorsal surface sub-model is separated from the initial 3D model A, and the plantar imprint sub-point cloud is separated from the initial 3D point cloud B. Finally, the dorsal surface sub-model and the plantar imprint sub-point cloud are spliced ​​to form the full foot 3D data.

[0182] See also Figure 7 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a fifth embodiment of the present invention, the method comprising:

[0183] S301, performing three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A;

[0184] S302, generating an initial three-dimensional point cloud B using a multi-view image corresponding to the footprint mold scene;

[0185] S303, separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model;

[0186] S304, separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud;

[0187] S305, generating a corresponding plantar imprint sub-model based on the plantar imprint sub-point cloud, and splicing the plantar imprint sub-model with the dorsal surface sub-model to form a three-dimensional model of the entire foot.

[0188] See also Figure 8 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a sixth embodiment of the present invention, the method comprising:

[0189] S401, generating an initial three-dimensional point cloud A using a multi-view image corresponding to the foot standing scene;

[0190] S402, performing three-dimensional reconstruction using multi-view images corresponding to the footprint mold scene to obtain an initial three-dimensional model B;

[0191] S403, separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain a dorsal surface sub-point cloud;

[0192] S404, separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model;

[0193] S405, splicing the dorsal surface sub-point cloud with the plantar imprint sub-model to form full foot three-dimensional data.

[0194] It should be understood that the 3D reconstruction is performed based on the multi-view images corresponding to the standing scene of the foot to obtain the initial 3D point cloud A, and the initial 3D model B is generated based on the multi-view images corresponding to the footprint mold scene. Then the dorsal surface sub-point cloud is separated from the initial 3D point cloud A, and the plantar imprint sub-model is separated from the initial 3D model B. Finally, the dorsal surface sub-point cloud and the plantar imprint sub-model are spliced ​​to form the full foot 3D data.

[0195] See also Fig. 9 , provides a method for three-dimensional reconstruction of a foot based on multi-view images according to a seventh embodiment of the present invention, the method comprising:

[0196] S401, performing three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional point cloud A;

[0197] S402, generating an initial three-dimensional model B using multi-view images corresponding to the footprint mold scene;

[0198] S403, separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain a dorsal surface sub-point cloud;

[0199] S404, separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model;

[0200] S405, generating a corresponding dorsal surface sub-model based on the dorsal surface sub-point cloud; splicing the dorsal surface sub-model with the plantar imprint sub-model to form a three-dimensional model of the entire foot.

[0201] See also Fig.10 , provides a foot three-dimensional reconstruction system based on multi-view images according to embodiment eight of the present invention, including a reconstruction module 501, a separation module 502 and a splicing module 503.

[0202] The reconstruction module 501 is used to perform three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet; and is also used to perform three-dimensional reconstruction using multi-view images corresponding to the footprint mold scene to obtain an initial three-dimensional model B, wherein the multi-view images corresponding to the footprint mold scene are obtained by shooting the double footprints in a footprint box from multiple perspectives, and the double footprints in the footprint box are left by a single person standing with both feet in the footprint box.

[0203] The separation module 502 is used to separate the dorsal surface submodel from other submodels in the initial three-dimensional model A to obtain the dorsal surface submodel; and is also used to separate the plantar imprint submodel from other submodels in the initial three-dimensional model B to obtain the plantar imprint submodel.

[0204] The splicing module 503 is used to splice the dorsal surface sub-model and the plantar imprint sub-model to form a three-dimensional model of the entire foot.

[0205] It can be understood that the three-dimensional foot reconstruction system based on multi-view images provided in Example 8 of the present invention corresponds to the three-dimensional foot reconstruction method based on multi-view images provided in the aforementioned Example 1. The relevant technical features of the three-dimensional foot reconstruction system based on multi-view images in Example 8 can refer to the relevant technical features of the three-dimensional foot reconstruction based on multi-view images in the aforementioned Example 1, which will not be repeated here.

[0206] In the ninth embodiment of the present invention, the reconstruction module 501 is used to generate an initial three-dimensional point cloud A using a multi-view image corresponding to the foot standing scene; and to generate an initial three-dimensional point cloud B using a multi-view image corresponding to the footprint mold scene;

[0207] The separation module 502 is used to separate the dorsal surface sub-point cloud from other sub-point clouds in the initial three-dimensional point cloud A to obtain the dorsal surface sub-point cloud; and to separate the plantar imprint sub-point cloud from other sub-point clouds in the initial three-dimensional point cloud B to obtain the plantar imprint sub-point cloud;

[0208] The stitching module 503 is used to stitch the dorsal surface sub-point cloud and the plantar imprint sub-point cloud to form a three-dimensional point cloud of the entire foot.

[0209] It can be understood that the three-dimensional foot reconstruction system based on multi-view images provided in Example 9 of the present invention corresponds to the three-dimensional foot reconstruction method based on multi-view images provided in the aforementioned Example 2. The relevant technical features of the three-dimensional foot reconstruction system based on multi-view images in Example 9 can refer to the relevant technical features of the three-dimensional foot reconstruction method based on multi-view images in the aforementioned Example 2, which will not be repeated here.

[0210] In the tenth embodiment of the present invention, the reconstruction module 501 is used to perform three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A; and to generate an initial three-dimensional point cloud B using multi-view images corresponding to the footprint mold scene.

[0211] The separation module 502 is used to separate the dorsal surface sub-model from other sub-models in the initial three-dimensional model A to obtain the dorsal surface sub-model; and to separate the plantar imprint sub-point cloud from other sub-point clouds in the initial three-dimensional point cloud B to obtain the plantar imprint sub-point cloud.

[0212] The splicing module 503 is used to splice the dorsal surface sub-model with the plantar imprint sub-point cloud to form full-foot three-dimensional data.

[0213] It can be understood that the three-dimensional foot reconstruction system based on multi-view images provided in the tenth embodiment of the present invention corresponds to the three-dimensional foot reconstruction method based on multi-view images provided in the aforementioned fourth embodiment. The relevant technical features of the three-dimensional foot reconstruction system based on multi-view images in the tenth embodiment can refer to the relevant technical features of the three-dimensional foot reconstruction method based on multi-view images in the aforementioned fourth embodiment, which will not be repeated here.

[0214] In the eleventh embodiment of the present invention, the reconstruction module 501 is used to perform three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional point cloud A; and to generate an initial three-dimensional model B using multi-view images corresponding to the footprint mold scene.

[0215] The separation module 502 is used to separate the dorsal surface sub-point cloud from other sub-point clouds in the initial three-dimensional point cloud A to obtain the dorsal surface sub-point cloud; and to separate the plantar imprint sub-model from other sub-models in the initial three-dimensional model B to obtain the plantar imprint sub-model.

[0216] The splicing module 503 is used to splice the dorsal surface sub-point cloud with the plantar imprint sub-model to form full-foot three-dimensional data.

[0217] It can be understood that the three-dimensional foot reconstruction system based on multi-view images provided in the eleventh embodiment of the present invention corresponds to the three-dimensional foot reconstruction method based on multi-view images provided in the aforementioned sixth embodiment. The relevant technical features of the three-dimensional foot reconstruction system based on multi-view images in the eleventh embodiment can refer to the relevant technical features of the three-dimensional foot reconstruction method based on multi-view images in the aforementioned sixth embodiment, which will not be repeated here.

[0218] The method and system for reconstructing a foot based on multi-view images provided in an embodiment of the present invention obtain a data portion representing the dorsal surface of the foot from the three-dimensional data representing the scene of the footprint mold, and obtain a data portion representing the plantar imprint from the three-dimensional data representing the scene of the foot standing, so that other redundant data portions are separated, and the data portion representing the plantar imprint and the data portion representing the dorsal surface of the foot are integrated and spliced, so that the foot can be fully represented in three dimensions. Compared with the existing three-dimensional data that can only represent the dorsal surface of the foot, the foot data can be expressed richer and more realistically.

[0219] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0220] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0221] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for three-dimensional reconstruction of a foot based on multi-view images, characterized in that: include: Performing three-dimensional reconstruction using multi-view images corresponding to a foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet; The multi-view images corresponding to the footprint mold scene are used for three-dimensional reconstruction to obtain an initial three-dimensional model B; Separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model; Separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model; The dorsal surface sub-model and the plantar imprint sub-model are spliced ​​to form a three-dimensional model of the entire foot.

2. The method for three-dimensional reconstruction of a foot based on multi-view images according to claim 1, characterized in that: The three-dimensional reconstruction using multi-view images corresponding to the foot standing scene to obtain an initial three-dimensional model A includes: Using a camera device to shoot at multiple angles around the two feet or one foot of a person standing on the floor, to obtain multi-view dorsal image data; Based on the multi-view dorsal imaging data, an initial three-dimensional model A is constructed.

3. The method for three-dimensional reconstruction of a foot based on multi-view images according to claim 1, characterized in that: The three-dimensional reconstruction using multi-view images corresponding to the footprint mold scene to obtain an initial three-dimensional model B includes: Based on the camera device, multi-angle shooting is performed around a footprint box with double-foot prints to obtain multi-angle sole image data, wherein the double-foot prints are left by a single person standing in the footprint box with two feet; Based on the multi-view plantar imaging data, an initial three-dimensional model B is constructed.

4. A method for three-dimensional reconstruction of a foot based on multi-view images, characterized in that: include: Generate an initial three-dimensional point cloud A using multi-view images corresponding to the foot standing scene; An initial three-dimensional point cloud B is generated using multi-view images corresponding to the footprint mold site; Separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud; Separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud; The dorsal surface sub-point cloud and the plantar imprint sub-point cloud are spliced ​​to form a three-dimensional point cloud of the entire foot.

5. The method for three-dimensional reconstruction of a foot based on multi-view images according to claim 4, characterized in that: The step of separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud comprises: Segment the dorsal surface sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud A by using a pre-trained point cloud segmentation model A, extract the dorsal surface sub-point cloud and / or delete the other sub-point clouds; The step of separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud comprises: The plantar imprint sub-point cloud and other sub-point clouds in the initial three-dimensional point cloud B are segmented by using a pre-trained point cloud segmentation model B, and the plantar imprint sub-point cloud is extracted and / or other sub-point clouds are deleted.

6. A method for three-dimensional reconstruction of a foot based on multi-view images, characterized in that: include: The multi-view images corresponding to the foot standing scene are used for three-dimensional reconstruction to obtain an initial three-dimensional model A; An initial three-dimensional point cloud B is generated using multi-view images corresponding to the footprint mold site; Separating the dorsal surface sub-model of the foot in the initial three-dimensional model A from other sub-models to obtain the dorsal surface sub-model; Separating the plantar imprint sub-point cloud in the initial three-dimensional point cloud B from other sub-point clouds to obtain the plantar imprint sub-point cloud; The dorsal surface sub-model is spliced ​​with the plantar imprint sub-point cloud to form full-foot three-dimensional data.

7. The method for three-dimensional reconstruction of a foot based on multi-view images according to claim 6, characterized in that: The step of splicing the dorsal surface sub-model with the plantar imprint sub-point cloud to form full-foot three-dimensional data includes: generating a corresponding plantar imprint sub-model based on the plantar imprint sub-point cloud; The plantar imprint sub-model and the dorsal surface sub-model are spliced ​​together to form a three-dimensional model of the entire foot.

8. A method for three-dimensional reconstruction of a foot based on multi-view images, characterized in that: include: Generate an initial three-dimensional point cloud A using multi-view images corresponding to the foot standing scene; The multi-view images corresponding to the footprint mold scene are used for three-dimensional reconstruction to obtain an initial three-dimensional model B; Separating the dorsal surface sub-point cloud in the initial three-dimensional point cloud A from other sub-point clouds to obtain the dorsal surface sub-point cloud; Separating the plantar imprint sub-model in the initial three-dimensional model B from other sub-models to obtain the plantar imprint sub-model; The dorsal surface sub-point cloud is spliced ​​with the plantar imprint sub-model to form full-foot three-dimensional data.

9. The method for three-dimensional reconstruction of a foot based on multi-view images according to claim 8, characterized in that: The step of splicing the dorsal surface sub-point cloud with the plantar imprint sub-model to form full-foot three-dimensional data includes: generating a corresponding dorsal surface sub-model based on the dorsal surface sub-point cloud; The dorsal surface sub-model is spliced ​​with the plantar imprint sub-model to form a three-dimensional model of the entire foot.

10. A foot 3D reconstruction system based on multi-view images, characterized in that: include: A reconstruction module, for performing three-dimensional reconstruction using multi-view images corresponding to a foot standing scene to obtain an initial three-dimensional model A, wherein the foot standing scene refers to a single person standing on the ground with both feet; and for performing three-dimensional reconstruction using multi-view images corresponding to a footprint mold scene to obtain an initial three-dimensional model B; A separation module, used for separating the dorsal surface sub-model in the initial three-dimensional model A from other sub-models; It is also used to separate the plantar imprint sub-model from other sub-models in the initial three-dimensional model B; The splicing module is used to splice the dorsal surface sub-model and the plantar imprint sub-model to form a three-dimensional model of the entire foot.

Images