A biological model construction method and a three-dimensional size measurement method

Abstract

The application provides a biological model construction method and device, electronic equipment and storage medium, which are used for improving the problem that the accuracy of obtaining a biological model is low. The method comprises the following steps: acquiring a plurality of grid topologies, the plurality of grid topologies are obtained by three-dimensional scanning and modeling of a real biological part, and any two of the plurality of grid topologies have different postures and shapes; acquiring a template topology, and performing normalized alignment on the postures and scales of the plurality of grid topologies according to the template topology to obtain a plurality of shape topologies with the same posture and scale; performing average value calculation on the plurality of shape topologies to obtain a mean value topology, and extracting a feature vector of each shape topology in the plurality of shape topologies to obtain a plurality of feature vectors; and constructing a parameterized biological model according to the mean value topology and the plurality of feature vectors.

Description

Technical Field

[0001] This application relates to the technical fields of computer vision and computer graphics, and more specifically, to a method, apparatus, electronic device, and storage medium for constructing biological models. Background Technology

[0002] Currently, point cloud data is obtained by using depth cameras or 3D cameras to scan the feet of real users in 3D, or by using laser scanning technology. Then, a biological model is created based on this point cloud data. However, in practice, it has been found that the point cloud data obtained from 3D scanning or laser scanning contains many noisy or fragmented points, resulting in low accuracy of the obtained biological model. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, electronic device and storage medium for constructing biological models, in order to improve the problem of low accuracy in obtaining biological models.

[0004] In a first aspect, embodiments of this application provide a method for constructing a biological model, comprising: acquiring multiple mesh topologies, wherein the multiple mesh topologies are obtained by three-dimensional scanning and modeling of real biological parts, and any two of the multiple mesh topologies have different poses and shapes; acquiring a template topology, and normalizing and aligning the poses and scales of the multiple mesh topologies according to the template topology to obtain multiple shape topologies with the same pose and scale; calculating the average value of the multiple shape topologies to obtain a mean topology, and extracting the feature vector of each shape topology to obtain multiple feature vectors; and constructing a parameterized biological model based on the mean topology and the multiple feature vectors.

[0005] In the above implementation process, the pose and scale of multiple mesh topologies obtained from 3D scanning and modeling are normalized and aligned to obtain multiple shape topologies with the same pose and scale. Then, a parametric biological model is constructed based on the mean topology of multiple shape topologies and multiple feature vectors. This effectively avoids the situation where low-precision biological models are obtained due to directly using point cloud data including noise points. Furthermore, normalizing and aligning the pose and scale of multiple mesh topologies can avoid the impact of misaligned pose and scale on the biological model, thereby effectively improving the accuracy of the obtained biological model.

[0006] In one implementation of the first aspect, the pose and scale of multiple mesh topologies are normalized and aligned according to the template topology to obtain multiple shape topologies with the same pose and scale. This includes: performing a rigid transformation on the mesh topologies according to the template topology to obtain the rigid transformation parameters of the mesh topologies; performing pose alignment on the mesh topologies according to the rigid transformation parameters of the mesh topologies to obtain the pose-aligned mesh topologies; and performing a non-rigid transformation and nearest-neighbor point alignment on the pose-aligned mesh topologies under the constraint of the landmark points of the template topology and the landmark points of the mesh topologies to obtain the shape topologies.

[0007] In the above implementation process, the mesh topology is aligned according to the rigid transformation parameters of the mesh topology to obtain the aligned mesh topology. Under the constraint of the landmark points of the template topology and the landmark points of the mesh topology, the aligned mesh topology is subjected to non-rigid transformation and nearest neighbor point alignment, thereby avoiding the situation of no alignment, non-rigid transformation or nearest neighbor point alignment, and effectively improving the accuracy of obtaining the shape topology.

[0008] In one implementation of the first aspect, a non-rigid transformation and nearest-neighbor alignment are performed on the orientation-aligned mesh topology to obtain a shape topology, including: performing a non-rigid transformation on the orientation-aligned mesh topology using a non-rigid iterative nearest-neighbor algorithm to obtain a transformed shape topology; and performing nearest-neighbor alignment on the transformed shape topology to obtain a shape topology.

[0009] In the above implementation process, a non-rigid iterative nearest-neighbor algorithm is used to perform a non-rigid transformation on the mesh topology after orientation alignment to obtain the transformed shape topology; and the nearest-neighbor alignment is performed on the transformed shape topology, thereby avoiding the situation where there is no non-rigid transformation or nearest-neighbor alignment, and effectively improving the accuracy of obtaining the shape topology.

[0010] In one implementation of the first aspect, after constructing a parameterized biological model based on the mean topology and multiple eigenvectors, the method further includes: performing three-dimensional reconstruction of the parameterized biological model to obtain a reconstructed biological model; and measuring the reconstructed biological model to obtain the three-dimensional dimensions of the organism parts.

[0011] Secondly, embodiments of this application provide a three-dimensional size measurement method, comprising: acquiring multiple images to be processed, and performing semantic segmentation on biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines, wherein the images to be processed are obtained by taking pictures of biological parts from multiple different angles; acquiring a parametric biological model using the biological model construction method described above, projecting the parametric biological model from multiple angles to obtain the contour line projected onto the parametric biological model from each of the multiple angles; performing three-dimensional reconstruction of the parametric biological model based on the distance loss value between the image segmentation line corresponding to each of the multiple angles and the contour line projected onto the parametric biological model from each angle to obtain a reconstructed biological model; and measuring the reconstructed biological model to obtain the three-dimensional size of the biological parts. In the above implementation process, the parameterized biological model is reconstructed in three dimensions based on the distance loss value between each image segmentation line and the corresponding contour line in multiple image segmentation lines, and the reconstructed biological model is measured. This effectively avoids the situation where there are many noise points in the point cloud data. Instead, the distance loss value required for fitting is calculated by using the foot outer contour image segmentation line with very few noise points and the foot contour line determined by the model, thereby effectively improving the accuracy of the three-dimensional dimensions obtained by measuring the reconstructed foot model.

[0012] In one implementation of the second aspect, the biological part is the human foot; before projecting the parameterized biological model from multiple angles to obtain the outline of the parameterized biological model, the method further includes: identifying the coordinates of multiple marker points around the human foot in each of the multiple images to be processed, the coordinates of the multiple marker points being set on the test paper stepped on by the human foot; for each of the multiple images to be processed, determining the camera parameters of each image to be processed based on the coordinates of the multiple marker points, and reconstructing the projection angle of the parameterized biological model onto the plane corresponding to each image to be processed based on the camera parameters of each image to be processed, thereby obtaining multiple angles.

[0013] In the above implementation process, by identifying the coordinates of multiple marker points around the human foot in each of the multiple images to be processed, and then determining the camera parameters of each image to be processed based on the coordinates of the multiple marker points, and reconstructing the projection angle of the parameterized biological model onto the corresponding plane of each image to be processed based on the camera parameters of each image to be processed, the problem of poor model accuracy caused by the difficulty in reconstructing the projection angle and the reliance on point cloud data fitting is avoided, thereby effectively improving the modeling accuracy of the parameterized biological model.

[0014] In one implementation of the second aspect, the parametric biological model is reconstructed in three dimensions based on the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parametric biological model from each angle, thereby obtaining a reconstructed biological model. This includes: adjusting the rigid transformation parameters and / or fusion deformation parameters of the parametric biological model according to camera parameters to change the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parametric biological model from each angle; and determining the parametric biological model with a distance loss value less than a preset threshold as the reconstructed biological model after three-dimensional reconstruction.

[0015] In the above implementation process, the reconstructed biological model after 3D reconstruction is determined by changing the distance loss value between each image segmentation line and the contour line corresponding to each image segmentation line. This avoids the problem of poor model accuracy when using point cloud data fitting, and thus effectively improves the modeling accuracy of the parametric biological model.

[0016] Thirdly, embodiments of this application provide a biological model construction apparatus, comprising: a mesh topology acquisition module for acquiring multiple mesh topologies, wherein the multiple mesh topologies are obtained by three-dimensional scanning and modeling of real biological parts, and any two of the multiple mesh topologies have different poses and shapes; a shape topology acquisition module for acquiring a template topology, and normalizing and aligning the poses and scales of the multiple mesh topologies according to the template topology to obtain multiple shape topologies with the same pose and scale; a feature vector acquisition module for calculating the average value of the multiple shape topologies to obtain a mean topology, and extracting the feature vector of each shape topology to obtain multiple feature vectors; and a biological model construction module for constructing a parameterized biological model based on the mean topology and the multiple feature vectors.

[0017] In one implementation of the third aspect, the shape topology acquisition module includes: a transformation parameter acquisition module, used to perform a rigid transformation on the mesh topology based on the template topology to obtain the rigid transformation parameters of the mesh topology; a mesh topology alignment module, used to perform attitude alignment on the mesh topology based on the rigid transformation parameters of the mesh topology to obtain the attitude-aligned mesh topology; and a topology transformation alignment module, used to perform a non-rigid transformation and nearest neighbor point alignment on the attitude-aligned mesh topology under the constraint of the landmark points of the template topology and the landmark points of the mesh topology to obtain the shape topology.

[0018] In one implementation of the third aspect, the topology transformation alignment module includes: a topology non-rigid transformation module, used to perform a non-rigid transformation on the mesh topology after attitude alignment using a non-rigid iterative nearest point algorithm to obtain the transformed shape topology; and a nearest neighbor alignment module, used to perform nearest neighbor alignment on the transformed shape topology to obtain the shape topology.

[0019] In one implementation of the third aspect, the biological model construction device further includes: a biological model reconstruction module for performing three-dimensional reconstruction of the parameterized biological model to obtain a reconstructed biological model; and a three-dimensional dimension acquisition module for measuring the reconstructed biological model to obtain the three-dimensional dimensions of the biological parts.

[0020] Fourthly, embodiments of this application provide a three-dimensional size measurement device, comprising: an image acquisition module for acquiring multiple images to be processed and performing semantic segmentation on biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines, wherein the images to be processed are obtained by taking pictures of biological parts from multiple different angles; a biological model projection module for acquiring a parametric biological model using the biological model construction method described above, and projecting the parametric biological model from multiple angles to obtain the contour line projected onto the parametric biological model from each of the multiple angles; a three-dimensional model reconstruction module for performing three-dimensional reconstruction of the parametric biological model based on the distance loss value between the image segmentation line corresponding to each of the multiple angles and the contour line projected onto the parametric biological model from each angle to obtain a reconstructed biological model; and a three-dimensional model measurement module for measuring the reconstructed biological model to obtain the three-dimensional dimensions of the biological parts.

[0021] In one implementation of the fourth aspect, the biological part is the human foot; the three-dimensional size measuring device further includes: a marker coordinate recognition module, used to identify the coordinates of multiple marker points around the human foot in each of the multiple images to be processed, the multiple marker point coordinates being set on the test paper stepped on by the human foot; and a projection angle restoration module, used to determine the camera parameters of each image to be processed based on the coordinates of the multiple marker points, and restore the projection angle of the parameterized biological model onto the plane corresponding to each image to be processed based on the camera parameters of each image to be processed, thereby obtaining multiple angles.

[0022] In one implementation of the fourth aspect, the model 3D reconstruction module includes: a model parameter adjustment module, used to adjust the rigid transformation parameters and / or fusion deformation parameters of the parameterized biological model according to the camera parameters, so as to change the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parameterized biological model at each angle; and a biological model determination module, used to determine the parameterized biological model with a distance loss value less than a preset threshold as the reconstructed biological model after 3D reconstruction.

[0023] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method described above.

[0024] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the method described above. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 The diagram shown is a flowchart illustrating the biological model construction method provided in an embodiment of this application.

[0027] Figure 2 The diagram shows a grid topology diagram of six human feet provided in an embodiment of this application;

[0028] Figure 3 The diagram shown is a comparison of the pose alignment before and after the alignment provided in the embodiments of this application;

[0029] Figure 4 The diagram shown is a planar schematic of one type of test paper provided in an embodiment of this application;

[0030] Figure 5 The diagram shown is a flowchart illustrating the three-dimensional dimension measurement method provided in an embodiment of this application.

[0031] Figure 6 The illustration shows a schematic diagram of photographing the human foot from different angles, provided in an embodiment of this application.

[0032] Figure 7 The diagram shown is a schematic representation of one type of measuring spindle provided in an embodiment of this application;

[0033] Figure 8 The diagram shows a schematic of the structure of the biological model construction device provided in the embodiments of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0035] It is understood that the terms "first" and "second" in the embodiments of this application are used to distinguish similar objects. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" are not necessarily different.

[0036] Before introducing the biological model construction method provided in the embodiments of this application, let's first introduce some concepts involved in the embodiments of this application:

[0037] Computer vision is a science that studies how to enable machines to "see," that is, to use cameras and computers to replace human eyes to identify, track and measure targets, and to further process the images into images that are more suitable for human eyes to observe or to be transmitted to instruments for detection.

[0038] Principal Component Analysis (PCA), also known as principal component analysis, is a statistical analysis and dataset simplification method in multivariate statistical analysis. PCA uses orthogonal transformations to linearly transform the observations of a series of potentially related variables, projecting them onto a series of linearly unrelated variables.

[0039] Singular Value Decomposition (SVD) is an important matrix decomposition in linear algebra. In some respects, SVD is similar to the diagonalization of symmetric or Hermitian matrices based on eigenvectors. However, although these two matrix decompositions are related, they are still significantly different. The eigenvector decomposition of symmetric matrices is based on spectral analysis, while SVD is a generalization of spectral analysis theory to arbitrary matrices.

[0040] It should be noted that the biological model construction method provided in this application can be executed by an electronic device. Here, an electronic device refers to a device terminal or server with the function of executing computer programs. Device terminals include, for example, smartphones, personal computers, tablets, personal digital assistants, or mobile internet devices. A server refers to a device that provides computing services through a network. Servers include, for example, x86 servers and non-x86 servers. Non-x86 servers include, for example, mainframes, minicomputers, and UNIX servers.

[0041] The following describes the applicable scenarios for this biological model construction method. These scenarios include, but are not limited to, 3D scanning and modeling of real biological body parts (such as human feet, hands, heads, or waists) to obtain multiple mesh topologies. For ease of understanding and explanation, the human foot will be used as an example for detailed explanation. Then, this biological model construction method can be used to construct a parametric biological model, and the 3D dimensions of this parametric biological model can be measured using a 3D dimensional measurement method, thereby improving the accuracy of the 3D dimensions obtained from measuring the parametric biological model.

[0042] Please see Figure 1 The illustrated flowchart illustrates the biological model construction method provided in this application embodiment. The main idea of ​​this method is to normalize and align the poses and scales of multiple mesh topologies obtained through 3D scanning and modeling to obtain multiple shape topologies with the same pose and scale. Then, a parametric biological model is constructed based on the mean topology of the multiple shape topologies and multiple feature vectors. This effectively avoids the situation where directly using point cloud data including noise points leads to a low-precision biological model. Furthermore, normalizing and aligning the poses and scales of multiple mesh topologies avoids the impact of misaligned poses and scales on the biological model, thereby effectively improving the accuracy of the obtained biological model. Specifically, the above-mentioned biological model construction method may include:

[0043] Step S110: Obtain multiple mesh topologies. These multiple mesh topologies are obtained by 3D scanning and modeling of real biological parts. Any two mesh topologies have different poses and shapes.

[0044] Please see Figure 2 The diagram shows six mesh topologies of human feet provided in this embodiment of the application. These six mesh topologies can be viewed as variations in foot model shape in different principal directions based on a mean topology (a model composed of the average value of all mesh topologies). These mesh topologies can be expressed using the following formula: Where mean represents the mean topology, vec i Let val represent the i-th eigenvector. iThis represents the fusion deformation parameter (also known as the fusion deformation coefficient or BlendShape coefficient) corresponding to the i-th feature vector.

[0045] An example implementation of step S110 above is as follows: The aforementioned real biological body part can be a real user's foot. Therefore, 3D scanning and 3D modeling of the human foot can be performed from different angles to obtain multiple mesh topologies from different angles. These multiple mesh topologies are then stored in a file system. Any two of these mesh topologies have different poses and shapes. When multiple mesh topologies are needed, they can be retrieved from the file system, which can include file systems such as GFS, HDFS, Ceph, Lustre, MogileFS, MooseFS, FastDFS, TFS, and GridFS.

[0046] Step S120: Obtain the template topology, and normalize and align the pose and scale of multiple mesh topologies according to the template topology to obtain multiple shape topologies with the same pose and scale.

[0047] The specific implementation of step S120 above may include:

[0048] Step S121: Obtain the template topology and perform a rigid transformation on the mesh topology based on the template topology to obtain the rigid transformation parameters of the mesh topology.

[0049] The implementation of step S121 above can be exemplified by: obtaining a template topology pre-designed by the designer from the file system, and performing a rigid transformation on the mesh topology based on the template topology. Specifically, multiple 3D point coordinates are marked on the template topology to form a first set of point coordinates, and multiple corresponding 3D point coordinates are also marked on the corresponding positions on the mesh topology to form a second set of point coordinates. Singular value decomposition (SVD) can then be performed on the first set of point coordinates and the corresponding second set of point coordinates to obtain the rigid transformation parameters of the mesh topology. These rigid transformation parameters can include rotation parameters, scaling parameters, and translation parameters. In practice, after obtaining the template topology, it can also be preprocessed. For example, the template topology can be folded, folding the right foot's template topology to match the posture of the left foot. Landmark points can also be set at key or semantically significant locations on the template topology. These landmark points can then be used to guide the model's deformation.

[0050] Step S122: Align the mesh topology with attitude according to the rigid transformation parameters of the mesh topology to obtain the attitude-aligned mesh topology.

[0051] Please see Figure 3The illustration shows a comparison diagram before and after posture alignment provided in the embodiments of this application; the implementation of step S122 above is as follows: considering that the direction and size of the real human foot stepping on the test paper will vary from person to person, a rigid transformation parameter can be added to the parameterized biological model. The rigid transformation parameter can characterize the rigid transformation of the real human foot and the template topology, such as enlargement or reduction, tilting and offset. The parameterized biological model with added rigid transformation parameter can be expressed by the following formula: Among them, rigid rot The rigid parameter represents the rotation parameter in the rigid transformation parameters. scale This refers to the scaling parameter in the rigid transformation parameters, rigid. trans This represents the translation parameter in the rigid transformation parameters, where mean represents the mean topology, and vec represents the mean topology. i Let val represent the i-th eigenvector. i This represents the fusion deformation parameter (also known as the fusion deformation coefficient or BlendShape coefficient) corresponding to the i-th eigenvector. Therefore, the size, angle, and position of a real human foot can be more accurately represented based on the rigid transformation parameters of the mesh topology, resulting in a more precise shape topology.

[0052] Step S123: Under the constraints of the landmark points of the template topology and the landmark points of the mesh topology, perform non-rigid transformation and nearest neighbor point alignment on the orientation-aligned mesh topology to obtain the shape topology.

[0053] The implementation of step S123 above may include: under the constraint of the landmark points of the template topology and the landmark points of the mesh topology, a non-rigid iterative nearest-point algorithm can be used to perform a non-rigid transformation on the orientation-aligned mesh topology to obtain the transformed shape topology. Here, the first energy equation can be designed as E1 = w0 * smooth + w1 * landmark; where E1 represents the first energy equation, w0 represents the weight parameter of the non-rigid transformation, smooth represents the smooth transformation requirement in the model deformation, w1 represents the weight parameter constrained by the landmark points of the template topology and the mesh topology, and landmark represents the transformation requirement of the model's key point positions under the constraint of the landmark points of the template topology and the mesh topology.

[0054] Then, the first energy equation is transformed to obtain the second energy equation. Based on the second energy equation, the transformed shape topology is aligned with its nearest neighbors to obtain the final shape topology. The second energy equation can be expressed as E2 = w0*smooth + w1*landmark + w2*closest; where E2 represents the second energy equation, w0 represents the weight parameters of the non-rigid transformation, smooth represents the non-rigid transformation operation, w1 represents the weight parameters constrained by the landmark points of the template topology and the landmark points of the mesh topology, landmark represents the rigid transformation operation constrained by the landmark points of the template topology and the mesh topology, w2 represents the weight parameters of the nearest neighbor alignment, and closest represents the nearest neighbor alignment operation. Under the constraint of the nearest neighbor alignment operation, the weight of the nearest neighbor part can be gradually increased. After several iterations, the model will increasingly resemble the shape of the scanned data.

[0055] Step S130: Calculate the average value of multiple shape topologies to obtain the mean topology, and extract the feature vector of each shape topology from the multiple shape topologies to obtain multiple feature vectors.

[0056] For example, the implementation of step S130 above involves: calculating the average value of multiple shape topologies to obtain the mean topology. Assuming there are 1000 shape topologies, principal component analysis (PCA) feature compression can be performed on these 1000 shape topologies with the same orientation at a certain preset ratio (e.g., one-tenth), resulting in 100 compressed feature vectors. These feature vectors express several main directions of the human foot shape transformation (e.g., ...). Figure 2 The document shows six main directions, but there are many more main directions in practice.

[0057] Step S140: Construct a parameterized biological model based on the mean topology and multiple feature vectors.

[0058] For example, the implementation of step S140 above can be described as follows: Considering the different projection angles of the model (i.e., the projection angles corresponding to each image to be processed are different), the process of taking a picture to obtain the image to be processed can be regarded as the process of projecting the model onto the plane of the test paper. Therefore, projecting the model onto the plane of the test paper is a process of transforming the model from the world coordinate system to the pixel coordinate system. Finally, the parameterized biological model constructed based on the mean topology and multiple feature vectors can be expressed by the formula:

[0059]

[0060] Among them, framerot This indicates the rotation parameter of the camera that took the photo to be processed; frame trans The rigid parameter represents the translation parameter of the camera that took the photo to be processed. rot The rigid parameter represents the rotation parameter in the rigid transformation parameters. scale This refers to the scaling parameter in the rigid transformation parameters, rigid. trans This represents the translation parameter in the rigid transformation parameters, where mean represents the mean topology, and vec represents the mean topology. i Let val represent the i-th eigenvector. i This represents the fusion deformation parameter (also known as the fusion deformation coefficient or BlendShape coefficient) corresponding to the i-th eigenvector. Therefore, the pose (e.g., topology, angle, and position) and scale of multiple mesh topologies can be normalized and aligned according to the rigid transformation parameters of the mesh topology to obtain multiple shape topologies with the same pose.

[0061] Please see Figure 4 The diagram shows a planar schematic of one type of test paper provided in an embodiment of this application; optionally, after constructing a parameterized biological model, post-processing operations (such as model repair and posture transformation) can be performed on the parameterized biological model.

[0062] Optionally, after constructing the parameterized biological model or performing post-processing, the parameterized biological model can be used for 3D dimension measurement. The specific process of 3D dimension measurement may include: having a real user stand on the test paper (i.e., the human foot standing on the test paper) and taking photos of the human foot from different angles to obtain multiple images to be processed. The markers on the test paper can be found in [reference needed]. Figure 4 The system uses six markers, such as N, P, Q, O, Z, and I, representing six different directions. Then, semantic segmentation is performed on the biological parts in each of the multiple images to be processed, obtaining multiple image segmentation lines. These images are obtained by capturing the biological parts from multiple different angles. A parametric biological model is obtained using the biological model construction method described above. This model is then projected from multiple angles, obtaining the contour line projected onto the model from each angle. Based on the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the model, the parametric biological model is reconstructed in 3D, resulting in a reconstructed biological model. Finally, the reconstructed biological model is measured to obtain the 3D dimensions of the biological parts.

[0063] In the above implementation process, the parametric biological model is reconstructed in three dimensions by using the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parametric biological model at each angle. The reconstructed biological model is then measured. This effectively avoids the situation where there are many noise points in the point cloud data. Instead, the distance loss value required for fitting is calculated using the foot outer contour image segmentation line with very few noise points and the foot contour line determined by the model. This effectively improves the accuracy of the three-dimensional dimensions obtained by measuring the reconstructed foot model.

[0064] Please see Figure 5 The illustrated flowchart shows a three-dimensional size measurement method provided in an embodiment of this application. Optionally, after constructing the parameterized biological model above, the parameterized biological model constructed above can also be used for three-dimensional size measurement. The three-dimensional size measurement method here may specifically include:

[0065] Step S210: Acquire multiple images to be processed, and perform semantic segmentation on the biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines.

[0066] The aforementioned biological body parts can be human feet, hands, toes, head, or waist, etc. For ease of understanding and explanation, the following will use human feet as an example. The images to be processed are obtained by taking pictures of the biological body parts from different angles.

[0067] Please see Figure 6 The illustration shows a schematic diagram of taking pictures of a human foot from different angles according to an embodiment of this application; the implementation of step S210 above is, for example, having a real user stand on the test paper (i.e., the human foot is standing on the test paper) and taking pictures of the human foot from different angles to obtain multiple images to be processed, and performing semantic segmentation on the biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines.

[0068] After step S210, step S220 is executed: a parameterized biological model is obtained using the biological model construction method described above, and the parameterized biological model is projected from multiple angles to obtain the contour line projected onto the parameterized biological model from each of the multiple angles.

[0069] The specific implementation of step S220 above may include: before projecting the parameterized biological model from multiple angles, the projection relationship angle between the parameterized biological model and the plane corresponding to each image to be processed can be restored first. For example, the coordinates of multiple marker points around the human foot in each of the multiple images to be processed can be identified, and these marker point coordinates are set on the test paper where the human foot is stepping. For each of the multiple images to be processed, the camera parameters of each image to be processed are determined based on the coordinates of the multiple marker points, and the projection relationship angle between the parameterized biological model and the plane corresponding to each image to be processed is restored based on the camera parameters of each image to obtain multiple angles.

[0070] After step S220, step S230 is executed: based on the distance loss value between the image segmentation line corresponding to each of the multiple angles and the contour line projected onto the parameterized biological model by each angle, the parameterized biological model is reconstructed in three dimensions to obtain the reconstructed biological model.

[0071] The specific implementation of step S230 above may include:

[0072] Step S231: Adjust the rigid transformation parameters and / or fusion deformation parameters of the parameterized biological model according to the camera parameters to change the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parameterized biological model at each angle.

[0073] The process of adjusting the parameterized biological model to change the distance loss value in step S231 above is represented as follows:

[0074]

[0075] Where, min represents minimizing the distance loss between each image segmentation line and its corresponding contour line, and frame rot This indicates the rotation parameter of the camera that took the photo to be processed; frame trans The rigid parameter represents the translation parameter of the camera that took the photo to be processed. rot The rigid parameter represents the rotation parameter in the rigid transformation parameters. scale This refers to the scaling parameter in the rigid transformation parameters, rigid. trans This represents the translation parameter in the rigid transformation parameters, where mean represents the mean topology, and vec represents the mean topology. i Let val represent the i-th eigenvector. iThis represents the fusion deformation parameter (also known as the fusion deformation coefficient or BlendShape coefficient) corresponding to the i-th feature vector.

[0076] Step S232: The parameterized biological model with a distance loss value less than a preset threshold is determined as the reconstructed biological model after three-dimensional reconstruction.

[0077] For example, in implementing step S232 above: when the distance loss value is less than a preset threshold, or when the distance loss value no longer decreases, the parameterized biological model at this time is determined as the reconstructed biological model after 3D reconstruction. It is understood that since many parameters no longer change after 3D reconstruction, the reconstructed biological model after 3D reconstruction can be determined only by the changing parameters. Therefore, the reconstructed biological model after 3D reconstruction at this time can be represented as follows: Among them, rigid scale This represents the scaling parameter in the rigid transformation parameters, where mean represents the mean topology, and vec... i Let val represent the i-th eigenvector. i This represents the fusion deformation parameter (also known as the fusion deformation coefficient or BlendShape coefficient) corresponding to the i-th feature vector.

[0078] After step S230, step S240 is performed: the reconstructed biological model is measured to obtain the three-dimensional dimensions of the biological parts.

[0079] Please see Figure 7 The diagram shown is one of the measurement axes provided in the embodiments of this application. For example, the implementation of step S240 above is as follows: based on an existing reconstructed biological model, the measurement axis of the foot model (including the measurement abscissa and measurement ordinate) can be determined. Then, the reconstructed biological model is measured according to the measurement axis to obtain the three-dimensional dimensions of the biological part (such as the foot length and foot width in the three-dimensional dimensions).

[0080] In the above implementation process, a parametric model that varies according to rigid transformation parameters and fusion deformation parameters is first obtained. Then, based on the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parametric biological model at each angle, the parametric model is reconstructed in three dimensions to obtain a high-precision reconstructed foot model. This effectively avoids the situation where there are many noise points in the point cloud data. Instead, the distance loss value required for fitting is calculated using the foot outer contour image segmentation line with very few noise points and the foot contour line determined by the parametric model. This effectively improves the accuracy of the three-dimensional dimensions obtained by measuring the reconstructed foot model.

[0081] Please see Figure 8The diagram shown is a structural schematic of a biological model construction apparatus provided in an embodiment of this application. This application provides a biological model construction apparatus 300, comprising:

[0082] The mesh topology acquisition module 310 is used to acquire multiple mesh topologies, which are obtained by 3D scanning and modeling of real biological parts. Any two of the multiple mesh topologies have different poses and shapes.

[0083] The shape topology acquisition module 320 is used to acquire template topology and normalize and align the pose and scale of multiple mesh topologies based on the template topology to obtain multiple shape topologies with the same pose and scale.

[0084] The feature vector acquisition module 330 is used to calculate the average value of multiple shape topologies, obtain the mean topology, and extract the feature vector of each shape topology from the multiple shape topologies to obtain multiple feature vectors.

[0085] Biological model building module 340 is used to build parameterized biological models based on mean topology and multiple feature vectors.

[0086] Optionally, in this embodiment of the application, the shape topology acquisition module includes:

[0087] The transformation parameter acquisition module is used to perform a rigid transformation on the mesh topology based on the template topology to obtain the rigid transformation parameters of the mesh topology.

[0088] The mesh topology alignment module is used to perform attitude alignment on the mesh topology based on the rigid transformation parameters of the mesh topology, so as to obtain the attitude-aligned mesh topology.

[0089] The topology transformation and alignment module is used to perform non-rigid transformation and nearest neighbor point alignment on the orientation-aligned mesh topology under the constraints of the landmark points of the template topology and the landmark points of the mesh topology, so as to obtain the shape topology.

[0090] Optionally, in this embodiment of the application, the topology transformation alignment module includes:

[0091] The topology non-rigid transformation module is used to perform non-rigid transformation on the topology of the orientation-aligned mesh using a non-rigid iterative nearest point algorithm to obtain the transformed shape topology.

[0092] The nearest neighbor alignment module is used to align the nearest neighbor points of the transformed shape topology to obtain the shape topology.

[0093] Optionally, in embodiments of this application, the biological model construction apparatus further includes:

[0094] The biological model reconstruction module is used to perform three-dimensional reconstruction of parametric biological models to obtain reconstructed biological models.

[0095] The 3D dimension acquisition module is used to measure the reconstructed biological model and obtain the 3D dimensions of the biological parts.

[0096] This application embodiment also provides a three-dimensional dimension measuring device, including:

[0097] The image acquisition module is used to acquire multiple images to be processed and perform semantic segmentation on the biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines. The images to be processed are obtained by taking pictures of the biological parts from different angles.

[0098] The biological model projection module is used to obtain a parameterized biological model using the biological model construction method described above, and to project the parameterized biological model from multiple angles to obtain the contour line projected onto the parameterized biological model from each of the multiple angles.

[0099] The model 3D reconstruction module is used to perform 3D reconstruction of the parametric biological model based on the distance loss value between the image segmentation line corresponding to each of the multiple angles and the contour line projected onto the parametric biological model from each angle, thereby obtaining the reconstructed biological model.

[0100] The model 3D measurement module is used to measure the reconstructed biological model and obtain the 3D dimensions of the biological parts.

[0101] Optionally, in this embodiment, the biological part is the human foot; the three-dimensional dimension measuring device further includes:

[0102] The marker coordinate recognition module is used to identify the coordinates of multiple marker points around the human foot in each of the multiple images to be processed. The coordinates of the multiple marker points are set on the test paper where the human foot is stepping.

[0103] The projection angle restoration module is used to determine the camera parameters of each image to be processed based on the coordinates of multiple marker points, and restore the projection angle of the parameterized biological model onto the plane corresponding to each image to be processed based on the camera parameters of each image to be processed, thereby obtaining multiple angles.

[0104] Optionally, in this embodiment of the application, the model 3D reconstruction module includes:

[0105] The model parameter adjustment module is used to adjust the rigid transformation parameters and / or fusion deformation parameters of the parameterized biological model according to the camera parameters, so as to change the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parameterized biological model at each angle.

[0106] The biological model determination module is used to determine the parameterized biological model whose distance loss value is less than a preset threshold as the reconstructed biological model after 3D reconstruction.

[0107] It should be understood that this device corresponds to the above-described biological model construction method embodiment and is capable of performing the various steps involved in the above method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0108] This application also provides an electronic device, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions, when executed by the processor, perform the method described above.

[0109] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the method described above. The computer-readable storage medium can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0110] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0111] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, as provided in the embodiments of this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending primarily on the functions involved.

[0112] Furthermore, the functional modules of each embodiment in this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0113] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0114] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A method for constructing a biological model, characterized in that, include: Multiple mesh topologies are obtained by 3D scanning and modeling of real biological parts. Any two of the multiple mesh topologies have different poses and shapes. The biological parts include human feet. The template topology of the human foot is obtained, and the pose and scale of the multiple mesh topologies are normalized and aligned according to the template topology of the human foot to obtain multiple shape topologies with the same pose and scale. The average value of the multiple shape topologies is calculated to obtain the mean topology, and the feature vector of each shape topology in the multiple shape topologies is extracted to obtain multiple feature vectors; A parameterized biological model is constructed based on the mean topology and the multiple feature vectors; The step of normalizing and aligning the pose and scale of the multiple mesh topologies based on the template topology of the human foot to obtain multiple shape topologies with the same pose includes: performing a rigid transformation on the mesh topology based on the template topology of the human foot to obtain the rigid transformation parameters of the mesh topology; performing pose alignment on the mesh topology based on the rigid transformation parameters to obtain the pose-aligned mesh topology; and, under the constraint of the landmark points of the template topology of the human foot and the landmark points of the mesh topology, performing a non-rigid transformation and nearest neighbor alignment on the pose-aligned mesh topology to obtain the shape topology. After constructing the parameterized biological model based on the mean topology and the plurality of feature vectors, the method further includes: performing semantic segmentation on the biological parts in the image to be processed to obtain image semantic segmentation lines, wherein the image to be processed is obtained by taking pictures of the biological parts from multiple different angles; performing three-dimensional reconstruction of the parameterized biological model based on the distance loss value between the image semantic segmentation lines and the contour lines projected onto the parameterized biological model from the multiple angles to obtain a reconstructed biological model; and measuring the reconstructed biological model to obtain the three-dimensional dimensions of the biological parts.

2. The method according to claim 1, characterized in that, The process of performing a non-rigid transformation and nearest-neighbor alignment on the orientation-aligned mesh topology to obtain the shape topology includes: The non-rigid iterative nearest point algorithm is used to perform a non-rigid transformation on the orientation-aligned mesh topology to obtain the transformed shape topology. The transformed shape topology is aligned with its nearest neighbor points to obtain the shape topology.

3. A three-dimensional dimension measurement method, characterized in that, include: Multiple images to be processed are acquired, and semantic segmentation is performed on the biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines. The images to be processed are obtained by taking pictures of the biological parts from multiple different angles. A parameterized biological model is obtained using the biological model construction method as described in claim 1, and the parameterized biological model is projected from the plurality of angles to obtain the contour line projected onto the parameterized biological model from each of the plurality of angles; Based on the distance loss value between the image segmentation line corresponding to each of the plurality of angles and the contour line projected onto the parameterized biological model by each angle, the parameterized biological model is reconstructed in three dimensions to obtain the reconstructed biological model. The reconstructed biological model is measured to obtain the three-dimensional dimensions of the biological parts.

4. The method according to claim 3, characterized in that, Before projecting the parameterized biological model from the multiple angles to obtain the outline of the parameterized biological model, the method further includes: The coordinates of multiple marker points around the human foot in each of the multiple images to be processed are identified, and the coordinates of the multiple marker points are set on the test paper under the human foot; For each of the plurality of images to be processed, the camera parameters of each image to be processed are determined based on the coordinates of the plurality of marker points, and the projection angle of the parameterized biological model onto the plane corresponding to each image to be processed is restored based on the camera parameters of each image to be processed, thereby obtaining the plurality of angles.

5. The method according to claim 4, characterized in that, The step of performing three-dimensional reconstruction of the parametric biological model based on the distance loss value between the image segmentation line corresponding to each of the plurality of angles and the contour line projected onto the parametric biological model from each angle to obtain the reconstructed biological model includes: The rigid transformation parameters and / or fusion deformation parameters of the parameterized biological model are adjusted according to the camera parameters to change the distance loss value between the image segmentation line corresponding to each angle and the contour line projected onto the parameterized biological model at each angle. The parameterized biological model whose distance loss value is less than a preset threshold is determined as the reconstructed biological model after three-dimensional reconstruction.

6. A biological model construction device, characterized in that, include: The mesh topology acquisition module is used to acquire multiple mesh topologies, which are obtained by 3D scanning and modeling of real biological parts. Any two of the multiple mesh topologies have different postures and shapes, and the biological parts include human feet. A shape topology acquisition module is used to acquire the template topology of the human foot, and to normalize and align the pose and scale of the multiple mesh topologies based on the template topology of the human foot to obtain multiple shape topologies with the same pose and scale. The feature vector acquisition module is used to calculate the average value of the multiple shape topologies to obtain the mean topology, and extract the feature vector of each shape topology from the multiple shape topologies to obtain multiple feature vectors. A biological model construction module is used to construct a parameterized biological model based on the mean topology and the multiple feature vectors; The step of normalizing and aligning the pose and scale of the multiple mesh topologies based on the template topology of the human foot to obtain multiple shape topologies with the same pose includes: performing a rigid transformation on the mesh topology based on the template topology of the human foot to obtain the rigid transformation parameters of the mesh topology; performing pose alignment on the mesh topology based on the rigid transformation parameters to obtain the pose-aligned mesh topology; and, under the constraint of the landmark points of the template topology of the human foot and the landmark points of the mesh topology, performing a non-rigid transformation and nearest neighbor alignment on the pose-aligned mesh topology to obtain the shape topology. After constructing the parameterized biological model based on the mean topology and the plurality of feature vectors, the method further includes: performing semantic segmentation on the biological parts in the image to be processed to obtain image semantic segmentation lines, wherein the image to be processed is obtained by taking pictures of the biological parts from multiple different angles; performing three-dimensional reconstruction of the parameterized biological model based on the distance loss value between the image semantic segmentation lines and the contour lines projected onto the parameterized biological model from the multiple angles to obtain a reconstructed biological model; and measuring the reconstructed biological model to obtain the three-dimensional dimensions of the biological parts.

7. A three-dimensional dimension measuring device, characterized in that, include: The image acquisition module is used to acquire multiple images to be processed and perform semantic segmentation on the biological parts in each of the multiple images to be processed to obtain multiple image segmentation lines. The images to be processed are obtained by taking pictures of the biological parts from different angles. A biological model projection module is used to obtain a parameterized biological model using the biological model construction method as described in claim 1, and to project the parameterized biological model from the plurality of angles to obtain the outline of each of the plurality of angles projected onto the parameterized biological model. The model 3D reconstruction module is used to perform 3D reconstruction of the parameterized biological model based on the distance loss value between the image segmentation line corresponding to each of the plurality of angles and the contour line projected onto the parameterized biological model by each angle, so as to obtain the reconstructed biological model. The model 3D measurement module is used to measure the reconstructed biological model and obtain the 3D dimensions of the biological parts.

8. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method as described in any one of claims 1 to 5.

Images