3D parameterization-based reconstruction and prediction method and electronic equipment

Abstract

The invention discloses a reconstruction and prediction method based on 3D parameterization. The method comprises the following steps: obtaining 2D images of one or more visual angles of an object to be measured; inputting the 2D image into a target parameter prediction model to obtain a shape parameter vector of the object to be measured; reconstructing a 3D model of the to-be-measured object based on the shape parameter vector and the shape base vector; performing geometric analysis on the 3D model of the to-be-measured object to obtain key physical parameters of the to-be-measured object; wherein the shape base vector is extracted by adopting a dimension reduction technology and is used for representing a main deformation mode of a category object to which the to-be-measured object belongs. The invention further discloses corresponding electronic equipment and a computer readable storage medium. According to the method, unified topology modeling is carried out on objects, deep learning is carried out by using a small number of multi-view images, 3D shape modeling and physical parameter prediction can be carried out on various objects under the condition of limited data, and the method has the characteristics of high modeling efficiency, 3D shape low-dimensional representation, shape change interpretability and low cost.

Description

Technical Field

[0001] The present invention belongs to the technical field of 3D model reconstruction, and specifically relates to a 3D parametric reconstruction and prediction technology. Background Art

[0002] The reconstruction of 3D models has wide applications in the fields of computer vision, graphics, industrial design, robotics, etc. In the prior art, for the reconstruction of 3D models and the prediction of physical parameters, mainly high-quality pictures are taken, and 3D models are constructed based on these pictures, and then the physical parameters (such as length, width, height, etc.) of the objects in the pictures are predicted based on the 3D models. The key to this kind of method is the construction from 2D pictures to 3D models.

[0003] The existing 3D model reconstruction methods mainly include 3D scanning technology, multi-view stereo vision technology, and neural radiance field (NeRF) method, but they still have many limitations. Among them, 3D scanning technology can usually directly generate point cloud data. On the one hand, it depends on expensive and complex laser or structured light devices to obtain the three-dimensional coordinate points on the object surface. On the other hand, a large number of three-dimensional coordinate points on the object surface need to be collected during the scanning process to form a dense point cloud. Therefore, it is difficult to be promoted on a large scale and at low cost. The core of multi-view stereo vision technology is to reconstruct the same scene from multiple perspectives of multiple cameras, and reconstruct the three-dimensional structure of the scene represented by dense or sparse point clouds based on image feature matching. This technology has high requirements for image quality, view coverage, and matching algorithms, and has a large amount of calculation and long time consumption, making it difficult to meet the requirements of real-time and high efficiency. Although the neural radiance field (NeRF) method can generate high-quality 3D reconstructions and novel view syntheses, it has a high dependence on training data and computing resources and is difficult to be effectively implemented in scenarios with limited data. In addition, in the existing 3D model reconstruction methods based on deep learning, on the one hand, a large amount of data is required for deep learning, and the internal mechanisms of many deep learning models are not transparent, making it difficult to clearly explain the shape changes, which is not conducive to understanding and application. Summary of the Invention

[0004] In view of this, the present invention proposes a 3D parametric reconstruction and prediction method. By performing unified topological modeling on an object and using a small number of multi-view images for deep learning, it can realize three-dimensional shape modeling and physical parameter prediction on a variety of objects (including animals, plants, and non-biological objects) under the condition of limited data, and has the characteristics of high modeling efficiency, low-dimensional representation of 3D shapes, interpretability of shape changes, and low cost. Further, the present invention also provides an electronic device and a computer-readable storage medium for implementing the above method.

[0005] The technical solution of the present invention is as follows:

[0006] The first aspect of the present invention discloses a 3D parametric reconstruction and prediction method, including the following steps:

[0007] S1: Obtain 2D images of one or more perspectives of the object to be measured;

[0008] S2: Input the 2D images into the target parameter prediction model to obtain the shape parameter vector of the object to be measured;

[0009] S3: Based on the shape parameter vector and the shape basis vectors of the objects belonging to the category of the object to be measured extracted in advance, reconstruct the 3D model of the object to be measured;

[0010] S4: Perform geometric analysis on the 3D model of the object to be measured to obtain the key physical parameters of the object to be measured;

[0011] Wherein, the shape basis vectors are extracted by using dimensionality reduction technology and are used to characterize the main deformation modes of the objects belonging to the category of the object to be measured.

[0012] As an alternative, there are k shape basis vectors, and each shape basis vector is used to represent a main deformation mode of the objects belonging to the category of the object to be measured; the k shape basis vectors constitute a basis vector space, which can explain more than a preset proportion of the deformation data of the objects belonging to the category of the object to be measured; the shape parameter vector has k projection coefficients and is obtained by projection based on the basis vector space.

[0013] As an alternative, the dimensionality reduction technology includes the linear principal component analysis (PCA) method, the non-linear locally linear embedding method (LLE), and the non-linear isometric mapping (Isomap).

[0014] As an alternative, in step S2, the construction process of the target parameter prediction model includes the following steps:

[0015] S21: Provide multiple target objects belonging to the same category as the object to be measured;

[0016] S22: Respectively obtain 2D images of multiple perspectives of each target object, and the 2D images can reflect the size and shape characteristics of the target object;

[0017] S23: Respectively measure each target object to obtain physical parameters that can reflect the size and shape characteristics of the target object;

[0018] S24: Make training data based on the 2D images and physical parameters, and the training data includes the 2D images and shape parameter vectors corresponding to each target object;

[0019] S25: Construct an initial parameter prediction model that can fuse multi-perspective features;

[0020] S26: Train the initial parameter prediction model through the training data set to obtain the target parameter prediction model.

[0021] As an alternative, in step S24, the process of generating training data includes:

[0022] A1: Select one of the target objects, and establish a reference 3D model based on the 2D image and physical parameters of the selected target object;

[0023] A2: Based on the 2D images and physical parameters of the other target objects except the selected target object, perform shape deformation on the reference 3D model to obtain a corresponding derivative 3D model, and the derivative 3D model and the reference 3D model have a consistent mesh topology structure;

[0024] A3: Based on the coordinate differences of the vertices at the corresponding positions of each derivative 3D model and the reference 3D model, use dimensionality reduction technology to extract k shape basis vectors of the target object, and the k shape basis vectors form a basis vector space;

[0025] A4: Based on the derivative 3D model and the basis vector space, obtain the shape parameter vector of the derivative 3D model through projection;

[0026] A5. Use the 2D images and shape parameter vectors of each target object as training data.

[0027] As an alternative, the dimensionality reduction technology is the principal component analysis method; step A3 includes:

[0028] A31: Calculate the coordinate differences of the vertices at the corresponding positions of each derivative 3D model and the reference 3D model respectively to obtain multiple groups of high-dimensional difference vectors;

[0029] A32: Stretch each group of high-dimensional difference vectors into one dimension by row or column to form multiple groups of one-dimensional difference vectors of 3N dimensions;

[0030] A33: Perform PCA statistics on the obtained difference vectors to extract the shape basis vector b of the target object j , the basis vector space; the basis vector space contains k shape basis vectors b j

[0031] The projection method in step A4 is:

[0032] Project the one-dimensional difference vectors of each 3D model onto the basis vector space, and calculate the shape parameter vectors β of each 3D model respectively; each 3D model includes the derivative 3D model and the newly established 3D model; the shape parameter vector β contains k projection coefficients β j .

[0033] As an alternative, in step A2, shape deformation is performed on the reference 3D model, which specifically includes: using modeling software to edit the reference 3D model while ensuring that the total number of triangular facets and the total number of vertices of the triangular facets constituting the model remain unchanged.

[0034] As an alternative, in step S2, the initial parameter prediction model includes an input layer, a feature extraction unit, a feature fusion unit, a shape parameter regression unit, and an output layer; the input layer is used to receive 2D images from multiple perspectives; the feature extraction unit is used to extract features from the 2D images; the feature fusion is used to fuse the extracted image features to obtain an overall shape feature; the shape parameter regression unit is used to map the overall feature to a shape parameter vector; the output layer is used to output the shape parameter vector; during the model training process, by continuously optimizing the model weight parameters, when the model loss function value tends to converge or the number of iterations reaches a preset value, the target model is obtained; the model loss function value is the mean square error between the predicted value of the shape parameter vector and the true value of the corresponding shape parameter vector in the training set.

[0035] As an alternative, in step S3, the 3D model of the object to be measured is linearly reconstructed based on the following formula:

[0036]

[0037] In the formula, M(β) is the 3D model of the object to be measured; is the 3D reference model of the object category to which the object to be measured belongs; b j is the j-th shape basis vector of the target object; β j is the projection coefficient of the object to be measured on the j-th shape basis vector.

[0038] As an alternative, step S4 specifically includes: importing the 3D model of the object to be measured into a preset library function to calculate the key physical parameters of the object to be measured; the preset library function includes at least one of Meshlab, Blender, and Open3D.

[0039] The second aspect of the present invention discloses a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of the method according to the first aspect of the present invention or any one of its alternative solutions are implemented.

[0040] The third aspect of the present invention discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method according to the first aspect of the present invention or any one of its alternative solutions are implemented.

[0041] The present invention has the following beneficial effects:

[0042] (1) By establishing a 3D model with a unified topological structure (abbreviated as "unified grid") and combining the PCA statistical method, the present invention maps the three-dimensional grid deformation of the target object to a low-dimensional shape parameter space, compactly describes the object shape, realizes 3D parametric modeling, and can predict the physical parameters (such as volume, weight, geometric dimensions, etc.) of the object in a limited data scenario. Compared with traditional large-scale voxel or point cloud methods, the present invention can reconstruct a complete 3D model by regressing a small number of parameters. Through this low-dimensional parameterization, the calculation efficiency can be greatly improved, and the interpretability of shape changes can be enhanced.

[0043] (2) The present invention pre-establishes a 3D model (unified grid) with a unified topological structure to ensure that all object deformations are carried out on the same number of triangular patches and the number of vertices of triangular patches (grid vertices), without the need for frequent topological rearrangement or reliance on non-rigid registration.

[0044] (3) By performing principal component analysis on the grid deformation in the training dataset, the present invention extracts the main deformation modes (i.e., shape parameter vectors), reduces the high-dimensional 3D grid (tens of thousands of vertices) to a low-dimensional shape parameter space (tens or hundreds of dimensions), realizes shape space dimensionality reduction, and the shape of the new object can be regarded as a linear combination of these modes, greatly simplifying the representation and processing of high-dimensional 3D data, having the advantage of strong real-time performance, and being very suitable for AR / VR, online detection, or robot real-time interaction scenarios. Moreover, the reduced-dimensional shape parameter space is more robust to noise and can also prevent the network from overfitting to singular shapes.

[0045] (4) The present invention has the advantages of high calculation efficiency, strong interpretability, and adaptability to various object shapes, overcomes the dependence of the prior art on expensive equipment, a large amount of data, or complex algorithms, and provides an efficient and general solution for various 3D modeling and parameter prediction tasks (such as animal and plant morphology assessment, industrial component detection, robot interaction, etc.), with very strong practicality.

[0046] (5) The 3D parametric reconstruction based on unified topological modeling of the present invention can not only meet the purpose of the present invention for physical parameter prediction, but also has derivative value, can be directly applied to subsequent task extensions such as rendering, measurement, animation, collision detection, physical simulation, etc., and interpolate or regress other physical quantities in the same space. Compared with point clouds or implicit representations, which require additional mesh reconstruction, the workload is greatly reduced. This also makes the invention more scalable and industrially valuable in actual 3D scene applications. Description of the Drawings

[0047] Figure 1 It is a flowchart of the 3D parametric reconstruction and prediction method for Embodiment 1;

[0048] Figure 2 Schematic flowchart of the method for constructing the target parameter prediction model in Embodiment 2;

[0049] Figure 3 Schematic flowchart of the method for preparing the model training data in Embodiment 3;

[0050] Figure 4 Schematic flowchart of the method for extracting the shape base vectors and calculating the shape parameter vectors in Embodiment 4;

[0051] Figure 5 Schematic grid diagram of the reference mushroom model;

[0052] Figure 6 Schematic diagram of a partial derivative mushroom model with a unified topological structure;

[0053] Figure 7 Schematic diagram of the neural network model structure;

[0054] Figure 8 Schematic diagram of 3D shape reconstruction and parameter extraction. Detailed implementation manners

[0055] Based on the idea of "parametric model + unified grid + PCA dimensionality reduction", the present invention combines the unified topological modeling of an object by a modeler and the deep learning of a small number of multi-view images by a neural network, and can achieve high-precision three-dimensional (3D) shape modeling and physical parameter prediction of an object in a situation with limited data, and realize an interpretable low-dimensional representation of shape changes.

[0056] The technical solutions of the present invention will be described in more detail below with reference to specific embodiments and the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not used to limit the protection scope of the present invention.

[0057] For ease of description, only the parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0058] As Figure 1 shown, Embodiment 1 of the present invention discloses a 3D parametric reconstruction and prediction method, which mainly includes the following steps:

[0059] Step 1: Obtain 2D images of one or more views of the object to be measured;

[0060] Step 2: Input the 2D image into the target parameter prediction model to obtain the shape parameter vector of the object to be measured;

[0061] Step 3: Reconstruct the 3D model of the object to be measured based on the shape parameter vector and the shape basis vectors of the objects belonging to the category of the object to be measured extracted in advance;

[0062] Step 4: Perform geometric analysis on the 3D model of the object to be measured to obtain the key physical parameters of the object to be measured.

[0063] Among them, the shape basis vectors are extracted using dimensionality reduction technology and are used to reflect the deformation modes of the objects belonging to the category of the object to be measured.

[0064] It can be understood that the shape basis vectors b extracted by dimensionality reduction technology j There are k of them, where k is a natural number greater than or equal to 1. Each extracted shape basis vector b j The deformation mode represented is also the main deformation mode among the numerous deformation modes of the objects belonging to the category of the object to be measured. In other words, the k deformation modes can explain more than a preset proportion of the deformation data of the objects belonging to the category of the object to be measured, usually reaching 90% or even higher. The k shape basis vectors b j Form a basis vector space. The shape parameter vector has k projection coefficients β j , and can be obtained by projection based on the basis vector space.

[0065] It should be noted that the dimensionality reduction technology used in the present invention may include the linear principal component analysis (PCA) method, or may also include the non-linear locally linear embedding method (LLE), the non-linear isometric mapping (Isomap). The present invention does not limit this.

[0066] Furthermore, as Figure 2 shown, Embodiment 2 of the present invention discloses a method for constructing a target parameter prediction model, which mainly includes the following steps:

[0067] Step 21. Multi-view image data acquisition.

[0068] Provide multiple objects belonging to the same category, and take pictures of each object from multiple angles to obtain multiple two-dimensional (2D) images of different views, and the obtained 2D images cover the overall size and shape characteristics of the object as much as possible.

[0069] In a specific embodiment, the object is a single mushroom (abbreviated as "mushroom"). Since the structure of the object is relatively simple, 30 mushrooms can be selected for shooting. The 30 mushrooms are photographed from the front, left side, right side, top view, back and other angles by a camera to obtain sufficient image information, and these multi-view 2D mushroom images obtained can reflect the overall shape characteristics of the mushrooms.

[0070] It should be noted that the number of target objects selected in the present invention usually only needs to be dozens to hundreds. Compared with deep learning which requires thousands or even hundreds of thousands, the requirement for the quantity of prior data is greatly reduced.

[0071] Step 22. Physical parameter measurement.

[0072] Measure the physical parameters of each target object respectively. The physical parameters to be measured can be selected according to the user's concern about the target object and the user's needs, and are usually physical parameters reflecting the size and shape characteristics of the target object. Among them, the physical parameters reflecting the size of the target object, such as size parameters like length, width, height, volume, surface area, etc., and the physical parameters reflecting the shape characteristics of the target object, such as curvature, maximum diameter, etc. These physical parameters usually include the key physical parameters that can reflect the main deformation modes of the target object.

[0073] Continuing with the previous specific embodiment, after shooting, the size parameters such as the cap diameter, cap thickness, stalk length, overall height, etc. of these 30 mushrooms can be measured on-site at the shooting location. At the same time, shape parameters such as the cap surface area, average stalk diameter, maximum stalk diameter, etc. can be calculated based on these size parameters, and the measured physical parameters are recorded.

[0074] It can be understood that the present invention does not limit the selection of the physical parameters to be measured, and can be flexibly adjusted according to the characteristics of the target object and the user's concern about the target object and the user's needs. And the measurement referred to in this step includes not only direct measurement but also simple calculations based on the measured parameters.

[0075] Step 23. Training data production.

[0076] Establish the mapping relationship between the 2D images of each target object and the shape parameter vectors of the 3D models respectively, so that each 3D model can find the corresponding coordinates in the parameter space, which is also equivalent to establishing the mapping relationship between "2D image - shape parameter vector - 3D model - physical parameter". The 2D images corresponding to each target object and the shape parameter vectors of their 3D models together constitute a training data set. Among them, the 2D images are used as the input of the subsequent parameter prediction model, and the shape parameter vectors of the corresponding 3D models are used for the calculation of the loss function to provide supervision information for the training of the parameter prediction model.

[0077] Continuing with the previous specific embodiment, establish the mapping relationship between the 2D mushroom images of 30 mushrooms and the shape parameter vectors of the 3D mushroom models, and use them as the training data set for the subsequent model training.

[0078] Based on this, it can be seen that since the deformation range of the present invention is concentrated in the principal component subspace a priori, when performing deep learning or regression, the present invention only needs to regress the shape parameter β. The dimension of the shape parameter vector β, that is, the number of principal components, is much smaller than the tens of thousands of dimensions of the original 3D mesh, thereby reducing the data scale requirement. Therefore, only a small amount of training data is needed during the deep learning process. Compared with the conventional 3D point cloud / voxel method, the dependence of the present invention on hardware devices and massive data is significantly reduced, and it is easier to achieve accurate 3D shape reconstruction in a limited data scenario.

[0079] Step 24: Construction of the initial parameter prediction model.

[0080] After obtaining the training dataset, the initial parameter prediction model can be established, or it can also be carried out synchronously. The present invention has no special requirements for the selection of the initial parameter prediction model. Generally, as long as it can satisfy multi-view feature fusion. For example, a convolutional neural network (CNN) or other models that fuse multi-view features can be used.

[0081] Combined Figure 7 As shown, the initial model selected by the present invention mainly consists of an input layer, a feature extraction unit, a feature fusion unit, a shape parameter regression unit, and an output layer. Among them, the input layer is used to receive one or more normalized 2D images of different views of the target object; the feature extraction unit is used for feature extraction of the 2D images; the feature fusion unit is used to fuse the features of multiple views to form a fusion feature for representing the overall shape feature; the shape parameter regression unit is used to map the fusion feature to the shape parameter vector and output it through the output layer. Specifically, network structures such as ConvNext, EfficientNet, RegNet, and Swin Transformer can be selected.

[0082] In a specific embodiment, for example, based on the combination use of ResNet and MLP, first extract the depth features of images from different perspectives through the ResNet network; then fuse (weight or average) the depth features of multi-perspective images through MLP (Multi-Layer Perceptron) to obtain the fused comprehensive features; finally, map the comprehensive features to the shape parameter vector through the output layer at the end of MLP for 3D reconstruction or physical parameter measurement. Another example is when using the Swin Transformer network. First, divide the images from different perspectives into blocks and perform transformer encoding to output a local-global combined image representation; then use mechanisms such as attention to perform multi-perspective feature fusion on the Transformer encoding representation to form an overall feature; finally, output the shape parameter vector through a small regression network (such as a fully connected layer). Another example is to perform feature extraction of multi-perspective images through an energy-saving and high-precision feature extractor such as the EfficientNet network; then simply average or weight and combine the multi-perspective features to obtain the fused global shape features, which also has good effects; finally, convert the global shape features into the shape parameter vector through one or several fully connected networks.

[0083] Step 25. Model training.

[0084] It can be understood that before extracting features from the 2D image of the target object, it is usually necessary to normalize the image to a similar size or resolution so that the network has a consistent scale reference when inputting. If the shooting distance or camera parameters are different, calibration objects can also be placed in the image or other calibration means can be used for size calibration. The present invention does not limit this.

[0085] The present invention trains the parameter prediction model with training data, and uses the loss function value to characterize the prediction effect of the parameter prediction model on the shape parameter vector. During the training process, the loss function value is continuously reduced to ensure the geometric consistency of the target object as much as possible. The present invention can select the mean square error (MSE) of the shape parameter vector as the loss function value to measure the difference between the predicted value of the shape parameter vector and the true value of the shape parameter vector in the training set. During the training process of the parameter prediction model, the present invention can select optimization algorithms such as Adam and SGD to iteratively update the network weights of the parameter prediction model until the loss function value tends to converge or the number of iterations reaches the threshold, and then stop training to obtain the target parameter prediction model.

[0086] Based on the trained target parameter prediction model, the present invention can predict the shape parameter vector in the 3D model of the object to be measured, and then realize the reconstruction of the 3D model of the object to be measured and the prediction of key physical parameters.

[0087] Continuing with the previous specific embodiment, based on the trained mushroom parameter prediction model, the 3D model reconstruction and key physical parameter prediction of all mushrooms of the same type can be achieved. Specifically, any mushroom can be photographed to obtain one or more 2D mushroom pictures; the obtained 2D mushroom pictures are input into the mushroom parameter prediction model to obtain the shape parameter vector of the mushroom; the obtained shape parameter vector is used as known parameters, and the corresponding 3D model can be calculated through formula (2); then, by calling library functions, the key physical parameters of the mushroom can be calculated, such as parameters like the cap area, cap length, cap width, stalk length, average stalk diameter, maximum stalk diameter, overall height, etc. Figure 8 The reconstructed 3D shape of the mushroom is shown. By calling library functions, the key physical parameters obtained are as follows: {'cap area': 1.0526140914811715, 'cap length': 1.1448038816452026, 'cap width': 1.168502688407898,'stalk length': 0.9566632509231567, 'average stalk diameter': 0.2134493738412857,'maximum stalk diameter': 0.3072476089000702, 'overall height': 1.7578308582305908}.

[0088] It should be noted that the 3D parametric reconstruction and prediction method provided by the present invention is not only applicable to jointless objects such as mushrooms, but can also be extended to other similar objects, and even to objects with skeletons but without pose modeling (such as certain animals and plants, industrial parts, etc.). When modeling various objects, the key lies in ensuring that all training samples have a consistent mesh topology and performing corresponding principal component extraction through statistical methods. Relying on the prior parametric shape space and a small number of multi-view images, the present invention can accurately predict the 3D model and key physical dimensions of objects, greatly reducing the dependence on expensive 3D scanning equipment or massive data, and having obvious advantages in limited data scenarios.

[0089] Furthermore, as Figure 3 shown, Embodiment 3 of the present invention discloses a method for making training data of an object parameter prediction model, which is also one of the cores of the present invention, and specifically includes the following sub-steps:

[0090] Step A1: Manually construct a reference 3D model.

[0091] Select one of the objects. A professional 3D modeler can manually create a high-precision 3D model of the object according to the corresponding 2D image and physical parameters of the object using modeling software such as Maya and Blender, and use the established high-precision 3D model (also called "3D mesh") as the reference 3D model (also called "reference 3D mesh").

[0092] It is understandable that 3D modeling software usually performs modeling in a standard 3D mesh interface. The 3D models established are composed of multiple interconnected triangular patches. Each triangular patch has 3 vertices, and adjacent triangular patches share the same vertex. Therefore, the 3D model of an object is also called the 3D mesh of the object.

[0093] Step A2: Obtain multiple derivative 3D models through shape editing.

[0094] The modeler can perform linear deformation modeling based on the 2D images and physical parameters of other target objects, using the reference 3D model as the basis to obtain multiple derivative 3D models with different forms corresponding to each target object one by one. Specifically, in the 3D modeling software, the reference 3D model can be edited, for example, by operations such as dragging and stretching. On the premise of keeping the 3D mesh topology unchanged, the shape of the reference 3D model is deformed to obtain multiple derivative 3D models. This can avoid frequent non-rigid registration or topology rearrangement in statistical learning (such as PCA) and the training of subsequent parameter prediction models. Moreover, the mesh structures of these 3D models are coherent and unified, and the size parameters, surface area, volume, etc. of the object can be easily measured, and visualization, comparison, or statistical analysis can be intuitively performed.

[0095] It is understandable that in the present invention, related expressions such as unchanged topology structure, having a unified topology structure, having a unified mesh, and consistent topology structure all have the same meaning. And the unchanged topology structure means that the number of triangular patches and vertices of each 3D model is the same. By manually creating and editing 3D models by the 3D modeler, the consistency of the training data in the mesh topology can be ensured, and subsequently, the low-dimensional representation of PCA is used to improve the accuracy and generalization ability of 3D shape reconstruction and key physical parameter prediction.

[0096] It is worth noting that on the premise of ensuring that the number of triangular patches and vertices remains unchanged, the modeler generates more deformation samples by diversely editing the reference 3D model. And all deformations are performed on a fixed mesh structure, there is no problem of patch or vertex rearrangement, and the 3D mesh with consistent topology structure can generate a complete renderable mesh, thus enriching the training data set and further improving the generalization ability of the model to new forms. In the prior art, the point cloud itself has no mesh topology and requires additional algorithms (such as Poisson Surface Reconstruction, Poisson reconstruction, etc.) to fill in the surface, and the quality is not necessarily stable; NeRF is implicit volume rendering, and when extracting the mesh, voxelization or isosurface extraction needs to be done, and the steps are complex and it is impossible to ensure an ideal topology.

[0097] Step A3: Extract shape basis vectors.

[0098] Here, PCA (principal components analysis) is used as an example. It aims to use the idea of dimensionality reduction to replace the original large number of variables with fewer variables, and can reflect most of the information of the original multiple variables. In this step, the coordinate differences of the vertices at the corresponding positions in each derived 3D model and the reference 3D model are first calculated; then, based on these vertex coordinate differences, the PCA statistical method is used to extract k shape basis vectors b of the target object. j .

[0099] It is understandable that in other embodiments, other dimensionality reduction techniques may also be used, including linear or nonlinear ones. For example, nonlinear LLE (local linear embedding) method and Isomap (isodistance mapping) method can also perform dimensionality reduction analysis on high-dimensional grid deformation vectors. Among them, LLE can capture more complex curved manifold changes in shape space by retaining the linear structure in the local neighborhood; Isomap is based on approximate geodesic distance to maintain the overall geometric structure and can handle strong nonlinear deformations. Both can also replace PCA as a dimensionality reduction method in the present invention.

[0100] Step A4: Calculate the shape parameter vector.

[0101] Based on the obtained shape basis vectors, the shape parameter vectors of each target object are calculated by projection. Specifically, the shape parameter vectors β corresponding to each model including the reference 3D model and the derived 3D model are calculated, and β has k projection coefficients β j .

[0102] It should be noted that in the PCA dimensionality reduction of the present invention, statistical analysis is mainly considered for a large number of high-dimensional difference vectors (such as grid coordinate difference vectors) to find the orthogonal basis vectors that can explain the maximum data variance, so as to extract the most significant deformation patterns (i.e., the main deformation patterns) and reduce the dimension. Specifically, it can be considered from the following principles: 1) PCA seeks to project in the direction with the largest variance in the dataset, corresponding to the eigenvector with the largest eigenvalue, to retain the most shape change information, so it is also called the maximum variance principle; 2) Each principal component is orthogonal to each other, avoiding redundancy and correlation, and ensuring the independence between deformation patterns (such as "becoming taller" and "becoming fatter" can be represented separately); 3) Sort the eigenvalues from large to small, and generally intercept the first k principal components. The cumulative variance ratio of these k principal components reaches 90% - 95%, avoiding high-dimensional noise and overfitting, so that both the main geometric changes can be retained and the data dimension can be greatly compressed; 4) From a practical perspective, the grid coordinate differences are centralized to exclude the influence of overall translation, and it is required that the deformation patterns can learn the most critical directions even in the case of a small amount of data. The linear statistical method of PCA is simple, efficient and interpretable. Based on the above principles, the principal component vectors extracted by PCA are the "shape basis vectors", which can not only summarize the main changes but also significantly reduce the dimension that the subsequent model needs to process.

[0103] It can be understood that PCA will generate multiple principal components, and each principal component corresponds to an eigenvalue (indicating the size of the data variance that the principal component can explain). These eigenvalues are sorted from large to small, and usually the first few principal components can explain most of the shape changes. For example, if the first k principal components together can explain 95% of the data changes, then only these k principal components can be retained, and the explanatory power of the remaining principal components is relatively small. Retaining them will instead introduce noise or lead to too high a dimension. Therefore, usually the part of the eigenvalues (i.e., the principal components) that can jointly explain the total data variance reaching a preset ratio (such as 95%) is selected to avoid subsequent overcomplexity or noise superposition caused by selecting too many principal components, that is, the number of principal components finally retained is determined by the preset threshold of the "cumulative variance contribution rate".

[0104] It can be understood that if k principal components are extracted through the PCA statistical method, each principal component represents a main deformation pattern, such as the length becoming longer or shorter, the overall height becoming higher, the edge becoming thicker, a certain part bulging, etc. Among them, represents the low-dimensional space (k-dimensional) where the shape parameter vector is located. The shape parameter vector β is a k-dimensional vector, so it is denoted as Among them, each component β jThey all correspond to the projection coefficients in a certain principal component direction, which are used to represent the degree of change of an object under this deformation mode. β is also the projection coefficient of each sample or new input on these main deformation modes, which means that the shape parameter vector β includes at least more than one component (usually 5 to 50), so as to be able to express the diverse geometric changes of the target object.

[0105] It should be noted that the present invention uses dimensionality reduction techniques such as PCA to map a 3D mesh with tens of thousands of vertices in high dimensions to a low-dimensional shape parameter space (tens or at most hundreds of dimensions), and only a small number of parameters need to be regressed to reconstruct a complete 3D mesh. Different from the prior art where NeRF or point cloud processing requires a large number of samplings or matches in a three-dimensional field, this parametric model of the present invention only regresses or optimizes low-dimensional parameters, and both training and inference are more efficient. Moreover, in real applications, in fact, the deformation of the target object (such as a mushroom or a fish) is often limited and regular. Incorporating it into a low-dimensional space can reduce the dependence on high-dimensional noise. Based on this, the present invention uses dimensionality reduction techniques to limit all legal deformations in a low-dimensional subspace, which means that the network or algorithm can only search in these several main deformation directions and is not likely to appear singular shapes or outrageous deformations. Therefore, it has a strong deformation prior. The model limits all legal deformations in the low-dimensional space obtained by statistical learning, enhancing the robustness to noise and data loss. Even if some viewpoints are missing, a reasonable 3D shape can still be obtained. Moreover, due to the prior of the shape parameter space, it is difficult for the network to fit to extreme or strange shapes, reducing the blind overfitting to abnormal data and being less likely to overfit to abnormal geometric shapes.

[0106] Continuing with the previous specific embodiment, select a relatively long and most standard or most average mushroom among 30 mushrooms. The modeler creates a 3D model (3D mesh) of this mushroom in the mesh according to multiple 2D images of the mushroom and the measured physical parameters through Blender modeling software. The obtained model can be called the reference mushroom model. The reference mushroom model is a 3D mesh composed of the vertices of 5000 triangular patches and 10,000 triangular patches, as Figure 5 shown.

[0107] Then, the modeler continues to refer to the 2D images and physical parameters of the other 29 mushrooms and performs operations such as dragging and stretching on the reference mushroom model in the modeling software to obtain 29 derivative mushroom models, combined with Figure 6As shown. It should be noted that during operations such as dragging and stretching, it is necessary to ensure that the topological structures of the derived mushroom models and the reference mushroom model are the same, that is, both have 5000 triangular patches and 10000 vertices, so that each model shares the same mesh topology. Then, by comparing the coordinate differences of the vertices in each derived mushroom model and the reference mushroom model, k principal components, that is, the shape basis vectors b of the mushroom, are extracted using the PCA statistical method based on these vertex coordinate differences. j These shape basis vectors are formed into a shape mixing matrix B. S ; and the shape parameter vectors β of all mushroom models are calculated, and each shape parameter vector β contains k projection coefficients β. j For each derived mushroom β. i ={β. i,1 , β. i,2 , β. i,3 , β. i,4 , β. i,5}, corresponding to the projections of these principal components.

[0108] In the data accuracy stage, several mushroom meshes with the same topological structure are obtained, including deformation samples such as different stalk lengths, cap sizes, and cap thicknesses. Each mesh is unfolded into a 3N-dimensional vector, and the difference {d. i} is obtained by subtracting the reference mushroom. The shape basis vectors of the mushroom are extracted through PCA statistics. Assuming that 5 principal components are obtained, that is, k = 5, these 5 principal components can already explain more than 95% of the mushroom shape changes. Therefore, only these principal component vectors need to be retained to describe most of the deformation patterns of the mushroom, without having to deal with subsequent smaller noise effects in the high-dimensional coordinates, so as to achieve the purpose of fully describing the shape changes and simplifying the model at the same time.

[0109] In this embodiment, the 5 principal components reflecting the main deformation patterns of the mushroom include: a) the length of the stalk, for example, the stalk becomes longer or shorter; b) the diameter of the cap, for example, the diameter of the cap increases or decreases; c) the thickness of the cap, for example, the top bulges or collapses, etc.; d) the thickness of the stalk, for example, the radius of the central stalk changes; e) the curvature of the cap edge, for example, the edge of the mushroom curls down or turns outwards. Each principal component provides a key deformation pattern of the mushroom.

[0110] A5: Finally, the 2D images of each target object and the calculated shape parameter vectors are used as training data to train the constructed initial parameter prediction model to obtain the target parameter prediction model.

[0111] In other embodiments, when the type of the target object is an animal, the present invention can remove unnecessary dynamic factors such as bones / joints, only retain the parts that play a major role in shape changes, and focus on describing the geometric deformation of the target object. The same can be achieved by PCA statistics. If several principal components are obtained by PCA statistics, each principal component represents a major deformation mode, and the shape parameters are the projection coefficients of the model in the directions of these principal components. The present invention extracts the main feature components of the data through the PCA statistical method, that is, the PCA shape basis vectors, and can realize the dimensionality reduction of high-dimensional data.

[0112] Further, as Figure 4 shown, taking the PCA method as an example in Embodiment 4 of the present invention, a method for extracting shape basis vectors and calculating shape parameter vectors is disclosed, which mainly includes the following steps:

[0113] Step A31: Calculate the coordinate differences between the vertices at the corresponding positions in each derivative 3D model and the reference 3D model respectively, to obtain multiple groups of high-dimensional difference vectors.

[0114] Among them, the coordinate difference calculation formula is as follows:

[0115]

[0116] In the formula, represents the coordinate difference between the vertex at the t-th position in the i-th derivative 3D model and the vertex at the t-th position in the reference 3D model; V it represents the vertex coordinate at the t-th position in the derivative 3D model; represents the vertex coordinate at the t-th position in the reference 3D model.

[0117] Assume that the 3D mesh of the target object has N vertices, t = 1, 2, L, N, and each vertex contains three coordinates (x, y, z). The coordinate difference of the 1st vertex at the 1st position is (Δx1, Δy1, Δz1), the coordinate difference of the 2nd vertex at the 2nd position is (Δx2, Δy2, Δz2), ……, the coordinate difference of the N-th vertex at the N-th position is (Δx N , Δy N , Δz N ), and after arranging them in sequence, a group of 3N high-dimensional difference vectors are formed, that is, an N×3 difference vector matrix with N rows and 3 columns. Repeat this process for all derivative 3D models to obtain multiple groups of N×3 difference vector matrices (high-dimensional difference vectors).

[0118] Step A32: Stretch each group of high-dimensional difference vectors obtained in Step A31 into one dimension by rows or columns to form multiple groups of 3N-dimensional one-dimensional difference vectors.

[0119] Taking the i-th derived 3D model as an example, the N×3 difference vector matrix of the corresponding vertices of the i-th derived 3D model and the reference 3D model is expanded row by row or column by column, and a one-dimensional difference vector d of 3N dimensions is obtained i , that is, {Δx1, Δy1, Δz1, Δx2, Δy2, Δz2, …, Δx N , Δy N , Δz N}, Repeating this process for all derived 3D models, multiple groups of one-dimensional difference vectors of 3N dimensions are obtained. Based on this, a set of difference vector datasets can be constructed for subsequent high-dimensional statistical analysis of PCA

[0120] It can be understood that the 3D mesh here can be a known 3D mesh or a new 3D mesh that appears later. Among them, the new mesh usually refers to a 3D mesh that appears later and is not in the original derived dataset, or a 3D mesh that the original derived mesh needs to recalculate, but all satisfy having a unified topological structure

[0121] Step A33: Perform PCA statistics on the obtained difference vectors to extract the shape basis vectors b j of the target object, forming a basis vector space

[0122] In this step, PCA statistics are performed on the one-dimensional difference vectors of 3N dimensions obtained in step A32 to obtain several principal components, that is, the shape basis vectors of the target object, and finally a shape basis vector matrix containing the principal component vectors is formed

[0123] Specifically, first the difference vector is transformed into a centered vector , where is the mean vector (taking the average of ); then the covariance of the centered vector set is calculated to obtain a covariance matrix; finally, eigenvectors (orthogonal basis vectors) corresponding to the largest eigenvalues (covariance values) are selected as the shape basis vectors and combined into a shape mixing matrix , that is, the basis vector space , where is the number of principal components, is the -th shape basis vector, , , which represents a main deformation mode. Among them, the covariance calculation formula is: (2); is the number of 3D meshes

[0126] Step A34: Obtain the shape parameter vector of the derived 3D model through projection.

[0127] Project the one-dimensional difference vectors of each 3D model into the basis vector space, and calculate the shape parameter vectors β of each 3D model respectively. For a known derived 3D model, the shape parameter vector β can be obtained by projection; for a new 3D mesh, the difference between the vertices of the new 3D mesh and the corresponding vertices of the reference 3D model can also be calculated first, and then projected into the basis vector space to obtain the shape parameter vector β.

[0128] Specifically, the shape parameter vector of the i-th derived 3D model The projection coefficient of the i-th derived 3D model on the j-th shape basis vector is β i,j , therefore, the shape parameter vector β of the i-th derived 3D model i contains k projection coefficients, that is, the shape parameter vector β of each 3D model contains k projection coefficients β j .

[0129] It should be noted that the shape basis vector b j is the principal component extracted by PCA statistics and is used to describe the main deformation modes that an object may undergo. Its number k is much smaller than the original dimension 3N. The shape parameter vector β is used to characterize the projection coefficients of each derived 3D model or new mesh on these shape basis vectors, that is, how much deformation each principal component contributes. In other words, the shape basis vector b j is like a base, and the shape parameter vector β is like coordinates. The three-dimensional shape of the object can be reconstructed by linear superposition of the two.

[0130] It should also be noted that the shape mixing matrix B S is a matrix that stores several shape basis vectors (also called "principal component vectors", "base deformation vectors", "basis vectors"), usually with dimensions where N is the number of mesh vertices (each vertex has three-dimensional coordinates, a total of 3N dimensions), and k is the number of selected principal components. It can be understood that since there are N vertices and each vertex contains three coordinates (x, y, z), each shape basis vector b j is a 3N-dimensional vector used to describe the coordinate offset of the reference mesh. The shape parameter vector β is a linear combination of the projection coefficients (such as β i,j ) on these shape basis vectors b j , usually with 5 to 50 dimensions, representing the components of the current object in the directions of each principal component, and is a low-dimensional vector.

[0131] It should be noted that, compared with the need for additional mesh reconstruction in point clouds or implicit representations, the parameter prediction model constructed by the present invention only needs to regress a small number of shape parameters, and the network inference speed is fast, which is very suitable for AR / VR, online detection or robot real-time interaction scenarios.

[0132] Further, in another embodiment, for step 3, based on the shape parameter vector and the shape basis vectors of the objects of the category to which the object to be measured belongs that have been pre-extracted, a complete 3D mesh can be reconstructed to realize the restoration of the 3D shapes of the objects.

[0133] In the embodiment, the grid vertex coordinates can be restored using a linear combination formula, and the calculation formula is as follows:

[0134]

[0135] In the formula, M i (β) is the 3D model of the i-th object; is the reference 3D model of the object (representing the average shape of the object, that is, the reference 3D model constructed by the modeler); b j is the j-th shape basis vector of the object; β i,j is the projection coefficient of the i-th object on the j-th shape basis vector.

[0136] Based on the above, the difference between each derived 3D model and the reference 3D model is calculated on the basis of "one-to-one" vertices, without a certain vertex corresponding to all vertices. This ensures that under the unified topology, the PCA statistical analysis has a good correspondence, and based on formula (3), the three-dimensional shape (i.e., 3D model) of the object can be efficiently reconstructed.

[0137] Further, in another embodiment, for step 4, after obtaining the 3D model of the object to be measured, the 3D model of the object to be measured is imported into the preset library function called, and the key physical parameters of the object to be measured can be obtained by performing geometric analysis. Among them, the preset library function can be common tools such as Meshlab, Blender, Open3D, etc., and the required key physical parameters can be returned by selecting relevant instructions or APIs. For example, dimensional parameters such as length, width, height, volume, surface area, etc.

[0138] In summary, the present invention combines three core technical means of "parameterized model + unified grid + PCA dimensionality reduction". By manually creating and editing 3D models by 3D modelers, it ensures the consistency of training data in the grid topology structure. With the help of dimensionality reduction technology representation, it improves the accuracy and generalization ability of 3D shape reconstruction and key physical parameter prediction, and realizes high-precision, high-efficiency, and interpretable 3D model reconstruction and physical parameter prediction for various objects in the context of limited data, providing an efficient, interpretable, and low-cost technical path for many application fields such as industrial inspection, agricultural product evaluation, animal and plant morphology analysis, and multi-view interaction.

[0139] Furthermore, another embodiment of the present invention also discloses an electronic device, which mainly includes a processor and a memory. Among them, the memory is used to store program codes and transmit the program codes to the processor; the processor is used to execute the 3D parameterized reconstruction and prediction method provided by the present invention according to the instructions in the program codes.

[0140] The electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) or the program loaded from the storage device into the random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0141] Furthermore, another embodiment of the present invention also provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the 3D parameterized reconstruction and prediction method provided by the present invention.

[0142] In some embodiments of the present invention, the above-mentioned computer-readable medium may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, (but not limited to) an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist separately without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device can run the 3D parametric reconstruction and prediction method provided by the present invention.

[0143] For the convenience and brevity of description, for the specific working processes of the above-mentioned electronic device, the program code in the computer-readable storage medium, and the computer program product, those skilled in the art can refer to the content in the foregoing method embodiments, and in combination with common general knowledge and conventional techniques in the art, can clearly understand the corresponding implementation manners, and will not be elaborated herein.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D parametric reconstruction and prediction method, characterized in that, It includes the following steps: S1: Obtain 2D images of one or more perspectives of the object to be measured; S2: Input the 2D images into the target parameter prediction model to obtain the shape parameter vector of the object to be measured; S3: Based on the shape parameter vector and the shape basis vectors of the objects belonging to the category of the object to be measured extracted in advance, reconstruct the 3D model of the object to be measured; S4: Conduct geometric analysis on the 3D model of the object to be measured to obtain the key physical parameters of the object to be measured; The shape basis vectors are extracted using dimensionality reduction techniques and are used to characterize the main deformation modes of the objects belonging to the category of the object to be measured.

2. The 3D parameterization reconstruction and prediction method according to claim 1, wherein There are k shape basis vectors, and each shape basis vector is used to represent a main deformation mode of the objects belonging to the category of the object to be measured; the k shape basis vectors form a basis vector space, which can explain more than a preset proportion of the deformation data of the objects belonging to the category of the object to be measured; The shape parameter vector has k projection coefficients and is obtained by projection based on the basis vector space.

3. The 3D parametric reconstruction and prediction method according to claim 1 or 2, characterized in that, The dimensionality reduction techniques include principal component analysis, locally linear embedding, and non-linear isometric mapping.

4. The 3D parametric reconstruction and prediction method according to claim 1 or 2, characterized in that The step In S2, the construction process of the target parameter prediction model includes the following steps: S21: Provide multiple target objects belonging to the same category as the object to be measured; S22: Respectively obtain 2D images of multiple perspectives of each target object, and the 2D images can reflect the size and shape characteristics of the target object; S23: Respectively measure each target object to obtain physical parameters that can reflect the size and shape characteristics of the target object; S24: Make training data based on the 2D images and physical parameters, and the training data includes the 2D images and shape parameter vectors corresponding to each target object; S25: Construct an initial parameter prediction model that can fuse multi-perspective features; S26: Train the initial parameter prediction model through the training data set to obtain the target parameter prediction model.

5. The 3D parametric reconstruction and prediction method according to claim 4, wherein In the step S24, the production process of the training data includes: A1: Select one of the target objects and establish a reference 3D model based on the 2D image and physical parameters of the selected target object; A2: Based on the 2D images and physical parameters of the other target objects except the selected target object, perform shape deformation on the reference 3D model to obtain the corresponding derivative 3D models, and the derivative 3D models and the reference 3D model have the same mesh topology structure; A3: Based on the coordinate differences of the vertices at the corresponding positions of each derivative 3D model and the reference 3D model, use dimensionality reduction techniques to extract k shape basis vectors of the target object, and the k shape basis vectors form a basis vector space; A4: Based on the derivative 3D model and the basis vector space, obtain the shape parameter vector of the derivative 3D model by projection; A5: Use the 2D images and shape parameter vectors of each target object as training data.

6. The 3D parametric reconstruction and prediction method according to claim 1 or 2, characterized in that, In the step S2, The initial parameter prediction model includes an input layer, a feature extraction unit, a feature fusion unit, a shape parameter regression unit, and an output layer; wherein, the input layer is used to receive 2D images from multiple perspectives; the feature extraction unit is used to extract features from the 2D images; the feature fusion is used to fuse the extracted image features to obtain the overall shape features; the shape parameter regression unit is used to map the overall features to a shape parameter vector; the output layer is used to output the shape parameter vector; During the model training process, by continuously optimizing the model weight parameters, when the model loss function value tends to converge or the number of iterations reaches a preset value, the target parameter prediction model is obtained; wherein, the model loss function value is the mean square error between the predicted value of the shape parameter vector and the true value of the corresponding shape parameter vector in the training set.

7. The 3D parametric reconstruction and prediction method according to claim 1 or 2, characterized in that In the step S3, the 3D model of the object to be measured is reconstructed based on the following formula: In the formula, M(β) is the 3D model of the object to be measured; is the 3D reference model of the object belonging to the category of the object to be measured; b j is the j-th shape basis vector of the target object; β j is the projection coefficient of the object to be measured on the j-th shape basis vector.

8. The 3D parametric reconstruction and prediction method according to claim 1 or 2, characterized in that The step S4 specifically includes: importing the 3D model of the object to be measured into a preset library function, and calculating the key physical parameters of the object to be measured; the preset library function includes at least one of Meshlab, Blender, and Open3D.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the 3D parametric reconstruction and prediction method according to any one of claims 1 to 8.

10. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the 3D parametric reconstruction and prediction method according to any one of claims 1 to 8.

Images