Three-dimensional shape generation method based on physical information constraint and related equipment

Abstract

The invention provides a physical information constraint-based three-dimensional shape generation method and related equipment, and relates to the technical field of three-dimensional shape generation, a batch of training samples containing real geometric shapes and real physical attributes are sampled in a real three-dimensional object data set, and a batch of latent vectors are sampled from predefined mathematical probability distribution; training the constructed generative model by using a training sample and a latent vector to obtain a three-dimensional shape generative model; inputting target three-dimensional object data into a three-dimensional shape generation model, generating a geometric shape and a physical attribute through a physical enhanced three-dimensional shape generation module, and evaluating the visual authenticity and physical rationality of the generated geometric shape and physical attribute through a multi-modal judgment module; therefore, the geometrical shape and the physical attribute corresponding to the target three-dimensional object data are obtained, and the defects of the existing three-dimensional generation technology in the aspect of physical authenticity are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional shape generation, and in particular to a three-dimensional shape generation method based on physical information constraints and related equipment. Background Art

[0002] In recent years, with the development of deep learning technology, 3D shape generation has made significant progress. Mainstream methods such as generative adversarial networks generate 3D shapes (point clouds, voxels, meshes, or neural field representations) through adversarial training of generators and discriminators, achieving great success in visual quality. Diffusion models generate 3D data from noise through an iterative denoising process, with excellent performance in terms of generation diversity and quality. Methods based on neural radiance fields and their variants are mainly used to synthesize new 3D views and scene reconstructions from 2D images, capable of generating detailed geometry and appearance. However, these mainstream methods have the following main problems and limitations:

[0003] 1. Lack of physical authenticity: Most existing 3D generative models focus primarily on geometric accuracy and visual fidelity, while ignoring the physical properties of the generated objects (such as mass distribution, material properties, stability, structural strength, etc.). As a result, the generated models may exhibit behaviors that do not conform to physical laws in virtual environments and cannot be directly applied to scenarios requiring physical interaction or simulation.

[0004] 2. Difficulty integrating physical constraints: It is challenging to directly integrate complex physical laws (such as Newtonian mechanics, material mechanics, fluid dynamics, etc.) into deep learning models. Physical simulations are often computationally expensive and difficult to efficiently integrate with the training process of generative models.

[0005] 3. Insufficient controllability: It is difficult for users to precisely control the physical properties of the generated shapes. For example, even if a visual model of a chair can be generated, it is difficult to ensure that its structure is stable or conforms to the mechanical properties of specific materials.

[0006] 4. Dataset limitations: Large-scale 3D model datasets with precise physical property annotations are relatively scarce, which limits the application of supervised learning methods in physical information modeling methods. Summary of the Invention

[0007] The present invention provides a three-dimensional shape generation method based on physical information constraints and related equipment, the purpose of which is to make up for the shortcomings of existing three-dimensional generation technology in terms of physical authenticity.

[0008] To achieve the above object, the present invention provides a method for generating a three-dimensional shape based on physical information constraints, comprising:

[0009] Step 1: Sample a batch of training samples containing real geometric shapes and real physical properties from the real 3D object dataset, and sample a batch of latent vectors from a predefined mathematical probability distribution;

[0010] Step 2: Use the training samples and latent vectors to train the constructed generative model to obtain a 3D shape generative model;

[0011] Step 3: Input the target 3D object data into the 3D shape generation model to generate the 3D shape, and obtain the geometric shape and physical properties corresponding to the target 3D object data;

[0012] The generative model includes a physics-enhanced 3D shape generation module for generating geometric shapes and physical properties, and a multimodal discrimination module for evaluating the visual realism and physical plausibility of the generated geometric shapes and physical properties.

[0013] Furthermore, the generative model also includes an information encoding module for processing the input conditional information into a conditional embedding vector;

[0014] The output end of the information encoding module is connected to the second input end of the physical enhanced three-dimensional shape generation module.

[0015] More specifically, step 2 includes:

[0016] A batch of conditional information is sampled from the predefined conditional information distribution and input into the information encoding module to generate a conditional embedding vector;

[0017] The conditional embedding vector and a batch of latent vectors sampled from a predefined mathematical probability distribution are input into the physically enhanced 3D shape generation module to generate fake samples;

[0018] Input the training samples and fake samples into the multimodal discriminator to evaluate the visual authenticity and physical rationality, and obtain the evaluation results;

[0019] According to the evaluation results, the total loss of the multimodal discrimination module and the physically enhanced 3D shape generation module is calculated, and the parameters of the multimodal discrimination module and the physically enhanced 3D shape generation module are updated by the total loss of the multimodal discrimination module and the physically enhanced 3D shape generation module to obtain the 3D shape generation model.

[0020] Furthermore, the multimodal discriminator module includes a visual discriminator and a physical discriminator;

[0021] The input end of the visual discriminator and the input end of the physical discriminator are both connected to the output end of the physical enhanced 3D shape generation module;

[0022] The visual discriminator is used to evaluate the visual realism, rationality of geometric structure and fineness of details of the generated geometric shapes;

[0023] The physical discriminator is used to evaluate the rationality and consistency of the generated geometric shapes and physical properties at the physical level.

[0024] Furthermore, the multimodal discrimination module also includes a physical simulator;

[0025] The physical simulator is used to simulate the input geometric shape and physical properties as well as a set of external forces to obtain simulation results;

[0026] The output of the physical simulator is connected to the input of the physical discriminator.

[0027] Specifically, the total loss of the physically enhanced 3D shape generation module is expressed as:

[0028]

[0029] Among them, L G represents the total loss value of the physical enhanced 3D shape generation module, w adv_vis 、w adv_phy 、w pc Both represent loss weights, Both represent the adversarial loss of the physically enhanced 3D shape generation module, L pc Indicates loss of physical consistency.

[0030] Furthermore, the calculation expression of physical consistency loss is:

[0031] L pc =λ mat L mat +λ stab L stab +λ sim_dev L sim_dev +λ PDE L PDE

[0032] Among them, L pc represents the physical consistency loss value, λ mat ,λ stab ,λ sim_dev ,λ PDE Both represent non-negative weight hyperparameters, L mat Represents the loss of material property range, L stab represents the loss of static stability, L sim_dev Indicates the physical simulation deviation loss, L PDE Represents the residual loss of the physical equation.

[0033] Furthermore, the total loss of the multimodal discriminant module is calculated as:

[0034]

[0035] Among them, L D Represents the total loss value of the multimodal discrimination module, Both represent the adversarial loss of the multimodal discriminative module.

[0036] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, a three-dimensional shape generation method based on physical information constraints is implemented.

[0037] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, a three-dimensional shape generation method based on physical information constraints is implemented.

[0038] The above solution of the present invention has the following beneficial effects:

[0039] The present invention samples a batch of training samples containing real geometric shapes and real physical properties from a real three-dimensional object data set, and samples a batch of latent vectors from a predefined mathematical probability distribution; uses the training samples and latent vectors to train the constructed generative model to obtain a three-dimensional shape generation model; inputs the target three-dimensional object data into the three-dimensional shape generation model, generates geometric shapes and physical properties through a physically enhanced three-dimensional shape generation module, and evaluates the visual authenticity and physical rationality of the generated geometric shapes and physical properties through a multimodal discrimination module, thereby obtaining the geometric shapes and physical properties corresponding to the target three-dimensional object data; compared with the existing technology, the present invention evaluates the visual authenticity and physical rationality of the geometric shapes and physical properties generated by the physically enhanced three-dimensional shape generation module through a multimodal discrimination module, thereby making up for the shortcomings of the existing three-dimensional generation technology in terms of physical authenticity.

[0040] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of the structure of a generation model in an embodiment of the present invention;

[0043] Figure 3 Schematic diagram of the structure of the terminal device in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0045] In the description of the present invention, it should be noted that the terms "first", "second" and "third" are only used for descriptive purposes and should not be understood as indicating or implying relative importance.

[0046] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0047] In addition, the technical features involved in different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0048] In response to existing problems, the present invention provides a three-dimensional shape generation method based on physical information constraints and related equipment.

[0049] like Figure 1 As shown, an embodiment of the present invention provides a three-dimensional shape generation method based on physical information constraints, comprising:

[0050] Step 1: Sample a batch of training samples containing real geometric shapes and real physical properties from the real 3D object dataset, and sample a batch of latent vectors from a predefined mathematical probability distribution;

[0051] Step 2: Use the training samples and latent vectors to train the constructed generative model to obtain a 3D shape generative model;

[0052] Step 3: Input the target three-dimensional object data into the three-dimensional shape generation model to generate the three-dimensional shape, and obtain the geometric shape and physical properties corresponding to the target three-dimensional object data. The geometric shape can be a point cloud set, a voxel grid, a parameterized surface or a signed distance function, and the physical property can be the density distribution or elastic modulus of each geometric part or spatial position.

[0053] In an embodiment of the present invention, the latent vector z refers to a multidimensional numerical vector, which is randomly sampled from a predefined mathematical probability distribution N(0,1) as a source of generating diversity and is used to support the generation of diverse three-dimensional shapes. The predefined mathematical probability distribution N(0,1) refers to a Gaussian distribution with a mean of 0 and a standard deviation of 1, such as a normal distribution.

[0054] Specifically, if Figure 2 As shown, the generative model includes a physically enhanced 3D shape generation module for generating geometric shapes and physical properties, and a multimodal discrimination module for evaluating the visual authenticity and physical rationality of the generated geometric shapes and physical properties, which are connected in sequence.

[0055] Most preferably, the generative model further comprises an information encoding module for processing the input conditional information into a conditional embedding vector;

[0056] The output end of the information encoding module is connected to the second input end of the physical enhanced three-dimensional shape generation module.

[0057] In the embodiment of the present invention, the conditional information can be a label describing the required object category, a detailed text description, or a desired range of physical properties. When the conditional information does not exist, the information encoding module can be removed or a fixed zero vector can be used to replace the conditional information and input it into the information encoding module. When the conditional information exists, the information encoding module processes the conditional information into a fixed-dimensional conditional embedding vector, which is expressed as follows:

[0058]

[0059] Where c represents the input conditional information, which can be a text string, a category index, a set of target physical parameters, etc. CE(·) represents the conditional encoding function. Depending on the type of conditional information, a text encoder based on the Transformer architecture can be selected for processing text strings, a simple embedding lookup table for processing category labels, or a small multi-layer perceptron for processing numerical physical parameters. emb Represents the conditional embedding vector, which is used to represent the core information in the conditional information. represents the real vector space, d c Indicates the dimension size of the preset conditional embedding vector, such as 256 or 512.

[0060] Specifically, the physically enhanced 3D shape generation module is the core module of the generative model. Its goal is to simultaneously generate the geometric shape and physical properties of a 3D object based on the input latent vector and conditional embedding vector. The expression is:

[0061] (X shape ,P phy )=G(z,cemb )

[0062] Wherein, z represents the latent vector, which indicates the randomness introduced in the generation process, so that the model can generate diverse shapes. G(·) represents the generator. In the embodiment of the present invention, the generator is implemented using a deep neural network, such as an architecture based on PointNet, Transformer, or Implicit Neural Representations (INR). X shape Represents the output 3D geometry, which can have multiple representations:

[0063] Point cloud collection: a collection of unordered points Represents each point cloud p i =(x i ,y i ,z i ) represents a sampling point on the object surface, and N represents the total number of point clouds;

[0064] Voxel grid: A three-dimensional grid where each grid cell is assigned a value as a voxel, indicating whether the voxel is occupied by an object;

[0065] Parametric surface: a surface mesh defined by vertices and faces;

[0066] Signed distance function: A function f(x,y,z) that returns the closest distance to the surface of an object for any point in space.

[0067] P phy Represents physical properties, whose structure depends on X shape Representation and physical properties that need to be modeled:

[0068] If X shape is the point cloud, P phy It can be for each point p i An assigned set of physical property values, such as the material density ρ at that point i , elastic modulus E i Wait, at this time,

[0069] If X shape is the signed distance function or voxel grid, P phy It can be one or more continuous or discrete physical property fields aligned with the geometric field, such as the density field ρ(x,y,z) or temperature field T(x,y,z) at each voxel or spatial query point.

[0070] In the embodiment of the present invention, the interior of the physical enhanced three-dimensional shape generation module can be designed as a multi-branch or multi-head neural network structure, in which one branch or head is dedicated to generating the geometric shape X.shape , while the other one or more branches or heads are responsible for predicting the corresponding physical properties P phy and ensure that the geometry X shape and physical properties P phy The inherent correlation between them.

[0071] In order to ensure that the generated three-dimensional geometric shapes and physical properties are visually realistic and physically reasonable, the embodiment of the present invention adopts a multimodal discrimination module.

[0072] Most preferably, the multimodal discriminator module includes a visual discriminator and a physical discriminator;

[0073] The input end of the visual discriminator and the input end of the physical discriminator are both connected to the output end of the physical enhanced 3D shape generation module;

[0074] The visual discriminator is used to evaluate the visual realism, rationality of geometric structure and fineness of details of the generated geometric shapes;

[0075] The physical discriminator is used to evaluate the rationality and consistency of the generated geometric shapes and physical properties at the physical level.

[0076] Specifically, the multimodal discrimination module also includes a physical simulator;

[0077] The physical simulator is used to simulate the input geometric shape and physical properties as well as a set of external forces to obtain simulation results;

[0078] The output of the physical simulator is connected to the input of the physical discriminator.

[0079] In this embodiment of the present invention, the visual discriminator and the physical discriminator can be trained independently or share some underlying network layers. The visual discriminator is responsible for evaluating the visual realism, rationality of the geometric structure, and fineness of details of the generated geometric shapes, and determining whether they are similar to three-dimensional objects in the real world. The expression is:

[0080] s vis =D vis (X shape )

[0081] Among them, X shape Represents the input geometry. During the training process, this data is the real geometry in the training sample. During the actual reasoning process, this data is the geometry output by the physical enhanced 3D shape generation module. vis(·) represents the visual discriminant function, which is usually a deep neural network. Its architecture depends on the representation of the geometric shape. For example, for a point cloud set, a 3D point cloud network (Point Net) or a Transformer classifier is used. For a voxel grid, a 3D convolutional network is used. For a rendered image, a 2D convolutional network can be used. It represents the scalar value output by the visual discriminator, s vis Represents the geometric shape X shape The probability or score of being judged as "real" (rather than "generated"), where the higher the score, the more visually realistic it is;

[0082] The physical discriminator is responsible for evaluating the rationality and consistency of the generated geometric shapes and physical properties at the physical level. Its expression is:

[0083] s phy =D phy (X shape ,P phy )or s phy =D phy (R sim )

[0084] Among them, P phy Indicates the physical properties of the input. During the training process, this data is the real physical properties of the training sample. During the actual reasoning process, this data is the physical properties output by the physical enhanced 3D shape generation module. phy (·) represents the physical discriminant function, which can be implemented in many ways:

[0085] Directly discriminate attribute distribution: D phy It can be a neural network that learns the physical properties P of real objects phy The statistical distribution of (possibly also dependent on the geometry X shape ), and judge the generated P phy Whether it conforms to the learned distribution;

[0086] Judgment based on simulation results: In this case, first generate (X shape ,P phy ) and a set of preset external forces or boundary conditions Forces ext (e.g. gravity, external loads) are input to a (lightweight, differentiable or approximate) physical simulator Sim(·), which outputs the simulation results R sim , such as the deformation of the object, stress distribution, whether it remains stable, etc. Then, D phy Receive this simulation result R sim As input, and judge whether the simulation behavior is real or in accordance with the expected physical criteria, the simulation result R simIt can be the displacement field of an object under a specific load, a stress tensor field, or a Boolean value / scalar indicating whether the object is stable;

[0087] s phy It is a scalar value output by the physical discriminator, which is used to represent the probability or score that the input geometry and physical properties or simulation results are judged to be "physically reasonable".

[0088] Specifically, step 2 includes:

[0089] A batch of conditional information is sampled from the predefined conditional information distribution and input into the information encoding module to generate a conditional embedding vector;

[0090] The conditional embedding vector and a batch of latent vectors sampled from a predefined mathematical probability distribution are input into the physically enhanced 3D shape generation module to generate fake samples;

[0091] Input the training samples and fake samples into the multimodal discriminator to evaluate the visual authenticity and physical rationality, and obtain the evaluation results;

[0092] According to the evaluation results, the total loss of the multimodal discrimination module and the physically enhanced 3D shape generation module is calculated, and the parameters of the multimodal discrimination module and the physically enhanced 3D shape generation module are updated by the total loss of the multimodal discrimination module and the physically enhanced 3D shape generation module to obtain the 3D shape generation model.

[0093] In the embodiment of the present invention, the conditional embedding vector and a batch of latent vectors sampled from a predefined mathematical probability distribution are input into the physical enhanced 3D shape generation module to generate a fake sample (X fake ,P fake )=G(z,c emb ).

[0094] Specifically, in order to effectively train the physically enhanced 3D shape generation module and the multimodal discrimination module, the embodiment of the present invention adopts a composite loss function composed of multiple weighted loss terms, which mainly includes adversarial loss and physical consistency loss;

[0095] The adversarial loss is used to drive the physically enhanced 3D shape generation module to produce outputs that can "fool" the multimodal discriminant module, while driving the multimodal discriminant module to improve its ability to distinguish between real samples and generated samples. The visual discriminator and the physical discriminator each correspond to an adversarial loss term.

[0096] Among them, the adversarial loss L of the visual discriminator adv_vis The calculation expression is:

[0097]

[0098] Adversarial loss L for physical discriminator adv_phy The calculation expression is:

[0099]

[0100] in, represents the mathematical expectation operator, X real ~p data Represents the dataset p from real 3D objects data A real geometric shape X sampled from real ,(X real ,P real )~p data Represents the dataset p from real 3D objects data A real geometric shape X sampled from real and real physical properties P real , z~p z Represents a predefined mathematical probability distribution p z A latent vector z,c~p sampled from c Represents a conditional information sampled from a predefined conditional information distribution. f1(·) and f2(·) both represent the specific form of the adversarial loss function. For example, in the standard SGAN, for the discriminator, f1(x) = -log(x) and f2(x) = -log(1-x); for the generator, the f1 term is usually not directly used for the generator loss, and f2(x) becomes -log(x). Other forms such as WGAN-GP loss and Hinge Loss can also be used. In this case, the forms of f1 and f2 will change accordingly. For example, for the generator loss of WGAN-GP,

[0101] The physical consistency loss is used to directly impose physical laws or desired physical properties on the output of the physics-enhanced 3D shape generation module. It can contain one or more of the following specific sub-items, which are selected and combined according to the application scenario and the physical phenomenon to be modeled. Specific sub-items include:

[0102] Material property range loss L mat , which is used to penalize the generated physical property values ​​that are beyond the predefined reasonable range or do not conform to the input condition information. The calculation expression is:

[0103]

[0104] Where K represents the total number of physical attribute types considered, Represents the value of the kth physical property generated by the input training sample (z, c) or its statistics, such as the average value, the value at a specific location, [min k ,maxk ] represents the minimum and maximum values ​​of the preset valid range of the kth physical property, such as the density range of wood, Represents an indicator function. When the condition (physical property value is out of range) is met, its value is 1, otherwise it is 0. Represents a measure Exceed [min k ,max k ] function, for example Where clamp(v,min,max) limits v to the interval [min,max], w k represents the weight of the k-th material property loss;

[0105] Static stability loss L stab : For self-supporting objects, such as furniture and building structures, this loss is used to evaluate their structural stability under a standard gravity field. The calculation expression is:

[0106]

[0107] in, density represents the geometric shape, physical properties and corresponding density distribution generated by the input (z, c), respectively. StabilityMetric(·) represents a metric function used to quantify the stability of an object. For example, it can calculate whether the center of mass (CoM) of an object is inside its support polygon (SP). If the CoM is inside the SP, the function returns a positive value (indicating stability, the larger the value, the more stable it is). If the CoM is outside the SP, it returns a negative value (indicating instability, the larger the absolute value, the more unstable it is). ReLU(·) represents the rectified linear unit activation function. The reason for using ReLU(·) here is that it is expected to generate a positive loss when StabilityMetric is negative (unstable), and a loss of zero when stable.

[0108] Physical simulation deviation loss L sim_dev :If the expected behavior or simulation result R of the target object in a specific physical scenario can be obtained target , which may come from real-world experimental data, the output of high-precision simulation software, or user-defined goals, then this loss term is used to penalize the generated geometry and physical properties (X shape ,P phy ) The simulation results R obtained by using a differentiable or approximate physical simulator Sim(·) in the same scenario sim and the target result R tar get The deviation between them is calculated as:

[0109]

[0110] Among them, scenario represents a specific physical scenario, including external forces And the corresponding target simulation results Sim(·) represents the physical simulator function, Represents the p-th power of the Lp norm. Usually, the L1 norm (p=1) or L2 norm (p=2) is used to measure R sim and R target the differences between;

[0111] Physical equation residual loss L PDE :For some physical systems whose behavior can be accurately described by partial differential equations (PDEs), such as fluids whose stress-strain relationship of elastic materials follows the Navier-Stokes equations, the residuals of these PDEs can be used as loss terms. This draws on the core idea of ​​​​Physics-Informed Neural Networks (PINNs), which encourages the output of the physically enhanced 3D shape generation module to be regarded as a candidate solution to the PDE, satisfying these governing equations as much as possible within the domain of definition:

[0112]

[0113] Where G(z,CE(c)) represents the field function output by the physical enhanced 3D shape generation module, such as the displacement field u(x) and the velocity field v(x). represents a partial differential operator acting on a field function, such as For elasticity, Ω represents the spatial domain in which the physical problem is defined, ∫ Ω ||·|| 2 dx represents the integration of the L2 norm square of the PDE residual over the spatial domain Ω, which measures the degree to which the PDE is satisfied within the domain. Represents the boundary condition operator acting on the field function, g(x) is defined on the boundary The known boundary condition values ​​on Indicates that at the border The L2 norm square of the boundary condition residual is integrated to measure the degree to which the boundary condition is satisfied.

[0114] In summary, the calculation expression of physical consistency loss is:

[0115] L pc =λ mat L mat +λ stab Lstab +λ sim_dev L sim_dev +λ PED L PED

[0116] Among them, L pc represents the physical consistency loss value, λ mat ,λ stab ,λ sim_dev ,λ PDE Both represent non-negative weight hyperparameters used to balance the importance of different physical constraints. mat Represents the loss of material property range, L stab represents the loss of static stability, L sim_dev Indicates the physical simulation deviation loss, L PDE Represents the residual loss of the physical equation.

[0117] Specifically, the total loss of the physically enhanced 3D shape generation module is calculated as:

[0118]

[0119] Among them, L G represents the total loss value of the physical enhanced 3D shape generation module, w adv_vis 、w adv_phy 、w pc Both represent loss weights, Both represent the adversarial loss of the physically enhanced 3D shape generation module, L pc Indicates loss of physical consistency.

[0120] Specifically, the calculation expression of the total loss of the multimodal discriminant module is:

[0121]

[0122] Among them, L D Represents the total loss value of the multimodal discrimination module, Both represent the adversarial loss of the multimodal discriminative module.

[0123] In this embodiment of the present invention, if a Wasserstein Generative Adversarial Networks-Gradient Penalty (WGAN-GP) model is used, it is necessary to add a gradient penalty term to the loss calculation process of the multimodal discrimination module, and the resulting expression is:

[0124]

[0125] Among them, L GPrepresents the gradient penalty term, λ GP Represents the weight of the penalty term, which is used to stabilize the training of the WGAN-GP model.

[0126] Specifically, the parameters of the multimodal discriminant module and the physically enhanced 3D shape generation module are updated through the total loss of the multimodal discriminant module and the physically enhanced 3D shape generation module, including:

[0127] The parameters of the multimodal discriminant module are updated by the total loss of the multimodal discriminant module. The process is as follows:

[0128] The training sample (X real ,P real )(or just X real To D vis ,(X real ,P real ) to D phy ) and the generated fake samples (X fake ,P fake ) or just X fake Input to the visual discriminator D vis ,(X fake ,P fake ) is input to the physical discriminator D phy middle;

[0129] According to the output of the discriminator, calculate the total loss L of the discriminator D (include and possible gradient penalty term L GP );

[0130] Calculate L by back propagation algorithm D Compared with the multimodal discrimination module (including D vis and D phy ) parameters and use optimizers such as Adam and Stochastic Gradient Descent (SGD) to update the parameters of the multimodal discriminant module, with the goal of minimizing L D (or maximize its original objective function), usually, the multimodal discriminant module is updated multiple times (for example, k times) corresponding to the generator update once;

[0131] The parameters of the physically enhanced 3D shape generation module are updated by the total loss of the physically enhanced 3D shape generation module. The process is as follows:

[0132] Fix the parameters of the multimodal discrimination module;

[0133] The latent vector z and the conditional embedding vector are input into the physical enhanced 3D shape generation module to obtain the generated fake sample (X fake ,P fake );

[0134] The generated X fake Input to the visual discriminator D vis , (X fake ,P fake ) is input to the physical discriminator D phy ;

[0135] Calculate the total loss L of the physics-enhanced 3D shape generation module G , which includes adversarial feedback from the multimodal discriminative module and the physical consistency loss L pc , by L mat , L stab , L sim_dev , L PDE etc.

[0136] Calculate L by back propagation algorithm G The gradient of the parameters of the physically enhanced 3D shape generation module is calculated and the parameters of the physically enhanced 3D shape generation module are updated using an optimizer with the goal of minimizing L G ;

[0137] By repeating the above updating process until the generative model reaches the preset number of training rounds, the loss function converges, or the performance on the validation set no longer improves, a 3D shape generation model is obtained.

[0138] The embodiment of the present invention takes the example of generating a 3D furniture model with stable structure and reasonable material properties (such as density) to further illustrate the provided method, as follows:

[0139] Assume that the information encoding module CE model and physically enhanced 3D shape generation module G model It has been pre-trained and loaded. The inference process steps are as follows:

[0140] Model and parameter settings:

[0141] CE model :The input category index c index Convert to conditional embedding vector c emb , which can be realized as c emb =EmbeddingLayer(c index ).

[0142] G model : Receives the latent vector z and the conditional embedding c emb As input, output geometry X shape , for example, a point cloud of N points with dimensions of BatchSize×N×3 and corresponding physical properties P phy_densityFor example, the density of each point has the dimension of BatchSize×N×1, and the calculation expression is:

[0143] (X shape ,P phy_density )=G(z,c emb )

[0144] Physically enhanced 3D shape generation module G model The internal layer may contain a multi-layer perceptron (MLP) for feature extraction and transformation, as well as a specific output layer for generating shape and density. The density value may be scaled according to the target range [min_density, max_density] after passing an activation function such as Sigmoid, for example, P phy_density =min_density+Sigmoid(output)·(max_density-min_density), or directly take the absolute value P phy_density =|output| to ensure its physical meaning, such as positive value;

[0145] Set relevant parameters, such as the total number of categories, embedding dimension, latent vector dimension, number of points in the generated point cloud, etc.

[0146] Prerequisites:

[0147] Specify the category name of the target object, such as "chair", obtain its corresponding category index target_category_index, and convert the index into a tensor form;

[0148] Condition information encoding:

[0149] Pass CE model Encode the category index tensor into a conditional embedding vector c emb_tensor ;

[0150] c emb,tensor =CE model (target_category,index,tensor)

[0151] Latent vector sampling:

[0152] Randomly sample one or a batch of latent vectors z from a standard normal distribution tensor ;

[0153] Physics-enhanced 3D shape generation module reasoning:

[0154] G model Set to evaluation mode, such as G model.eval() to turn off specific layers used during training, such as the Dropout layer.

[0155] In a mode where gradients are not calculated, for example, torch.no g rad(), which performs the forward propagation of the physically enhanced 3D shape generation module:

[0156] X shape_tensor ,P phy_density_tensor =G model (z tensor ,C emb_tensor )

[0157] Data post-processing:

[0158] The tensor output by the physics-enhanced 3D shape generation module is usually converted to a Numpy array on the host side (Central Processing Unit, CPU) on the Graphics Processing Unit (GPU) for subsequent physical evaluation or visualization. If a batch of data is processed, one sample is selected for analysis. The expression is:

[0159] X shape_numpy =X shape_tensor [0].cpu().numpy()

[0160] P phy_density_numpy =P phy_density_tensor [0].cpu().numpy()

[0161] Physical Assessment (optional post-processing inspection):

[0162] Stability assessment: Use the aforementioned check s tability algorithm / function, input X shape_numpy and P phy_density_numpy , get the stability score stability_score;

[0163] Material density range check: define an auxiliary algorithm check m aterial d ensity r Ange, enter P phy_density_numpy As well as the preset minimum density min_val and maximum density max_val, the algorithm checks whether the density values ​​of all points are in the interval [min_val, max_val];

[0164]

[0165] Output or record the results of these assessments;

[0166] Visualization (optional):

[0167] Use appropriate visualization tools to visualize the generated point cloud X shape_numpy Shown, such as libraries based on Matplotlib, Open3D, etc.

[0168] Through the above process, a three-dimensional state model with corresponding geometric shape and physical properties (such as density) can be generated according to the user-specified category, and its physical rationality (such as stability and material properties) can be preliminarily evaluated.

[0169] To verify the effectiveness of the method provided by the embodiment of the present invention, the method provided by the embodiment of the present invention is compared with a baseline 3D generative model that does not include explicit physical constraints, such as a standard 3D Generative Adversarial Network (3D-GAN) or a diffusion model without physical guidance. The process is as follows:

[0170] Evaluation Metrics:

[0171] Visual / geometric quality:

[0172] Chamfer Distance (CD): measures the average closest point distance between the generated shape and the true shape (or a set of reference shapes). The smaller the value, the better.

[0173] F-Score: The harmonic mean of precision and recall at the distance threshold d, which measures the matching degree of the point cloud. The larger the value, the better.

[0174] Manual evaluation: Domain experts or crowdsourced users subjectively score the visual realism and reasonableness of details of the generated shapes;

[0175] Physical plausibility:

[0176] Stability Success Rate (SSR): For objects such as furniture, the percentage of objects that do not fall over under gravity is tested in a standard physics simulation environment (such as PyBullet). The larger the value, the better.

[0177] Material Property Rate (MPR): The percentage of samples whose generated material property (e.g., density) values ​​fall within a predefined reasonable or target range. The higher the value, the better.

[0178] Stress Constraint Satisfaction Rate (SCSR): For the mechanical parts in Example 2, under preset standard load conditions, the percentage of samples whose maximum von Mises stress does not exceed the safety threshold verified by FEA simulation. The larger the value, the better.

[0179] Average Physical Metric Deviation (APMD): For example, for stress, we can calculate mean(max(0,σ max -StressLimit)) measures the average degree of exceeding the constraint, the smaller the value, the better;

[0180] Generate diversity:

[0181] For example, computing the average pairwise distance in the feature space of the generated sample set, or using other standard diversity metrics;

[0182] Expected experimental results (example table comparing with baseline model):

[0183] The performance comparison of the generative model provided by the embodiment of the present invention and the baseline model on the furniture generation task is shown in Table 1 below:

[0184] Table 1 Performance comparison results

[0185]

[0186] As shown in Table 1 above, this method significantly outperforms existing technologies that do not consider or only implicitly consider physical constraints in all physical evaluation indicators, while maintaining good visual quality and generation diversity.

[0187] The embodiment of the present invention samples a batch of training samples containing real geometric shapes and real physical properties from a real three-dimensional object dataset, and samples a batch of latent vectors from a predefined mathematical probability distribution; uses the training samples and latent vectors to train the constructed generative model to obtain a three-dimensional shape generation model; inputs the target three-dimensional object data into the three-dimensional shape generation model, generates geometric shapes and physical properties through a physically enhanced three-dimensional shape generation module, and evaluates the visual authenticity and physical rationality of the generated geometric shapes and physical properties through a multimodal discrimination module, thereby obtaining the geometric shapes and physical properties corresponding to the target three-dimensional object data; compared with the existing technology, the embodiment of the present invention evaluates the visual authenticity and physical rationality of the geometric shapes and physical properties generated by the physically enhanced three-dimensional shape generation module through a multimodal discrimination module, thereby making up for the shortcomings of the existing three-dimensional generation technology in terms of physical authenticity.

[0188] The embodiment of the present invention further provides a terminal device, such as Figure 3 As shown, the terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 3 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100. When the processor D100 executes the computer program D102, the above-mentioned three-dimensional shape generation method based on physical information constraints is implemented.

[0189] The terminal device D10 can be a computing device such as a desktop computer, a notebook, a PDA, a server, a server cluster, a cloud server, etc. The terminal device may include, but is not limited to, a processor D100 and a memory D101. It will be understood by those skilled in the art that Figure 3 This is merely an example of the terminal device D10 and does not constitute a limitation on the terminal device D10 . The terminal device D10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device D10 may also include input and output devices, network access devices, etc.

[0190] The processor D100 may be a central processing unit (CPU), or may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0191] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card, etc. equipped on the terminal device D10. Furthermore, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0192] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0193] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0194] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the computer program implements a three-dimensional shape generation method based on physical information constraints.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to a construction device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0196] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A three-dimensional shape generation method based on physical information constraints, characterized in that: include: Step 1: Sample a batch of training samples containing real geometric shapes and real physical properties from the real 3D object dataset, and sample a batch of latent vectors from a predefined mathematical probability distribution; Step 2: training the constructed generative model using the training samples and the latent vector to obtain a three-dimensional shape generative model; Step 3: Inputting the target three-dimensional object data into the three-dimensional shape generation model to generate a three-dimensional shape, thereby obtaining a geometric shape and physical properties corresponding to the target three-dimensional object data; The generation model includes a physics-enhanced three-dimensional shape generation module for generating geometric shapes and physical properties, and a multimodal discrimination module for evaluating the visual authenticity and physical rationality of the generated geometric shapes and physical properties.

2. The three-dimensional shape generation method based on physical information constraints according to claim 1, characterized in that: The generative model further includes an information encoding module for processing input conditional information into a conditional embedding vector; The output end of the information encoding module is connected to the second input end of the physical enhanced three-dimensional shape generation module.

3. The method for generating three-dimensional shapes based on physical information constraints according to claim 1, characterized in that: The step 2 includes: Sampling a batch of conditional information from a predefined conditional information distribution and inputting it into the information encoding module to generate a conditional embedding vector; Input the conditional embedding vector and a batch of latent vectors sampled from a predefined mathematical probability distribution into the physically enhanced 3D shape generation module to generate fake samples; Inputting the training sample and the fake sample into the multimodal discriminator to perform visual authenticity and physical rationality evaluation to obtain an evaluation result; The total loss of the multimodal discrimination module and the physically enhanced three-dimensional shape generation module is calculated according to the evaluation results, and the parameters of the multimodal discrimination module and the physically enhanced three-dimensional shape generation module are updated according to the total loss of the multimodal discrimination module and the physically enhanced three-dimensional shape generation module to obtain a three-dimensional shape generation model.

4. The method for generating a three-dimensional shape based on physical information constraints according to claim 3, characterized in that: The multimodal discrimination module includes a visual discriminator and a physical discriminator; The input end of the visual discriminator and the input end of the physical discriminator are both connected to the output end of the physical enhanced three-dimensional shape generation module; The visual discriminator is used to evaluate the visual realism, rationality of the geometric structure and fineness of details of the generated geometric shapes; The physical discriminator is used to evaluate the rationality and consistency of the generated geometric shapes and physical properties at the physical level.

5. The method for generating a three-dimensional shape based on physical information constraints according to claim 4, characterized in that: The multimodal discrimination module further includes a physical simulator; The physical simulator is used to simulate the input geometric shape and physical properties and a set of external forces to obtain simulation results; The output end of the physical simulator is connected to the input end of the physical discriminator.

6. The method for generating three-dimensional shapes based on physical information constraints according to claim 3, characterized in that: The calculation expression of the total loss of the physical enhanced three-dimensional shape generation module is: Among them, L G represents the total loss value of the physical enhanced 3D shape generation module, w adv_vis 、w adv_phy 、w pc Both represent loss weights, Both represent the adversarial loss of the physically enhanced 3D shape generation module, L pc Indicates loss of physical consistency.

7. The method for generating three-dimensional shapes based on physical information constraints according to claim 3, characterized in that: The calculation expression of the physical consistency loss is: L pc =λ mat L mat +λ stab L stab +λ sim_dev L sim_dev +λ PDE L PDE Among them, L pc represents the physical consistency loss value, λ mat ,λ stab ,λ sim_dev ,λ PDE Both represent non-negative weight hyperparameters, L mat Represents the loss of material property range, L stab represents the loss of static stability, L sim_dev Indicates the physical simulation deviation loss, L PDE Represents the residual loss of the physical equation.

8. The method for generating three-dimensional shapes based on physical information constraints according to claim 3, characterized in that: The calculation expression of the total loss of the multimodal discrimination module is: Among them, L D Represents the total loss value of the multimodal discrimination module, Both represent the adversarial loss of the multimodal discriminative module.

9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the three-dimensional shape generation method based on physical information constraints according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the three-dimensional shape generation method based on physical information constraints according to any one of claims 1 to 8 is implemented.

Images