A 3D biomimetic generative system and method based on an interpolation operator

Abstract

The present invention discloses a three-dimensional bionic object generation system based on an interpolation operator, the system comprising a data preprocessing module, a deep generation model, a sample vector storage library, and a post-processing module. The present invention also discloses a three-dimensional bionic object generation method based on the generation system, the method comprising: step 1: establishing a training set; step 2: establishing a deep generation model, and training the deep generation model through the training set; the deep generation model comprises an implicit autoencoder and a latent vector generation model; the implicit autoencoder comprises an encoder 3DCNN and a decoder IM‑Decoder; the latent vector generation model adopts an interpolation operator; step 3: using the deep generation model to generate a three-dimensional bionic object.

Description

Technical Field

[0001] The invention belongs to the field of machine learning and intelligent technology, and relates to a three-dimensional bionic object generation system and method based on an interpolation operator. Background Art

[0002] Bionics is the science of building technical systems by imitating the principles of biological systems, or making artificial technical systems have or be similar to the characteristics of biological systems. The main research method of bionics is to propose theoretical models and realize the bionic principles to functions through mathematical simulation. Bionics Design is to simulate the structure, function, form, color and other characteristics of nature and biological systems to carry out creative design. Bionics design starts from people's psychological needs and takes the designer's personal bionic concept as its sustenance to create design products full of biological interest and natural forms, which in turn serve humans, thus completing a natural state from man to man cycle. Creative bionic product design is carried out by imitating nature or being inspired by nature and biology. On the basis of observing and refining the characteristics of natural objects, designers use creative form design techniques such as association, exaggeration and simplification, combined with factors such as design objects and product usage scenarios to form design plans, and then go through in-depth design suitable for production and manufacturing technology needs to complete the product design process. According to the different ways of applying bionics, bionic design is mainly divided into product form bionics, functional bionics, structural bionics, color bionics, texture bionics, etc. Morphological bionics is widely used in modern product design, such as daily necessities, small appliances, furniture, transportation, etc. Today's designers have turned their product modeling innovations to the field of bionic design.

[0003] Outstanding and creative bionic product design is related to the designer's own talent, and is accidental and uncertain, such as Colani's bionic product design. In order to produce more excellent creative bionic designs, academia and industry have been trying to explore the laws of bionic morphological creative design with the help of computer technology for many years. Before applying deep generative models to product morphological bionic design, academia has established a modeling database and extraction rules through evolutionary design, used evolutionary algorithms to define fitness and product feature models, and extracted the contour feature elements of bionic objects to generate diversified design solutions on product contours. Recent studies have attempted to use deep learning technology to automatically synthesize the contour of bionic products. In particular, Simiao Yu et al. first applied deep generative models such as generative adversarial networks to the bionic design of product appearance. They mathematically defined the bionic design problem from the perspective of deep learning, and thus designed a set of deep generative models to form the bionic design of product contours.

[0004] The deep generative model that combines autoencoders and generative adversarial networks (GANs) has shown unique advantages and great potential in the fields of plane image generation, restoration, and style transfer. In the field of three-dimensional design, in 2016, Wu et al. from MIT proposed a specific method for applying generative adversarial networks to three-dimensional shape generation. The trained model can restore a photo of a chair into a three-dimensional model of a chair. The voxel-based 3D data model was used, and the concept of latent vector space in three-dimensional shape representation was first proposed. Panos et al. used autoencoders and the intermediate latent generative adversarial network (LatentGAN) to embed and represent three-dimensional point cloud data. The autoencoder was used to directly train a latent vector space based on the three-dimensional data set, and then the bijection between Euclidean space and three-dimensional shapes was used to complete a series of three-dimensional generation tasks, and the existence of the intermediate LatentGAN ensured the diversity of the generated results. Chen et al. used an implicit method to optimize the 3D model fitting effect of the deep generative model LatentGAN, making the effect of the three-dimensional model generated based on interpolation more continuous and smooth, and reused the three-dimensional data of voxels, and used the MarchingCube algorithm to convert the voxel data into a more widely used three-dimensional grid data. However, deep generative models have not yet been applied in the process of bionic product form design. Summary of the invention

[0005] The present invention provides a three-dimensional bionic object generation system based on a deep generation model, the system comprising:

[0006] The data preprocessing module is used to input the 3D shape data set represented by the bionic object and the design target, expand the data set through design target-based and performance-based data enhancement, and obtain the signed distance field of the 3D model required for model training through mesh2voxel and SDF sampler;

[0007] A deep generative model, comprising an implicit autoencoder, a potential generative adversarial network / interpolation operator / creative generator solver; the implicit autoencoder comprises an encoder 3DCNN and a decoder IM-Decoder;

[0008] A sample vector repository, which is used to store latent vector point sets, and the latent vectors in the repository are used to obtain three-dimensional bionic generated objects through an implicit decoder;

[0009] A post-processing module is used to post-process the three-dimensional bionic product to obtain a three-dimensional bionic design.

[0010] In the three-dimensional bionic generated object generation system based on the deep generative model proposed by the present invention, the implicit autoencoder is used to fit the signed distance field of the three-dimensional model to perform the first stage of training; in order to obtain a generated object that meets the design requirements and has high fidelity, it is necessary to search for a suitable latent vector point set in the latent space; the latent vector point set is obtained by Gaussian sampling after training a potential generative adversarial network, or is obtained by interpolating latent vector pairs obtained by an encoder through the three-dimensional model of the bionic object and the design target in a given training set, or is obtained by a creative generated object solver; wherein the model to be trained is subjected to the second stage of training after the first stage of encoding and decoding network training is completed to obtain a callable latent vector generation model.

[0011] 3DCNN Encoder

[0012]

[0013]

[0014] Implicit Decoder

[0015] layer Jump link source Input Shape Activation Function Output shape Label Input latent vector and query coordinate point (128+3) (131) a Fully connected (131) LReLU Fully connected a (2048+131) LReLU Fully connected a (1024+131) LReLU Fully connected a (512+131) LReLU Fully connected a (256+131) LReLU Fully connected (128) Sigmoid (1)

[0016] Latent Generative Adversarial Networks

[0017] Generator

[0018] layer Activation Function Output shape Input latent vector (128) Fully connected LReLU (2048) Fully connected LReLU (2048) Fully connected Sigmoid (128)

[0019] Discriminator

[0020] layer Activation Function Output shape Input latent vector (128) Fully connected LReLU (2048) Fully connected LReLU (2048) Fully connected (1)

[0021] The present invention also proposes a method for generating a three-dimensional bionic object based on the above system, comprising the following steps:

[0022] Step 1: Create a training set;

[0023] Step 2: Establish a deep generative model and train the deep generative model through the training set; the deep generative model includes an implicit autoencoder and a latent vector generative model; the implicit autoencoder includes an encoder 3DCNN and a decoder IM-Decoder; the latent vector generative model uses an interpolation operator;

[0024] Step 3: Generate a three-dimensional bionic object using the deep generative model.

[0025] Step 4: The three-dimensional bionic product is post-processed in collaboration with the designer to obtain a three-dimensional bionic design.

[0026] The step 1 of the present invention includes: constructing an identification conversion path from a 3D file format to a voxel model format, specifically including the following steps:

[0027] Step 1.1: The .obj file of the 3D mesh is voxelized using binvox and converted into a .binvox file;

[0028] Step 1.2: The voxelized file is flood-filled to obtain point value pairs, which is equivalent to discrete sampling of the isosurface;

[0029] Step 1.3: Use the encoder in the implicit autoencoder to learn the mapping of three-dimensional voxels to 128-dimensional latent vectors, and the decoder learns the mapping of the 128-dimensional latent vector plus the three-dimensional voxel position to the signed distance field of the voxel; the signed distance field obtains three-dimensional grid data through the marching cube algorithm; the signed distance field obtains voxel data through thresholding.

[0030] Step 1 of the present invention further includes: a training set enhancement step, which includes design target-based data enhancement and performance-based data enhancement, increasing the total amount of the training set from 113+6000 to more than 20k.

[0031] The data enhancement based on design goals includes: design-oriented data enhancement focuses on transferring design knowledge to the preparation of data sets to meet design requirements. For example, in the designer's design process, the factor of direction is also taken into account. After evaluating existing bionic products and brainstorming, generative design considering multiple directions enables the present invention to explore more possibilities. Considering the symmetry of the target product and the trade-off between diversity and efficiency, the present invention chooses to apply rotation transformations in seven directions to the bionic object data set to achieve diversified hybrid results.

[0032] The performance-based data enhancement includes: performance-oriented data enhancement aims to improve the performance of deep generative models. The generator is affected by unbalanced training data, especially only hundreds of animal data sets. Non-rigid deformation, such as stretching transformation, is an effective way to alleviate this problem. The present invention stretches the training data set on three orthogonal axes, with degrees ranging from 0.25 to 2.

[0033] Step 1 of the present invention further includes: increasing the resolution of the generated voxel file from 16^3 to 32^3 to 64^3.

[0034] It includes: the model generates a signed distance field (SDF), and by controlling the sampling density of the SDF, it allows the generation of voxels of arbitrary resolution. However, the level of detail of the network fitting depends on the resolution of the voxel file used.

[0035] During training, progressive training is used, first training 100 rounds on a 16^3 resolution voxel file, then training 200 rounds on a 32^3 resolution voxel file, and finally training 1000 rounds on a 64^3 resolution voxel file. Training can be terminated during the training process to control the generation of a 3D model with multi-scale details.

[0036] Compared with the traditional decoder based on convolutional neural network, the IM-Decoder can make the latent vector space and the generated three-dimensional voxels more continuous, thereby obtaining a more continuous interpolation effect.

[0037] The loss function of the IM-Decoder model depends on the sampling method, where w p is the weight of the weighted sampling method, which depends on the sparsity of the sampling points. The loss function is the weighted average square error between the true value label and the predicted label of each point. Let S be the point set of the target shape, and the loss function L(θ) is:

[0038]

[0039] Where θ is the network weight, f θ For network mapping, is the true value of the actual signed distance field sampled at point p.

[0040] In the present invention, the latent vector generation model can also use an interpolation operator. The structure of the interpolation operator: given the bionic object latent vector x1 and the design target latent vector x2, the interpolation vector is expressed as x=γx1+(1-γ)x2, where γ∈[0,1] is obtained by uniformly sampling γ to obtain the corresponding interpolation latent vector, and finally the corresponding generated product is obtained through the decoder. The implicit field obtained by mapping the interpolation latent vector through the generator is the creative generated product, and the triangular mesh file is obtained through the marchingcube algorithm.

[0041] The present invention establishes the interpolation operator comprising the following steps:

[0042] Step 2.1: Train the autoencoder to learn low-dimensional representations;

[0043] Step 2.2: Train an interpolation operator in the latent vector space between the encoder and decoder of the autoencoder;

[0044] Step 2.3: After the training set is prepared, the autoencoder is trained. The autoencoder is trained using progressive training to make the generated model have more details.

[0045] The interpolation operator can directly obtain a closed solution through the training set data without the need for training.

[0046] In the present invention, the latent vector generation model may also adopt a potential adversarial generation network.

[0047] In the latent-GANs framework, both the generator and the discriminator contain three fully connected layers, where the activation functions of the first two layers are leaky-relu and the last layer is sigmoid function. The network is used to fit the latent vector point set obtained by the encoder of the training set through adversarial training. Finally, the generator can output the latent vector that obeys the distribution of the training point set in the latent space of the training set by inputting the standard normal distribution.

[0048] The potential generative adversarial network includes a generator and a discriminator, and the potential generative adversarial network is used to fit the distribution of embedded points of a training set in a latent space.

[0049] The present invention establishes a deep generative model comprising the following steps:

[0050] Step 2.1: Train the autoencoder to learn low-dimensional representations;

[0051] Step 2.2: Train a generative adversarial network consisting of a generator and a discriminator in the latent vector space between the encoder and decoder of the autoencoder;

[0052] Step 2.3: After the training set is prepared, the autoencoder and the adversarial generative network are trained in two stages respectively.

[0053] The two-stage training means that in the first stage, the autoencoder is trained by progressive training so that the generated model has more details; in the second stage, the adversarial generative network in the latent vector space is trained by the generator and the discriminator so that the results generated by the generator are more realistic and diverse.

[0054] After the autoencoder and adversarial generation network are trained, the embedding map AE is obtained S→Z , which includes the encoder of the autoencoder and the generative mapping G Z→S , which contains the generator of the generative adversarial network and the decoder of the autoencoder.

[0055] In the present invention, the latent vector generation model can also use a creative generator solver. The creative generator solver includes: a heuristic objective function, the heuristic objective function includes a heuristic sampling algorithm and a latent space optimization algorithm; the heuristic sampling algorithm samples the latent space; the generation model is trained by a weighted training method, the reasoning is reduced to complete the black box optimization, and the latent vector generator is obtained.

[0056] The invention provides a creative product solver comprising the following steps:

[0057] Step 2.1: Train the autoencoder to learn low-dimensional representations;

[0058] Step 2.2: Deploy the idea generator solver in the latent vector space between the encoder and decoder of the autoencoder;

[0059] Step 2.3: After the training set is prepared, the autoencoder and idea generation solver are trained in two stages respectively.

[0060] The two-stage training means: in the first stage, the autoencoder is trained by progressive training to make the generated model have more details; in the second stage, the creative generator is trained by sampling the latent space by a heuristic sampling method, and then the score of the sampling point is calculated by a heuristic objective function, and the high-scoring sampling point set is used to train the kernel density estimation of the non-parametric estimation. At the same time, the high-scoring sampling point set is used as the set for subsequent evaluation.

[0061] Considering that the previous method has a high computational overhead, the high-scoring sampling points are used as part of the training set to train the generative model of the document Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining. Due to the effectiveness of the weighted training method, black-box optimization can be completed with less reasoning to obtain the latent vector generator. The latent vector generator can automatically input 32-dimensional training outputs that obey the standard multivariate Gaussian distribution and have high-scoring sample points in the heuristic objective function. These sample points can generate creative products that are considered to be of high quality after passing through the implicit decoder IM-Decoder.

[0062] The hybrid heuristic objective function f(x) consists of a connected component counting function and a binary classifier for distinguishing between design targets and bionic objects. The connected component counting function makes the objective function f(x) black box, that is, the general closed form or differential information is unknown.

[0063]

[0064] where δ is the threshold used to discretize the continuous signed distance field, N r is the voxel resolution. θ (z,p) is the network prediction value at point p in the signed distance field obtained by decoding z, Represents the whole number.

[0065] Given the embedded representation of the bionic object and the design goal respectively and M, N are the number of training set samples of bionic objects and design targets. Zd and zb are just naming symbols, and k is a sequence number from 1 to M or N. ijk is basically used in this way.

[0066] Train each perceptual classifier f c (z) = argmax i P(Y=i|z). The loss function is:

[0067]

[0068] in: is a latent vector, Y represents a binary random variable, ideally f c (z)∈{0,1}, the actual differentiable network value is continuous f c (z)∈[0,1]. The goal of this invention is to encourage the generation of a trade-off between biomimetic objects and design target categories. When the classifier uncertainty reaches the maximum value, percept() should reach the optimal value, so it is defined as follows:

[0069] Percept(z)=H(Z)=-f c (z)logf c (z)-(1-f c (z))log(1-f c (z))

[0070] The hybrid heuristic objective function is finally expressed as:

[0071] S(z)=Percept(z)+α1N cg (z)

[0072] Where α1 is the coefficient.

[0073] The latent space optimization algorithm (LSO) maps the latent vector point set to the high value area of ​​the objective function S(z). The objective function f is evaluated as few times as possible, and finally the evaluation point sequence is obtained. The algorithm currently used is (Austin Tripp, Erik Daxberger, and José Miguel Hernández-Lobato. 2020. Sample efficient optimization in the latent space of deep generative models via weighted retraining. Advances in Neural Information Processing Systems 33 (2020).) This method ultimately obtains a latent vector generator that outputs a latent vector in the high value region of the objective function S(z) by sampling from a simple prior distribution.

[0074] In order to evaluate the effect of latent space optimization algorithm, heuristic sampling method and kernel density estimation are used to form the true value point set based on Bayesian optimization.

[0075] The heuristic sampling method is a prime interval sampler. First, the range of the high-quality sample latent vector point set is narrowed to the convex hull of the training data set latent vector. Selecting several middle points from the interpolation is an acceptable creative exploration method suitable for the ready-to-use evaluation of the convex hull S. If the present invention selects each pair of and Interpolate and sample z1 and z2 N times, and select the middle m points in each pair. The sample set As shown below:

[0076]

[0077] Where N represents the number of interpolation sampling times; λ ranges from 1 to m.

[0078] The amount of samples obtained in this way is very large. In order to reduce the sampling amount, the present invention finds that sampling with a prime number interval can keep the sample set The algebraic structure of . Sample set is a cyclic group of order m. This sampling yields The subset of is still a cyclic group of order m, and the subset obtained by methods such as uniform sampling converges to this structure probabilistically.

[0079] The final sample set is estimated by kernel density estimation, where the bandwidth is 0.1. After that, the creative generation can be obtained by sampling through kernel density estimation.

[0080] The present invention proposes a three-dimensional bionic object generation method based on a deep generative model. The deep generative model can use one of the potential generative adversarial network, the creative object solver, and the interpolation operator alone, or a combination of multiple ones can be used simultaneously.

[0081] The present invention also provides an application of the above method in three-dimensional representation, and the application specifically includes the following process:

[0082] Step 1: Learn the embedding of the three-dimensional shape through the encoder, where the embedding refers to the transformation into an element in a low-dimensional space;

[0083] Step 2: Searching in the low-dimensional space according to specific rules to find a series of elements in the low-dimensional space that integrate semantic features of different three-dimensional shapes; the specific rules include interpolation, latent vector generation by potential generative adversarial network, and creative generator solver;

[0084] Step 3: Use the implicit encoder-decoder to learn the mapping and remap the elements in the low-dimensional space back to the three-dimensional shape to obtain the three-dimensional shape of the object.

[0085] In the present invention, the design in the three-dimensional bionic design refers to the design obtained by optimizing the product. Both "product" and "synthetic" correspond to the English term synthetic, specifically referring to the product generated by the generator.

[0086] Generator. The generator is a neural network model used for generating tasks. In the model training stage, it is necessary to learn an implicit representation space Z for the space X where the multimedia content in the given training set is located, and the Z space is a low-dimensional Euclidean space. In the model reasoning stage, the present invention has x=f θ (z), that is, given a latent space sampling point z, the generator can generate data of the corresponding modality.

[0087] The implicit autoencoder includes a 3DCNN encoder and an implicit decoder to fit the signed distance field of the 3D model and reconstruct the signed distance field. After the first stage of training, it is represented as a 3D shape dataset. The loss function is the reconstruction error of the implicit field sampling points. The specific network structure is shown in the table.

[0088] The intermediate hidden layer of the autoencoder in the present invention is a low-dimensional Euclidean space, and the semantics therein is extracted by deep learning methods. Specific method: Assume that the low-dimensional space is R n A subset of , defined as the latent vector space Z, given a sample of the design target domain Samples from the bionic object domain Assume that the distribution of the design target domain D is p(d) and the distribution of the bionic object domain B is p(b). The first step is to train the encoder AE S→Z, to learn from the sample the subset Z that embeds the distribution p(d) and p(b) into the low-dimensional latent vector space Z B With Z D In the above example, we assume that their distribution is p(z b ) and p(z d ). The second step is to distribute p(z b ) and p(z d ) to search in the middle ground, and obtain the latent representation space Z of the creative generation domain through searching I , assuming its distribution is p(z i ), in p(z i ) to obtain a series of samples The third step is to use the generator G of the trained generative model Z→S The sample Mapped into a three-dimensional shape domain, thus obtaining a series of three-dimensional shapes with bionic creative inspiration value.

[0089] The present invention uses an interpolation method in latent space to synthesize novel bionic design forms. Related work on representational learning has shown that three-dimensional shapes can be transformed by operating on latent vectors. The model trained by the present invention can also generate two latent vectors z1 and z2 for two different objects through an encoder, and then input the interpolated latent vector z′=γz1+(1-γ)z2 (0<γ<1) into a generator to generate three-dimensional shapes. These three-dimensional shapes reflect a smooth transition between objects as γ increases.

[0090] The potential generative adversarial network is used to map the normal distribution samples of N(0,1) into new sample points that fit the latent vector distribution of the training set.

[0091] Gaussian sampling is used to directly sample N(0,1) sample points in the latent space. The results are not as good as the potential generative adversarial network.

[0092] Select appropriate bionic objects and design goals from the voxel models in the input dataset. S→Z Obtain latent vectors, interpolate between these latent vectors, and obtain a large number of intermediate products, which are then handed over to designers for selection. In the subsequent generation process, different angles of combination are tried to obtain a richer creative inspiration model.

[0093] Gaussian distribution in the latent space of the latent-GAN model is also a method for generating novel bionic design forms. After sampling is completed, the designer intervenes to select the bionic objects and design goals with ideal model learning effects, in preparation for the subsequent call of the model and the generation of bionic products. In order to make the generated results and designers more acceptable, the present invention adopts the expansion of the data set and direction adjustment to produce more diverse three-dimensional shapes and morphological details. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 It is a schematic diagram of the process of the present invention.

[0095] Figure 2 It is a schematic diagram of the method for generating three-dimensional bionic objects based on the deep generative model of the present invention.

[0096] Figure 3 It is a flow chart of the two-stage training method of the present invention.

[0097] Figure 4 It is a schematic diagram of the data representation method of four three-dimensional shapes of the present invention and the conversion method therebetween.

[0098] Figure 5 It is the overall framework diagram of the model call of the present invention.

[0099] Figure 6 It is a hypothetical diagram of the creative generation problem with bionic design as the goal in an embodiment of the present invention.

[0100] Figure 7 It is a data format diagram of the result generated by the embodiment of the present invention.

[0101] Figure 8 It is a schematic diagram of the three-dimensional form of a typical product of an embodiment of the present invention, design inspiration points and design development direction.

[0102] Fig. 9 It is a design scheme diagram of the embodiment of the present invention completed under the inspiration of the generated object.

[0103] Fig.10 It is a comparison diagram of the morphology of some products of the embodiments of the present invention and the design morphology of internationally excellent bionic children's home chairs.

[0104] Fig.11 It is a comparison diagram of the design scheme inspired by some products of the embodiments of the present invention and the design of internationally excellent bionic children's home chair.

[0105] Fig.12 It is a schematic diagram of scoring the results generated by the solver of some creative products in the embodiments of the present invention.

[0106] Fig.13 It is a schematic diagram of the structure of the creative generator solver of the present invention. DETAILED DESCRIPTION

[0107] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0108] Example 1

[0109] The embodiment of the present invention selects children aged 3-6 as the design object, bionic home chair as the design target, dogs that are more familiar to people in life as the bionic object, and urban home interior as the application scenario. The designers involved in this design include a Red Dot Product Design Award winner, a product designer who has worked in a preschool education institution, a product designer who has a 5-year-old child at home, and a team of computer engineers composed of graduate students and teachers from the School of Computer Science of East China Normal University.

[0110] Based on the work, designers and computer engineers jointly built a bionic design resource library. The open source model library of chairs was used in training, and simple, undecorated backrest chairs were used as preparation for the training of the generative model. Dog models with obvious features and consistent postures were selected to build a bionic modeling library.

[0111] This project has been trained multiple times according to the set interpolation calculation of the 3D deep learning model. Each generated object consists of three views and model files. After 9 training and calls, more than 15,000 generated objects have been generated for calling, evaluation and screening. Designers and computer engineers have continuously revised and advanced from the following 5 aspects, so that the subsequent generated objects have higher bionic design form value.

[0112] The first stage is the three-dimensional deep learning product call and optimization measure stage, as shown in the following table:

[0113] Table 1. 3D deep learning generated product calls and optimization measures

[0114]

[0115]

[0116] Deep learning generated objects include plane contours and three-dimensional morphological files. The bionic morphological value assessment of the generated objects is divided into two aspects: bionic morphological creative value assessment and model generation quality assessment. Designers use the generated objects to quickly select objects with high bionic design value based on whether the generated objects reflect the important characteristics of the original bionic objects, the correspondence between the generated object morphological structure and the bionic object structure and product structure. Algorithm engineers evaluate the generated objects from the perspective of the completion effect of deep learning, find out the abnormal phenomena and causes such as distortion, fracture, and default in the generated objects, and observe and evaluate the impact of each interpolation, parameter adjustment, and data set on the generated results. After multiple adjustments and screenings in the generation process, about 5 types of generated objects with high bionic morphological value were obtained, which can be further optimized into design solutions. The generated three-dimensional morphology and plane contours will be of great inspiration to designers.

[0117] The second stage is to further optimize the 3D generated objects with high bionic morphological value into design solutions. The designer's work at this stage is divided into two parts: 1. Selecting generated objects with high bionic design morphological value, optimizing the morphology to form a design solution; 2. Inspired by a single or multiple generated objects, the designer directly generates secondary creativity to form a new design solution. The designer's sensitivity to bionic morphology, morphological processing skills and product design experience play a key role in this process. Computer engineers need to fully understand the designer's selection criteria and assist the designer in completing the screening work and converting the generated object format and model.

[0118] The designers adjusted and optimized the generated model based on product structure, functional design, etc., and formed multiple design plans for children's home chairs.

[0119] In order to evaluate the value of deep generative models in the creative generation of three-dimensional bionic forms, the present invention compares the generated forms selected by designers in this work with the forms of well-known bionic product designs, such as Fig.10 shown.

[0120] The comparison shows that, apart from the problems of surface roughness and incompleteness of local morphology, the morphology of the generated objects is very close to that of most existing bionic product designs, and conforms to the bionic modeling rules of abstract bionics, intentional bionics, and concrete bionics; the generated objects are also close to bionic objects in terms of posture dynamics, typical characteristics of animals, etc. The generated objects have rich morphology, basically covering the existing creative morphological variations of bionic design.

[0121] In the embodiment of the present invention, three product designers observed the characteristics of the generated products and integrated their own ideas to form 10 design schemes in a short period of time, as well as multiple design creative sketches, such as Fig.11As shown. From the distribution of the scheme, multiple generated objects can inspire designers, and after form optimization, they can form a design scheme; from the perspective of the creative inspiration points of the scheme, designers can get creative inspiration from a single generated object, or integrate the creative points of more than two generated objects to form a new creative form. This process proves that the generated objects can give designers effective creative inspiration and assist designers in forming design schemes. In contrast, the designs of international designers are better in form processing skills. Designers also got inspiration from the outline of the generated objects and formed a design scheme.

[0122] Example 2

[0123] The potential autoencoder can only fit the distribution of training data, and may not necessarily obtain a structure that integrates the generated objects. However, sometimes there is a certain degree of design abstraction in just reconstructing the structure. In this embodiment, a three-dimensional bionic design is obtained by using a potential generative adversarial network, and the specific process is as follows:

[0124] The latent generative adversarial network includes a generator and a discriminator, and the latent generative adversarial network is used to fit the distribution of embedded points of a training set in a latent space.

[0125] During the training phase, the generator inputs 128-dimensional Gaussian noise that follows a normal distribution of N(0,1) and outputs a generated 128-dimensional vector. The discriminator inputs the generated vector and the embedded point set of the training set in the latent space to classify whether the vector category belongs to the training set or is generated by the generator. The two sides conduct adversarial training, and after the training is completed, the generator can input Gaussian noise to output a latent vector that is close to the distribution of the embedded points of the training set in the latent space.

[0126] In the calling phase, the 128-dimensional vector sampled from the normal distribution of N(0,1) is mapped to a new sample point that fits the distribution of the latent vector of the training set.

[0127] Generator

[0128] layer Activation Function Output shape Input latent vector (128) Fully connected LReLU (2048) Fully connected LReLU (2048) Fully connected Sigmoid (128)

[0129] Discriminator

[0130]

[0131]

[0132] like Figure 1 As shown in Figure 1, the potential autoencoder can only fit the distribution of training data, and may not necessarily obtain a fusion structure between the generated products. However, sometimes there is a certain degree of design abstraction in just reconstructing the structure.

[0133] Example 3

[0134] like Fig.12 As shown in the figure, from left to right and from top to bottom, the scores of the creative generated product solver are from high to low. It can be seen that high-scoring products can often integrate the morphological features of the bionic object and the design target. However, products in the middle stage can only fit the distribution of the training data, and may not necessarily obtain the structure of integration between the products. Products in the low-scoring stage often show a lot of fragmentation, which affects the functionality of the product.

[0135] In this embodiment, a creative generated object solver is used to obtain a three-dimensional bionic design, and the specific process is as follows:

[0136] First, the latent space is sampled by a heuristic sampling method, and then the score of the sampling point is calculated by a heuristic objective function. The high-scoring sampling point set is used to train the kernel density estimation of non-parametric estimation. At the same time, the high-scoring sampling point set is used as the set for subsequent evaluation.

[0137] Considering that the previous method has a high computational overhead, the high-scoring sampling points are used as part of the training set to train the generative model of the document Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retraining. Due to the effectiveness of the weighted training method, black-box optimization can be completed with less reasoning to obtain the latent vector generator. The latent vector generator can automatically input 32-dimensional training outputs that obey the standard multivariate Gaussian distribution and have high-scoring sample points in the heuristic objective function. These sample points can generate creative products that are considered to be of high quality after passing through the implicit decoder IM-Decoder.

[0138] Example 4

[0139] The invention proposes a method for generating a three-dimensional bionic object based on a deep generative model, wherein the deep generative model can be used alone or in combination of a potential generative adversarial network, a creative object solver, and an interpolation operator. When multiple models are used at the same time, the accuracy of the three-dimensional bionic design is improved.

[0140] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A three-dimensional bionic object generation system based on an interpolation operator, characterized in that: The system adopts a three-dimensional bionic object generation method based on an interpolation operator, and the system comprises: The data preprocessing module is used to input the 3D shape data set represented by the bionic object and the design target, expand the data set through design target-based and performance-based data enhancement, and obtain the signed distance field of the 3D model required for model training through mesh2voxel and SDF sampler; The data enhancement based on the design goal includes: applying rotation transformation in seven directions to the bionic object dataset to achieve diversified hybrid results; The performance-based data augmentation includes: stretching the training dataset on three orthogonal axes; A deep generative model, comprising an implicit autoencoder and an interpolation operator; the implicit autoencoder comprises an encoder 3DCNN and a decoder IM-Decoder; The structure of the interpolation operator: given the bionic object latent vector x1 and the design target latent vector x2, the interpolation vector is expressed as x=γx1+(1-γ)x2; where γ∈[0,1] is obtained by uniformly sampling γ to obtain the corresponding interpolation latent vector, and finally the corresponding generated product is obtained through the decoder; the implicit field obtained by mapping the interpolation latent vector through the generator is the creative generated product, and the triangular mesh file is obtained through the marchingcube algorithm; The establishment of the interpolation operator comprises the following steps: Step 2.1: Train the autoencoder to learn low-dimensional representations; Step 2.2: Train an interpolation operator in the latent vector space between the encoder and decoder of the autoencoder; Step 2.3: After the training set is prepared, the autoencoder is trained; progressive training is used to train the autoencoder so that the generated model has more details; The interpolation operator directly obtains a closed solution through the training set data without the need for training; The sample vector repository is used to store latent vector point sets. The latent vectors in the repository are used to obtain three-dimensional bionic generated objects through implicit decoders.

2. The three-dimensional bionic object generation system based on the interpolation operator according to claim 1, characterized in that: Further including: A post-processing module is used to post-process the three-dimensional bionic product to obtain a three-dimensional bionic design.

3. The three-dimensional bionic object generation system based on the interpolation operator according to claim 1, characterized in that: The implicit autoencoder is used to fit the signed distance field of the three-dimensional model. In order to obtain a high-fidelity generated product that meets the design requirements, it is necessary to search for a suitable latent vector point set in the latent space; the latent vector point set is obtained by interpolating the latent vector pairs obtained by the encoder through the three-dimensional model of the bionic object and the design target in the given training set.

4. A method for generating a three-dimensional bionic object using the generation system according to any one of claims 1 to 3, characterized in that: The following steps are involved: Step 1: Create a training set; Step 2: Establish a deep generative model and train the deep generative model through the training set; the deep generative model includes an implicit autoencoder and a latent vector generative model; the implicit autoencoder includes an encoder 3DCNN and a decoder IM-Decoder; the latent vector generative model uses an interpolation operator; Step 3: Generate a three-dimensional bionic object using the deep generative model.

5. The method for generating a three-dimensional bionic object according to claim 4, characterized in that: The method further comprises step 4: post-processing the three-dimensional bionic product in collaboration with the designer to obtain a three-dimensional bionic design.

6. The method for generating a three-dimensional bionic object according to claim 4, characterized in that: The step 1 includes: constructing an identification conversion path from a 3D file format to a voxel model format, specifically including the following steps: Step 1.1: The .obj file of the 3D mesh is voxelized using binvox and converted into a .binvox file; Step 1.2: The voxelized file is flood-filled to obtain point value pairs, which is equivalent to discrete sampling of the isosurface; Step 1.3: Use the encoder in the implicit autoencoder to learn the mapping of 3D voxels to 128-dimensional latent vectors, and the decoder learns the mapping of 128-dimensional latent vectors plus 3D voxel positions to the signed distance field of the voxel; the signed distance field is used to obtain 3D grid data through the marching cube algorithm; the signed distance field is used to obtain voxel data through thresholding; and / or, In the step 1, the performance-based data enhancement includes: stretching the training data set on three orthogonal axes with a degree ranging from 0.25 to 2.

7. The method for generating a three-dimensional bionic object according to claim 4, wherein: The step 1 further includes: improving the resolution of the generated voxel file, which includes generating a signed distance field by the model, and allowing voxels of arbitrary resolution to be generated by controlling the sampling density of the signed distance field.

8. The method for generating a three-dimensional bionic object according to claim 4, wherein: The loss function of the IM-Decoder model depends on the sampling method; the loss function is the weighted mean square error between the true value label and the predicted label of each point; let S be the point set of the target shape, and the loss function L(θ) is: Among them, θ is the network weight, f θ For network mapping, is the true value of the actual signed distance field sampled at point p; w p is the weight of the weighted sampling method, which depends on the sparsity of the sampling points.

Images