3D printing method and system for building biological 3D model based on large model

Abstract

The invention discloses a 3D printing method and system for building a biological 3D model based on a large model, and the method comprises the following steps: S1, receiving input data which comprises a text description, a single image or a multi-view image of a pre-generated biological 3D model; s2, performing large model training on the input data to generate a biological 3D model; s3, receiving the biological 3D model and preprocessing the biological 3D model; and S4, selecting a biological material, and printing the obtained biological 3D model by utilizing 3D printing equipment to obtain a biological model entity. According to the method, the complex biological 3D model can be generated in a short time, and the generation efficiency of the 3D model is greatly improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of 3D printing, and particularly relates to a 3D printing method and system for generating a biological 3D model based on large model training. BACKGROUND

[0002] In the current 3D model generation technical field, traditional modeling methods mainly rely on professional modeling software such as 3ds Max, Maya, etc. Although these software are powerful, the operation interface is complex and the tools are numerous. The modeling personnel not only need to spend a lot of time to learn the functions of the software, but also need to have a solid foundation in art, spatial imagination and a deep understanding of the structure of objects. With the continuous expansion of the application of 3D printing technology in the fields of biological medicine, scientific research and education, the demand for high-quality and high-precision biological 3D models has shown explosive growth. In the field of biological medicine, doctors need to quickly obtain accurate biological models according to the specific conditions of patients for surgical planning, customized medical devices, etc. In the field of scientific research and education, researchers need a large number of biological models of different types for teaching demonstration, scientific research, etc. However, the traditional 3D model generation method cannot meet these growing needs, and therefore, for the field, there is an urgent need to develop a method for quickly and accurately generating biological 3D models. SUMMARY

[0003] To solve the technical problems of low 3D model generation efficiency and difficulty in biological model construction in the prior art, the application provides a 3D printing method and system for constructing a biological 3D model based on a large model. The application realizes the rapid and accurate generation of a biological 3D model by using an AI large model, and converts it into a physical object through 3D printing technology, thereby improving the production efficiency and quality of the biological model.

[0004] The technical solutions adopted by the application are as follows: A 3D printing method for constructing a biological 3D model based on a large model, comprising the following steps: S1, receiving input data, including a text description of a pre-generated 3D model, a single image or multiple-view images; S2, training the input data through a large model to generate a biological 3D model; S3, receiving the biological 3D model and preprocessing it; S4, selecting a biological material and printing the obtained biological 3D model using a 3D printing device to obtain a biological model physical object.

[0005] Preferably, in step S2, the large model training is specifically as follows: The received input data (including a text description and an image such as a single image or multi-view images) are subjected to semantic feature vector extraction by a pre-trained CLIP model (which can adopt a dual tower architecture known in the art, including a Transformer text encoder and a ViT image encoder); the extracted semantic feature vector is used as conditional information to generate an implicit representation of an initial 3D model by a pre-trained diffusion model, wherein the diffusion model can adopt a U-Net architecture known in the art, including DownBlock, UpBlock, residual connection and conditional embedding module, and the training target is to minimize the noise prediction loss function; the generated initial 3D model implicit field is subjected to differential position encoding to improve the details of the model; the position information subjected to differential position encoding is input into a multi-layer perceptron of the neural radiance field, and the multi-layer perceptron adopts a fully connected architecture known in the art, including an input layer, a hidden layer and an output layer, and outputs the predicted volume density and color through the multi-layer perceptron; the predicted volume density and color are calculated using a volume rendering formula, and the implicit field is optimized by a rendering loss; and the rendered result is subjected to explicit mesh extraction.

[0006] Preferably, the CLIP model is based on a Transformer architecture, and through contrastive learning, the image and the text have similar semantic features in a common feature space.

[0007] Preferably, the diffusion model is a conditional diffusion model, and its noise prediction loss function is defined as: wherein E is the mathematical expectation (i.e., taking the average of the joint distribution of t, x0, t is the diffusion time step, x0is the original 3D model, is a random noise, is a noise prediction network, and c is a conditional embedding vector.

[0008] Preferably, the differential position encoding used to improve the details of the model is defined as: wherein x is the 3D space coordinate, L is the encoding frequency level, is the high-frequency component.

[0009] Preferably, the volume rendering formula used to optimize the implicit field is defined as: wherein r(t) is a camera ray defined by an origin o and a direction d, t f , t n is the far-end boundary and the near-end of the light ray, is the density field, c is the color field, and T(t) is the transmittance.

[0010] Preferably, the explicit mesh extraction is performed by using a differentiable Marching Cubes algorithm to convert the rendered implicit field into a mesh model with explicit topology.

[0011] Preferably, the preprocessing is model repairing and slicing processing of the generated 3D biological model.

[0012] Preferably, in the step of selecting suitable biological materials, the biological materials are determined according to the use, structural characteristics of the biological model and the performance of the 3D printing device, and the biological materials include various biological resins.

[0013] Preferably, the 3D printing device is a light-curing 3D printer which uses photosensitive resin as printing material and realizes printing of the 3D biological model by layer-by-layer curing of the photosensitive resin through ultraviolet irradiation.

[0014] Preferably, after the step S4, a step S5 of post-processing the obtained biological model physical object is performed.

[0015] Preferably, the post-processing includes secondary curing, polishing and polishing.

[0016] The application also discloses a 3D printing system for constructing a biological 3D model based on a large model, which is used to execute the above method and comprises the following modules: A data input module is used to receive input data, and the input data includes a text description, a single image or multi-view images of a pre-generated biological 3D model; A biological 3D model construction module is used to train the input data through a large model to construct a biological 3D model; A preprocessing module is used to receive the biological 3D model and perform preprocessing on the biological 3D model; A 3D printing module is used to select biological materials and print the obtained biological 3D model by using a 3D printing device to obtain a biological model physical object.

[0017] Compared with the prior art, the application has the following remarkable technical effects.

[0018] The application utilizes an AI large model system composed of a CLIP model, a diffusion model and a neural radiation field (NeRF) as the core, combines key technologies such as differentiable position encoding and volume rendering optimization, and greatly improves the generation efficiency of 3D models compared with traditional modeling methods, and can generate complex biological 3D models in a short time. The large model trained based on a large amount of data can more accurately capture the details of biological structures, so that the generated 3D biological model is more in line with the actual application requirements. The biological model printed through the slicing processing and light curing 3D printing process is of reliable quality, and provides strong support for research and application in the fields of biological medicine and scientific research and education. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A 3D printing method for constructing a biological 3D model based on a large model according to the preferred embodiment of the application is shown in the flowchart.

[0020] Figure 2 A process diagram of large model training according to the preferred embodiment of the application is shown.

[0021] Figure 3 A flowchart of model preprocessing according to the preferred embodiment of the application is shown.

[0022] Figure 4 A physical model of the printed model according to the preferred embodiment of the application is shown.

[0023] Figure 5 A 3D printing system block diagram for constructing a biological 3D model based on a large model according to the preferred embodiment of the application is shown. DETAILED DESCRIPTION

[0024] The preferred embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0025] As shown in the figure, the 3D printing method for constructing a biological 3D model based on a large model according to the preferred embodiment of the application, the specific steps are as follows: Figure 1 1. Input data 2. Model preprocessing In this embodiment, the data input method of this step uses pictures and text descriptions of biological structures. When using pictures of biological structures as input data, high-resolution medical imaging equipment such as dental X-ray machines and CT scanners can be used to obtain clear images of biological structures such as teeth and bones. Professional photography equipment can also be used to obtain appearance pictures of biological samples under specific lighting and shooting angle conditions. These pictures need to be formatted and preprocessed to remove noise and enhance contrast, etc. to meet the acceptance requirements of the model. After the model receives the picture data, it identifies and analyzes the features of the biological structure such as contour, shape, and internal texture through the built-in image recognition algorithm. If language description is chosen as input data, the features of the biological structure are described in detail and accurately. For example, a language information about a specific bone should include the name of the bone, the general shape, the size range, and the key structural features. The large model uses natural language processing technology to convert the language information into structured data, understands the key information about the biological structure, and generates a 3D model based on it. For example, for a language description "human femur, long tubular shape, middle section thinner, both ends enlarged, length about 40 cm, diameter about 2-3 cm".

[0026] As shown in Figure 2 , in this embodiment, the process of training the large model is as follows: 2.1) Extract semantic feature vector The semantic feature vector contains rich semantic content in the image or text, which includes the category, attribute, spatial relationship, and other key information of the object. In the subsequent model training process, especially when the diffusion model generates the initial 3D model, the semantic feature vector will serve as important conditional information to guide the entire generation process towards the expected semantic direction. At the same time, the semantic feature vector also plays an indispensable role in calculating the loss function, which helps to optimize the training effect of the model.

[0027] To accurately extract the semantic feature vector, the following method is used in this embodiment: First, load the pre-trained CLIP model, which has two core components: image encoder and text encoder. In actual operation, the image is input into the image encoder and the text is input into the text encoder. After processing by the encoder, the corresponding image features and text features are obtained. In order to align these two features in the same feature space for better extraction of semantic feature vectors, this embodiment introduces a contrastive learning loss function. By maximizing the similarity of matching pairs through contrastive learning, the specific loss function expression is: where, is the feature vector obtained by mapping the image input through the image encoder, which contains the key feature information of the image; is the feature vector obtained by mapping the text input through the text encoder, which contains the semantic features of the text; sim represents the similarity function, which is used to measure the similarity between two vectors; is the temperature parameter, which adjusts the strength of contrastive learning and affects the model training effect; N is the sample size, which reflects the sample size involved in the calculation, and the selection of the sample size will have a certain effect on the accuracy and stability of the model training.

[0028] In this embodiment, the key step of generating the initial 3D model is realized based on the pre-trained diffusion model, and the core goal is to minimize the noise prediction loss function, which is achieved by organically integrating the semantic feature vector and the random noise vector.

[0029] First, based on the semantic feature vector c extracted by the CLIP model, a conditional diffusion model is constructed. The core component of this model is the noise prediction network In the input condition processing, a cross-attention mechanism is used for injection. The specific calculation formula is: where, is the learnable projection matrix, which is continuously optimized during model training to improve the processing ability and prediction accuracy of the model for input information.

[0030] Then, the initial data is gradually added to the noise, and the t-step data is: where, represents a Gaussian distribution, I represents an identity matrix, represents a noise term, is a noise scheduling parameter, is a predefined noise scheduling parameter, which is calculated based on the predefined noise scheduling parameter . The noise scheduling parameter plays a key role in the entire noise addition process, determining the rate and manner of noise addition, and thus affecting the quality of the initial 3D model generated by the model.

[0031] Finally, the denoising operation is performed to minimize the noise prediction loss function, and the expression of the noise prediction loss function is: During the model generation process, noise interference with the initial data is gradually removed through continuous iterative denoising, ultimately obtaining an initial 3D model that meets the requirements. .

[0032] The core idea of ​​differentiable positional encoding is to map low-dimensional positional information to a high-dimensional space. The purpose of this is to provide neural networks with more frequency information, enabling them to better process and understand positional information, thereby significantly improving the model's ability to capture details and contributing to the generation of more accurate and high-quality 3D models.

[0033] For the initial 3D model It can be represented as a series of point coordinates in three-dimensional space. For any point in three-dimensional space with coordinate x, its differentiable position code is defined as: Where x is the 3D spatial coordinate and L is the coding frequency level. These are high-frequency components. These high-frequency components can capture detailed features in the location information. L is the number of encoding layers, and its value directly determines the dimension of the encoded features. Generally speaking, the more layers there are, the higher the dimension of the encoded features, the richer the high-frequency information that the model can capture, and the stronger its ability to depict the details of the 3D model.

[0034] After encoding the x-coordinate, the same encoding formula is used to perform sine and cosine transforms of different frequencies on the y and z coordinates of any point in three-dimensional space. Finally, the encoding results of the three components are concatenated to form a high-dimensional encoding vector. This encoding vector contains rich positional information of the point in three-dimensional space, providing strong support for subsequent model processing.

[0035] After differentiable position encoding is completed, the encoded position information is passed as input to a multilayer perceptron (MLP) neural network for neural radiation fields. The MLP neural network performs deep processing and analysis of the input information through fully connected layers or convolutional layers. Through these calculations, the MLP neural network predicts and outputs the volume density at that point. And color c, the specific calculation expression is: in, is the result of differentiable positional encoding on position p, which carries rich position information; d is the viewing direction vector, which reflects the viewing direction of the observation point, and different viewing directions will affect the attribute prediction of 3D model points; MLP represents the neural network model of NeRF, which plays a core role in the entire attribute prediction process. Through continuous learning and optimization, it can accurately predict the volume density and color attributes of each point, and these attribute values are the key parameters for calculating pixel color, and their accuracy directly affects the quality of the final generated 3D model. 2.5) Volume rendering formula calculation The goal of volume rendering is to generate multi-view 2D images, and the implicit field is optimized through rendering loss, so as to further improve the quality and realism of the 3D model generated by the model. In the specific operation process, first, a ray r is emitted from the camera position to the scene, and a series of discrete sampling points are obtained by sampling along the path of the ray within the near-far plane After obtaining the sampling points, the volume rendering formula needs to be calculated, and the definition of the volume rendering formula is: However, since the integral in the above formula is continuous, it is difficult to directly calculate accurately in the actual calculation process, so it needs to be discretized. The pixel color C(r) is calculated by discretely integrating the sampling points: Among them, the transmittance reflects the attenuation of the light ray during propagation through different sampling points; the adjacent sampling point spacing , the size of the spacing will affect the accuracy of the discrete integral; is the volume density and color of the sampling point output by the above multi-layer perception (MLP) neural network.

[0036] After completing the volume rendering operation, the output result is the implicit field. In order to obtain a 3D model with a clear topological structure and geometric shape, it is also necessary to extract the explicit 3D structure from the continuous density field obtained through training. This embodiment uses a differentiable Marching Cubes algorithm to achieve this goal.

[0037] First, the continuous space is divided into a uniform voxel grid, and the resolution of the voxel grid is set to , and the coordinates of each voxel vertex are: Then, the density threshold determination is performed, which aims to determine which voxels intersect with the target surface, so as to screen the voxels to be processed. For this purpose, a density threshold is set According to the comparison result of the density value of the vertex of the voxel with the threshold, the state of each vertex of the voxel is marked: if , it is marked as "inside"; otherwise, it is marked as "outside".

[0038] After screening the voxels to be processed, the vertex interpolation algorithm is used to accurately locate the spatial coordinates of the intersection points of the surface on the voxel edge across the threshold, to provide accurate geometric data for the generation of triangular patches. The specific interpolation calculation formula is: Wherein, are the density values of the two endpoints of the voxel edge, and satisfy ; are the coordinates of the two endpoints of the voxel edge in the three-dimensional space. The intersection point coordinates calculated by the formula can accurately determine the position of the surface.

[0039] Finally, triangular patches are combined according to the intersection point positions obtained by interpolation, and explicit triangular meshes are gradually constructed, to finally form a continuous surface mesh. On the basis of constructing the surface mesh, the surface normal n is calculated through the gradient of the density field, to further enhance the geometric details. The calculation formula of the surface normal is: Through the above complete process, the rendered implicit field can be successfully converted into a mesh model with a clear topological structure, which provides a solid foundation for the subsequent processing and application of 3D models.

[0040] For example, Figure 3As shown, the model preprocessing is as follows: import the AI large model based on the input data generated biological 3D model file (common formats such as STL, OBJ, etc.) into the model repair or slicing software; further optimize and perfect the generated preliminary model, such as topology, mesh structure optimization and surface refinement, feature enhancement and other detail optimization; adjust the position and direction of the model on the printing platform, and scale the model as needed to meet the printing requirements; set the printing parameters including the number of printing layers, layer height, filling density and other parameters; add support structure to ensure the stability of the model during printing and prevent collapse; use the slicing software to slice the 3D model into a series of two-dimensional slices according to the set parameters, and generate the printing path and instructions for each layer; before generating the final printing file, check the slicing effect through the preview function of the slicing software to check whether there are errors or unreasonable places, such as whether the model slices are complete, whether the support structure is reasonable, etc.; after confirming that the slicing effect is correct, save the generated slicing file in a format that the 3D printer can recognize (such as G code), and then transfer it to the 3D printer for printing.

[0041] Select a light-cured 3D printer as the printing equipment, and select appropriate biological materials according to the obtained biological 3D model and actual requirements. The light-cured 3D printer uses photosensitive resin as the printing material, and its working principle is to use ultraviolet radiation to make the photosensitive resin solidify layer by layer in a specific area. During printing, set the printing parameters according to the characteristics of the photosensitive resin and the structural characteristics of the 3D biological model. For example, for a tooth model with complex structure, set the exposure time to 2-5 seconds to ensure that the resin can be fully cured and not overexposed to cause model deformation; set the layer thickness to 0.05-0.1 millimeters to ensure that the details of the model can be clearly presented; set the lifting speed to 3-5 millimeters per second to ensure smooth printing process and avoid affecting the model quality due to excessive speed or slow speed. Through these precise parameter settings, a high-precision, smooth-surface biological model can be printed to meet the strict requirements of the biological medical, scientific research and other fields.

[0042] After printing, the physical model is post-processed. Remove the support structure on the surface of the model, wash off the residual uncured resin on the surface of the model using an ultrasonic cleaning machine, then place the model in a curing oven for secondary curing, and finally perform surface polishing and other treatments to make the surface smoother and meet the actual use requirements.

[0043] Printed model physical object: According to the above-mentioned embodiment based on large model training to generate a biological 3D model, a malleus model object is successfully printed, as shown in Figure 4 The overall size of the malleus model is highly consistent with the standard size of the real malleus of the human body. After accurate measurement, the length is about 8.44 mm, the width is about 3.19 mm, and the thickness is about 2.53 mm. The model has a fine appearance and fully presents the unique shape of the malleus, including the head, neck, handle, and protrusions and other structural details. The proportions and shapes of each part are highly consistent with the malleus structure in human anatomy. As a key auditory ossicle in the middle ear structure of the human body, the malleus has a complex and delicate morphological structure. The printed malleus model object highly restores the real structure of the malleus, providing a high-quality physical model for biomedical research, medical teaching demonstration, and other application scenarios, fully verifying the effectiveness and accuracy of the method in the field of 3D printing of complex biological structures.

[0044] As shown in Figure 5 The embodiment discloses a 3D printing system for constructing a biological 3D model based on a large model, which is used to execute the above method, and includes the following modules: A data input module is used to receive input data, and the input data includes a text description, a single image, or multi-view images of a pre-generated biological 3D model; A biological 3D model construction module is used to train the input data through a large model to construct a biological 3D model; A preprocessing module is used to receive and preprocess the biological 3D model; A 3D printing module is used to select biological materials and print the obtained biological 3D model using a 3D printing device to obtain a biological model object.

[0045] Other contents of the embodiment can refer to the above method embodiment.

[0046] In summary, the present application discloses a 3D printing method and system based on large model training to generate a 3D model. The present application first acquires an existing AI large model, which has the ability to generate a 3D model according to input data. Then, the AI large model is input with data related to a biological model, and receives a biological 3D model generated by the AI large model based on the input data and preprocesses it. Then, according to the obtained biological 3D model and actual requirements, appropriate biological materials are selected, and finally, a 3D printing device is used to print the obtained biological 3D model to obtain a biological model object, and the obtained biological model object is post-processed. In summary, the present application uses an AI large model to replace the traditional modeling method, which can generate a complex biological 3D model in a short time, greatly improving the generation efficiency of the 3D model.

[0047] The above description only illustrates the preferred embodiments of the present application and principles of the present application. For those skilled in the art, according to the idea provided by the present application, there will be changes in the specific embodiments, and these changes should also be considered as the protection scope of the present application.

Claims

1. A 3D printing method for constructing biological 3D models based on large models, characterized in that, Includes the following steps: S1. Receive input data, which includes text descriptions and images of the pre-generated biological 3D model; S2. Train the input data using a large model to generate a biological 3D model; S3. Receive the biological 3D model and preprocess it; S4. Select biological materials and use 3D printing equipment to print the obtained biological 3D model to obtain the biological model physical object.

2. The 3D printing method for generating biological 3D models based on large model training according to claim 1, characterized in that, In step S2, the large model training is specifically as follows: The received input data is processed by extracting semantic feature vectors using a pre-trained CLIP model. Using these extracted semantic feature vectors as conditional information, an implicit representation of the initial 3D model is generated using a pre-trained diffusion model, where the training objective of the diffusion model is to minimize the noise prediction loss function. Differentiable positional encoding is used to enhance the detail of the generated initial 3D model's implicit field. The positional information after differentiable positional encoding is then passed as input to a multilayer perceptron for neural radiation fields, which outputs the predicted volume density and color. The predicted volume density and color are calculated using a volume rendering formula, and the implicit field is optimized using rendering loss. Explicit mesh extraction is then performed on the rendered result.

3. The 3D printing method for generating biological 3D models based on large model training according to claim 2, characterized in that, The CLIP model is based on the Transformer text encoder and the ViT image encoder. Through comparative learning, it enables images and text to have similar semantic features in a common feature space.

4. The 3D printing method for generating biological 3D models based on large model training according to claim 2, characterized in that, The noise prediction loss function of the aforementioned diffusion model is defined as: Where t is the diffusion time step, and x0 is the original 3D model. It is random noise. For noise prediction networks, c This is the conditional embedding vector.

5. The 3D printing method for generating biological 3D models based on large model training according to claim 2, characterized in that, The differentiable positional encoding is defined as follows: Where x is a 3D spatial coordinate. L For coding frequency level, For high-frequency components, l represents the range from 0 to... L An integer equal to -1.

6. The 3D printing method for generating biological 3D models based on large model training according to claim 2, characterized in that, The volume rendering formula is defined as follows: Where r(t) is the camera ray, defined by the origin o and direction d, t f t n For the far boundary and the near end of the light, Let c be the density field, c be the color field, and T(t) be the transmittance.

7. The 3D printing method for generating biological 3D models based on large model training according to claim 2, characterized in that, The explicit mesh extraction employs a differentiable Marching Cubes algorithm to convert the rendered implicit field into a mesh model with a defined topological structure.

8. The 3D printing method for generating biological 3D models based on large model training according to claim 1, characterized in that, In step S3, the preprocessing specifically involves model repair and slicing of the generated biological 3D model.

9. The 3D printing method for generating biological 3D models based on large model training according to claim 1, characterized in that, After step S4, proceed to step S5 to perform post-processing on the obtained biological model.

10. A 3D printing system for constructing biological 3D models based on large models, used to perform the method as described in any one of claims 1-9, characterized in that, Includes the following modules: Data input module: used to receive input data, including text descriptions of the pre-generated biological 3D model, single images, or multi-view images; Biological 3D Model Building Module: Used to build biological 3D models by training input data through a large model; Preprocessing module: used to receive the biological 3D model and preprocess it; 3D printing module: Used to select biological materials and print the resulting 3D biological model using 3D printing equipment to obtain the physical biological model.

Images