A rapid reconstruction system for 3D printed models based on neural radiation fields

Abstract

This invention discloses a rapid reconstruction system for 3D printed models based on neural radiation fields, belonging to the field of 3D printing technology. It includes a dataset acquisition module, a dataset processing module, a 3D model reconstruction module, and a 3D printed entity module. This invention, by setting up these modules, not only solves the problems of low modeling efficiency, high barriers to entry, and high technical difficulty associated with forward modeling software, but also addresses the issues of high cost and difficulty in reverse point cloud modeling. Furthermore, applying neural networks to 3D printing model acquisition significantly improves model acquisition efficiency, providing valuable reference for promoting the development of the 3D printing industry.

Description

Technical Field

[0001] This invention belongs to the field of 3D printing technology, specifically relating to a rapid reconstruction system for 3D printed models based on neural radiation fields. Background Technology

[0002] 3D printing model reconstruction refers to using 3D scanning equipment to scan an object, obtain the object's 3D data, and then using 3D modeling software to process the scanned data by peeling, cutting, filling, and reconstructing, ultimately forming a complete 3D digital model.

[0003] Currently, there are two methods for 3D printed model reconstruction: one is forward modeling using software. This method requires designers to have professional modeling theory knowledge and be proficient in using modeling software. It is relatively complex and inefficient, making it difficult to apply to the construction of complex 3D models. The other is reverse 3D scanning modeling. This method requires expensive equipment to acquire point cloud data, and the disorder and semantic segmentation of the subsequent point cloud data bring great difficulties to subsequent visualization and interaction.

[0004] In view of this, a rapid reconstruction system for 3D printed models based on neural radiation fields is proposed to solve the above problems. Summary of the Invention

[0005] To address the problems mentioned in the background section, this invention provides a rapid reconstruction system for 3D printed models based on neural radiation fields. This system not only solves the problems of low modeling efficiency, high barriers to entry, and high technical difficulty associated with forward modeling software, but also addresses the issues of high cost and difficulty in reverse point cloud modeling. Furthermore, by applying neural networks to 3D printing model acquisition, it significantly improves model acquisition efficiency, providing valuable reference for promoting the development of the 3D printing industry.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a rapid reconstruction system for 3D printed models based on neural radiation fields, comprising: a dataset acquisition module, a dataset processing module, a 3D model reconstruction module, and a 3D printed entity module, wherein:

[0007] The dataset acquisition module acquires an image dataset of the entity to be 3D printed;

[0008] The dataset processing module preprocesses the image dataset to obtain a background-free image dataset and the camera pose of the corresponding images.

[0009] The 3D model reconstruction module reconstructs the 3D model based on the camera pose of the backgroundless image dataset and corresponding images using the neural radiation field, thereby obtaining the 3D model of the entity to be 3D printed.

[0010] The 3D printing entity module performs 3D printing of the entity based on the 3D model of the entity to be 3D printed.

[0011] Furthermore, the dataset acquisition module includes a green screen, a rotating platform, camera equipment, and a computer, wherein:

[0012] The green screen provides a shooting background for the object to be 3D printed;

[0013] The rotating platform carries and rotates the entity to be 3D printed.

[0014] The camera equipment captures images of the 3D-printed entity on the rotating platform to obtain the image dataset.

[0015] The computer is electrically connected to the rotating platform, provides power to the rotating platform, and controls the rotational angular velocity of the rotating platform. It is also electrically connected to the camera equipment, provides power to the camera equipment, and receives the image dataset captured by the camera equipment.

[0016] Furthermore, the dataset processing module includes an image denoising module, an image background removal module, and a camera pose calculation module, wherein:

[0017] The image denoising module removes noise from the image dataset to obtain a denoised image dataset.

[0018] The image background removal module removes the shooting background of the denoised image dataset to obtain a background-free image dataset.

[0019] The camera pose calculation module performs motion recovery structure calculation on the backgroundless image dataset to obtain the corresponding camera pose for each backgroundless image in the backgroundless image dataset.

[0020] Furthermore, the camera pose calculation module includes a feature extraction module, a feature matching module, and a sparse reconstruction module, wherein:

[0021] The feature extraction module obtains sparse feature points based on the background-free image dataset;

[0022] The feature matching module performs feature matching on the backgroundless images in the backgroundless image dataset based on the sparse feature points to obtain feature matching data.

[0023] The sparse reconstruction module performs incremental reconstruction based on the feature matching data to obtain the corresponding camera poses of each backgroundless image in the backgroundless image dataset.

[0024] Furthermore, the 3D model reconstruction module includes a neural network training module and a volume rendering module, wherein:

[0025] The neural network training module trains the multilayer perceptron based on the backgroundless image dataset and the corresponding camera poses of each backgroundless image in the backgroundless image dataset, to obtain the color and voxel density corresponding to each point in the modeling image.

[0026] The volume rendering module integrates the sampling points on the light source based on the color and voxel density corresponding to each point in the modeling image to obtain the color of each pixel, and renders the modeling image based on the color of each element.

[0027] Furthermore, the volume rendering module selects sampling points on the light rays based on two radiation field networks: a coarse network module and a fine network module, wherein:

[0028] The coarse network module performs a uniform sampling of points on the light ray to obtain a small number of points for rendering and training the network;

[0029] The fine network module estimates the distribution of light rays based on the volume density value predicted by the coarse network module, and performs secondary fine sampling on the points on the light rays based on the estimated light ray distribution to obtain a large number of points for rendering and training the network.

[0030] Furthermore, the volume rendering module integrates the sampling points on the light rays to obtain the color of each pixel using the following formula:

[0031]

[0032] In the formula: d represents the unit vector of the ray direction, r(t) represents the position t on the r-ray, c() represents the color reflected by the particle as seen from the direction of the r-ray in specific three-dimensional coordinates, σ() represents the particle voxel density at position t on the r-ray in specific three-dimensional coordinates, T(t) represents the cumulative amount of the r-ray, and t n and t f It is the boundary between near and far on the r-ray, where:

[0033]

[0034] In the formula: r(s) represents the position of s on the r-ray, σ() represents the particle voxel density at the position of s on the r-ray in specific three-dimensional coordinates, and t n t and t are the boundaries between near and far on the r-ray.

[0035] Furthermore, the 3D printing entity module includes a software module and a 3D printer, wherein:

[0036] The software modules include MeshLab software and BambuStudio software. MeshLab software is connected to the volume rendering module and performs file format conversion on the 3D model of the entity to be 3D printed to obtain an STL format file. BambuStudio software is connected to MeshLab software and performs slicing processing on the STL format file to obtain a sliced ​​model.

[0037] The 3D printer uses the slice model to 3D print the physical object.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] This invention sets up a dataset acquisition module, a dataset processing module, a 3D reconstruction module, and a 3D printing entity module. It can not only solve the problems of low modeling efficiency, high threshold, and high technical difficulty of forward modeling software, but also solve the problems of high cost and difficulty of point cloud processing in reverse point cloud modeling. At the same time, applying neural networks to the model acquisition of 3D printing can greatly improve the model acquisition efficiency, which has good reference value for promoting the development of the 3D printing industry. Attached Figure Description

[0040] Figure 1 This is a framework diagram of the rapid reconstruction system for 3D printed models based on neural radiation fields, as described in this invention.

[0041] Figure 2 This is a flowchart of the method for rapid reconstruction of 3D printed models based on neural radiation fields according to the present invention.

[0042] Figure 3 This is a schematic diagram of the data acquisition module of the present invention;

[0043] In the diagram: 1. Dataset acquisition module; 101. Green screen; 102. Rotating platform; 103. Camera equipment; 104. Computer;

[0044] 2. Dataset processing module; 201. Image denoising module; 202. Image background removal module; 203. Camera pose calculation module;

[0045] 2031. Feature Extraction Module; 2032. Feature Matching Module; 2033. Sparse Reconstruction Module;

[0046] 3. 3D model reconstruction module; 301. Neural network training module; 302. Volume rendering module;

[0047] 3021, Coarse network module; 3022, Fine network module;

[0048] 4. 3D printing physical module; 401. Software module; 402. 3D printer. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Please see Figure 1-3 This invention provides the following technical solution: a rapid reconstruction system for 3D printed models based on neural radiation fields, comprising: a dataset acquisition module 1, a dataset processing module 2, a 3D model reconstruction module 3, and a 3D printed entity module 4, wherein:

[0051] Dataset acquisition module 1 is set up near the indoor entity to be 3D printed to acquire images of the upper, middle and lower rings of the entity, i.e., the image dataset of the entity to be 3D printed. The number of images in each ring should be as large as possible, the overlap area between two adjacent images should be as large as possible, and the rotation angle between two adjacent images should not exceed 30°.

[0052] Data set processing module 2 is connected to data set acquisition module 1 to preprocess the image data set to obtain background-free image data set and the camera pose of the corresponding image;

[0053] The 3D model reconstruction module 3 is connected to the dataset processing module 2. Based on the neural radiation field, it reconstructs the 3D model of the backgroundless image dataset and the corresponding image camera pose to obtain the 3D model of the entity to be 3D printed.

[0054] The 3D printing entity module 4 is communicatively connected to the 3D model reconstruction module 3, and performs 3D printing of the entity based on the 3D model of the entity to be 3D printed.

[0055] Specifically, the dataset acquisition module 1 includes a green screen 101, a rotating platform 102, camera equipment 103, and a computer 104, wherein:

[0056] Green Screen 101 provides a shooting background for the object to be 3D printed, making it easier to remove the background in post-processing.

[0057] The rotating platform 102 carries and rotates the 3D printed object at a specific angular velocity. When photographing the 3D printed object, the shooting angle can be kept constant while the 3D printed object rotates, so as to ensure that each picture is taken under the same light source.

[0058] The camera equipment 103 captures images of the top, middle and bottom rings of the entity to be 3D printed on the rotating platform 102 to obtain an image dataset.

[0059] Computer 104 is electrically connected to rotating platform 102, provides power to rotating platform 102, and controls the rotational angular velocity of rotating platform 102. At the same time, it is electrically connected to camera equipment 103, provides power to camera equipment 103, and receives image datasets captured by camera equipment 103.

[0060] Specifically, the dataset processing module 2 includes an image denoising module 201, an image background removal module 202, and a camera pose calculation module 203, wherein:

[0061] Image denoising module 201 removes noise from the image dataset and obtains a denoised image dataset based on the natural light shooting conditions of the indoor scene.

[0062] Image background removal module 202 removes the shooting background from the denoised image dataset and obtains a clean background-free image dataset based on the shooting background.

[0063] The camera pose calculation module 203 performs motion recovery structure calculation on the backgroundless image dataset to obtain the corresponding camera pose of each backgroundless image in the backgroundless image dataset.

[0064] Specifically, the camera pose calculation module 203 includes a feature extraction module 2031, a feature matching module 2032, and a sparse reconstruction module 2033, wherein:

[0065] Feature extraction module 2031 obtains sparse feature points based on background-free image dataset and describes them numerically;

[0066] Feature matching module 2032 performs feature matching on backgroundless images in a backgroundless image dataset based on sparse feature points to obtain feature matching data. Feature matching includes exhaustive matching, sequential matching, vocabulary tree matching, spatial matching, and transitive matching.

[0067] Extensive matching is used to match a relatively small number of image datasets, and can quickly reconstruct a model with good results.

[0068] Sequence matching is used to match image datasets arranged in a sequence, such as video sequences shot by a camcorder. In this case, consecutive frames have visual overlap. Sequence matching will match each image with its visually closest image. It is important to note that the images must be named strictly in order.

[0069] Lexical tree matching is used to match large image datasets. Each image is matched with the visually closest image using a spatially reordered vocabulary tree.

[0070] Spatial matching matches each image with its nearest image in space. The spatial location needs to be given manually in the data structure. If the prior location information is accurate, this method is recommended.

[0071] Transitive matching uses existing feature matching transitivity relationships to generate a more complete matching graph;

[0072] The sparse reconstruction module 2033 reads feature matching data from the database into memory, selects two optimal initial viewpoints for incremental reconstruction, and can observe the real-time process during incremental reconstruction, including camera pose and sparse points, to obtain the corresponding camera pose for each backgroundless image in the backgroundless image dataset.

[0073] Specifically, the 3D model reconstruction module 3 includes a neural network training module 301 and a volume rendering module 302, wherein:

[0074] The neural network training module 301 uses the background-free image dataset and the corresponding camera poses of each background-free image in the dataset as input to train the multilayer perceptron, obtaining the color and voxel density corresponding to each point in the modeling image. The input to the background-free image dataset is the 3D position information (x, y, z) of the image, and the corresponding camera pose input to each background-free image in the dataset is the orientation and viewing angle (θ, φ) of the points in the image. The multilayer perceptron is a neural network model, with each layer containing multiple hidden units. It maps the input vector to a high-dimensional space for hidden representation, and then combines multiple hidden layers and a non-linear activation function to finally map the hidden representation back to the output vector. This mapping function is F: (x... The output color is the color information R, G, B of a point in space, and the output voxel density is the particle density σ of a point in space, reflecting the probability that a ray will terminate at that point. A ray connecting the camera optical center to the object's spatial coordinates is simulated as a ray. The greater the particle density, the greater the probability that the ray will terminate at that point, and the closer that point is to the object's surface. Since the multilayer perceptron is only sensitive to learning low-frequency information and performs poorly in high-frequency changes, and the color of an object is precisely high-frequency information, this will lead to poor training results. Therefore, position encoding is added during the training process of the multilayer perceptron. High-frequency functions can be used to map the input to a higher-dimensional space so that data containing high-frequency changes can be better fitted.

[0075] The volume rendering module 302 integrates the sampling points on the light source based on the color and voxel density of each point in the modeling image to obtain the color of each pixel, and renders the modeling image based on the color of each element.

[0076] Specifically, the volume rendering module 302 selects sampling points on the light rays based on two radiation field networks: a coarse network module 3021 and a fine network module 3022, wherein:

[0077] The coarse network module 3021 performs a uniform sampling of points on the light ray to obtain a small number of points for rendering and training the network;

[0078] The fine network module 3022 estimates the distribution of light rays based on the volume density value predicted by the coarse network module 3021. Based on the estimated light ray distribution, it performs secondary fine sampling on the points on the light rays to obtain a large number of points for rendering and training the network.

[0079] Specifically, the volume rendering module 302 calculates the color of each pixel by integrating the sampling points on the light source using the following formula:

[0080]

[0081] In the formula: d represents the unit vector of the ray direction, r(t) represents the position t on the r-ray, c() represents the color reflected by the particle as seen from the direction of the r-ray in specific three-dimensional coordinates, σ() represents the particle voxel density at position t on the r-ray in specific three-dimensional coordinates, T(t) represents the cumulative amount of the r-ray, and t n and t f It is the boundary between near and far on the r-ray, where:

[0082]

[0083] In the formula: r(s) represents the position of s on the r-ray, σ() represents the particle voxel density at the position of s on the r-ray in specific three-dimensional coordinates, and t n t and t are the boundaries between near and far on the r-ray.

[0084] Specifically, the 3D printing entity module 4 includes a software module 401 and a 3D printer 402, wherein:

[0085] Software module 401 includes MeshLab software and BambuStudio software. MeshLab software is connected to volume rendering module 302. The volume rendering module 302 outputs files in OBJ format. The 3D model of the entity to be 3D printed is converted to STL format. BambuStudio software is connected to MeshLab software. 3D printer 402 is selected and set. The settings of 3D printer 402 include printing materials and printing parameters. If the laboratory has PLA, PETG, carbon fiber and ABS types, PLA is selected as the printing material. The printing parameters include printing accuracy, first layer height and support type. The printing accuracy is set to 0.4mm, the first layer height is set to 0.2mm, and the support type is set to ordinary support without brim. After the 3D printer 402 is set, the STL format file is sliced ​​to obtain a sliced ​​model.

[0086] 3D printer 402 is used to 3D print sliced ​​models to obtain physical objects.

[0087] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A rapid reconstruction system for 3D printed models based on neural radiation fields, characterized in that, include: The dataset acquisition module (1), dataset processing module (2), 3D model reconstruction module (3), and 3D printing entity module (4) are as follows: The dataset acquisition module (1) acquires the image dataset of the entity to be 3D printed; The dataset processing module (2) preprocesses the image dataset to obtain a background-free image dataset and the camera pose of the corresponding image; The three-dimensional model reconstruction module (3) reconstructs the three-dimensional model based on the camera pose of the backgroundless image dataset and the corresponding image using the neural radiation field, and obtains the three-dimensional model of the entity to be 3D printed. The 3D printing entity module (4) performs 3D printing of the entity based on the three-dimensional model of the entity to be 3D printed.

2. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 1, characterized in that: The dataset acquisition module (1) includes a green screen (101), a rotating platform (102), camera equipment (103), and a computer (104), wherein: The green screen (101) provides a shooting background for the entity to be 3D printed; The rotating platform (102) carries and rotates the entity to be 3D printed. The camera equipment (103) captures images of the 3D printed entity on the rotating platform (102) to obtain the image dataset; The computer (104) is electrically connected to the rotating platform (102), provides power to the rotating platform (102), and controls the rotational angular velocity of the rotating platform (102). At the same time, it is electrically connected to the camera equipment (103), provides power to the camera equipment (103), and receives the image dataset captured by the camera equipment (103).

3. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 1, characterized in that: The dataset processing module (2) includes an image denoising module (201), an image background removal module (202), and a camera pose calculation module (203), wherein: The image denoising module (201) removes noise from the image dataset to obtain a denoised image dataset; The image background removal module (202) removes the shooting background of the denoised image dataset to obtain a background-free image dataset; The camera pose calculation module (203) performs motion recovery structure calculation on the backgroundless image dataset to obtain the corresponding camera pose of each backgroundless image in the backgroundless image dataset.

4. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 3, characterized in that: The camera pose calculation module (203) includes a feature extraction module (2031), a feature matching module (2032), and a sparse reconstruction module (2033), wherein: The feature extraction module (2031) obtains sparse feature points based on the background-free image dataset; The feature matching module (2032) performs feature matching on the backgroundless images in the backgroundless image dataset based on the sparse feature points to obtain feature matching data. The sparse reconstruction module (2033) performs incremental reconstruction based on the feature matching data to obtain the corresponding camera pose of each backgroundless image in the backgroundless image dataset.

5. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 1, characterized in that: The 3D model reconstruction module (3) includes a neural network training module (301) and a volume rendering module (302), wherein: The neural network training module (301) trains the multilayer perceptron based on the backgroundless image dataset and the corresponding camera poses of each backgroundless image in the backgroundless image dataset to obtain the color and voxel density corresponding to each point in the modeling image. The volume rendering module (302) integrates the sampling points on the light source based on the color and voxel density corresponding to each point in the modeling image to obtain the color of each pixel, and renders the modeling image based on the color of each element.

6. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 5, characterized in that: The volume rendering module (302) selects sampling points on the light rays based on two radiation field networks, namely a coarse network module (3021) and a fine network module (3022), wherein: The coarse network module (3021) performs a uniform sampling of points on the light ray to obtain a small number of points for rendering and training the network; The fine network module (3022) estimates the distribution of light rays based on the volume density value predicted by the coarse network module (3021), and performs secondary fine sampling on the points on the light rays based on the estimated light ray distribution to obtain a large number of points for rendering and training the network.

7. The rapid reconstruction system for 3D printed models based on neural radiation fields according to claim 1, characterized in that: The 3D printing entity module (4) includes a software module (401) and a 3D printer (402), wherein: The software module (401) includes MeshLab software and BambuStudio software. MeshLab software is connected to the volume rendering module (302) and performs file format conversion on the three-dimensional model of the entity to be 3D printed to obtain an STL format file. BambuStudio software is connected to MeshLab software and performs slicing processing on the STL format file to obtain a sliced ​​model. The 3D printer (402) 3D prints the sliced ​​model to obtain the physical object.

Images