3D printing path rendering method and device based on style migration network

Abstract

The invention discloses a 3D printing path rendering method and device based on a style migration network. The 3D printing path rendering method comprises the following steps: constructing a diversified data set for renderer training; the renderer takes the tensor obtained by converting the original path diagram and the printing height as input, uses U-Net as a backbone network to generate multi-scale texture features, and inputs the generated simulation path diagram into a true and false discriminator and a height classifier; the true and false discriminator takes the simulation path diagram and the true path diagram as input and forms confrontation with the image quality generated by the renderer, so that the renderer generates a high-quality rendering result; and the height classifier judges and classifies the printing height of the simulated path diagram, and recovers printing height information in the simulated path diagram, so that the renderer generates a rendering result with corresponding printing height style textures. According to the method and device, high-precision rendering can be carried out according to the printing path diagram, the printing height and the printing path, and the visual quality and fidelity of the rendered image are improved.

Description

Technical Field

[0001] This invention relates to the field of 3D (Three Dimensions) printing technology, and in particular to a 3D printing path rendering method and apparatus based on style transfer networks. Background Technology

[0002] In recent years, with the deepening of Industry 4.0, 3D printing technology has experienced rapid development and widespread adoption. 3D printers are moving towards consumer-grade applications and expanding their reach to end users. Currently, many key parameters in the 3D printing process (such as layer height, infill density, and print path) support user-defined settings. To ensure printing quality, users typically need to preview the 3D printing results to optimize parameters.

[0003] Some common 3D printing slicing software, such as Ultimaker Cura and Bambu Studio, offer comprehensive 3D printing path rendering methods. However, they lack simulation of real-world 3D printing textures, resulting in unrealistic rendering effects. To achieve lifelike 3D printing rendering, researchers have proposed various methods. For example, one method uses a diffusion model to transfer the style and texture of the printing material to the object being rendered, thus rendering the 3D printing effect of that object. However, this type of method can only transfer the texture style of the 3D model surface and cannot generate corresponding path renderings based on user-defined printing parameters, making it difficult to achieve editable and high-fidelity 3D printing path visualization. Summary of the Invention

[0004] The purpose of this invention is to provide a 3D printing path rendering method and apparatus based on style transfer networks. This method and apparatus can perform high-precision rendering based on the printing path map, printing height and printing path, thereby improving the visual quality and fidelity of the rendered image.

[0005] The objective of this invention is achieved through the following technical solution: A 3D printing path rendering method based on style transfer networks, the method comprising: Step 1: First, construct a diverse dataset for renderer training; the diverse dataset includes original-real path image pairs consisting of original path maps and real path maps; Step 2: The renderer takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator and the height classifier. Step 3: The real / fake discriminator takes the simulated path map and the real path map as input, and works against the renderer to generate high-quality image quality, so that the renderer can generate high-quality rendering results. Step 4: The height classifier judges and classifies the printing height of the simulated path map, restores the printing height information in the simulated path map, and enables the renderer to generate a rendering result with a style texture corresponding to the printing height.

[0006] A 3D printing path rendering device based on a style transfer network, the device comprising: A dataset construction module is used to construct diverse datasets for renderer training; the diverse datasets include original-real path image pairs consisting of original path graphs and real path graphs; The renderer module takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator module and the height classifier module. The real / fake discriminator module takes the simulated path map and the real path map as input and works against the renderer module in terms of the quality of the generated image, so that the renderer module can generate high-quality rendering results. The height classifier module is used to determine and classify the print height of the simulated path map, recover the print height information in the simulated path map, and enable the renderer module to generate a rendering result with a style texture corresponding to the print height.

[0007] As can be seen from the technical solution provided by the present invention, the above method and apparatus can perform high-precision rendering based on the printing path diagram, printing height and printing path, thereby improving the visual quality and fidelity of the rendered image. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A schematic diagram of the 3D printing path rendering method based on style transfer network provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the network structure of the true / false discriminator described in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the device described in an embodiment of the present invention; Figure 4 This is a schematic diagram of the rendering effect of the renderer described in an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0011] like Figure 1 The diagram shows a flowchart of a 3D printing path rendering method based on a style transfer network provided in an embodiment of the present invention. The method includes: Step 1: First, construct a diverse dataset for renderer training; the diverse dataset includes original-real path image pairs consisting of original path maps and real path maps; In this step, a fixed printing path G-code file is first used to print the same single-layer bottom sheet at different printing heights. These sheets are then scanned with a paper scanner to obtain the actual path map. The G-code file is the executable file input into the 3D printer, which contains instructions for print head movement, extrusion, etc., to guide the 3D printer to print related items.

[0012] Then, an original path map with the same size and path as the real path map is generated, and represented by black lines on a white background. Then, the original path map and the real path map are cropped synchronously using sliding windows of the same size and consistent position to obtain original-real path image pairs, which serve as diverse datasets for rendering training.

[0013] For example, to preserve the style and texture information of the real print result to the greatest extent, a fixed G-code file is printed by a consumer-grade Fused Deposition Modeling (FDM) printer at print heights of 0.2mm, 0.3mm, and 0.4mm, respectively, to obtain three different printed sheets containing bottom path textures. These sheets are then scanned by a consumer-grade paper scanner to convert them into digital images containing bottom path textures, which are called real path maps. A perspective transformation is applied to the real path map so that the length and width of these images are both integer multiples of 128. The white background is used as the background color and the black lines represent the printing path. The original path map with the same size as the real path map is generated by combining the G-code file used for printing. A 128-square sliding window is used to simultaneously and randomly crop the original path image and the real path image, forming original-real path image pairs for training the renderer. In the specific implementation, 2000 image pairs can be cropped from the real image and the original image at different printing heights, for a total of 6000 image pairs, which are used as training samples for the renderer.

[0014] Step 2: The renderer takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator and the height classifier. In this step, the renderer There are two inputs, including the original path graph. The height tensor spanned by the printed height value has the same dimensions as the original path map. Original path map for High-resolution color images, with height tensors After splicing, a tensor is obtained. Input into the renderer; The renderer network has n layers, using the nth layer. Layer and First The U-Net network structure, which connects all channels of the layer through a jumper structure, can take into account both low-frequency coarse-grained and high-frequency fine-grained feature information in the image, thereby improving the realism of the simulation.

[0015] Low-level features can preserve details in the real path map, including path boundaries, textures, and noise; high-level features can capture key path contour information. Among them, the size is The input image-height tensor will pass through 7 downsampling modules and 6 upsampling modules. Each downsampling module consists of a convolutional layer, an instance normalization layer, a ReLU activation function, and a Dropout random deactivation function stacked in sequence. The Dropout random deactivation function is turned off during training and turned on during use. Each upsampling module merges the output of the corresponding downsampling module with the output of the previous layer as input. It consists of a transposed convolutional layer, an instance normalization layer, a ReLU activation function, and a Dropout random deactivation function stacked in sequence. The Dropout random deactivation function is turned off during training and turned on during use. The result of the seventh upsampling is passed through the Tanh hyperbolic tangent function and a convolutional layer to generate a sample of size [size missing]. Simulated path diagram ,Right now .

[0016] Step 3: The real / fake discriminator takes the simulated path map and the real path map as input, and works against the renderer to generate high-quality image quality, so that the renderer can generate high-quality rendering results. In this step, the true / false discriminator Original path map With simulation path diagram Or actual path diagram The concatenated tensor is used as input; the true / false discriminator Original path map With simulation path diagram The merged input is considered false, as shown below: ; the original path map Compared with the actual path map If the merged input condition is true, it is represented as: Among them, the original path map As a true / false detector Preconditions for determining the authenticity of input images can accelerate model convergence and improve the accuracy of the discriminator. like Figure 2 The diagram shown is a schematic representation of the network structure of the true / false discriminator according to an embodiment of the present invention. First, divide the input image into horizontal and vertical sections. Non-overlapping, size [missing information] Image blocks, in which The image is divided into N parts horizontally and M parts vertically. For each image patch, a downsampling process consisting of 6 convolutional layers, instance normalization layers, and ReLU activation functions is performed, with a size of... The feature tensor is used as the output, and the loss function is calculated with the all-zero or all-one tensor of the same size; where the all-zero tensor represents false and the all-one tensor represents true. True / False Discriminator By optimizing network parameters to reduce the difference between simulated image discrimination results and fake results, as well as the difference between real image discrimination results and real results, the renderer is able to compete with the renderer in terms of generated image quality, thereby enabling the renderer to generate high-quality rendering results.

[0017] Step 4: The height classifier judges and classifies the printing height of the simulated path map, restores the printing height information in the simulated path map, and enables the renderer to generate a rendering result with a style texture corresponding to the printing height.

[0018] In this step, the height classifier Original path map With simulation path diagram The concatenated tensor is used as input, which then undergoes a downsampling process consisting of three convolutional layers, an instance normalization layer, and a ReLU activation function. Finally, it passes through a fully connected layer to form a feature vector of length 3. This feature vector is used in conjunction with the simulated path map generated by the renderer. The height tensor input at time Calculate the cross-entropy loss to enable the renderer to generate a rendering result with a style texture corresponding to the print height.

[0019] In practice, during the renderer's training process, the renderer generates a simulated path graph. Corresponding real path diagram Calculate reconstruction loss True / False Discriminator Calculate the discriminant loss Height classifier Calculate classification loss ; When optimizing the renderer, the real / fake detector With renderer parameters frozen, renderer loss function The definition is as follows: ; ; ; ; in, Original path map With height tensor The tensor obtained after splicing; For renderer For tensors The generated simulated path graph; True / False Discriminator Original path map With simulation path diagram The merged input is judged as false; Represents a height classifier Original path map With simulation path diagram The spliced ​​tensors form an eigenvector of length 3; The function outputs a tensor of all ones with the same size as the input tensor. Indicates a 1-dimensional tensor; , and They are respectively , and The weighting factor; MSE represents the mean squared error; BCE represents the binary cross-entropy loss; True / False Discriminator Simulated path diagram If it is determined to be fake, the corresponding real path diagram will be displayed. If the result is deemed genuine, the authenticity discriminator will be optimized. At this time, the renderer's parameters are frozen to optimize the real / fake detector. loss function The definition is as follows: ; ; ; ; in, True / False Discriminator Original path map Compared with the actual path map The merged input condition is true; The function outputs a tensor of all zeros with the same size as the input tensor. This represents a tensor containing all zeros. This represents the discriminant loss in the simulated path graph; Represent the actual path graph to determine the loss; , They are respectively , The weighting factors, in the default settings , ; Height classifier For the simulated path diagram Print height Judgment as In optimizing the height classifier At this time, the renderer's parameters are frozen to optimize the height classifier. loss function The definition is as follows: ; ; ; in, Classification loss The weighting factors, in the default settings ; Considering the mutual influence between the various modules, the renderer, real / fake discriminator, and height classifier are optimized alternately at each stage of the training process.

[0020] Based on the above method embodiments, this invention also provides a 3D printing path rendering device based on a style transfer network, such as... Figure 3 The diagram shown is a structural schematic of the device according to an embodiment of the present invention. The device includes: A dataset construction module is used to construct diverse datasets for renderer training; the diverse datasets include original-real path image pairs consisting of original path graphs and real path graphs; The renderer module takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator module and the height classifier module. The real / fake discriminator module takes the simulated path map and the real path map as input and works against the renderer module in terms of the quality of the generated image, so that the renderer module can generate high-quality rendering results. The height classifier module is used to determine and classify the print height of the simulated path map, recover the print height information in the simulated path map, and enable the renderer module to generate a rendering result with a style texture corresponding to the print height.

[0021] The specific implementation methods of each module in the above device are described in the method embodiment.

[0022] In specific implementations, the rendering method and training framework described in this embodiment of the invention can be implemented based on PyTorch, and the conversion of G-code into the original path graph is implemented based on OpenCV. During training, Adam with default hyperparameters is used as the optimizer, and StepLR is used as the learning rate scheduler.

[0023] After training, the G-code to be rendered will be converted into a raw path map, which, along with a print height ranging from 0.2mm to 0.4mm, will be input into the renderer. The renderer will then render it as a print path with realistic print texture. Figure 4 The diagram shown is a rendering effect diagram of the renderer described in the embodiment of the present invention. When the renderer is used to render the printing path at different printing heights, it can be found that the overall outline of the rendered path is consistent, the path edges, gaps, highlights and other high-frequency details are clearly preserved, and the style information corresponding to different heights is distinct and consistent with the real image at the corresponding height.

[0024] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0025] This invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method.

[0026] This invention also provides a computer storage medium storing a plurality of instructions adapted for loading and executing the method by a processor.

[0027] In summary, the method and apparatus described in the embodiments of the present invention have the following advantages: 1. This invention proposes a scheme for constructing a paired image dataset. By simultaneously cropping the 3D printing path map of real printing-scanning and the original image drawn according to G-code, image pairs are cropped, laying the foundation for the training of the renderer. 2. To improve the rendering quality of the renderer, this embodiment of the invention designs a real / fake discriminator and a height classifier. The real / fake discriminator and the renderer compete on the quality of the generated image, forcing the renderer to generate a higher quality rendering result; the height classifier performs variational inference on the height of the generated image, forcing the renderer to generate a style texture corresponding to the printing height.

[0028] Furthermore, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware, and the corresponding program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0029] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A 3D printing path rendering method based on style transfer networks, characterized in that, The method includes: Step 1: First, construct a diverse dataset for renderer training; the diverse dataset includes original-real path image pairs consisting of original path maps and real path maps; Step 2: The renderer takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator and the height classifier. Step 3: The real / fake discriminator takes the simulated path map and the real path map as input, and works against the renderer to generate high-quality image quality, so that the renderer can generate high-quality rendering results. Step 4: The height classifier judges and classifies the printing height of the simulated path map, restores the printing height information in the simulated path map, and enables the renderer to generate a rendering result with a style texture corresponding to the printing height.

2. The 3D printing path rendering method based on style transfer networks according to claim 1, characterized in that, In step 1, the same single-layer bottom sheet is first printed at different printing heights using a fixed printing path G-code file. These sheets are then scanned with a paper scanner to obtain the actual path map. The G-code file is the executable file input into the 3D printer to guide the 3D printer in printing the relevant items. Then, an original path map with the same size and path as the real path map is generated, and represented by black lines on a white background. Then, the original path map and the real path map are cropped synchronously using sliding windows of the same size and consistent position to obtain original-real path image pairs, which serve as diverse datasets for rendering training.

3. The 3D printing path rendering method based on style transfer networks according to claim 1, characterized in that, In step 2, the renderer There are two inputs, including the original path graph. The height tensor spanned by the printed height value has the same dimensions as the original path map. Original path map for High-resolution color images, with height tensors After splicing, a tensor is obtained. Input into the renderer; The renderer network has n layers, using the nth layer. Layer and First The U-Net network structure is a jumper structure that connects all channels in the layer. This U-Net network structure can take into account both low-frequency coarse-grained and high-frequency fine-grained feature information in the image.

4. The 3D printing path rendering method based on style transfer networks according to claim 3, characterized in that, Low-level features can preserve details in the real path map, including path boundaries, textures, and noise; high-level features can capture key path contour information. Among them, the size is The input image-height tensor will pass through 7 downsampling modules and 6 upsampling modules. Each downsampling module consists of a convolutional layer, an instance normalization layer, a ReLU activation function, and a Dropout random deactivation function stacked in sequence. The Dropout random deactivation function is turned off during training and turned on during use. Each upsampling module merges the output of the corresponding downsampling module with the output of the previous layer as input. It consists of a transposed convolutional layer, an instance normalization layer, a ReLU activation function, and a Dropout random deactivation function stacked in sequence. The Dropout random deactivation function is turned off during training and turned on during use. The result of the seventh upsampling is processed by the Tanh hyperbolic tangent function and a convolutional layer to generate a sample of size [size missing]. Simulated path diagram ,Right now .

5. The 3D printing path rendering method based on style transfer networks according to claim 1, characterized in that, In step 3, the true / false discriminator Original path map With simulation path diagram Or actual path diagram The concatenated tensor is used as input; true / false discriminator Original path map With simulation path diagram The merged input is considered false, as shown below: ; the original path map Compared with the actual path map If the merged input condition is true, it is represented as: ; where the original path map As a true / false detector Prerequisites for determining the authenticity of an input image; True / False Discriminator First, divide the input image into horizontal and vertical sections. Non-overlapping, size [missing information] Image blocks, in which The image is divided into N parts horizontally and M parts vertically. For each image patch, a downsampling process consisting of 6 convolutional layers, instance normalization layers, and ReLU activation functions is performed, with a size of... The feature tensor is used as the output, and the loss function is calculated with the all-zero or all-one tensor of the same size; where the all-zero tensor represents false and the all-one tensor represents true. True / False Discriminator By optimizing network parameters to reduce the difference between simulated image discrimination results and fake results, as well as the difference between real image discrimination results and real results, the renderer is able to compete with the quality of generated images.

6. The 3D printing path rendering method based on style transfer networks according to claim 5, characterized in that, In step 4, the height classifier Original path map With simulation path diagram The concatenated tensor is used as input, which then undergoes a downsampling process consisting of three convolutional layers, an instance normalization layer, and a ReLU activation function. Finally, it passes through a fully connected layer to form a feature vector of length 3. This feature vector is used in conjunction with the simulated path map generated by the renderer. The height tensor input at that time Calculate the cross-entropy loss to enable the renderer to generate a rendering result with a style texture corresponding to the print height.

7. The 3D printing path rendering method based on style transfer networks according to claim 6, characterized in that, During the renderer's training process, the renderer generates a simulated path map. Corresponding real path diagram Calculate reconstruction loss True / False Discriminator Calculate discriminant loss Height classifier Calculate classification loss ; When optimizing the renderer, the real / fake detector With renderer parameters frozen, renderer loss function The definition is as follows: ； ； ； ； in, Original path map With height tensor The tensor obtained after splicing; For renderer For tensors The generated simulated path graph; True / False Discriminator Original path map With simulation path diagram The merged input is judged as false; Represents a height classifier Original path map With simulation path diagram The spliced ​​tensors form an eigenvector of length 3; The function outputs a tensor of all ones with the same size as the input tensor. Indicates a 1-dimensional tensor; , and They are respectively , and The weighting factor; MSE represents the mean squared error; BCE represents the binary cross-entropy loss; True / False Discriminator Simulated path diagram If it is determined to be fake, the corresponding real path diagram will be displayed. If the result is deemed genuine, the authenticity discriminator will be optimized. At this time, the renderer's parameters are frozen to optimize the real / fake detector. loss function The definition is as follows: ； ； ； ； in, True / False Discriminator Original path map Compared with the actual path map The merged input condition is true; The function outputs a tensor of all zeros with the same size as the input tensor. This represents a tensor containing all zeros. This represents the discriminant loss in the simulated path graph; Represent the actual path graph to determine the loss; , They are respectively , Weighting factors; Height classifier For the simulated path diagram Print height Judgment as In optimizing the high-level classifier At this time, the renderer's parameters are frozen to optimize the height classifier. loss function The definition is as follows: ； ； ； in, Classification loss Weighting factors; Considering the mutual influence between the various modules, the renderer, real / fake discriminator, and height classifier are optimized alternately at each stage of the training process.

8. A 3D printing path rendering device based on a style transfer network, characterized in that, The device includes: A dataset construction module is used to construct diverse datasets for renderer training; the diverse datasets include original-real path image pairs consisting of original path graphs and real path graphs; The renderer module takes the tensor obtained by converting the original path map and the printed height as input, uses U-Net as the backbone network to generate multi-scale texture features, and inputs the generated simulated path map into the real / fake discriminator module and the height classifier module. The real / fake discriminator module takes the simulated path map and the real path map as input and works against the renderer module in terms of the quality of the generated image, so that the renderer module can generate high-quality rendering results. The height classifier module is used to determine and classify the print height of the simulated path map, recover the print height information in the simulated path map, and enable the renderer module to generate a rendering result with a style texture corresponding to the print height.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the method of any one of claims 1 to 7.

Images