A method and device for denoising a rendered image, a method and device for rendering a model, and a storage medium

Abstract

Embodiments of the present application provide a rendering image denoising method, a model rendering method, a device, equipment and a storage medium. The method comprises: obtaining an original rendering image obtained by rendering, the original rendering image comprising a color layer and at least one auxiliary layer; inputting the auxiliary layer into a corresponding first encoding-decoding network for processing to obtain a processed auxiliary layer, the first encoding-decoding network being a deep learning network for denoising the corresponding auxiliary layer; performing splicing processing based on the color layer and the processed auxiliary layer to obtain a spliced rendering image; inputting the spliced rendering image into a second encoding-decoding network for processing to obtain a final rendering image, the second encoding-decoding network being a deep learning network for denoising the spliced rendering image. The denoising method provided by the present application can better highlight the effective information in the auxiliary layer, thereby obtaining better denoising effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for denoising rendered images, a method for rendering models, an apparatus, a device, and a storage medium. Background Technology

[0002] In fields such as interior design, design schemes are typically presented using spatial renderings. Currently, most mainstream renderers employ Monte Carlo-based ray tracing methods; the higher the ray tracing sampling rate, the higher the rendering quality, but the longer the rendering time.

[0003] For scenes with many 3D models, high-definition textures, and complex lighting, such as interior design scenes, directly generating high-quality rendered images requires a lot of time and computing resources. Therefore, in practical use, in order to reduce rendering time and computing resources, a low sampling rate is usually used for rendering first to obtain a low-quality, noisy rendered image, and then the rendered image is denoised.

[0004] However, filtering and noise reduction methods based on traditional image algorithms (such as Gaussian filtering and mean filtering) suffer from poor noise reduction performance in scenarios with complex content and high requirements for rendering effects. Summary of the Invention

[0005] This application provides a method for denoising rendered images, a method for rendering models, an apparatus, a device, and a storage medium to solve the technical problem that the denoising effect of rendering images based on traditional image algorithms is poor in the prior art.

[0006] In a first aspect, embodiments of this application provide a method for denoising rendered images, including:

[0007] Obtain the original rendered image, which includes a color layer and at least one auxiliary layer;

[0008] The auxiliary layer is input into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer. The first encoding / decoding network is a deep learning network used to denoise the corresponding auxiliary layer.

[0009] Based on the color layer and the processed auxiliary layer, a stitched rendering image is obtained;

[0010] The stitched rendering image is input into a second encoding / decoding network for processing to obtain the final rendering image. The second encoding / decoding network is a deep learning network used to denoise the stitched rendering image.

[0011] Secondly, embodiments of this application provide a model rendering method, including:

[0012] Obtain the rendering request sent by the terminal;

[0013] In response to the rendering request, the corresponding scene model data is rendered to obtain the original rendering image;

[0014] A final rendered image is obtained based on the original rendered image, wherein the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain a processed auxiliary layer, wherein the first encoding / decoding network is a deep learning network for denoising the corresponding auxiliary layer; stitching the color layer and the processed auxiliary layer together to obtain a stitched rendered image; and inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, wherein the second encoding / decoding network is a deep learning network for denoising the stitched rendered image.

[0015] The final rendered image is sent to the terminal.

[0016] Thirdly, embodiments of this application provide a model rendering method, including:

[0017] In response to the user's rendering operation, a rendering request is sent to the server;

[0018] The server receives the final rendered image returned by the server according to the rendering request; the final rendered image is obtained by the server based on the original rendered image obtained by rendering the scene model data, the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer, the first encoding / decoding network being a deep learning network for denoising the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, the second encoding / decoding network being a deep learning network for denoising the stitched rendered image;

[0019] The final rendered image is then displayed to the user.

[0020] Fourthly, embodiments of this application provide a rendering image noise reduction device, comprising:

[0021] The acquisition module is used to acquire the original rendered image obtained from the rendering process. The original rendered image includes a color layer and at least one auxiliary layer.

[0022] The first noise reduction processing module is used to input the auxiliary layer into the corresponding first encoding and decoding network for processing to obtain the processed auxiliary layer. The first encoding and decoding network is a deep learning network used to reduce noise in the corresponding auxiliary layer.

[0023] The stitching module is used to perform stitching processing based on the color layer and the processed auxiliary layer to obtain a stitched rendering image;

[0024] The second noise reduction processing module is used to input the stitched rendering image into the second encoding and decoding network for processing to obtain the final rendering image. The second encoding and decoding network is a deep learning network used to reduce noise in the stitched rendering image.

[0025] Fifthly, embodiments of this application provide a model rendering apparatus, including:

[0026] The acquisition module is used to acquire the rendering request sent by the terminal;

[0027] The rendering module is used to render the corresponding scene model data to obtain the original rendering image in response to the rendering request.

[0028] A noise reduction module is used to obtain a final rendered image based on the original rendered image, wherein the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain a processed auxiliary layer, wherein the first encoding / decoding network is a deep learning network for noise reduction of the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; and inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, wherein the second encoding / decoding network is a deep learning network for noise reduction of the stitched rendered image;

[0029] The final rendered image is sent to the terminal.

[0030] Sixthly, embodiments of this application provide a model rendering apparatus, including:

[0031] The sending module is used to send rendering requests to the server in response to user rendering operations;

[0032] A receiving module is configured to receive the final rendered image returned by the server according to the rendering request; the final rendered image is obtained by the server based on the original rendered image obtained by rendering the scene model data, the original rendered image including a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer, the first encoding / decoding network being a deep learning network for denoising the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, the second encoding / decoding network being a deep learning network for denoising the stitched rendered image;

[0033] The display module is used to display the final rendered image to the user.

[0034] In a seventh aspect, embodiments of this application provide a computing device, including: a memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the first aspects.

[0035] Eighthly, embodiments of this application provide a server, including: a memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the second aspects.

[0036] Ninthly, embodiments of this application provide a server, including: a memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of the third aspects.

[0037] In a tenth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method as described in any one of the first aspects.

[0038] Eleventhly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the method as described in any one of the second aspects.

[0039] In a twelfth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon that, when executed, implements the method as described in any one of the third aspects.

[0040] This application also provides a computer program, which, when executed by a computer, is used to implement the method as described in any of the first aspects.

[0041] This application also provides a computer program, which, when executed by a computer, is used to implement the method described in any of the second aspects.

[0042] This application also provides a computer program that, when executed by a computer, is used to implement the method described in any of the third aspects.

[0043] In this embodiment, for the original rendered image including a color layer and at least one auxiliary layer, a first encoding / decoding network is first used to denoise the auxiliary layer. Then, the color layer and the processed auxiliary layer are stitched together to obtain a stitched rendered image. Finally, a second encoding / decoding network is used to denoise the stitched rendered image. The first and second encoding / decoding networks are deep learning networks, enabling intelligent denoising of the rendered image. Compared to traditional image denoising algorithms, this is more intelligent and yields better denoising results. Furthermore, by denoising the auxiliary layer, noise information in the auxiliary layer can be reduced while retaining effective information, making the effective information in the auxiliary layer clearer. Therefore, compared to the original rendered image, the stitched rendered image obtained based on the color layer and the processed auxiliary layer contains clearer effective information related to the auxiliary layer. Thus, the final rendered image obtained by denoising the stitched rendered image better highlights the effective information in the auxiliary layer compared to the final rendered image obtained by directly denoising the original rendered image, thereby achieving a better denoising effect. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this application;

[0046] Figure 2 This is a schematic flowchart of a rendering image noise reduction method provided in an embodiment of this application;

[0047] Figure 3 A block diagram of the rendering image noise reduction scheme provided in the embodiments of this application;

[0048] Figure 4 A schematic flowchart of a rendering image noise reduction method provided in another embodiment of this application;

[0049] Figure 5 This is a schematic diagram illustrating a noise reduction process for a rendered image provided in an embodiment of this application.

[0050] Figure 6A This is a magnified view of a portion of the rendered image.

[0051] Figure 6B To Figure 6A The rendered image shown is a partial magnified view of the final rendered image obtained without using the auxiliary layer noise reduction module in this embodiment;

[0052] Figure 6C To Figure 6A The rendered image shown is a partial enlarged view of the final rendered image obtained using the noise reduction scheme provided in the embodiments of this application;

[0053] Figure 7 A schematic flowchart of a model rendering method provided in an embodiment of this application;

[0054] Figure 8 A schematic flowchart of a model rendering method provided in another embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the structure of a rendering image noise reduction device provided in an embodiment of this application;

[0056] Figure 10 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0057] Figure 11 This is a schematic diagram of the structure of a model rendering apparatus provided in an embodiment of this application;

[0058] Figure 12 This is a schematic diagram of the structure of a server provided in one embodiment of this application;

[0059] Figure 13 This is a schematic diagram of the structure of a model rendering apparatus provided in another embodiment of this application;

[0060] Figure 14 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0063] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0064] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0065] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0066] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0067] Figure 1 This is a schematic diagram illustrating an application scenario of the rendering image noise reduction method provided in an embodiment of this application, such as... Figure 1As shown, this application scenario may include a first device 11 and a second device 12. The first device 11 may run a renderer, which can render a low-quality, noisy image (hereinafter referred to as the original render image). In practical applications, the renderer can use a Monte Carlo-based ray tracing algorithm to obtain the render image. Ray tracing is a rendering algorithm in 3D computer graphics that obtains a model of the ray path by tracing the rays that interact with optical surfaces. Furthermore, to reduce rendering time and computational resources, the renderer can use a low sampling rate to render, resulting in a low-quality, noisy render image. The second device 12 can denoise the original render image obtained by the first device 11 to obtain the final render image.

[0068] Typically, noise reduction can be achieved by using traditional image algorithms. However, in scenarios with complex content and high requirements for rendering effects, the noise reduction effect is often poor.

[0069] To address the technical problem of poor denoising performance when using traditional image denoising algorithms on original rendered images, this application embodiment describes a method for denoising an original rendered image comprising a color layer and at least one auxiliary layer. First, a first encoding / decoding network is used to denoise the auxiliary layer. Then, the color layer and the processed auxiliary layer are stitched together to obtain a stitched rendered image. Finally, a second encoding / decoding network is used to denoise the stitched rendered image to obtain the final rendered image. The first and second encoding / decoding networks are deep learning networks, enabling intelligent denoising of the rendered image through deep learning. Compared to traditional image denoising algorithms, this method is more intelligent and achieves better denoising results. Furthermore, by denoising the auxiliary layer, noise information in the auxiliary layer can be reduced while retaining the effective information, making the effective information contained in the auxiliary layer clearer. Therefore, compared with the original rendering image, the stitched rendering image obtained based on the color layer and the processed auxiliary layer contains clearer effective information related to the auxiliary layer. Thus, the final rendering image obtained by denoising the stitched rendering image can better highlight the effective information in the auxiliary layer compared with the final rendering image obtained by directly denoising the original rendering image, thereby achieving a better denoising effect.

[0070] It should be noted that, Figure 1 The fact that the rendered image and the noise reduction of the rendered image are performed by different devices is only an example. It is understood that in other embodiments, they can also be performed by the same device.

[0071] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0072] Figure 2 This is a schematic flowchart of a rendering image noise reduction method provided in an embodiment of this application. This embodiment can be applied to... Figure 1 The second device 12 in the process can be executed by the processor of the second device 12. For example... Figure 2 As shown, the method in this embodiment may include:

[0073] Step 21: Obtain the original rendered image, which includes a color layer and at least one auxiliary layer.

[0074] Step 22: Input the auxiliary layer into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer. The first encoding / decoding network is a deep learning network used to denoise the corresponding auxiliary layer.

[0075] Step 23: Perform a stitching process based on the color layer and the processed auxiliary layer to obtain the stitched rendering image;

[0076] Step 24: Input the stitched rendering image into the second encoding / decoding network for processing to obtain the final rendering image. The second encoding / decoding network is a deep learning network used to denoise the stitched rendering image.

[0077] In this embodiment, the original rendered image may be a low-quality, noisy rendered image obtained by the renderer. In one embodiment, the original rendered image may be a rendered image obtained by the renderer using a ray tracing-based rendering method. The ray tracing sampling rate can be denoted as the first sampling rate. The original rendered image obtained based on the first sampling rate is a low-quality, noisy rendered image. For example, the renderer may obtain the original rendered image based on scene model data. The scene model data may include 3D modeling data of multiple objects and environmental data. The environmental data may include light source positions, camera positions, etc. Taking an interior design scene as an example, the multiple objects may specifically be multiple furniture objects.

[0078] The original rendered image includes a color layer and at least one auxiliary layer. The color layer, also called a color layer, contains pixel values ​​that represent the color information of the corresponding pixel. For example, the color layer can be an image with three channels: red (R), green (G), and blue (B). The auxiliary layer refers to any layer in the original rendered image other than the color layer that contains information that is crucial for noise reduction. For example, in one embodiment, when noise reduction requires focusing on high-frequency details to minimize their loss, the auxiliary layer can be a non-color layer containing high-frequency detail information.

[0079] In one embodiment, at least one auxiliary layer may include a diffuse layer and / or a bump normal layer. The diffuse layer may also be called a diffuse layer, and the pixel values ​​in the diffuse layer are used to represent the diffuse color of the corresponding pixel, that is, the diffuse intrinsic color without illumination. The bump normal layer may also be called a bump normal layer, and the pixel values ​​in the bump normal layer are used to represent the normal information of the corresponding pixel.

[0080] In this embodiment, after obtaining the original rendered image, the auxiliary layer can be input into the corresponding encoder-decoder network (hereinafter referred to as the first encoder-decoder network) for processing to obtain the processed auxiliary layer. In this application, an encoder-decoder network consists of an encoder network and a corresponding decoder network. The two networks compress and expand the feature dimensions through downsampling and upsampling, respectively. A classic encoder-decoder network is U-Net.

[0081] The first encoding / decoding network is a deep learning network. Since the first encoding / decoding network has undergone deep learning, it can learn to reduce noise in the corresponding auxiliary layer. Therefore, using the first encoding / decoding network to process the auxiliary layer can reduce the noise in the auxiliary layer.

[0082] A single first encoder-decoder network can be trained based on a corresponding sample set (hereinafter referred to as the first sample set). The first sample set includes multiple sample pairs, and each sample pair may include a first sample auxiliary layer and a second sample auxiliary layer. The first sample auxiliary layer is an auxiliary layer in the first original rendered image, and the second sample auxiliary layer is an auxiliary layer in the second original rendered image. The first and second original rendered images are rendered images obtained by rendering the same data to be rendered, and the quality of the second original rendered image is greater than that of the first original rendered image. In one embodiment, the first and second original rendered images can be rendered images obtained based on a ray tracing method, and the sampling rate of obtaining the first original rendered image is lower than the sampling rate of obtaining the second original rendered image.

[0083] In one embodiment, the method of training the first encoder-decoder network may include: constructing a first encoder-decoder network, wherein training parameters are set in the first encoder-decoder network; inputting a first sample auxiliary layer into the corresponding first encoder-decoder network for processing to obtain a processed first sample auxiliary layer; and iteratively adjusting the training parameters based on the difference between the processed first sample auxiliary layer and the corresponding second sample auxiliary layer until the difference meets a preset requirement.

[0084] For example, suppose the first encoding / decoding network is the encoding / decoding network corresponding to the diffuse layer. Each sample pair in the first sample set corresponding to the first encoding / decoding network may include a diffuse layer in the first original rendering image and a diffuse layer in the second original rendering image. The diffuse layer in the first original rendering image can be input into the first encoding / decoding network for processing to obtain the processed diffuse layer. Based on the difference between the processed diffuse layer and the diffuse layer in the second original rendering image, the training parameters are iteratively adjusted until the difference meets the preset requirements.

[0085] In one embodiment, when training the first encoder-decoder network, L1 loss and image edge loss can be used, and the loss of the first encoder-decoder network can satisfy the following formula (1).

[0086] Edge image.

[0087] Optionally, after obtaining the original rendering image, at least one auxiliary layer in the original rendering image can be directly input into the corresponding first encoding / decoding network for processing.

[0088] Alternatively, after obtaining the original rendering image, if the quality of the auxiliary layer in the original rendering image is not high, the auxiliary layer can be input into the corresponding first encoding / decoding network for processing. This helps to save processing resources and improve processing efficiency. The quality can be used to characterize whether the details are clear; the clearer the details, the higher the quality.

[0089] Based on this, in one embodiment, the step of inputting the auxiliary layer into the corresponding first encoding / decoding network for processing may specifically include: inputting at least one auxiliary layer whose quality is all below a quality threshold into the corresponding first encoding / decoding network for processing.

[0090] For example, suppose at least one auxiliary layer includes a diffuse layer and a normal layer. If the quality of the diffuse layer is lower than the quality threshold, the diffuse layer is input into the corresponding first encoder-decoder network for processing. If the quality of the normal layer is lower than the quality threshold, the normal layer is input into the corresponding first encoder-decoder network for processing. If the quality of both the diffuse layer and the normal layer is higher than the quality threshold, then the diffuse layer and the normal layer do not need to be input into the first encoder-decoder network for processing.

[0091] In this embodiment, after obtaining the processed auxiliary layer, a stitching process can be performed based on the color layer and the processed auxiliary layer to obtain a stitched rendering image. Taking at least one auxiliary layer including a diffuse layer and a normal layer, and the diffuse layer, normal layer, and color layer being RGB three-channel images as an example, the number of channels in the stitched rendering image can be greater than or equal to 9.

[0092] Optionally, you can directly stitch the color layer and the processed auxiliary layer together to obtain a stitched rendering image.

[0093] Alternatively, the information that needs to be enhanced can be stitched together with the color layer and the processed auxiliary layer to obtain a stitched rendering image. Considering that edge information is easily blurred, an edge information layer can be added to enhance the edge information and make the final rendering image better emphasize high frequencies.

[0094] Based on this, in one embodiment, the method provided by this embodiment may further include: generating a corresponding edge information layer based on the processed auxiliary layer, wherein the pixel values ​​in the edge information layer are used to represent the gradient values ​​of the corresponding pixel points; the step of performing splicing processing based on the color layer and the processed auxiliary layer to obtain a spliced ​​rendering image may specifically include: splicing the color layer, the processed auxiliary layer and the edge information layer to obtain a spliced ​​rendering image.

[0095] Optionally, a corresponding edge information layer can be generated based on a single processed auxiliary layer. In one embodiment, generating a corresponding edge information layer based on a processed auxiliary layer may specifically include: determining the edge image of a certain processed auxiliary layer and using the edge image as the edge information layer.

[0096] Alternatively, an edge information layer can be generated based on all processed auxiliary layers. In one embodiment, generating the corresponding edge information layer based on the processed auxiliary layers may specifically include: determining the edge image of each processed auxiliary layer, and generating a corresponding edge information layer based on the edge image of each processed auxiliary layer. For example, the pixel values ​​of corresponding pixels in the edge images of all processed edge layers can be averaged to obtain the edge information layer.

[0097] In this embodiment of the application, after obtaining the stitched rendering image, the stitched rendering image can be input into another encoding and decoding network (hereinafter referred to as the second encoding and decoding network) for processing to obtain the final rendering image.

[0098] The second encoding / decoding network is a deep learning network. Because the second encoding / decoding network has undergone deep learning, it can learn to reduce noise in the rendered image. Therefore, using the second encoding / decoding network to process the stitched rendered image can reduce the noise in the stitched rendered image.

[0099] The second encoder-decoder network can be trained based on a corresponding sample set (hereinafter referred to as the second sample set). The second sample set includes multiple sample pairs, each of which can include a first sample rendering image and a second sample rendering image. The first sample rendering image is obtained based on a first sample color layer and a first sample auxiliary layer. The first sample color layer is the color layer in the first original rendering image, and the first sample auxiliary layer is the auxiliary layer in the first original rendering image. The second sample rendering image is obtained by concatenating the second sample color layer and the second sample auxiliary layer. The second sample color layer is the color layer in the second original rendering image, and the second sample auxiliary layer is the auxiliary layer in the second original rendering image. The first and second original rendering images are obtained by rendering the same data to be rendered, and the quality of the second original rendering image is greater than that of the first original rendering image. In one embodiment, the first and second original rendering images can be rendering images obtained based on a ray tracing method, and the sampling rate of obtaining the first original rendering image is lower than the sampling rate of obtaining the second original rendering image.

[0100] In one embodiment, the method of training the second encoder-decoder network may include: constructing a second encoder-decoder network, wherein training parameters are set in the second encoder-decoder network; inputting a first sample rendering image into the second encoder-decoder network for processing to obtain a processed first sample rendering image; and iteratively adjusting the training parameters based on the difference between the processed first sample rendering image and the corresponding second sample rendering image until the difference meets a preset requirement.

[0101] In one embodiment, the first sample rendering image in the second sample set can be obtained by stitching together a first sample color layer and a first sample auxiliary layer. In this case, after training the second encoder-decoder network based on the second sample set, the second encoder-decoder network can be fine-tuned based on a third sample set. The third sample set includes multiple sample pairs, each of which can include a third sample rendering image and a second sample rendering image. The third sample rendering image is obtained by stitching together a first sample color layer and a processed first sample auxiliary layer. The processed first sample auxiliary layer is obtained by inputting the first sample auxiliary layer into the corresponding (trained) first encoder-decoder network. For further explanation regarding the first sample color layer and the second sample rendering image, please refer to the foregoing description; it will not be repeated here.

[0102] Based on this, the above-mentioned method for training the second encoder-decoder network may also include: inputting the third sample rendering image into the second encoder-decoder network for processing to obtain the processed third sample rendering image; and iteratively adjusting the training parameters based on the difference between the processed third sample rendering image and the corresponding second sample rendering image until the difference meets the preset requirements.

[0103] In one embodiment, when training the second encoder-decoder network, L1 loss and image edge loss can be used, and the loss of the first encoder-decoder network can satisfy the following formula (2).

[0104] Edge image.

[0105] The rendering image denoising method provided in this embodiment obtains the original rendered image, which includes a color layer and at least one auxiliary layer. The auxiliary layer is input into a corresponding first encoding / decoding network for processing to obtain a processed auxiliary layer. The color layer and the processed auxiliary layer are then stitched together to obtain a stitched rendered image. This stitched rendered image is then input into a second encoding / decoding network for processing to obtain the final rendered image. This method achieves intelligent denoising of the rendered image through deep learning networks, which is more intelligent and has a better denoising effect compared to traditional image denoising algorithms. Furthermore, by denoising the auxiliary layer, noise information in the auxiliary layer can be reduced while retaining effective information, making the effective information contained in the auxiliary layer clearer. Therefore, compared to the original rendered image, the stitched rendered image obtained based on the color layer and the processed auxiliary layer contains clearer effective information related to the auxiliary layer. Thus, the final rendered image obtained by denoising the stitched rendered image better highlights the effective information in the auxiliary layer compared to the final rendered image obtained by directly denoising the original rendered image, thereby achieving a better denoising effect.

[0106] In one embodiment, the block diagram of the rendering image denoising scheme can be as follows: Figure 3 As shown, the noise reduction process for the rendered image can be as follows: Figure 4 As shown. Reference Figure 3 The rendering image denoising scheme mainly includes an auxiliary layer denoising module 31 and a color layer denoising module 32. Taking a 2K resolution image as an example, refer to... Figure 4In step 41, a 2K rendering task can be initiated. Then, in step 42, the rendering engine can execute a low sampling rate rendering task. The rendering result will generate several layers. In this embodiment, three layers are used: 1) a noisy color layer, 2) a diffuse layer, and 3) a bump normal layer. The diffuse layer and the bump normal layer can be collectively referred to as auxiliary layers. Then, in step 43, the quality of the auxiliary layers can be judged. If the quality of the auxiliary layers is less than the quality threshold (less than the quality threshold can be considered as low quality), then in step 44, the auxiliary layers can be input into the auxiliary layer denoising module 31 to denoise the auxiliary layers and obtain high-quality auxiliary layers, namely high-quality diffuse and bump normal layers. Then, in step 45, the color layer and the high-quality auxiliary layers can be input into the color layer denoising module 32 to generate the final rendered image. Finally, in step 46, the final rendered image can be output.

[0107] The auxiliary layer denoising module 31 removes noise from low-quality diffuse and bump normal layers. When the rendering sampling rate is low, the diffuse and normal layers also suffer from noise and loss of high-frequency details, affecting the denoising effect of the rendered image. Using a high-quality auxiliary layer generated by the auxiliary layer denoising module can significantly improve the denoising effect of subsequent noise layers, preserving more detail information in the image. Figure 5 As shown, the input to the auxiliary layer noise reduction module 31 can be a low-quality diffuse layer and a normal layer output by the renderer. These are input to an encoder-decoder network with the same structure but different weights, and the final output is a high-quality, almost noise-free diffuse layer and a normal layer. Here, the input size of the encoder-decoder network is H×W×3, where H and W are the length and width of the input image, and 3 represents the three RGB channels of the image.

[0108] The function of the color layer noise reduction module 32 is to remove noise from the rendered image. For example... Figure 5As shown, the color layer denoising module 32 can have four inputs: a color layer, a processed diffuse layer, a processed normal layer, and an edge information layer. The diffuse and normal layers are processed by the auxiliary layer denoising module, while the edge information layer is obtained by extracting and averaging the edge information from the diffuse and normal layers. These four layers are then concatenated and input into the encoding / decoding network, ultimately outputting a high-quality, almost noise-free rendered image. The structure of the encoding / decoding network used in the color layer denoising module 32 is basically the same as that in the auxiliary layer denoising module 31, except for the input size. Here, the input size of the encoding / decoding network is H×W×10, where H and W are the length and width of the input image, and 9 represents the three input layers: color, diffuse, and normal. Each layer has three RGB channels, and the remaining channel is a single-channel image of the edge information layer.

[0109] Before applying the noise reduction scheme provided in this embodiment to the original rendered image, a magnified view of a portion of the original rendered image can be referenced at 6A, such as... Figure 6A As shown, before denoising the original render image, there is a lot of noise and the overall image is quite blurry. For Figure 6A The original rendered image shown does not use the auxiliary layer denoising module 32 in this embodiment for auxiliary layer denoising. Instead, the auxiliary layer is directly input into the color layer denoising module 31 for denoising. A partial enlarged view of the resulting final rendered image can be referenced. Figure 6B , Figure 6B The final rendered image shown is... Figure 6A The rendered image shown is clearer, but the details are still somewhat blurry (you can focus on these details). Figure 6B (The area within the rectangular frame). For Figure 6A The original rendered image shown is accompanied by a magnified view of the final rendered image obtained by denoising using the denoising scheme provided in this embodiment. Figure 6C ,like Figure 6C The final rendered image shown is... Figure 6B The final render shown here clearly enhances the details of wall seams and tile joints (pay special attention to these details). Figure 6C (The area within the rectangular frame).

[0110] As can be seen, compared to directly using low-quality auxiliary layers, the solution provided in this embodiment, by adding an auxiliary layer denoising module, allows the auxiliary layer processed by the denoising module to contain more effective detail information of the image, effectively improving the quality of the final rendered image. Furthermore, traditional image denoising methods based on image algorithms are rule-based and difficult to apply to complex and rich rendering scenarios. The solution provided in this embodiment, however, is trained on a large amount of data, making it more efficient and versatile, thus resulting in better rendering effects.

[0111] Figure 7 This is a flowchart illustrating a model rendering method provided in an embodiment of this application. The method provided in this embodiment can be executed by a server, such as... Figure 7 As shown, the method provided in this embodiment may include:

[0112] Step 71: Obtain the rendering request sent by the terminal;

[0113] Step 72: In response to the rendering request, render the corresponding scene model data to obtain the original rendering image;

[0114] Step 73: Obtain the final rendered image based on the original rendered image. The original rendered image includes a color layer and at least one auxiliary layer. The final rendered image is obtained as follows: the auxiliary layer is input into the corresponding first encoder-decoder network for processing to obtain a processed auxiliary layer. The first encoder-decoder network is a deep learning network used for denoising the corresponding auxiliary layer. The color layer and the processed auxiliary layer are then stitched together to obtain a stitched rendered image. The stitched rendered image is then input into a second encoder-decoder network for processing to obtain the final rendered image. The second encoder-decoder network is a deep learning network used for denoising the stitched rendered image.

[0115] Step 74: Send the final rendered image to the terminal.

[0116] For example, scene model data can be obtained by the server through the following method: acquiring operation information of interior design operations sent by the terminal; and generating scene model data based on the operation information. The interior design operations are input by the user using the terminal for interior design, such as operations like dragging and dropping components, adjusting component positions, setting component attributes, setting camera positions, and setting light source positions. The server can generate scene model data including 3D model data and environmental data based on the operation information of the interior design operations. It should be noted that this application does not limit the specific method by which the server generates scene model data based on the operation information of the interior design operations.

[0117] A rendering request can be triggered by a rendering operation input by the user on the terminal. In response to the rendering request, the server can use the renderer to render the scene model data to obtain a raw rendering image. After obtaining the raw rendering image, the final rendering image can be obtained based on the raw rendering image. The raw rendering image includes a color layer and at least one auxiliary layer.

[0118] In one embodiment, obtaining the final rendered image based on the original rendered image may specifically include: inputting the auxiliary layer into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; and inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image.

[0119] In another embodiment, obtaining the final rendered image based on the original rendered image may specifically include: sending the original rendered image to other devices so that the other devices can input the auxiliary layer into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer; performing splicing processing based on the color layer and the processed auxiliary layer to obtain the spliced ​​rendered image; and inputting the spliced ​​rendered image into the second encoding / decoding network for processing to obtain the final rendered image.

[0120] It should be noted that the specific method for noise reduction of the original rendered image can be found in the description of the foregoing embodiments, and will not be repeated here.

[0121] The model rendering method provided in this embodiment obtains the rendering request sent by the terminal, responds to the rendering request, renders the corresponding scene model data to obtain the original rendering image, obtains the final rendering image based on the original rendering image, and sends the final rendering image to the terminal. This realizes the provision of an interior design rendering image with better noise reduction effect to the terminal in the interior design scene, which can better highlight the effective information in the auxiliary layer.

[0122] Figure 8 This is a flowchart illustrating a model rendering method provided in another embodiment of this application. The method provided in this embodiment can be executed by a terminal, such as... Figure 8 As shown, the method provided in this embodiment may include:

[0123] Step 81: In response to the user's rendering operation, a rendering request is sent to the server;

[0124] Step 82: Receive the final rendered image returned by the server based on the rendering request. The final rendered image is obtained by the server based on the original rendered image obtained from the scene model data. The original rendered image includes a color layer and at least one auxiliary layer. The final rendered image is obtained as follows: The auxiliary layer is input into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer. The first encoding / decoding network is a deep learning network used for denoising the corresponding auxiliary layer. The color layer and the processed auxiliary layer are stitched together to obtain a stitched rendered image. The stitched rendered image is input into a second encoding / decoding network for processing to obtain the final rendered image. The second encoding / decoding network is a deep learning network used for denoising the stitched rendered image.

[0125] Step 83: Display the final rendered image to the user.

[0126] It should be noted that, Figure 8 The illustrated embodiment is similar to Figure 7 The terminal-side implementation corresponding to the illustrated embodiment; for specific implementation details regarding the server-side and terminal-side, please refer to... Figure 7 The description in the illustrated embodiment.

[0127] The model rendering method provided in this embodiment sends a rendering request to the server in response to the user's rendering operation, receives the final rendered image returned by the server according to the rendering request, and displays the final rendered image to the user. This realizes the display of an interior design rendering image with better noise reduction effect that can better highlight the effective information in the auxiliary layers in the interior design scene.

[0128] Figure 9 This is a schematic diagram of the structure of a rendering image noise reduction device provided in an embodiment of this application; see attached drawing. Figure 9 As shown, this embodiment provides a noise reduction device that can perform the noise reduction method provided in the above embodiment. Specifically, the device may include:

[0129] The acquisition module 91 is used to acquire the original rendered image obtained from the rendering process. The original rendered image includes a color layer and at least one auxiliary layer.

[0130] The first noise reduction processing module 92 is used to input the auxiliary layer into the corresponding first encoding and decoding network for processing to obtain the processed auxiliary layer.

[0131] The splicing module 93 is used to perform splicing processing based on the color layer and the processed auxiliary layer to obtain a spliced ​​rendering image;

[0132] The second noise reduction processing module 94 is used to input the stitched rendering image into the second encoding and decoding network for processing to obtain the final rendering image.

[0133] Optionally, the stitching module 93 is also used to generate a corresponding edge information layer based on the processed auxiliary layer, wherein the pixel values ​​in the edge information layer are used to represent the gradient values ​​of the corresponding pixels.

[0134] The stitching module 9 is used to perform stitching processing based on the color layer and the processed auxiliary layer to obtain a stitched rendering image. Specifically, it includes stitching the color layer, the processed auxiliary layer and the edge information layer to obtain a stitched rendering image.

[0135] Optionally, the stitching module 93 is used to generate a corresponding edge information layer based on the processed auxiliary layers, specifically including: determining the edge image of each processed auxiliary layer, and generating a corresponding edge information layer based on the edge image of each processed auxiliary layer.

[0136] Optionally, the first noise reduction processing module 92 is used to input the auxiliary layer into the corresponding first encoding and decoding network for processing, specifically including: inputting the auxiliary layer with quality lower than the quality threshold from the at least one auxiliary layer into the corresponding first encoding and decoding network for processing.

[0137] Optionally, the at least one auxiliary layer includes a diffuse layer and / or a normal layer.

[0138] Optionally, the first encoder-decoder network can be trained in the following ways:

[0139] Construct the first encoder-decoder network, wherein training parameters are set in the first encoder-decoder network;

[0140] The first sample auxiliary layer is input into the corresponding first encoding / decoding network for processing to obtain the processed first sample auxiliary layer. The first sample auxiliary layer is the auxiliary layer in the first original rendering image.

[0141] Based on the difference between the processed first sample auxiliary layer and the corresponding second sample auxiliary layer, the training parameters are iteratively adjusted until the difference meets the preset requirements. The second sample auxiliary layer is an auxiliary layer in the rendered second original rendering image. The first original rendering image and the second original rendering image are rendering images obtained by rendering the same data to be rendered, and the quality of the second original rendering image is higher than that of the first original rendering image.

[0142] Optionally, the second encoder-decoder network can be trained in the following ways:

[0143] Construct the second encoder-decoder network, which is equipped with training parameters;

[0144] The first sample rendering image is input into the second encoding and decoding network for processing to obtain the processed first sample rendering image. The first sample rendering image is obtained based on the first sample color layer and the first sample auxiliary layer. The first sample color layer is the color layer in the first original rendering image, and the first sample auxiliary layer is the auxiliary layer in the first original rendering image.

[0145] Based on the difference between the processed first sample rendering image and the corresponding second sample rendering image, the training parameters are iteratively adjusted until the difference meets the preset requirements. The second sample rendering image is obtained by splicing the second sample color layer and the second sample auxiliary layer. The second sample color layer is the color layer in the rendered second original rendering image, and the second sample auxiliary layer is the auxiliary layer in the second original rendering image. The first original rendering image and the second original rendering image are rendering images obtained by rendering the same data to be rendered, and the quality of the second original rendering image is higher than that of the first original rendering image.

[0146] Optionally, the first sample rendering image is obtained by stitching together the first sample color layer and the first sample auxiliary layer;

[0147] The methods for training the second encoder-decoder network also include:

[0148] The third sample rendering image is input into the second encoding / decoding network for processing to obtain the processed third sample rendering image. The third sample rendering image is obtained by stitching together the first sample color layer and the processed first sample auxiliary layer. The processed first sample auxiliary layer is obtained by inputting the first sample auxiliary layer into the corresponding first encoding / decoding network for processing.

[0149] Based on the difference between the processed third sample rendering image and the corresponding second sample rendering image, the training parameters are iteratively adjusted until the difference meets the preset requirements.

[0150] Figure 9 The device shown can perform Figure 2 The method provided in the illustrated embodiment, for parts not described in detail in this embodiment, can be referred to the [examples / descriptions]. Figure 2 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 2 The descriptions in the illustrated embodiments will not be repeated here.

[0151] In one possible implementation, Figure 9 The structure of the device shown can be implemented as a computing device. For example... Figure 10 As shown, the computing device may include a processor 101 and a memory 102. The memory 102 stores data that supports the controller in performing the above-described functions. Figure 2 The program of the method provided in the illustrated embodiment, the processor 101 is configured to execute the program stored in the memory 102.

[0152] The program includes one or more computer instructions, wherein when executed by processor 101, the one or more computer instructions can perform the following steps:

[0153] Obtain the original rendered image, which includes a color layer and at least one auxiliary layer;

[0154] The auxiliary layer is input into the corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer.

[0155] Based on the color layer and the processed auxiliary layer, a stitched rendering image is obtained;

[0156] The stitched rendering image is input into the second encoding / decoding network for processing to obtain the final rendering image.

[0157] Optionally, the processor 101 is also used to perform the aforementioned Figure 2 All or part of the steps in the illustrated embodiments.

[0158] The computing device may also include a communication interface 103 for communicating with other devices or communication networks.

[0159] Figure 11 This is a schematic diagram of the structure of a model rendering apparatus provided in an embodiment of this application; see attached diagram. Figure 11 As shown, this embodiment provides a model rendering apparatus that can perform the above-described... Figure 7 The method provided in the illustrated embodiment, specifically, the apparatus may include:

[0160] The acquisition module 111 is used to acquire the rendering request sent by the terminal;

[0161] Rendering module 112 is used to render the corresponding scene model data to obtain the original rendering image in response to the rendering request;

[0162] The noise reduction module 113 is used to obtain a final rendering image based on the original rendering image, wherein the original rendering image includes a color layer and at least one auxiliary layer; the final rendering image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain a processed auxiliary layer, wherein the first encoding / decoding network is a deep learning network for noise reduction of the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendering image; and inputting the stitched rendering image into a second encoding / decoding network for processing to obtain the final rendering image, wherein the second encoding / decoding network is a deep learning network for noise reduction of the stitched rendering image;

[0163] The sending module 114 is used to send the final rendered image to the terminal.

[0164] Figure 11The device shown can perform Figure 7 The method provided in the illustrated embodiment, for parts not described in detail in this embodiment, can be referred to the [examples / descriptions]. Figure 7 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 7 The descriptions in the illustrated embodiments will not be repeated here.

[0165] In one possible implementation, Figure 11 The structure of the device shown can be implemented as a server. For example... Figure 12 As shown, the server may include a processor 121 and a memory 122. The memory 122 stores data that supports the controller in performing the above-described functions. Figure 7 The program of the method provided in the illustrated embodiment, the processor 121 is configured to execute the program stored in the memory 122.

[0166] The program includes one or more computer instructions, wherein when executed by processor 121, the one or more computer instructions can perform the following steps:

[0167] Obtain the rendering request sent by the terminal;

[0168] In response to the rendering request, the corresponding scene model data is rendered to obtain the original rendering image;

[0169] A final rendered image is obtained based on the original rendered image, wherein the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain a processed auxiliary layer, wherein the first encoding / decoding network is a deep learning network for denoising the corresponding auxiliary layer; stitching the color layer and the processed auxiliary layer together to obtain a stitched rendered image; and inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, wherein the second encoding / decoding network is a deep learning network for denoising the stitched rendered image.

[0170] The final rendered image is sent to the terminal.

[0171] Optionally, the processor 121 is also used to perform the aforementioned Figure 7 All or part of the steps in the illustrated embodiments.

[0172] The server structure may also include a communication interface 123 for communication between the server and other devices or communication networks.

[0173] Figure 13 This is a schematic diagram of the structure of a model rendering apparatus provided in another embodiment of this application; see attached diagram. Figure 13As shown, this embodiment provides a model rendering apparatus that can perform the above-described... Figure 8 The method provided in the illustrated embodiment, specifically, the apparatus may include:

[0174] Sending module 131 is used to send a rendering request to the server in response to the user's rendering operation;

[0175] The receiving module 132 is configured to receive the final rendered image returned by the server according to the rendering request; the final rendered image is obtained by the server based on the original rendered image obtained by rendering the scene model data, the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer, the first encoding / decoding network being a deep learning network for denoising the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, the second encoding / decoding network being a deep learning network for denoising the stitched rendered image;

[0176] Display module 133 is used to display the final rendered image to the user.

[0177] Figure 13 The device shown can perform Figure 8 The method provided in the illustrated embodiment, for parts not described in detail in this embodiment, can be referred to the [examples / descriptions]. Figure 8 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 8 The descriptions in the illustrated embodiments will not be repeated here.

[0178] In one possible implementation, Figure 13 The structure of the device shown can be implemented as a terminal. For example... Figure 14 As shown, the terminal may include a processor 141 and a memory 142. The memory 142 stores data that supports the controller in performing the above-described actions. Figure 8 The program of the method provided in the illustrated embodiment is configured to execute a program stored in memory 142.

[0179] The program includes one or more computer instructions, wherein when executed by processor 141, the one or more computer instructions can perform the following steps:

[0180] In response to the user's rendering operation, a rendering request is sent to the server;

[0181] The server receives the final rendered image returned by the server according to the rendering request; the final rendered image is obtained by the server based on the original rendered image obtained by rendering the scene model data, the original rendered image includes a color layer and at least one auxiliary layer; the final rendered image is obtained by: inputting the auxiliary layer into a corresponding first encoding / decoding network for processing to obtain the processed auxiliary layer, the first encoding / decoding network being a deep learning network for denoising the corresponding auxiliary layer; performing a stitching process based on the color layer and the processed auxiliary layer to obtain a stitched rendered image; inputting the stitched rendered image into a second encoding / decoding network for processing to obtain the final rendered image, the second encoding / decoding network being a deep learning network for denoising the stitched rendered image;

[0182] The final rendered image is then displayed to the user.

[0183] Optionally, the processor 141 is also used to perform the aforementioned Figure 8 All or part of the steps in the illustrated embodiments.

[0184] The terminal may also include a communication interface 143 for communication between the terminal and other devices or communication networks.

[0185] In addition, embodiments of this application also provide an interior design system, including: Figure 12 The server described in the illustrated embodiment and Figure 14 The terminal described in the illustrated embodiment.

[0186] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, performs the following... Figure 2 The method described in the illustrated embodiment.

[0187] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, performs the following... Figure 7 The method of the illustrated embodiment.

[0188] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, performs the following... Figure 8 The method of the illustrated embodiment.

[0189] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0190] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. This application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0191] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0192] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0193] These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0194] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0195] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0196] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, linked lists, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for denoising rendered images, characterized in that, include: Obtain the original rendering image obtained by rendering using a ray tracing algorithm. The original rendering image includes a color layer and at least one auxiliary layer. The at least one auxiliary layer includes a diffuse layer and / or a normal layer. The auxiliary layer in the original rendering image obtained by ray tracing is input into the corresponding first encoding and decoding network for noise reduction processing to obtain the noise-reduced auxiliary layer. The first encoding and decoding network is a deep learning network used to denoise the corresponding auxiliary layer. Based on the color layer in the original rendering image obtained by ray tracing algorithm, and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing algorithm, a stitched rendering image is obtained. The stitched rendering image is input into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on the corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image.

2. The method according to claim 1, characterized in that, The method further includes: Based on the auxiliary layer after noise reduction, a corresponding edge information layer is generated, and the pixel values ​​in the edge information layer are used to represent the gradient values ​​of the corresponding pixels. The process of stitching together the color layer in the original rendering image obtained by ray tracing and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing to obtain a stitched rendering image includes: stitching together the color layer, the noise-reduced auxiliary layer and the edge information layer to obtain a stitched rendering image.

3. The method according to claim 1, characterized in that, The auxiliary layer is input into the corresponding first encoder-decoder network for noise reduction processing, including: The auxiliary layers whose quality is below the quality threshold are input into the corresponding first encoding / decoding network for noise reduction.

4. The method according to claim 1, characterized in that, The methods for training the first encoder-decoder network include: Construct the first encoder-decoder network, wherein training parameters are set in the first encoder-decoder network; The first sample auxiliary layer is input into the corresponding first encoding / decoding network for noise reduction processing to obtain the noise-reduced first sample auxiliary layer. The first sample auxiliary layer is the auxiliary layer in the first original rendering image. Based on the difference between the first sample auxiliary layer and the corresponding second sample auxiliary layer after noise reduction, the training parameters are iteratively adjusted until the difference meets the preset requirements. The second sample auxiliary layer is an auxiliary layer in the second original rendering image. The first original rendering image and the second original rendering image are rendering images obtained by rendering the same data to be rendered, and the quality of the second original rendering image is higher than that of the first original rendering image.

5. The method according to claim 1, characterized in that, The methods for training the second encoder-decoder network include: Construct the second encoder-decoder network, which is equipped with training parameters; The first sample rendering image is input into the second encoding and decoding network for processing to obtain the processed first sample rendering image. The first sample rendering image is obtained based on the first sample color layer and the first sample auxiliary layer. The first sample color layer is the color layer in the first original rendering image, and the first sample auxiliary layer is the auxiliary layer in the first original rendering image. Based on the difference between the processed first sample rendering image and the corresponding second sample rendering image, the training parameters are iteratively adjusted until the difference meets the preset requirements. The second sample rendering image is obtained by splicing the second sample color layer and the second sample auxiliary layer. The second sample color layer is the color layer in the rendered second original rendering image, and the second sample auxiliary layer is the auxiliary layer in the second original rendering image. The first original rendering image and the second original rendering image are rendering images obtained by rendering the same data to be rendered, and the quality of the second original rendering image is higher than that of the first original rendering image.

6. The method according to claim 5, characterized in that, The first sample rendering image is obtained by stitching together the first sample color layer and the first sample auxiliary layer. The methods for training the second encoder-decoder network also include: The third sample rendering image is input into the second encoding / decoding network for processing to obtain the processed third sample rendering image. The third sample rendering image is obtained by stitching together the first sample color layer and the processed first sample auxiliary layer. The processed first sample auxiliary layer is obtained by inputting the first sample auxiliary layer into the corresponding first encoding / decoding network for processing. Based on the difference between the processed third sample rendering image and the corresponding second sample rendering image, the training parameters are iteratively adjusted until the difference meets the preset requirements.

7. A model rendering method, characterized in that, include: Obtain the rendering request sent by the terminal; In response to the rendering request, a ray tracing algorithm is used to render the corresponding scene model data to obtain the original rendering image; A final rendering image is obtained based on the original rendering image obtained by rendering using a ray tracing algorithm. The original rendering image includes a color layer and at least one auxiliary layer, the at least one auxiliary layer including a diffuse layer and / or a normal layer. The final rendering image is obtained by inputting the auxiliary layer in the original rendering image obtained by rendering using a ray tracing algorithm into a corresponding first encoder-decoder network for noise reduction processing to obtain the noise-reduced auxiliary layer. The first encoder-decoder network is a deep learning network used for noise reduction of the corresponding auxiliary layer. Based on the color layer in the original rendering image obtained by ray tracing algorithm, and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing algorithm, a stitched rendering image is obtained. The stitched rendering image is input into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on the corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image. The final rendered image is sent to the terminal.

8. A model rendering method, characterized in that, include: In response to the user's rendering operation, a rendering request is sent to the server; Receive the final rendered image returned by the server based on the rendering request; The final rendered image is obtained by the server based on the original rendered image obtained by rendering scene model data using a ray tracing algorithm. The original rendered image includes a color layer and at least one auxiliary layer. The at least one auxiliary layer includes a diffuse layer and / or a normal layer. The final rendered image is obtained by inputting the auxiliary layer in the original rendered image obtained by the ray tracing algorithm into a corresponding first encoder-decoder network for noise reduction processing to obtain the noise-reduced auxiliary layer. The first encoder-decoder network is a deep learning network used to perform noise reduction on the corresponding auxiliary layer. Based on the color layer in the original rendering image obtained by ray tracing algorithm, and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing algorithm, a stitched rendering image is obtained. The stitched rendering image is input into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on the corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image. The final rendered image is then displayed to the user.

9. A noise reduction device for rendered images, characterized in that, include: The acquisition module is used to acquire the original rendering image obtained by rendering using a ray tracing algorithm. The original rendering image includes a color layer and at least one auxiliary layer. The at least one auxiliary layer includes a diffuse layer and / or a normal layer. The first noise reduction module is used to input the auxiliary layer in the original rendering image obtained by rendering using the ray tracing algorithm into the corresponding first encoding and decoding network for noise reduction processing, so as to obtain the noise-reduced auxiliary layer. The first encoding and decoding network is a deep learning network used to reduce noise in the corresponding auxiliary layer. The stitching module is used to stitch together the color layer in the original rendering image obtained by rendering using the ray tracing algorithm and the noise-reduced auxiliary layer obtained by noise reduction processing of the auxiliary layer in the original rendering image obtained by rendering using the ray tracing algorithm, to obtain a stitched rendering image. The second noise reduction module is used to input the stitched rendering image into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on a corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image.

10. A model rendering apparatus, characterized in that, include: The acquisition module is used to acquire rendering requests sent by the terminal; The rendering module is used to respond to the rendering request by using a ray tracing algorithm to render the corresponding scene model data to obtain the original rendering image. A noise reduction module is used to obtain a final rendering image based on the original rendering image obtained by rendering using a ray tracing algorithm. The original rendering image includes a color layer and at least one auxiliary layer, the at least one auxiliary layer including a diffuse layer and / or a normal layer. The final rendering image is obtained by inputting the auxiliary layer in the original rendering image obtained by rendering using a ray tracing algorithm into a corresponding first encoder-decoder network for noise reduction processing to obtain the noise-reduced auxiliary layer. The first encoder-decoder network is a deep learning network used for noise reduction of the corresponding auxiliary layer. Based on the color layer in the original rendering image obtained by ray tracing algorithm, and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing algorithm, a stitched rendering image is obtained. The stitched rendering image is input into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on the corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image. A sending module is used to send the final rendered image to the terminal.

11. A model rendering apparatus, characterized in that, include: The sending module is used to send rendering requests to the server in response to user rendering operations; A receiving module is used to receive the final rendered image returned by the server according to the rendering request; The final rendered image is obtained by the server based on the original rendered image obtained by rendering scene model data using a ray tracing algorithm. The original rendered image includes a color layer and at least one auxiliary layer. The at least one auxiliary layer includes a diffuse layer and / or a normal layer. The final rendered image is obtained by inputting the auxiliary layer in the original rendered image obtained by the ray tracing algorithm into a corresponding first encoder-decoder network for noise reduction processing to obtain the noise-reduced auxiliary layer. The first encoder-decoder network is a deep learning network used to perform noise reduction on the corresponding auxiliary layer. Based on the color layer in the original rendering image obtained by ray tracing algorithm, and the noise-reduced auxiliary layer obtained by denoising the auxiliary layer in the original rendering image obtained by ray tracing algorithm, a stitched rendering image is obtained. The stitched rendering image is input into the second encoding and decoding network for noise reduction processing to obtain the final rendering image. The second encoding and decoding network learns the ability to reduce noise in the rendering image based on the corresponding sample set including multiple sample pairs. Each sample pair includes a first sample rendering image and a second sample rendering image. The display module is used to display the final rendered image to the user.

12. A computing device, characterized in that, include: A memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of claims 1 to 6.

13. A server, characterized in that, include: A memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in claim 7.

14. A terminal, characterized in that, include: A memory and a processor; wherein the memory stores one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in claim 8.

15. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed, implements the method as described in any one of claims 1 to 8.

Images