Three-dimensional 3D rendering method and apparatus

Abstract

A three-dimensional (3D) rendering method includes extracting a sampling point from a 3D scene, obtaining rendering result information of the sampling point by rendering the sampling point, and generating a rendering result image corresponding to the entire rendering based on the rendering result information of the sampling point and feature information of the 3D scene.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of Korean Patent Application No. 10-2019-0111734, filed on September 9, 2019, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0003] The following description relates to three-dimensional (3D) computer graphics technology. Background Technology

[0004] 3D rendering falls under the field of computer graphics, specifically the rendering of 3D models. 3D rendering is used in a variety of applications, such as 3D games, virtual reality, animation, and film. Examples of 3D rendering techniques include: ray tracing rendering, which tracks the path of light emitted from a light source as it reflects off the surface of an object; radiosity rendering, which reflects both direct illumination effects and indirect illumination effects caused by diffuse reflection or light corresponding to direct illumination reflected from an object; and rasterization rendering, which converts vector data into pixel-patterned images and renders 3D models. Summary of the Invention

[0005] This summary is provided to describe in simplified form the selection of concepts further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help define the scope of the claimed subject matter.

[0006] In one general aspect, a three-dimensional (3D) rendering method includes: extracting sampling points from a 3D scene; obtaining rendering result information of the sampling points by rendering the sampling points; and generating a rendering result image corresponding to the entire rendering based on the rendering result information of the sampling points and feature information of the 3D scene.

[0007] Extracting sampling points may include using a neural network-based sampling point determination model, which takes feature information of the 3D scene as input and outputs the location information of the sampling points from the 3D scene.

[0008] Extracting sampling points can include extracting the minimum number of sampling points required to generate a rendered image in a 3D scene.

[0009] Generating a rendered image may include: using a neural network-based rendered image generation model to obtain the rendered image, which uses the rendered image information of the sampling points and the feature information of the 3D scene as input, and outputs the rendered image.

[0010] The generated rendering result image can include: rendering result information based on sampling points and feature information of the 3D scene, and rendering result information of points in the 3D scene other than sampling points.

[0011] Estimating rendering result information for points other than the sampling points may include: estimating the first vignetting component of points other than the sampling points based on the rendering result information of the sampling points, estimating the second vignetting component of points other than the sampling points, and estimating rendering result information for points other than the sampling points based on the first vignetting component and the second vignetting component.

[0012] Estimating the second vignetting component may include estimating the second vignetting component through a graphics processing unit (GPU) pipeline.

[0013] The first vignetting component can be the diffuse reflection component, while the second vignetting component can be the specular reflection component.

[0014] Obtaining rendering result information can include rendering sampling points.

[0015] The feature information of a 3D scene may include any one or any combination of the following: depth information of the 3D model in the 3D scene, surface normal information of the 3D model, and texture information of the 3D model.

[0016] The sampling point can correspond to a portion of the vertices of a 3D model included in a 3D scene.

[0017] 3D scenes can be defined using texture atlas data. Obtaining rendering result information can include performing rendering based on texture atlas data.

[0018] Obtaining rendering result information may include obtaining rendering result information based on each of a portion or all of the rendering sample points of the virtual light source used for rendering.

[0019] Obtaining rendering result information may include obtaining rendering result information based on each of the rendering sample points of a portion or all of the rays used for rendering.

[0020] In another general aspect, a training method includes: acquiring a first rendered image by performing an entire rendering of a 3D scene; determining sampling points in the 3D scene using a sampling point determination model based on a first neural network, the sampling point determination model using feature information of the 3D scene as input; acquiring rendering result information of the determined sampling points by rendering the determined sampling points; acquiring a second rendered image corresponding to the entire rendering using a rendered image generation model based on a second neural network, the rendered image generation model using the rendered result information of the sampling points and feature information of the 3D scene as input; and adjusting the parameters of one or both of the sampling point determination model and the rendered image generation model based on the first and second rendered images.

[0021] Adjusting parameters may include: adjusting one or both of the parameters of the sampling point determination model and the rendering image generation model to reduce loss, based on whether the second rendering result image obtained from the rendering result information of the sampling points meets the preset conditions, the number of sampling points, and the separation between sampling points in the 3D scene.

[0022] In another general aspect, a 3D rendering apparatus includes at least one processor, wherein the at least one processor controls the 3D rendering apparatus to perform the following operations: extracting sampling points from a 3D scene to be rendered; obtaining rendering result information of the sampling points by rendering the sampling points; and generating a rendering result image corresponding to the entire rendering based on the rendering result information of the sampling points and feature information of the 3D scene.

[0023] In another general aspect, the 3D rendering apparatus includes: a first processor configured to extract sampling points from a 3D scene to be rendered using a neural network; and a second processor configured to generate a rendered image corresponding to the entire rendering based on rendering result information from the sampling points and feature information of the 3D scene.

[0024] The second processor can be configured to render sample points via a GPU pipeline, and the first processor can be configured to estimate rendering result information for points other than the sample points based on the rendering result information of the sample points.

[0025] The second processor can be configured to render sample points via a GPU pipeline, render points other than the sample points based on the rendering result information of the sample points, and generate a rendered image based on the results obtained by rendering the sample points and the points other than the sample points.

[0026] The first processor can be configured to: acquire rendering result information of the sampled points by rendering the sampled points, and estimate the rendering result information of points other than the sampled points based on the rendering result information of the sampled points. The second processor can be configured to generate a rendered image based on the rendering result information of the sampled points and the rendering result information of points other than the sampled points.

[0027] The first processor can be configured to extract sampling points based on virtual light source information and object information related to objects in the 3D scene, and use a neural network to estimate the rendering result information of the sampling points. The second processor can be configured to estimate the rendering result information of points other than the sampling points based on the rendering result information of the sampling points.

[0028] In another general aspect, a processor-implemented three-dimensional (3D) rendering method includes: extracting a minimum number of sampled pixels from a 3D scene for generating a result image; rendering the sampled pixels to generate rendering result information; inputting feature information of the 3D scene and rendering result information into a neural network-based rendering image generation model; and displaying the result image as the output of the rendering image generation model corresponding to the entire rendering.

[0029] The rendered image generation model can be configured to estimate the color values ​​of pixels other than the sampled pixels based on the rendered result information and feature information, and to determine the pixel values ​​of pixels included in the resulting image.

[0030] Other features and aspects will become clear from the following detailed description, drawings and claims. Attached Figure Description

[0031] Figure 1 An overview of a three-dimensional (3D) rendering apparatus is shown.

[0032] Figure 2 An overview of 3D rendering processing is shown.

[0033] Figure 3 and Figure 4 An example of training processing for a deep learning model is shown.

[0034] Figure 5 This is a flowchart illustrating an example of the operation of the training method.

[0035] Figure 6 This is a block diagram illustrating an example configuration of the training device.

[0036] Figure 7 This is a flowchart illustrating an example of how a 3D rendering method operates.

[0037] Figure 8 An example of 3D rendering processing is shown.

[0038] Figure 9A An example of 3D rendering processing is shown.

[0039] Figure 9B Another example of 3D rendering processing is shown.

[0040] Figure 10 This is a block diagram illustrating an example configuration of a 3D rendering apparatus.

[0041] Throughout the accompanying drawings and detailed description, unless otherwise described or provided, the same reference numerals shall be understood to refer to the same elements, features, and structures. The drawings may be drawn not to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be enlarged. Detailed Implementation

[0042] The following detailed description is provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding the disclosure of this application. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be significantly altered upon understanding the disclosure of this application, except for operations that must be performed in a certain order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0043] The features described herein may be implemented in various forms and are not to be construed as limited to the examples described herein. Rather, the examples described herein are provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein, which will become apparent upon understanding the disclosure of this application.

[0044] The following structural or functional descriptions of the examples disclosed in this disclosure are for illustrative purposes only, and the examples may be implemented in various forms. The examples are not intended to be limiting, but rather to cover various modifications, equivalents, and alternatives within the scope of the claims.

[0045] Although the terms “first” or “second” are used to describe various components, the components are not limited to these terms. These terms should only be used to distinguish one component from another. For example, within the scope of the conception of this disclosure, a “first” component may be referred to as a “second” component, or similarly, a “second” component may be referred to as a “first” component. It will be understood that when referring to a component being “connected” to another component, the component may be directly connected to or coupled to the other component, or there may be an intermediate component.

[0046] Unless the context clearly indicates otherwise, the singular form used herein is intended to include the plural form as well. It should also be understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated feature, integer, step, operation, element, component, or combination thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0047] Unless otherwise defined herein, the terms used herein (including technical or scientific terms) have the same meaning as commonly understood. Unless otherwise defined herein, terms as defined in commonly used dictionaries should be interpreted as having a meaning that matches the contextual meaning in the relevant field and should not be interpreted as having an ideal or overly formal meaning.

[0048] In the following description, examples will be illustrated with reference to the accompanying drawings, and the same reference numerals in the drawings always denote the same elements.

[0049] Figure 1 An overview of the three-dimensional (3D) rendering apparatus 100 is shown.

[0050] refer to Figure 1 The 3D rendering apparatus 100 is an apparatus for performing rendering on a 3D scene and can be implemented as, for example, a software module, a hardware module, or a combination thereof. The 3D rendering apparatus 100 generates a rendered image by rendering the 3D scene. For example, the 3D rendering apparatus 100 may include a graphics processing unit (GPU) and can use the GPU to generate a two-dimensional (2D) image from 3D directing stored in a file of the 3D scene. The 3D rendering apparatus 100 can be used in applications such as real-world rendering or augmented reality (AR) rendering.

[0051] A 3D scene is a space that describes the spatial relationships between objects. A 3D scene is a model of the physical world that can be mathematically represented using geometry, which describes all points in 3D space using coordinates. A 3D scene includes at least two objects. Objects can be modeled objects or 3D models of 3D models; however, the examples are not limited to these. A 3D scene simulates the realistic interactions between objects (e.g., rules of force control and physical laws).

[0052] Rendering is the process or technique of creating images from a 3D scene containing objects using computer programs. Rendering a 3D scene involves applying visual effects (e.g., lighting effects or shading effects) to the 3D scene in 3D computer graphics, based on light output from at least one virtual light source acting as a virtual light source. By applying lighting effects to the 3D scene, the colors of the 3D models included in the 3D scene can be determined. By applying shading effects to the 3D scene, transparency effects caused by occlusion, reflected light, or shading can be determined. These effects are based on the characteristics of the light emitted from the virtual light source (e.g., direction or color), the characteristics of the 3D model (e.g., color, surface normals, or texture), and the arrangement relationship between the visible light source and the 3D model.

[0053] Virtual light sources include, for example, direct virtual light sources and indirect virtual light sources (virtual point light sources (VPL)). A direct virtual light source is a light source that emits light directly into a 3D model, while an indirect virtual light source is a virtual light source that emits light in a region where light emitted from a direct virtual light source or other indirect virtual light sources is reflected, diffracted, or refracted.

[0054] The rendered result image generated by the rendering process of the 3D rendering device 100 is the result of rendering a 3D scene and includes, for example, digital images or raster graphics images.

[0055] The main elements of rendering are reflectivity and vignetting. The final rendering result can be determined based on reflectivity and vignetting. Reflectivity is given in the information about the 3D model and includes its color and texture information. Vignetting is complex to calculate because it is based on the texture of the 3D model and the arrangement of virtual light sources with the 3D model. In vignetting processing, all points in the 3D scene do not need to have the same density of vignetting. For example, for diffuse materials, the desired vignetting effect can be obtained even with coarse sampling points; however, for smooth materials or shadowed environments, denser sampling points are required.

[0056] Although the textures, virtual light source positions, and geometric properties of 3D models require sampling points of varying densities, predicting these density points is not easy due to their complex correlations. As illustrated in the example described below, a neural network-based deep learning model is trained on the results obtained through simulations of various rendering conditions, and the trained deep learning model is used to perform 3D rendering during actual 3D rendering. Therefore, sampling points in a 3D scene can be efficiently sampled without degrading the quality of the rendered image, and 3D rendering can be performed faster with lower complexity. For example, in the deep learning model training process, interpolation methods are trained to recover all rendered images and multiple sampling points to be sampled in the 3D scene; during actual 3D rendering, rendering of the main sampling points is performed, and all rendered results are recovered based on the rendered results. In this paper, the terms "point" and "sampling point" can be replaced by "pixel" and "sampled pixel."

[0057] Figure 2 An overview of 3D rendering processing is shown.

[0058] refer to Figure 2 In the 3D rendering process performed by the 3D rendering device, the following operations are performed: sampling points are sampled based on the training used for rendering, and the entire rendered image is reconstructed based on the rendering results of the sampled points. In the example, a portion of the sampled points, which are considered to be the main points in the 3D scene, can be rendered, and a rendered image corresponding to the entire rendering can be generated based on the rendering results of the sampled points.

[0059] In the aforementioned 3D rendering process, 3D scene information 210 is provided as information related to the 3D scene. 3D scene information 210 is information related to a pre-provided 3D scene to be rendered. 3D scene information 210 may include, for example, information related to the arrangement, viewpoint, texture mapping, lighting, or vignetting of the 3D model used to represent the virtual 3D scene.

[0060] 3D scene information 210 is input to a sampling point determination model 220 based on a first neural network, and the sampling point determination model 220 extracts or samples sampling points in the 3D scene based on the input 3D scene information 210. The sampling point determination model 220 may have a neural network structure, such as an autoencoder. The sampling points may correspond to predetermined positions in the 3D scene, or correspond to a portion of vertices of a 3D model included in the 3D scene.

[0061] In operation 230, the extracted sampling points are rendered. As a result of the rendering, the rendering result information of the sampling points is obtained. In the rendering process, shadow effects and vignetting effects are calculated based on the virtual light source set in the 3D scene information 210, and the color values ​​and / or shadow values ​​at the sampling point locations are determined as the rendering result information.

[0062] In the example, during the rendering of sampled points, vignetting can be performed based on sampled points corresponding to a subset of points and all given virtual light sources or all rays, and the color value of the sampled points can be determined. In another example, during the rendering of sampled points, vignetting can be performed based on sampled points and a subset of given virtual light sources or rays, and the color value of the sampled points can be determined.

[0063] When rendering sample points, feature information extracted from 3D scene information 210 and rendering result information for the sample points are input into a rendering image generation model 240 based on a second neural network. The rendering image generation model 240 may have a neural network structure, such as an autoencoder. The feature information extracted from 3D scene information 210 includes, for example, depth information, surface normal information, and texture information (or material information) associated with the 3D model. Depth information is information relating to the depth or distance to each point in the 3D model, and surface normal information is information relating to the normal direction components on the surface of the 3D model.

[0064] The rendering image generation model 240 determines the color values ​​of points other than the sampling points based on the input information and generates a rendered result image 250 corresponding to the entire rendering. In this example, the vignetting values ​​of all target points to be rendered are restored.

[0065] As described above, the 3D rendering device renders sampled points corresponding to a portion of the points in the 3D scene, rather than rendering all points in each image frame, and estimates or reconstructs the rendering results of other points based on the rendering results of the sampled points.

[0066] The sampling point determination model 220 and the rendering image generation model 240 are trained respectively through the training process described below, based on the scheme of extracting sampling points from the provided 3D scene and the scheme of generating a rendering result image corresponding to the entire rendering based on the rendering result of the pre-extracted sampling points.

[0067] Figure 3 An overview of the training process is shown.

[0068] refer to Figure 3The system provides a 3D scene 310 and its feature information as training data. The feature information of the 3D scene 310 includes, for example, information related to the color, depth, surface normals, texture, and smoothness of the 3D models included in the 3D scene 310, categories for distinguishing the types of 3D models, and / or material identifiers.

[0069] In operation 320, the training device renders a 3D scene 310 under various rendering conditions. These various rendering conditions are those that modify variable elements (e.g., lighting in the 3D scene 310, and the texture and geometry of the 3D models in the 3D scene 310). For example, the training device can obtain first rendered result images under various rendering conditions through realistic rendering simulations. For each of these first rendered result images, training processes, including operations 330 and 340, can be performed.

[0070] In operation 330, the training device performs training on the sampling scheme for the sampling points. The training device trains a sampling point determination model based on a first neural network to determine how to determine the number and location of sampling points extracted from the 3D scene 310. Through training processing, training is performed on the sampling scheme to generate a rendered result image corresponding to the entire rendering or to reconstruct the result of the entire rendering.

[0071] In operation 340, the training device performs training on a scheme to estimate the rendering results for points other than the sampled points, based on the rendering results of the sampled points. The training device trains a rendering image generation model based on a second neural network to generate a second rendered image corresponding to the entire rendering, based on the rendering result information of the sampled points and the feature information of the 3D scene 310. Through training processing, training is performed on the correlation between representative sampled points and other points, as well as the scheme for recovering the color values ​​of other points based on the sampled points.

[0072] The training process described above, including operations 330 and 340, is performed based on a comparison between a first rendered image corresponding to the rendering results of all points acquired in operation 320 and a second rendered image generated based on the rendering results of a subset of sampled points acquired in operation 340. The training process includes adjusting the parameters of the sampled point determination model and the rendered image generation model to reduce the difference between the second rendered image and the first rendered image, which will refer to... Figure 4 To be further described.

[0073] Figure 4 An example of training processing is shown.

[0074] refer to Figure 4It provides 3D scene information 410 about the 3D scene to be learned. 3D scene information 410 includes information related to the characteristics of the 3D models included in the 3D scene. 3D scene information 410 includes, for example, information related to the color, depth, surface normals, texture, and smoothness of the 3D models, categories used to distinguish the type of 3D models, or material identifiers.

[0075] The training device performs the entire rendering 420 based on 3D scene information 410. The training device obtains various first rendered result images by performing the entire rendering in each of various rendering conditions. For example, the training device obtains a first rendered result image by rendering all points (or pixels) of the 3D scene using a ray tracing scheme.

[0076] 3D scene information 410 is input to a sampling point determination model 430, which outputs information about sampling points to be sampled in the 3D scene, in the form of coordinates or a confidence map, based on calculations of neural network parameters. The confidence map provides confidence values ​​for the locations of points to be extracted as sampling points among all points in 3D space. Locations in the confidence map corresponding to the sampling point locations indicate confidence levels greater than or equal to a threshold. Based on the output of the sampling point determination model 430, sampling points are determined, representing a subset of points to be actually rendered among all points in 3D space.

[0077] In operation 440, the training device renders sample points and obtains the rendering result information. For example, the training device calculates a vignetting result for each sample point based on a ray tracing scheme and stores the calculated vignetting result as the result obtained by performing rendering. In this example, the color value of each sample point is calculated as the vignetting result.

[0078] The rendering result information of the sampled points, along with the feature information of the 3D scene information 410, is transmitted to the rendering image generation model 450. The feature information may include, for example, depth information, surface normal information, and texture information about the 3D model. The rendering image generation model 450 generates a second rendering result image 460 corresponding to the entire rendering based on the input information. The rendering image generation model 450 estimates the vignetting results for points other than the sampled points based on the input information, and generates the second rendering result image 460 based on the estimated vignetting results.

[0079] The trainer 470, included in the training device, trains the sampling point determination model 430 and the rendered image generation model 450 based on a first rendered image obtained through the entire rendering 420 and a second rendered image 460 obtained using the sampling point determination model 430 and the rendered image generation model 450. The first rendered image corresponds to the desired result used in the training process. The trainer 470 adjusts the parameters of the sampling point determination model 430 and the rendered image generation model 450 based on the difference between the first rendered image and the second rendered image 460.

[0080] In the example, the trainer 470 calculates the loss based on whether the second rendered image 460 can be recovered from the sampling points determined by the sampling point determination model 430, the concentration and separation between sampling points, and the number of points in the confidence graph that are greater than or equal to a threshold, and adjusts the parameters of the sampling point determination model 430 and the rendered image generation model 450 to reduce the loss.

[0081] Typically, as the number of sampling points extracted from the 3D scene increases, the number of errors occurring in the reconstruction of the second rendered image 460 decreases. Conversely, as the number of sampling points decreases, the number of errors occurring in the reconstruction of the second rendered image 460 increases. In other words, there is a trade-off between the number of sampling points and the number of errors. The training device trains a sampling point determination model 430 based on this trade-off to determine the optimal number and location of sampling points to generate the second rendered image 460, and trains a rendered image generation model 450 based on the rendering result information of the sampling points to generate a high-quality rendered image corresponding to the entire rendering.

[0082] Figure 5 This is a flowchart illustrating an example of the operation of the training method. The training method is performed by, for example, the training device described above.

[0083] refer to Figure 5 In operation 510, the training device obtains a first rendered result image by performing the entire rendering of the provided 3D scene. The training device renders the 3D scene under various rendering conditions and obtains the first rendered result image under various rendering conditions. For each of the first rendered result images, the training device repeatedly performs a series of processes from operation 520 to 550.

[0084] In operation 520, the training device determines the sampling points to be rendered in the 3D scene. The training device uses a sampling point determination model based on a first neural network to determine the sampling points in the 3D scene, which uses information related to the 3D scene as input. This information may include, for example, the color, depth, surface normals, texture, and smoothness of the 3D model, categories used to distinguish the type of the 3D model, and / or material identifiers. The sampling point determination model may sample the sampling points to be actually rendered in the 3D scene based on the information related to the 3D scene.

[0085] In operation 530, the training device obtains the rendering result information of the sample points by rendering the sample points. The training device calculates the vignetting result for each sample point and obtains the calculated vignetting result as the rendering result information.

[0086] In operation 540, the training device acquires a second rendered image corresponding to the entire rendering based on the rendering result information of the sampling points. The training device uses a rendering image generation model based on a second neural network to acquire the second rendered image. This rendering image generation model uses the rendering result information of the sampling points and the feature information of the 3D scene as input. The rendering image generation model estimates the vignetting results of points other than the sampling points based on the rendering result information of the sampling points and the feature information of the 3D scene, and generates the second rendered image based on the estimation results.

[0087] In operation 550, the training device adjusts the parameters of any one or any combination of the sampling point determination model and the rendered image generation model based on the first rendered result image and the second rendered result image. For example, the training device adjusts the parameters of any one or any combination of the sampling point determination model and the rendered image generation model to reduce loss based on whether the second rendered result image obtained according to the rendering result information of the sampling points meets preset conditions, the number of determined sampling points, and the concentration or separation between the determined sampling points in the 3D scene. In this example, whether the second rendered result image meets preset conditions may include, for example, whether the image quality of the second rendered result image meets predetermined conditions, or whether the entire rendered result image can be recovered based on the sampling points.

[0088] Figure 6 This is a block diagram illustrating an example configuration of the training device 600.

[0089] refer to Figure 6 The training device 600 is an apparatus configured to train a deep learning model that can be used in 3D rendering. The deep learning model includes, for example, a sampling point determination model based on a first neural network and a rendered image generation model based on a second neural network. The training device 600 corresponds to the aforementioned training apparatus.

[0090] The training device 600 includes at least one processor 610 and a memory 620. The memory 620 is connected to the processor 610 and stores instructions executable by the processor 610, data computed by the processor 610, and / or data processed by the processor 610. Depending on the example, the training device 600 also includes a storage device 630, which stores data related to the 3D scene to be learned, data about rendering results, and data about the deep learning model.

[0091] The processor 610 controls the training device 600 to perform the above-described training process. For example, the processor 610 controls the training device 600 to perform the following operations: obtain a first rendered image by performing the entire rendering of a 3D scene; determine sampling points in the 3D scene using a sampling point determination model based on a first neural network, which uses feature information of the 3D scene as input; obtain rendering result information of the sampling points by rendering the extracted sampling points; obtain a second rendered image corresponding to the entire rendering using a rendering image generation model based on a second neural network, which uses the rendering result information of the sampling points and feature information of the 3D scene as input; and adjust the parameters of any one or any combination of the sampling point determination model and the rendering image generation model based on the first and second rendered images. The processor 610 calculates the loss caused by the difference between the first and second rendered images and adjusts the parameters of the sampling point determination model and the rendering image generation model to reduce the calculated loss. The processor 610 gradually adjusts the parameters of the sampling point determination model and the rendered image generation model to the desired values ​​by repeatedly performing the above training process on each first rendered result image.

[0092] Figure 7 This is a flowchart illustrating an example of the operation of a 3D rendering method. The 3D rendering method is performed, for example, by the 3D rendering apparatus described above.

[0093] refer to Figure 7 In operation 710, the 3D rendering apparatus extracts sampling points from the 3D scene to be rendered. In this example, the extracted sampling points may correspond to predetermined locations in the 3D scene. In another example, the extracted sampling points may correspond to a portion of vertices of a 3D model included in the 3D scene.

[0094] For example, a 3D rendering apparatus uses a sampling point determination model based on a first neural network to extract sampling points. This model takes feature information of the 3D scene as input and outputs the location information of the sampling points that can be extracted from the 3D scene. The 3D rendering apparatus uses the sampling point determination model to extract the minimum number of sampling points allowed to generate a rendered image in the 3D scene. The feature information of the 3D scene may include, for example, depth information, surface normal information, and texture information of the 3D models included in the 3D scene.

[0095] In operation 720, the 3D rendering apparatus obtains rendering result information for the sampled points by rendering the sampled points extracted in operation 710. For example, the 3D rendering apparatus renders sampled points extracted from points in the 3D scene in operation 710 and obtains rendering result information including vignetting result information for the sampled points. The rendering result information for the sampled points may include color information corresponding to each sampled point and obtained by rendering each sampled point.

[0096] In one example, the 3D rendering apparatus can obtain rendering result information for each sample point by rendering each sample point based on a portion or all of the virtual light source used for rendering. In another example, when the 3D rendering apparatus uses a ray tracing scheme to perform rendering, it can obtain rendering result information by rendering each sample point based on a portion or all of the light source used for rendering.

[0097] In another example, rendering processing is applied to pixel-based or vertex-based rendering, and also to texture atlas-based rendering. The 3D scene is determined using texture atlas data, and the 3D rendering apparatus obtains rendering result information by performing rendering based on the texture atlas data. Texture atlas data is a data format that unfolds a 3D scene and stores it as a 2D texture regardless of the viewpoint. For example, texture atlas data is obtained by representing multiple images that form a 3D scene as a single 2D texture image. Using texture atlas data can reduce resource consumption during rendering processing and improve rendering speed. Furthermore, examples of using texture atlas data have also been applied to lightmaps, and the scope of lightmap usage can be expanded through rendering methods.

[0098] In operation 730, the 3D rendering apparatus generates a rendered image corresponding to the entire rendering process based on the rendering result information of the sampled points and the feature information of the 3D scene. For example, the 3D rendering apparatus uses a rendered image generation model based on a second neural network to obtain the rendered image, which uses the rendering result information of the sampled points and the feature information of the 3D scene as input and outputs the rendered image.

[0099] The 3D rendering device uses a rendering image generation model to estimate the rendering results of points in the 3D scene other than the sampling points, based on the rendering result information of the sampling points and the feature information of the 3D scene.

[0100] In the example, the 3D rendering apparatus uses neural network-based deep learning to determine intermediate rendering results for other points, rather than determining final rendering result information (e.g., color values), and uses another scheme to determine the final rendering result information based on the intermediate rendering results. For example, the 3D rendering apparatus divides the process of acquiring vignetting components for other points, acquiring a first vignetting component using deep learning reconstruction and acquiring a second vignetting component using the GPU pipeline. The first vignetting component can be, for example, a diffuse component, while the second vignetting component can be, for example, a specular component. The 3D rendering apparatus estimates the first vignetting component for points other than the sampled points based on the rendering result information of the sampled points, estimates the second vignetting component for other points, and estimates the rendering result information for other points based on the first and second vignetting components. The first vignetting component corresponds to the intermediate rendering result, and the 3D rendering apparatus uses a trained neural network to estimate the first vignetting component for other points. The 3D rendering apparatus uses the GPU pipeline to estimate the second vignetting component for other points and estimates the rendering result information for other points by combining the first and second vignetting components.

[0101] The 3D rendering device extracts a small number of sampling points from the 3D scene through the above processing, and restores the rendering result image corresponding to the entire rendering based on the rendering result of the extracted sampling points, thereby performing 3D rendering with low complexity and extremely fast speed.

[0102] Figure 8 An example of 3D rendering processing is shown.

[0103] refer to Figure 8 In a broad sense, 3D rendering processing includes: processing 810 for extracting sampling points from a 3D scene 820, processing 850 for rendering the extracted sampling points, and processing 860 for generating a rendered image based on the rendering result information of the sampling points.

[0104] In operation 830 of processing 810, the 3D rendering apparatus extracts sampling points from the 3D scene 820 to be rendered. Furthermore, the 3D rendering apparatus extracts feature information 840 from the 3D scene 820, such as depth information, surface normal information, and texture information of the 3D model.

[0105] The 3D rendering apparatus extracts the minimum number of sampling points required to reconstruct the entire rendered image, based on depth, surface normal, and texture information of the 3D model included in the 3D scene 820. The 3D rendering apparatus uses a trained sampling point determination model to extract points from points in the 3D scene 820 that will be identified as sampling points. For example, the sampling point determination model is trained such that a relatively large number of sampling points are extracted in regions where surface normals change rapidly, and a relatively small number of sampling points are extracted in regions where surface normals change slightly. The 3D rendering apparatus uses the sampling point determination model to adaptively extract sampling points based on the 3D scene 820.

[0106] In Process 850, the 3D rendering device renders the extracted sample points. The color values ​​of these sample points are then determined through rendering.

[0107] In operation 880 of processing 860, the 3D rendering apparatus reconstructs a rendering result image corresponding to the entire rendering based on rendering result information 870 derived from the rendering through sampling points and feature information 840 derived from the 3D scene 820. The 3D rendering apparatus uses a trained rendering image generation model to obtain the rendering result image from the rendering result information 870 and feature information 840. The rendering image generation model estimates the color values ​​of points other than the sampling points based on the rendering result information 870 and feature information 840, and determines the pixel values ​​of pixels included in the rendering result image. As described above, the 3D rendering apparatus uses a portion of the rendering result to reconstruct other rendering results.

[0108] Figure 9A An example of 3D rendering processing is shown.

[0109] refer to Figure 9A 3D scene 905 is a space that describes the spatial relationships between objects (e.g., 3D model 910). The 3D rendering apparatus extracts a portion of points 915 from the 3D scene 905 as sampling points 920 for performing rendering. The 3D rendering apparatus uses a trained sampling point determination model to determine the positions of the sampling points 920 within the 3D scene 905. The sampling point determination model determines the positions of the minimum number of sampling points used to generate a rendered image in the provided 3D scene and provides information related to the determined positions of the sampling points.

[0110] The 3D rendering apparatus renders sample point 920 based on virtual light source 925 or light rays based on a ray tracing scheme. To render sample point 920, the 3D rendering apparatus uses part or all of the virtual light source 925, or part or all of the light rays. In processing the rendering of sample point 920, the 3D rendering apparatus does not calculate or reflect vignetting effects caused by a specific virtual light source 930 in the rendering result. By considering a part of the virtual light source or a part of the light rays, computational complexity can be reduced and 3D rendering can be performed faster.

[0111] The 3D rendering apparatus generates a rendering result image 935 corresponding to the entire rendering based on the rendering result information of sampling point 920 and the feature information of 3D scene 905. The 3D rendering apparatus uses a rendering image generation model to obtain the rendering result image 935, which uses the rendering result information of sampling point 920 and the feature information of 3D scene 905 as input to provide the rendering result image.

[0112] Figure 9B Another example of 3D rendering processing is shown.

[0113] exist Figure 9B In the example, sampling points are extracted from the vertices of the 3D model. For example, the sampling points may correspond to a subset of vertices of the 3D model included in the 3D scene.

[0114] In the provided 3D scene 950, the 3D model 955 includes multiple vertices 960. The 3D rendering apparatus extracts the vertices 965 to be rendered from the vertices 960 of the 3D model 955. The 3D rendering apparatus uses sampling points determined by taking the feature information of the 3D scene 950 as input to extract the vertices 965. Vertices 970 other than the extracted vertices 965 are not rendered.

[0115] The 3D rendering apparatus renders the extracted vertex 965 based on a virtual light source 975 or rays based on a ray tracing scheme. To render vertex 965, the 3D rendering apparatus uses part or all of the virtual light source 975, or part or all of the rays. For example, the 3D rendering apparatus does not calculate the vignetting effect caused by a predetermined virtual light source 980, or does not reflect the rendering results in the rendering of vertex 965.

[0116] Similar to Figure 9A For example, the 3D rendering apparatus generates a rendered image 985 corresponding to the entire rendering based on the rendering result information of the rendered vertex 965 and the feature information of the 3D scene 950. The 3D rendering apparatus uses a trained rendered image generation model to obtain the rendered image 985.

[0117] Figure 10This is a block diagram illustrating an example configuration of the 3D rendering device 1000.

[0118] refer to Figure 10 The 3D rendering apparatus 1000 includes at least one processor 1010 and a memory 1020. The 3D rendering apparatus 1000 corresponds to, for example, the 3D rendering apparatus described above.

[0119] The memory 1020 stores computer-readable instructions, information used in the 3D rendering process described above, and result information. For example, the memory 1020 includes non-transitory computer-readable media, such as high-speed random access memory (RAM) and / or non-volatile computer-readable storage media (e.g., at least one disk storage device, flash memory device, or other non-volatile solid-state storage device).

[0120] The processor 1010 controls the overall operation of the 3D rendering apparatus 1000 and processes at least one operation of the aforementioned 3D rendering method. Those skilled in the art will understand that the processor 1010 can be implemented as an array of multiple logic gates or another type of hardware. Furthermore, the processor 1010 includes at least one GPU to perform the rendering of the 3D scene.

[0121] When the processor 1010 executes the instructions stored in the memory 1020, the processor 1010 controls the 3D rendering device 1000 to perform at least one of the above-described operations. For example, the processor 1010 controls the 3D rendering device 1000 to perform the following operations: extracting sampling points from the 3D scene; obtaining rendering result information of the sampling points by rendering the extracted sampling points; and generating a rendering result image corresponding to the entire rendering based on the rendering result information of the sampling points and the feature information of the 3D scene.

[0122] To extract sampling points, processor 1010 uses a sampling point determination model based on a first neural network. This model takes feature information from the 3D scene as input and outputs the position information of the sampling points to be extracted from the 3D scene. Furthermore, to generate a rendered image, processor 1010 uses a rendered image generation model based on a second neural network. This model takes the rendered image generation information of the sampling points and the feature information of the 3D scene as input and outputs a rendered image based on the rendered image information and the feature information.

[0123] In order to obtain rendering result information of the sampling points, the processor 1010 renders each of the sampling points based on a portion or all of the virtual light source used for rendering, or based on a portion or all of the light used for rendering.

[0124] In addition, processor 1010 executes the above reference Figures 1 to 9B Any one or any combination of operations, and no further description will be repeated here.

[0125] For example, the processor 1010 included in the 3D rendering apparatus 1000 includes a first processor and a second processor that divide and process the entire rendering process. For example, the first processor may be a unit configured to perform rendering operations using a neural network, and the second processor may be a GPU. The first processor can extract sampling points from the 3D scene to be rendered using the neural network. The second processor can generate a rendered result image corresponding to the entire rendering based on the rendering result information of the extracted sampling points and the feature information of the 3D scene.

[0126] In the example, the first processor uses a neural network to extract sampling points, and the second processor obtains the rendering result information of the sampling points by rendering the sampling points with the aid of a GPU pipeline. The first processor estimates the rendering result information of points other than the sampling points based on the rendering result information of the sampling points, and the second processor generates a rendering result image corresponding to the entire rendering based on the rendering result information of the sampling points and the estimated rendering result information.

[0127] In another example, the first processor uses a neural network to extract sampling points, and the second processor obtains the rendering result information of the sampling points by rendering the sampling points using a GPU pipeline. The second processor renders points other than the sampling points based on the rendering result information of the sampling points, and generates a rendered image based on the results obtained by rendering the sampling points and other points.

[0128] In another example, the first processor uses a neural network to extract sampling points, and the second processor obtains the rendering result information of the extracted sampling points by rendering the sampling points. The first processor estimates the rendering result information of points other than the sampling points based on the rendering result information of the sampling points, and the second processor generates the rendered result image based on the rendering result information of the sampling points and the estimated rendering result information.

[0129] In another example, the first processor uses a neural network to estimate the rendering result information of the sampled points, and the second processor estimates the rendering result information of points other than the sampled points by performing interpolation based on the rendering result information of the sampled points. The first processor extracts sampled points based on virtual light source information and object information related to objects in the 3D scene, and uses a neural network to estimate the rendering result information of the sampled points. In this example, the virtual light source information includes the motion information of the virtual light source in the 3D rendering, and the object information includes the motion information of the objects. The second processor estimates the rendering result information of points other than the sampled points based on the rendering result information of the sampled points. The second processor estimates the color values ​​or feature values ​​of other points by interpolation based on the color values ​​or feature values ​​of the sampled points.

[0130] In yet another example, such as Figure 7 As shown, the 3D rendering apparatus 1000 uses a neural network to determine intermediate rendering results for points other than the sampling points, and uses another scheme to determine the final rendering result information based on the intermediate rendering results. In this example, the first processor uses a neural network to estimate the first vignetting component of other points. The second processor estimates the second vignetting component of other points through the GPU pipeline, and estimates the rendering result information of those other points based on the first and second vignetting components.

[0131] Depending on the example, the 3D rendering apparatus 1000 also includes a display device 1030. The display device 1030 can display the rendered image.

[0132] This article is about Figure 1 , Figure 6 and Figure 10The described 3D rendering apparatuses 100 and 1000, training apparatus 600, and other apparatuses, units, modules, devices, and other components are implemented by hardware components. Examples of hardware components that can be used to perform the operations described in this application, where appropriate, include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more hardware components for performing the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). The processor or computer may be implemented by one or more processing elements (e.g., logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field-programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond to and execute instructions in a defined manner to achieve a desired result). In one example, the processor or computer includes (or is connected to) one or more memories storing instructions or software executed by the processor or computer. Hardware components implemented by a processor or computer can execute instructions or software, such as an operating system (OS) and one or more software applications running on the OS, to perform the operations described in this application. Hardware components can also access, manipulate, process, create, and store data in response to the execution of instructions or software. For brevity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application, but multiple processors or computers may be used in other examples, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may implement a single hardware component, or two or more hardware components. The hardware components can have any one or more different processing configurations, examples of which include single processor, discrete processor, parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0133] Perform the operations described in this application Figures 2 to 5 , Figure 7 , Figure 8 , Figure 9A and Figure 9BThe methods illustrated are executed by computing hardware, such as one or more processors or computers implemented as described above, which execute instructions or software to perform the operations described in this application (operations performed by the methods). For example, a single operation or two or more operations may be performed by a single processor, two or more processors, or a processor and a controller. One or more operations may be performed by one or more processors or a processor and a controller, and one or more other operations may be performed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller may perform a single operation or two or more operations.

[0134] Instructions or software for controlling a processor or computer to implement hardware components as described above and to execute the methods described above are written as computer programs, code segments, instructions, or any combination thereof, for individually or collectively instructing or configuring the processor or computer to operate as a machine or special-purpose computer to perform the operations performed by the hardware components and the methods described above. In one example, the instructions or software include machine code that is directly executed by the processor or computer, such as machine code generated by a compiler. In another example, the instructions or software include high-level code that is executed by the processor or computer using an interpreter. A programmer of ordinary skill in the art can readily write the instructions or software based on the block diagrams and flowcharts shown in the accompanying drawings and the corresponding descriptions in the specification, which disclose algorithms for performing the operations performed by the hardware components and the methods described above.

[0135] Instructions or software that control a processor or computer to implement hardware components and perform the methods described above, along with any associated data, data files, and data structures, are recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDDs), solid-state drives (SSDs), flash memory, card-type memory (e.g., multimedia cards or microcards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tape, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state drives, and any other devices configured to: store instructions or software and any associated data, data files, and data structures in a non-transitory manner, and provide instructions or software and any associated data, data files, and data structures to a processor or computer such that the processor or computer can execute the instructions.

[0136] While this disclosure includes specific examples, it will be apparent to those skilled in the art that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is intended to apply to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order and / or if components in the described system, architecture, device, or circuit are combined in a different manner and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of this disclosure is not defined by the detailed description but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents are to be included in this disclosure.

Claims

1. A three-dimensional (3D) rendering method performed by a 3D rendering device, the 3D rendering method comprising: extracting sample points from a 3D scene; acquiring rendering result information of the sample points by rendering the sample points; and generating a rendering result image corresponding to an entire rendering based on the rendering result information of the sample points and feature information of the 3D scene, wherein the extracting of the sample points comprises extracting the sample points using a neural network-based sample point determination model that uses the feature information of the 3D scene as input and outputs position information of the sample points from the 3D scene, and wherein the feature information of the 3D scene includes texture information of a 3D model included in the 3D scene, such that the extracting of the sample points using the neural network-based sample point determination model comprises sampling the sample points in the 3D scene at various densities based on the texture information of the 3D model. The extracting of the sample points comprises extracting a minimum number of sample points that allow the rendering result image to be generated in the 3D scene.

2. The 3D rendering method of claim 1, wherein, The generating of the rendering result image comprises acquiring the rendering result image using a neural network-based rendered image generation model that uses the rendering result information of the sample points and the feature information of the 3D scene as input and outputs the rendering result image.

3. The 3D rendering method of claim 1, wherein, The generating of the rendering result image comprises estimating rendering result information of points in the 3D scene other than the sample points based on the rendering result information of the sample points and the feature information of the 3D scene.

4. The 3D rendering method of claim 1, wherein, The estimating of the rendering result information of the points other than the sample points comprises:

5. The 3D rendering method of claim 4, wherein, estimating a first vignetting component of the points other than the sample points based on the rendering result information of the sample points; estimating a second vignetting component of the points other than the sample points; and estimating the rendering result information of the points other than the sample points based on the first vignetting component and the second vignetting component. The estimating of the first vignetting component comprises estimating the first vignetting component using a neural network.

6. The 3D rendering method of claim 5, wherein, The estimating of the second vignetting component comprises estimating the second vignetting component by a graphics processing unit (GPU) pipeline.

7. The 3D rendering method of claim 5, wherein, 8. The 3D rendering method of claim 5, wherein the first vignetting component is a diffuse reflection component, and the second vignetting component is a specular reflection component. The feature information of the 3D scene includes any one or any combination of the following: depth information of a 3D model included in the 3D scene, and surface normal information of the 3D model.

9. The 3D rendering method of claim 1, wherein, The rendering result information of the sample points includes color information corresponding to each of the sample points acquired by rendering each of the sample points.

10. The 3D rendering method of claim 1, wherein, 11. The 3D rendering method of claim 1, wherein the sample points indicate predetermined positions in the 3D scene, and the rendering result information of the sample points includes vignetting result information of the sample points. The sample points correspond to a portion of vertices of a 3D model included in the 3D scene.

12. The 3D rendering method of claim 1, wherein, ​ 13. The 3D rendering method of claim 1, wherein the 3D scene is determined by texture atlas data, and obtaining the rendering result information comprises performing rendering based on the texture atlas data.

14. The 3D rendering method of claim 1, wherein, obtaining the rendering result information comprises obtaining the rendering result information by rendering each of the sampling points based on a portion or all of virtual light sources used for rendering.

15. The 3D rendering method of claim 1, wherein, obtaining the rendering result information comprises obtaining the rendering result information by rendering each of the sampling points based on a portion or all of light rays used for rendering.

16. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of claim 1.

17. A training method comprising: obtaining a first rendering result image by performing a full rendering of a three-dimensional (3D) scene; determining sampling points in the 3D scene using a first neural network-based sampling point determination model that uses feature information of the 3D scene as input; obtaining rendering result information of the sampling points by rendering the sampling points; obtaining a second rendering result image corresponding to the full rendering using a second neural network-based rendering image generation model that uses the rendering result information of the sampling points and the feature information of the 3D scene as input; and adjusting parameters of one or both of the sampling point determination model and the rendering image generation model based on the first rendering result image and the second rendering result image, wherein the first neural network-based sampling point determination model outputs position information of the sampling points from the 3D scene, and wherein the feature information of the 3D scene includes texture information of a 3D model included in the 3D scene, such that determining sampling points in the 3D scene using the first neural network-based sampling point determination model comprises sampling sampling points in the 3D scene at various densities based on the texture information of the 3D model. adjusting the parameters comprises adjusting the parameters of one or both of the sampling point determination model and the rendering image generation model to reduce a loss based on whether the second rendering result image obtained from the rendering result information of the sampling points satisfies a preset condition, a number of the sampling points, and a separation between the sampling points in the 3D scene.

18. The training method of claim 17, wherein, 19. A three-dimensional (3D) rendering apparatus comprising: at least one processor, wherein the at least one processor controls the 3D rendering apparatus to: extract sampling points from a 3D scene; obtain rendering result information of the sampling points by rendering the sampling points; and generate a rendering result image corresponding to a full rendering based on the rendering result information of the sampling points and feature information of the 3D scene, ​ wherein the extracting the sampling points comprises extracting the sampling points using a neural network-based sampling point determination model that uses the feature information of the 3D scene as input and outputs position information of the sampling points from the 3D scene, and wherein the feature information of the 3D scene includes texture information of a 3D model included in the 3D scene, such that the extracting the sampling points using the neural network-based sampling point determination model comprises sampling the sampling points in the 3D scene at various densities based on the texture information of the 3D model.

20. The 3D rendering device of claim 19, wherein, The generating the rendered result image comprises obtaining the rendered result image using a neural network-based rendered image generation model that uses the rendered result information of the sampling points and the feature information of the 3D scene as input and outputs the rendered result image.

21. The 3D rendering device of claim 19, wherein, The obtaining the rendered result information comprises obtaining the rendered result information by rendering each of the sampling points based on some or all of virtual light sources used for rendering.

22. The 3D rendering device of claim 19, wherein, The obtaining the rendered result information comprises obtaining the rendered result information by rendering each of the sampling points based on some or all of light rays used for rendering. 23.A three-dimensional (3D) rendering apparatus comprising: a first processor configured to extract sampling points from a 3D scene using a neural network; and a second processor configured to generate a rendered result image corresponding to an entire rendering based on rendered result information of the sampling points and feature information of the 3D scene, wherein the first processor is further configured to extract the sampling points using a neural network-based sampling point determination model that uses the feature information of the 3D scene as input and outputs position information of the sampling points from the 3D scene, and wherein the feature information of the 3D scene includes texture information of a 3D model included in the 3D scene, such that the extracting the sampling points using the neural network-based sampling point determination model comprises sampling the sampling points in the 3D scene at various densities based on the texture information of the 3D model. 24.The 3D rendering apparatus of claim 23, wherein the second processor is configured to render the sampling points through a graphics processing unit (GPU) pipeline, and the first processor is configured to estimate rendered result information of points other than the sampling points based on the rendered result information of the sampling points.

25. The 3D rendering device of claim 24, wherein, the second processor is configured to generate the rendered result image based on the rendered result information of the sampling points and the rendered result information of the points other than the sampling points.

26. The 3D rendering device of claim 23, wherein, the second processor is configured to: render the sampling points through a graphics processing unit (GPU) pipeline; render the points other than the sampling points based on the rendered result information of the sampling points; and generate the rendered result image based on results obtained by rendering the sampling points and the points other than the sampling points. ​ 27. The 3D rendering apparatus of claim 23, wherein the first processor is configured to: obtain rendering result information of the sampling points by rendering the sampling points; and estimate rendering result information of points other than the sampling points based on the rendering result information of the sampling points, and the second processor is configured to generate the rendered result image based on the rendering result information of the sampling points and the rendering result information of the points other than the sampling points.

28. The 3D rendering device of claim 23, wherein, the first processor is configured to estimate a first vignetting component of the points other than the sampling points using a neural network.

29. The 3D rendering device of claim 28, wherein, the second processor is configured to estimate a second vignetting component of the points other than the sampling points by a graphics processing unit (GPU) pipeline, and estimate the rendering result information of the points other than the sampling points based on the first vignetting component and the second vignetting component.

30. The 3D rendering device of claim 23, wherein, the second processor is a graphics processing unit (GPU).

31. The 3D rendering apparatus of claim 23, wherein the first processor is configured to: extract the sampling points based on virtual light source information and object information related to objects in the 3D scene; and estimate rendering result information of the sampling points using a neural network, and the second processor is configured to estimate rendering result information of points other than the sampling points based on the rendering result information of the sampling points.

32. A processor-implemented three-dimensional (3D) rendering method, comprising: extracting a minimum number of sampling pixels required for generating a result image from a 3D scene; rendering the sampling pixels to generate rendering result information; inputting feature information of the 3D scene and the rendering result information to a neural network-based rendered image generation model; and displaying the result image as an output of the rendered image generation model corresponding to an entire rendering, wherein extracting the sampling pixels comprises extracting the sampling pixels using a neural network-based sampling point determination model that uses feature information of the 3D scene as input and outputs position information of sampling pixels from the 3D scene, and wherein the feature information of the 3D scene includes texture information of a 3D model included in the 3D scene, such that extracting the sampling pixels using the neural network-based sampling point determination model comprises sampling sampling pixels in the 3D scene at various densities based on the texture information of the 3D model. the rendered image generation model is configured to estimate color values of pixels other than the sampling pixels based on the rendering result information and the feature information, and determine pixel values of pixels included in the result image. ​ 33. The 3D rendering method of claim 32, wherein, ​

Images