Graphics processing unit, graphics processing system and graphics processing method for performing interpolation

Abstract

A graphics processing system includes: a graphics processor that generates a G-buffer (geometry buffer) by performing deferred shading of a first pass; and a memory that stores the G-buffer, wherein the graphics processor includes: a graph manager that generates a first fragment graph indicating fragments to be shaded and generates a second fragment graph by adding fragments to the first fragment graph or removing fragments from the first fragment graph based on the G-buffer; and a shader that shades fragments according to the second fragment graph.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] Korean Patent Application No. 10-2018-0134442, filed on November 5, 2018 in the Korean Intellectual Property Office and entitled “GRAPHICS PROCESSING UNIT, GRAPHICS PROCESSING SYSTEM, AND GRAPHICS PROCESSING METHOD OF PERFORMING INTERPOLATION IN DEFERRED SHADING” is incorporated herein by reference. Technical Field

[0003] Embodiments relate to graphics processing, and more particularly, to a graphics processing unit, a graphics processing system, and a graphics processing method that perform interpolation in deferred shading. Background Art

[0004] Rendering may refer to the process of generating an image from a three-dimensional (3D) model or a two-dimensional (2D) model from a given viewpoint. As the demand for computer graphics increases, a graphics processing system may be used as dedicated hardware for accelerating rendering, and the rendering process in a computing system including a graphics processing system may be referred to as a graphics pipeline.

[0005] Fragment shading, one of the operations in the graphics pipeline, may require a lot of computational resources. To reduce the computational resources consumed in fragment shading, fragments may be selectively shaded and unshaded fragments may be interpolated from shaded peripheral fragments. Therefore, the final image quality according to the interpolation may be determined by the interpolation input, i.e., the shaded fragments. Summary of the invention

[0006] According to one aspect, a graphics processing system is provided, comprising: a graphics processor that generates a G-buffer (geometry buffer) by performing deferred shading of a first pass; and a memory that stores the G-buffer, wherein the graphics processor includes: a graph manager that generates a first fragment graph indicating fragments to be shaded and generates a second fragment graph by adding fragments to the first fragment graph or removing fragments from the first fragment graph based on the G-buffer; and a shader that shades fragments according to the second fragment graph.

[0007] According to another aspect, there is provided a graphics processing system, comprising: a graphics processor that generates a G-buffer (geometry buffer) by performing deferred shading of a first pass; and a memory that stores the G-buffer, wherein the graphics processor comprises: a shader that generates intermediate rendering data by shading fragments according to a fragment map indicating fragments to be shaded, and generates final rendering data by selectively shading the fragments based on the G-buffer; and an interpolation filter that generates a final image including shaded pixels by interpolating shading for unshaded pixels based on shaded pixels adjacent thereto in the intermediate rendering data and the final rendering data.

[0008] According to another aspect, there is provided a graphics processing system, comprising: a graphics processor that outputs a G-buffer (geometry buffer) by performing deferred shading of a first pass; and a memory that stores the G-buffer, wherein the graphics processor includes: a shader that generates intermediate rendering data by shading fragments according to a fragment map that indicates the fragments to be shaded, and generates a final image by providing the intermediate rendering data and the G-buffer stored in the memory to an artificial neural network, the artificial neural network being trained to output the final image from an image of the G-buffer and an image of the intermediate rendering data.

[0009] According to another aspect, a graphics processing method is provided, comprising: generating a G-buffer (geometry buffer) by performing deferred shading of a first pass; generating intermediate rendering data by performing rendering according to an initial fragment map indicating fragments to be shaded; determining that fragments to be interpolated are unshaded fragments based on the G-buffer and the intermediate rendering data; and interpolating the unshaded fragments based on adjacent shaded fragments.

[0010] According to another aspect, there is provided a graphics processing method, comprising: generating a G-buffer (geometry buffer) by performing deferred shading of a first pass; generating intermediate rendering data by performing rendering according to an initial fragment map indicating fragments to be shaded; and providing the intermediate rendering data and the G-buffer to an artificial neural network, wherein the artificial neural network is trained to output a final image from an image of the G-buffer and an image of the intermediate rendering data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Features will become apparent to those skilled in the art by describing in detail exemplary embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 illustrates a computing system according to an embodiment;

[0013] Figure 2 illustrates an example of a graphics pipeline according to an embodiment;

[0014] Figure 3 illustrates deferred shading according to an embodiment;

[0015] Figure 4 An example of a multi-pass rendering method according to an embodiment is illustrated;

[0016] Figure 5 illustrates a graphics processing system according to an embodiment;

[0017] Figure 6 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Figure 5 A flowchart of an example of the operation of a map manager;

[0018] Fig. 7A and Figure 7B A flowchart illustrating a method of generating a second segment graph according to an embodiment;

[0019] Figure 8 illustrates a block diagram of a graphics processing system according to an embodiment;

[0020] Fig. 9 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Figure 8 A flowchart of an example of the operation of a shader;

[0021] Fig.10 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Figure 8 A flowchart of an example of the operation of an interpolation filter;

[0022] Fig.11 FIG. 1 shows a diagram according to an embodiment of the present invention. Fig.10 Examples of images generated by the method;

[0023] Fig.12 illustrates a graphics processing system according to an embodiment;

[0024] Fig.13 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Fig.12 A flowchart of an example of the operation of a shader;

[0025] Fig.14 A flowchart illustrating a graphics processing method according to an embodiment;

[0026] Fig.15 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Fig.14 Flowchart of an example of operation S260 and operation S280;

[0027] Fig.16 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Fig.15 A flowchart of an example of operation S262;

[0028] Fig.17 FIG. 1 shows a schematic diagram of an embodiment of the present invention. Fig.14 Flowchart of an example of operation S260 and operation S280;

[0029] and

[0030] Fig.18 A flowchart illustrating a graphics processing method according to an embodiment is shown. DETAILED DESCRIPTION

[0031] Figure 1 The computing system 10 according to an embodiment is illustrated. The computing system 10 may refer to any computing system including a graphics processing system 11. Without limitation, the computing system 10 may be a stationary type computing system such as a desktop computer, a server, a television set-top box, a video game console, or a portable computing system such as a laptop computer, a mobile phone, a wearable device, a portable media player, a tablet personal computer (PC), an e-book reader, etc.

[0032] like Figure 1 As shown, the computing system 10 may include a graphics processing system 11, a display device 12, an input device 13, a central processing unit (CPU) 14, a system memory 15, and a system bus 16. In some embodiments, two or more components included in the computing system 10, such as two or more of the graphics processing system 11, the CPU 14, and the system bus 16, may be integrated into a single processing system. The system bus 16 may be configured as follows: Figure 1 Graphics processing system 11, input device 13, CPU 14, and system memory 15 are shown connected. Graphics processing system 11, input device 13, CPU 14, and system memory 15 may communicate with each other via system bus 16. In some embodiments, system memory 15 may be directly connected to CPU 14.

[0033] The CPU 14 may receive user input via the input device 13. In addition, the CPU 14 may execute a series of instructions (or programs) stored in the system memory 15, process data stored in the system memory 15, or direct the graphics processing system 11 to perform a specific job in the graphics pipeline. In some embodiments, the CPU 14 may include two or more cores.

[0034] The system memory 15 may store instructions and data to be processed by the CPU 14 and the graphics processing system 11. The system memory 15 may include any type of memory that stores information. For example, in some embodiments, the system memory 15 includes a volatile memory, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a mobile DRAM, a double data rate synchronous dynamic random access memory (DDR SDRAM), a low power DDR (LPDDR) SDRAM, a graphics DDR (GDDR) SDRAM, a Rambus dynamic random access memory (RDRAM), etc., or in some embodiments, may include a non-volatile memory, such as an electrically erasable programmable read-only memory (EEPROM), a flash memory, a phase change random access memory (PRAM), a resistive random access memory (RRAM), a nano floating gate memory (NFGM), a polymer random access memory (PoRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), etc.

[0035] like Figure 1 As shown, the system memory 15 may include an application 15_1, an application program interface (API) 15_2, and a GPU driver 15_3. The application 15_1 may generate calls relative to the API 15_2 to generate a desired result in the form of a sequence of graphics images, for example. In addition, the application 15_1 may provide an advanced shading program to the API 15_2 to be processed by the GPU driver 15_3. The advanced shading program may include at least one shading engine (or shader) (e.g., Figure 5 The API 15_2 may be implemented in the GPU driver 15_3. The GPU driver 15_3 may convert the high-level shading program into a machine code shading program optimized for the type of shading engine (e.g., vertex, geometry, fragment, etc.).

[0036] The graphics processing system 11 may receive instructions provided by the CPU 14, and may process the received instructions to render an image and display the image on the display device 12. Figure 1 As shown, the graphics processing system 11 may include a graphics processing unit 11_2 and a memory 11_4 and may be referred to as a graphics subsystem. The graphics processing unit 11_2 and the memory 11_4 may be connected to each other via a bus (e.g., a GPU bus). The graphics processing unit 11_2 may store data generated when processing instructions received from the CPU 14 in the memory 11_4, and may display the final image stored in the memory 11_4 on the display device 12.

[0037] The graphics processing unit 11_2 may include a multiprocessor capable of executing multiple threads simultaneously and may be referred to as a parallel processing unit (PPU). The multiprocessor may be programmed to perform various operations, and some multiprocessors may be used as a shading engine including one or more programmable shaders.

[0038] In some embodiments, the graphics processing unit 11_2 can perform delayed shading. Delayed shading can refer to a method of separating the shading of fragments (or pixels) from the calculation of complex geometry and lighting conditions. In the first pass, the data required for shading can be collected and information about color, diffusion, depth, normal, mirror, etc. can be rendered on the G buffer (geometry buffer) G_BUF. Next, the shader can calculate the lighting of each fragment (or pixel) to perform shading. Figure 1 As shown, the memory 11_4 may include a G buffer G_BUF, and in this specification, the G buffer G_BUF may refer to data generated by performing delayed shading of the first pass. Because the G buffer G_BUF shares the same geometry as the final image (that is, the final image generated by delayed shading), the final image and the G buffer G_BUF (or the image included in the G buffer G_BUF) may correspond to each other one-to-one. This will be referred to later. Figure 3 Describes an example of deferred shading.

[0039] In some embodiments, the graphics processing unit 11_2 may selectively shade fragments. For example, the graphics processing unit 11_2 may shade only some fragments and may calculate other fragments, that is, the shading of unshaded fragments, from the shaded peripheral fragments, that is, the shaded fragments adjacent thereto, via interpolation. Therefore, the computing resources consumed in shading (e.g., fragment shading) may be reduced, and as a result, the cost of the graphics processing system 11 may be reduced and / or the time taken to complete the graphics pipeline may be reduced. The method of performing selective shading and then performing interpolation as described above may be referred to as a multi-pass rendering method, and in the multi-pass rendering method, the quality of the final image may be determined based on the interpolated input, that is, based on which fragments are shaded.

[0040] In some embodiments, the graphics processing unit 11_2 may determine the fragments to be selectively colored based on the G buffer G_BUF in a multi-pass rendering method. For example, the graphics processing unit 11_2 may detect edges from the image of the G buffer G_BUF, determine the fragments corresponding to the detected edges as the fragments to be colored, and determine the fragments corresponding to the areas from which the edges are not detected as the fragments to be interpolated. As described above, because the G buffer G_BUF generated by the delayed shading of the first pass may include various pieces of information about the final image, the graphics processing unit 11_2 may reduce computing resources and also achieve a high-quality final image by implementing multi-pass rendering using the G buffer G_BUF.

[0041] The display device 12 may refer to any device capable of outputting a visible image and may display the final image provided by the graphics processing system 11. For example, the display device 12 may be implemented by using a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED display, an active matrix OLED (AMOLED) display, an electrochromic device (ECD), a digital mirror device (DMD), an actuated mirror device (AMD), a grid light valve (GLV), a plasma display panel (PDP), an electroluminescent display (ELD), a vacuum fluorescent display (VFD), etc., and may be in the form of a flat panel display, a curved display, a flexible display, etc.

[0042] Figure 2 An example of a graphics pipeline according to an embodiment is shown. In some embodiments, Figure 2 The graphics pipeline can use Figure 1 The graphics processing system 11 is executed. For example, Figure 1 In the computing system 10, the application 15_1 can provide data and commands to the graphics processing system 11 via the GPU driver 15_3, and in response to the data and commands, the graphics processing system 11 can execute the graphics pipeline. Figure 2 , the graphics pipeline can be divided into geometry conversion of operation S21, operation S22 and operation S23, rasterization S24 and fragment shading S25.

[0043] In operation S21, vertices may be generated. For example, vertices 0, 1, 2, and 3 may be generated to indicate a three-dimensional object. In operation S22, vertex shading may be performed. For example, the positions of the vertices generated in operation S21 may be defined by vertex shading. In operation S23, primitives may be generated. A primitive may refer to a point, a line, a polygon, etc. formed using at least one vertex. For example, when a primitive is a triangle, the primitive is defined by three connected vertices.

[0044] In operation S24, rasterization of the primitive may be performed. Rasterization of the primitive may refer to dividing the primitive into at least one fragment, for example, a basic unit for performing graphics processing on the primitive. The primitive generated in operation S23 may include information about vertices, and because fragments are formed between vertices in operation S23, graphics processing may be subsequently performed.

[0045] In operation S25, fragment shading may be performed. In operation S25, the fragments shaded may be related to pixels on the screen space, and the fragment shading may be referred to as pixel shading. For example, a fragment may correspond to a basic unit of graphics processing constituting a primitive, and a pixel may correspond to a basic unit of graphics processing performed after fragment shading. In this specification, both fragments and pixels may be used.

[0046] In operation S26, texturing may be performed. Texturing may refer to determining the color of a fragment (or pixel) by using a previously provided texture. Determining the color of all pixels by using a previously provided texture may use fewer computing resources than determining the color of all pixels by performing an operation. In some embodiments, textures of different resolutions may be provided in advance, or the color of a pixel may be determined when a texture is mixed according to the location and size of the target. Reflecting a texture in a fragment may be referred to as texture picking.

[0047] In operation S27, testing and blending may be performed. For example, with respect to pixels corresponding to the same position, the pixels to be finally displayed may be determined by depth testing. In addition, warping and clipping, etc. may be performed, and the value of the pixel to be finally displayed, for example, color, may be determined. Next, in operation S28, the final image generated in the previous operation may be stored in a frame buffer, and the final image stored in the frame buffer may be displayed.

[0048] As above reference Figure 1 As stated, Figure 1 The graphics processing unit 11_2 of can generate the final image based on the multi-pass rendering method by shading some fragments and then performing interpolation. Figure 2 In operation S25 of the graphics pipeline, some fragments may be selected (which may also be referred to as sampling of fragments), and the selected (or sampled) fragments may be colored, while unselected (or uncolored) fragments may be calculated from the colored fragments adjacent thereto. Here, by selecting the fragments based on the G buffer G_BUF, the selected fragments may reflect the final image, thus preventing image distortion or aliasing due to inappropriate interpolation.

[0049] Figure 3 FIG. 1 illustrates delayed shading according to an embodiment. Figure 1Delayed shading may include a first pass PA1 to generate a G buffer G_BUF and a second pass PA2 to generate a final image F_IMG. Figure 3 Deferred shading can be used Figure 1 Executed by graphics processing unit 11_2.

[0050] refer to Figure 3 In the first pass PA1, a G buffer G_BUF may be generated from the first input data D31, and in the second pass PA2, a final image F_IMG may be generated from the second input data D32 and the G buffer G_BUF. The first pass PA1 may include vertex shading VS1 and fragment shading FS1, and the second pass PA2 may also include vertex shading VS2 and fragment shading FS2. In some embodiments, the vertex shading VS2 and fragment shading FS2 of the second pass PA2 may be exclusively referred to as deferred shading.

[0051] In some embodiments, in the fragment shading FS1 of the first pass PA1, shading can be performed without considering the lighting. At the same time, in the fragment shading FS2 of the second pass PA2, shading can be performed considering the lighting. Therefore, the image included in the G buffer G_BUF can include geometry information regardless of the lighting, and in the second pass PA2, the lighting can be calculated only with respect to the fragments affected by the lighting. Delayed shading can allow easy control of shader resources, simplified graphics pipelines, and convenient rendering of complex light sources. In some embodiments, the second input data D32 (i.e., the second input data D32 from which some pieces of information are removed) simplified compared to the first input data D31 can also be provided to the second pass PA2 together with the G buffer G_BUF.

[0052] As a result of the first pass PA1, the G buffer G_BUF may include images corresponding to various types of information. Figure 3 As shown, the G buffer G_BUF may include a mirror image, a normal image, a depth image, a diffusion image, etc. Figure 3 The G-buffer G-BUF shown is only an example, and the G-buffer G_BUF may only include Figure 3 Some of the images shown and / or may include Figure 3 Other images not shown.

[0053] Figure 4 An example of a multi-pass rendering method according to an embodiment is shown. Specifically, Figure 4 An example of shading a 4x4 fragment in a primitive 41, eg, a triangle, is shown.

[0054] From the 4x4 fragments, four fragments F0, F1, F2 and F3 may be selected (or sampled). In some embodiments, the fragments may be selected according to at least one predefined pattern or according to depth information. For example, some of the consecutive fragments having a depth difference below a threshold may not be selected, while fragments having a depth difference relative to adjacent fragments greater than a threshold, such as detected edges, may be selected for shading. When shading is performed on fragments selected in a predefined pattern, for example, four fragments like the 4x4 fragment 42, only those four selected fragments have values. Next, interpolation may be performed on the remaining fragments in the 4x4 fragment 42 based on the values ​​of the four fragments so that all fragments of the 4x4 fragment 43 have values.

[0055] Figure 5 is a block diagram of a graphics processing system 50 according to an embodiment. Figure 1 As described above, the graphics processing unit 52 may perform interpolation on the fragment based on the G buffer G_BUF stored in the memory 54, and store the final image F_IMG in the memory 54. Figure 5 As shown, the graphics processing unit 52 may include a graph manager 52_2, a shader 52_4, and an interpolation filter 52_6.

[0056] The graph manager 52_2 may generate a fragment graph and provide the fragment graph to the shader 52_4. The fragment graph may indicate the fragments to be shaded, that is, the fragments to be shaded by using the shader. Figure 4 In the example of , the fragment map may indicate four fragments F0, F1, F2, and F3. In this specification, a fragment to be colored and defined by a fragment map may be referred to as being included in the fragment map, and another fragment, that is, a fragment to be interpolated may be referred to as not being included in the fragment map. Figure 5 As shown, the map manager 52_2 may include an edge detector ED, and may generate or correct a fragment map by using the edge detector ED. The map manager 52_2 may receive a G buffer G_BUF from the memory 54, provide an initial fragment map MAP0 and a second fragment map MAP2 to the shader 52_4, and receive intermediate rendering data I_RND from the shader 52_4. Figure 6 Describe the operation of the graph manager 52_2.

[0057] The shader 52_4 may color the fragments according to the fragment graph received from the graph manager 52_2. For example, the shader 52_4 may color the fragments indicated by the fragment graph (e.g., Figure 4In some embodiments, the shader 52_4 may receive the initial fragment map MAP0 from the map manager 52_2, and may generate the intermediate rendering data I_RND by performing fragment shading (or rendering) according to the initial fragment map MAP0. Therefore, the intermediate rendering data I_RND may include some fragments included in the initial fragment map MAP0 and colored and other fragments that are not colored.

[0058] The shader 52_4 may receive the second fragment map MAP2 from the map manager 52_2, and may generate final rendering data F_RND by performing fragment shading according to the second fragment map MAP2. Figure 6 As described, the second fragment map MAP2 may correspond to a fragment map corrected from the initial fragment map MAP0 and may indicate fragments suitable for the final image F_IMG as objects to be shaded based on the G buffer G_BUF. The shader 52_4 may provide the final rendering data F_RND generated according to the second fragment map MAP2 to the interpolation filter 52_6.

[0059] The interpolation filter 52_6 may receive the final rendering data F_RND, and may interpolate the uncolored fragments based on the colored fragments included in the final rendering data F_RND. The interpolation filter 52_6 may perform any interpolation method using adjacent colored fragments, and may perform interpolation so that, for example, the uncolored fragment has an average value of the values ​​of the colored peripheral fragments above and below and / or to the left and right of the uncolored fragment. In addition, in some embodiments, bilinear interpolation, bicubic interpolation, etc. may be performed. The interpolation filter 52_6 may generate a value such as Figure 4 All segments of the segment 43 have a final image F_IMG of value, and the final image F_IMG is stored in the memory 54.

[0060] Figure 6 According to the embodiment Figure 5 FIG manager 52_2 operation example flow chart. As mentioned above Figure 5 As described above, the graph manager 52_2 can generate a fragment graph and provide the fragment graph to the shader 52_4. Figure 5 describe Figure 6 Embodiment of the invention.

[0061] refer to Figure 6, in operation S62, the map manager 52_2 may generate an initial fragment map MAP0. In some embodiments, the map manager 52_2 may generate an initial fragment map MAP0 indicating some fragments according to a predefined pattern. In addition, in some embodiments, the map manager 52_2 may obtain depth data D_DEP generated by a depth pass, and generate an initial fragment map MAP0 based on the depth data D_DEP. For example, the map manager 52_2 may detect adjacent fragments having a depth difference greater than a threshold from the depth data D_DEP by using an edge detector ED, and add the detected fragments to the initial fragment map MAP0.

[0062] In operation S64, the map manager 52_2 may generate a first fragment map MAP1. In some embodiments, the map manager 52_2 may generate the first fragment map MAP1 by adding fragments to the initial fragment map MAP0 or removing fragments from the initial fragment map MAP0 based on the intermediate rendering data I_RND. For example, the map manager 52_2 may detect fragments whose differences between adjacent colored fragments in the intermediate rendering data I_RND are greater than a threshold value by using an edge detector ED, and may add the detected fragments to the first fragment map MAP1. In addition, the map manager 52_2 may remove fragments corresponding to areas from which no edges are detected from the first fragment map MAP1.

[0063] In operation S66, the map manager 52_2 may generate a second fragment map MAP2. In some embodiments, the map manager 52_2 may generate the second fragment map MAP2 by adding fragments to the first fragment map MAP1 or removing fragments from the first fragment map MAP1 based on the G buffer G_BUF. For example, the map manager 52_2 may detect edges from an image included in the G buffer G_BUF using an edge detector ED, and add fragments corresponding to the detected edges to the second fragment map MAP2. In addition, the map manager 52_2 may remove fragments corresponding to areas from which no edges are detected from the second fragment map MAP2. This will be referred to later. Fig. 7A and Figure 7B An example of operation S66 is described.

[0064] Fig. 7A and Figure 7B is a flowchart of a method for generating a second segment map MAP2 according to an embodiment. Figure 5 and Figure 6 As described above, the second fragment map MAP2 can be used by the shader 52_4 to generate the final rendering data F_RND. In some embodiments, Fig. 7A and Figure 7B The method can be Figure 6 An example of operation S66 may be performed by using Figure 5Executed by the graph manager 52_2. Figure 5 and Figure 6 describe Fig. 7A and Figure 7B Embodiment of the invention.

[0065] refer to Fig. 7A In some embodiments, a second fragment map MAP2 including fragments corresponding to edges detected from at least one image among the images included in the G buffer G_BUF may be generated. In addition, in some embodiments, the second fragment map MAP2 may be generated by removing fragments corresponding to regions from which no edges are detected in all images included in the G buffer G_BUF. For example, the map manager 52_2 may evaluate all fragments.

[0066] In operation S70, the second segment map MAP2 is set to the first segment map MAP1. Therefore, the second segment map MAP2 may include segments included in the initial segment map MAP0 and segments added based on the intermediate rendering data I_RND.

[0067] In operations S71_1, S72_1, S73_1, and S74_1, it may be respectively performed to check whether the diffusion image, the depth image, the normal image, and the mirror image are included in the G buffer G_BUF. When the above images are included in the G buffer G_BUF, edges may be detected from the images in each of operations S71_2, S72_2, S73_2, and S74_2, and then operation S75_1 may be performed. When one or more of the above images are not included in the G buffer G_BUFF, edge detection may be performed on the remaining images therein. Although in Fig. 7A Although only four images are illustrated in the example, in some embodiments, edge detection may be further performed on additional images (eg, color images).

[0068] In operation S75_1, a segment corresponding to the detected edge that is not in the first segment map MAP1 may be added to the second segment map MAP2. For example, when an edge is detected from at least one of the images included in the G-buffer G_BUF, the segment corresponding to the detected edge may be added to the second segment map MAP2. Therefore, the second segment map MAP2 may include a segment corresponding to the edge detected from the image included in the G-buffer G_BUF. In addition, a segment corresponding to any different edge detected from a different image in the G-buffer G_BUF may be added to the second segment map MAP2.

[0069] In operation S75_2, fragments corresponding to areas in the first fragment map MAP1 from which no edges are detected may be removed from the second fragment map MAP2. For example, when no edges are detected in the images included in the G buffer G_BUF, fragments corresponding to the undetected edges may be removed. The fragments removed from the second fragment map MAP2 may not be shaded by the shader 52_4, but may instead be calculated by using the interpolation filter 52_6. In some embodiments, operation S75_2 may be omitted, and therefore, the second fragment map MAP2 may be the same as the first fragment map MAP1, or may include the fragments included in the first fragment map MAP1 and additional fragments.

[0070] refer to Figure 7B , in some embodiments, the second fragment map MAP2 may be generated by evaluating the first fragment map MAP1. For example, the map manager 52_2 may evaluate fragments not included in the first fragment map MAP1, that is, fragments to be interpolated, and may generate the second fragment map MAP2 by correcting the first fragment map MAP1 based on the evaluation result. In some embodiments, the evaluation may be performed on a group of fragments including a plurality of fragments, and when the evaluation result is a failure, the fragments from which the edges are detected among the group of fragments may be added to the second fragment map MAP2. Therefore, the fragments may only be added to the second fragment map MAP2, but not removed therefrom.

[0071] In operation S76_1, a diffuse image may be checked. For example, by using an edge detector ED, the map manager 52_2 may determine whether a diffuse image exists in the G buffer G_BUF obtained from the memory 54. The edge detector ED may detect edges by using any method, for example, a search-based method and / or a zero-crossing method. When there is no diffuse image, operation S77_1 may be subsequently performed. When there is a diffuse image, edge detection of the diffuse image may be performed on fragments not included in the first fragment map MAP1. In operation S76_3, when an edge is detected from the diffuse image, the evaluation result in operation S80_2 may be determined to be a failure, and when an edge is not detected from the diffuse image, operation S77_1 may be subsequently performed. When the evaluation result is a failure in operation S80_2, the fragment causing the error may be added to the first fragment map MAP1 to generate the second fragment map MAP2.

[0072] When there is no diffusion image in the G buffer G_BUF or an edge is not detected in the diffusion image, the depth image may be checked in operation S77_1. For example, the map manager 52_2 may determine whether there is a depth image in the G buffer G_BUF obtained from the memory 54. When there is no depth image, operation S78_1 may be subsequently performed. When there is a depth image, edge detection of the depth image may be performed in operation S77_2 for fragments not included in the first fragment map MAP1. In operation S77_3, when an edge is detected from the depth image, the evaluation result in operation S80_2 may be determined to be a failure, and when an edge is not detected from the depth image, operation S78_1 may be subsequently performed. When the evaluation result in operation S80_2 is a failure, the fragment that caused the error may be added to the first fragment map MAP1 to generate the second fragment map MAP2.

[0073] When there is no depth image in the G buffer G_BUF or no edge is detected in the depth image, the normal image may be checked in operation S78_1. For example, the map manager 52_2 may determine whether there is a normal image in the G buffer G_BUF obtained from the memory 54. When there is no normal image, operation S79_1 may be subsequently performed. When there is a normal image, edge detection of the normal image may be performed in operation S78_2 for fragments not included in the first fragment map MAP1. In operation S78_3, when an edge is detected from the normal image, the evaluation result in operation S80_2 may be determined to be a failure, and when an edge is not detected from the normal image, operation S79_1 may be subsequently performed. When the evaluation result in operation S80_2 is a failure, the fragment that caused the error may be added to the first fragment map MAP1 to generate the second fragment map MAP2.

[0074] When there is no normal image in the G buffer G_BUF or no edge is detected in the normal image, the mirror image may be checked in operation S79_1. For example, the map manager 52_2 may determine whether there is a mirror image in the G buffer G_BUF obtained from the memory 54. When there is no mirror image, that is, when there is no image in the G buffer G_BUF, the evaluation result in operation S80_2 may be determined to be a failure. When there is a mirror image, edge detection of the mirror image may be performed in operation S79_2 for the fragments not included in the first fragment map MAP1. When an edge is detected in operation S79_3, the evaluation result in operation S80_2 may be determined to be a failure, and when an edge is not detected from the mirror image, the evaluation result in operation S80_1 may be determined to be a pass, that is, there is no need to change the first fragment map MAP1. When the evaluation result in operation S80_2 is a failure, the fragment causing the error may be added to the first fragment map MAP1 to generate the second fragment map MAP2.

[0075] Figure 8 FIG. 8 is a diagram of a graphics processing system 80 according to an embodiment. Figure 1 As described above, the graphics processing unit 82 may perform interpolation on the fragment based on the G buffer G_BUF stored in the memory 84, and store the final image F_IMG in the memory 84. Figure 8 As shown, the graphics processing unit 82 may include a texture unit 82_3, a shader 82_4, and an interpolation filter 82_6.

[0076] The texture unit 82_3 may receive the G buffer G_BUF and generate a G buffer image G_IMG, and provide the G buffer image G_IMG to the shader 82_4. The texture unit 82_3 may refer to each shader that performs texturing (eg, Figure 2 The shader 82_4 may be a hardware module, a software module, or a combination thereof of operation S26 of the embodiment of the present invention. The texture unit 82_3 may be referred to as a texture mapping unit (TMU) and may perform texture fetching. In some embodiments, the shader 82_4 may obtain an image included in the G buffer G_BUF, that is, a G buffer image G_IMG, via the texture unit 82_3.

[0077] The shader 82_4 may receive the G buffer image G_IMG and generate intermediate rendering data I_RND and final rendering data F_RND. Figure 4 Each of the unselected 4x4 fragments 42, the intermediate rendering data I_RND and the final rendering data F_RND may include some colored fragments and the remaining, i.e., uncolored fragments. The shader 82_4 may include an edge detector ED, and generates the final rendering data F_RND by detecting edges from the G buffer image G_IMG. Figure 5 In the example of FIG. 5 , the G buffer G_BUF can be reflected in the fragment graph by using the graph manager 52_2, and in Figure 8 In the example of FIG. 8 , the G buffer G_BUF can be reflected in the final rendering data F_RND by using the texture unit 82_3 and the shader 82_4. Fig. 9 The operation of the shader 82_4 is described.

[0078] The interpolation filter 82_6 may receive the intermediate rendering data I_RND and the final rendering data F_RND from the shader 82_4 and generate a final image F_IMG in which all fragments have values ​​from the intermediate rendering data I_RND and the final rendering data F_RND. Figure 5 The interpolation filter 82_6 may further receive the intermediate rendering data I_RND. The interpolation filter 82_6 may store the final image F_IMG in the memory 84. Fig.10 and Fig.11Describe the operation of the interpolation filter 82_6.

[0079] Fig. 9 According to the embodiment Figure 8 Flowchart of an example of the operation of the shader 82_4. Figure 8 As described above, the shader 82_4 may generate intermediate rendering data I_RND and final rendering data F_RND, and provide them to the interpolation filter 82_6 . Fig. 9 Operation S92 and operation S94 may be performed as follows. Fig. 9 The steps shown may be performed in parallel with each other, or in some embodiments may be performed sequentially. Figure 8 describe Fig. 9 Embodiment of the invention.

[0080] In operation S92, the shader 82_4 may generate intermediate rendering data I_RND from the geometry data D90. For example, the geometry data D90 may include information about vertices, primitives, etc., and the shader 82_4 may generate the intermediate rendering data I_RND by performing fragment shading (or rendering) based on the initial fragment map MAP0. In some embodiments, the shader 82_4 may generate the initial fragment map MAP0 or may receive the initial fragment map MAP0 from the outside. For example, the shader 82_4 may generate an initial fragment map MAP0 indicating some fragments according to a predefined pattern, or may obtain depth data D_DEP generated by a depth pass, such as edge detection, to generate the initial fragment map MAP0 based on the depth data D_DEP. When the initial fragment map MAP0 is generated based on a predefined regular pattern, the intermediate rendering data I_RND may correspond to an image with a low resolution.

[0081] In operation S94, the shader 82_4 may generate final rendering data F_RND from the geometry data D90 based on the G buffer G_BUF. For example, the shader 82_4 may obtain an image included in the G buffer G_BUF, i.e., the G buffer image G_IMG, via the texture unit 82_3, and detect edges from the G buffer image G_IMG. The shader 82_4 may color fragments (or pixels) corresponding to the detected edges, and may discard fragments (or pixels) of areas from which no edges are detected. Therefore, the image included in the final rendering data F_RND may include fragments (or pixels) having invalid values, and on the other hand have a high (or full) resolution. In some embodiments, the shader 82_4 may generate a discard flag (e.g., Fig.10The interpolation filter 82_6 may identify invalid fragments (or pixels) from the final rendering data F_RND based on the discard flag.

[0082] Fig.10 According to the embodiment Figure 8 A flowchart of an example of the operation of the interpolation filter 82_6, and Fig.11 FIG. 1 shows a diagram according to an embodiment of the present invention. Fig.10 An example of an image generated by the method. Figure 8 As described above, the interpolation filter 82_6 may receive the intermediate rendering data I_RND and the final rendering data F_RND from the shader 82_4, or may receive the discard flag D_FLG. Figure 8 describe Fig.10 and Fig.11 Embodiment of the invention.

[0083] refer to Fig.10 , in operation S96_2, the images may overlap. For example, Fig.11 As shown, the interpolation filter 82_6 may include the first image IMG11 included in the intermediate rendering data I_RND and the second image IMG12 included in the final rendering data F_RND. The interpolation filter 82_6 may identify invalid pixels from the second image IMG12 based on the discard flag. Fig.11 As indicated by the highlighted pixels, the first image IMG11 and the second image IMG12 may each include some colored pixels. The interpolation filter 82_6 may overlap the first image IMG11 and the second image IMG12 to generate an overlapped image IMG20. Fig.11 As shown, the overlapped image IMG20 may include all colored pixels (or pixels having values) in the first image IMG11 and the second image IMG12. Fig.11 As shown, when the intermediate rendering data I_RND is generated based on a predefined regular pattern, the first image IMG11 may correspond to an image of relatively low resolution compared to the final image F_IMG.

[0084] In operation S96_4, pixels may be calculated from peripheral pixels. For example, the interpolation filter 82_6 may generate each invalid pixel in the overlapped image IMG20 from pixels with valid values ​​adjacent thereto, i.e., values ​​of pixels having no value. The interpolation filter 82_6 may generate pixels by using any interpolation method, and therefore, may generate a final image F_IMG in which all pixels have valid values.

[0085] Fig.12is a block diagram of a graphics processing system 120 according to an embodiment. Figure 1 As described above, the graphics processing unit 122 may perform interpolation on the fragment based on the G buffer G_BUF stored in the memory 124, and store the final image F_IMG in the memory 124. Fig.12 As shown, the graphics processing unit 122 may include a shader 122_4.

[0086] The shader 122_4 may include an artificial neural network (ANN) and generate a final image F_IMG based on the G buffer G_BUF received from the memory 124. ANN may refer to a structure in which a set of interconnected artificial neurons (or neuron models) are implemented. An artificial neuron may generate output data by performing a simple operation on the input data, and the output data may be passed to another artificial neuron.

[0087] The shader 122_4 may include an ANN trained to output a final image F_IMG from an image of the G buffer G_BUF and an image of the intermediate rendering data I_RND. Therefore, the shader 122_4 may generate the intermediate rendering data I_RND, and obtain the final image F_IMG from the ANN by providing the intermediate rendering data I_RND and the G buffer G_BUF to the ANN. Fig.13 The operation of the shader 122_4 is described.

[0088] Fig.13 According to the embodiment Fig.12 Flowchart of an example of the operation of the shader 122_4. Fig.12 describe Fig.13 Embodiment of the invention.

[0089] In operation S132, the shader 122_4 may generate intermediate rendering data I_RND from the geometry data D130. For example, the geometry data D130 may include information about vertices, primitives, etc., and the shader 122_4 may generate the intermediate rendering data I_RND by performing fragment shading (or rendering) based on the initial fragment map MAP0. In some embodiments, the shader 122_4 may generate the initial fragment map MAP0 or may receive the initial fragment map MAP0 from the outside. For example, the shader 122_4 may generate an initial fragment map MAP0 indicating some fragments according to a predefined pattern, or may obtain depth data D_DEP generated by a depth pass, such as edge detection, and generate the initial fragment map MAP0 based on the depth data D_DEP.

[0090] In operation S134, the shader 122_4 may provide the intermediate rendering data I_RND and the G buffer G_BUF to the ANN. Fig.12As described above, the ANN is trained to output a final image F_IMG from the image of the intermediate rendering data I_RND and the image of the G buffer G_BUF. Therefore, the ANN can output a final image F_IMG corresponding to the image of the intermediate rendering data I_RND and the image of the G buffer G_BUF. In some embodiments, the ANN can further receive the geometry data D130, and the shader 122_4 can further provide the geometry data D130 to the ANN.

[0091] Fig.14 is a flowchart of a graphics processing method according to an embodiment. For example, Figure 5 Graphics processing system 50 or Figure 8 The graphics processing system 80 performs Fig.14 In the following, reference will be made to Figure 5 describe Fig.14 Embodiment of the invention.

[0092] In operation S220, a G buffer G_BUF may be generated. Figure 3 As described above, the G buffer G_BUF may be generated as a result of the first pass PA1 in deferred shading. The graphics processing unit 52 may process the input data (eg, Figure 3 The G buffer G_BUF may be generated based on the input data D31, and the G buffer G_BUF may include at least one image corresponding to the screen space.

[0093] In operation S240, intermediate rendering may be performed. Intermediate rendering may refer to generating intermediate rendering data I_RND. For example, the graphics processing system 50 may perform rendering based on the initial fragment map MAP0 to generate intermediate rendering data I_RND having some fragments with valid values. Operations S220 and S240 may be performed as follows. Fig.14 The illustrated operations are performed in parallel with each other or may be performed sequentially.

[0094] In operation S260, the fragment to be interpolated may be determined. The fragment to be interpolated may refer to a fragment that has not been colored by the shader. Both the intermediate rendering data I_RND and the image of the G buffer G_BUF corresponding to the screen space may be used to accurately determine the fragment to be interpolated. The fragment other than the fragment to be interpolated, that is, the fragment to be colored may be colored by using the shader 52_4 as described above. This will be referred to later. Fig.15 and Fig.17 An example of operation S260 is described.

[0095] In operation S280, the fragments to be interpolated based on the colored peripheral fragments may be calculated. In some embodiments, the fragments to be interpolated may be identified based on a fragment map. In some embodiments, the fragments to be interpolated may also be identified by overlapping an image including some colored fragments. Fig.15 and Fig.17 An example of operation S280 is described.

[0096] Fig.15 According to the embodiment Fig.14 Flow chart of an example of operation S260 and operation S280. Fig.14 As stated in Fig.15 In operation S260', the fragment to be interpolated may be determined, and in operation S280', the fragment to be interpolated may be calculated based on the colored peripheral fragments, for example, the colored fragments adjacent thereto, that is, the uncolored fragments. Fig.15 As shown, operation S260' may include multiple operations, for example, operations S261, S262, and S263, and operation S280' may include operation S281. In some embodiments, Fig.15 The method can be used Figure 5 The graphics processing system 50 is executed and will now refer to Figure 5 and Fig.14 describe Fig.15 Embodiment of the invention.

[0097] In operation S261, a first fragment map MAP1 is generated from the initial fragment map MAP0. For example, the map manager 52_2 may generate the initial fragment map MAP0 based on a preset mode and / or depth data, for example, edge detection. In addition, the map manager 52_2 may receive intermediate rendering data I_RND generated by rendering performed using the shader 52_4 according to the initial fragment map MAP0, and may generate the first fragment map MAP1 from the initial fragment map MAP0 based on the intermediate rendering data I_RND. In some embodiments, the first fragment map MAP1 may be generated by adding fragments corresponding to edges detected from the intermediate rendering data I_RND.

[0098] In operation S262, a second segment map MAP2 may be generated from the first segment map MAP1. For example, the map manager 52_2 may detect edges from the image included in the G buffer G_BUF in segments not in the first segment map MAP1, and may generate the second segment map MAP2 by adding segments corresponding to the detected edges. Fig.16 An example of operation S262 is described.

[0099] In operation S263, fragments may be shaded according to the second fragment map MAP2. For example, the shader 52_4 may receive the second fragment map MAP2 from the map manager 52_2 and shade the fragments included in the second fragment map MAP2. Thus, final rendering data F_RND may be generated that may include shaded fragments and unshaded fragments.

[0100] In operation S281, the fragments to be interpolated may be identified according to the second fragment map MAP2. For example, the interpolation filter 52_6 may receive the second fragment map MAP2 from the shader 52_4 or the map manager 52_2, and identify the fragments to be interpolated based on the second fragment map MAP2. Next, the identified fragments to be interpolated may be calculated from the shaded peripheral fragments.

[0101] Fig.16 According to the embodiment Fig.15 Flow chart of an example of operation S262. Specifically, Fig.16 Operation S262' indicates an operation that may be performed on all segments or may be performed only on segments that are not included in the first segment map MAP1. Fig.15 As stated in Fig.16 In operation S262', a second segment map MAP2 may be generated from the first segment map MAP1. Figure 5 describe Fig.16 Embodiment of the invention.

[0102] In operation S262_1, an image included in the G buffer G_BUF may be obtained. The G buffer G_BUF may include at least one image, and one of the images included in the G buffer G_BUF may be obtained. In operation S262_3, edge detection may be performed on the image obtained in operation S262_1. When an edge is detected, operation S262_9 may be performed subsequently. When an edge is not detected, operation S262_5 may be performed subsequently.

[0103] When an edge is detected, in operation S262_9, the segment may be added to the second segment map MAP2. For example, the segment corresponding to the detected edge may have a value different from that of the segments adjacent thereto. Therefore, the segment may be added to the second segment map MAP2.

[0104] On the other hand, when no edge is detected, it may be determined in operation S262_5 whether there is a next image in the G buffer G_BUF. When there is a next image, in operation S262_1, the next image included in the G buffer G_BUF may be obtained. On the other hand, when there is no next image, operation S262_7 may be subsequently performed.

[0105] In operation S262_7, when operation S262' is performed for all fragments, the fragment may be removed from the second fragment map MAP2. That is, the fragment in the first fragment map MAP1 corresponding to the area from which no edge is detected in all images included in the G buffer G_BUF may be removed from the second fragment map MAP2. Therefore, the fragment may be classified as a fragment to be interpolated.

[0106] Fig.17 According to the embodiment Fig.14 Flow chart of an example of operation S260 and operation S280. Fig.14 As stated in Fig.17 In operation S260 ″, the fragment to be interpolated may be determined, and in operation S280 ″, the interpolation of the fragment may be calculated based on the colored peripheral fragments. Fig.17 As shown, operation S260" may include operation S264 and operation S265, and operation S280" may include operation S282 and operation S283. In some embodiments, Fig.17 The method can be used Figure 8 The graphics processing system 80 is executed and will now refer to Figure 8 and Fig.14 describe Fig.17 Embodiment of the invention.

[0107] In operation S264, the final rendering data F_RND may be generated. For example, the shader 82_4 may obtain an image included in the G buffer G_BUF, that is, the G buffer image G_IMG, via the texture unit 82_3, and detect an edge from the G buffer image G_IMG. The shader 82_4 may color pixels corresponding to the detected edges and discard pixels from areas from which no edges are detected. Therefore, the image included in the final rendering data F_RND may include invalid pixels. Next, in operation S265, the fragment to be interpolated may be determined. For example, the shader 82_4 may determine pixels corresponding to areas from which no edges are detected in all images included in the G buffer G_BUF as pixels to be interpolated. In some embodiments, the shader 82_4 may generate a discard flag D_FLG indicating the pixel to be interpolated.

[0108] In operation S282, the images may overlap. For example, the interpolation filter 82_6 may overlap the image of the interpolation filter 82_6 and the image of the final rendering data F_RND, and thus, as described above with reference to Fig.11 As described above, an overlapping image may be generated. Next, pixels may be calculated based on the colored peripheral pixels in operation S283. For example, the interpolation filter 82_6 may calculate invalid pixels from peripheral pixels having valid values ​​by using any interpolation method, and as a result, a final image F_IMG may be generated.

[0109] Fig.18 is a flowchart of a graphics processing method according to an embodiment. For example, Fig.12 The graphics processing system 120 performs Fig.18 method. Fig.14 An example comparison can be found in Fig.18 In the following, we will refer to Fig.12 describe Fig.18 of the embodiments, and in reference to Fig.18 References will be omitted in the description Fig.14 Provide a description of the details.

[0110] In operation S320, similar to Fig.14 In operation S220, a G buffer G_BUF may be generated. In addition, in operation S340, similar to Fig.14 In operation S240, intermediate rendering may be performed. Thus, a G buffer G_BUF and intermediate rendering data I_RND may be generated.

[0111] In operation S360, the intermediate rendering data I_RND and the G buffer G_BUF may be provided to the ANN. Fig.12 As described above, the ANN may be included in the shader 122_4 and may be trained to output a final image F_IMG from the image of the G buffer G_BUF and the image of the intermediate rendering data I_RND. In some embodiments, the ANN may be trained to output not only the image of the G buffer G_BUF and the image of the intermediate rendering data I_RND but also the geometric shape data used in the intermediate rendering (e.g., Fig.13 The final image F_IMG is outputted based on the geometry data D130 of the ANN, and thus, the shader 122_4 may further provide the geometry data to the ANN. Next, the final image F_IMG may be obtained from the ANN in operation S380.

[0112] One or more embodiments provide a graphics processor, a graphics processing system, and a graphics processing method in which fragments to be shaded are accurately detected to improve final image quality via interpolation.

[0113] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and interpreted in a general and descriptive sense only and not for purposes of limitation. In some cases, it will be apparent to those skilled in the art upon filing this application that features, characteristics, and / or elements described with respect to a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described with respect to other embodiments, unless otherwise specifically indicated. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the spirit and scope of the invention as set forth in the claims below.

Claims

1. A graphics processing system, comprising: A graphics processor, wherein the graphics processor generates a geometry buffer G-buffer by performing a first pass of deferred shading; and Memory, storage G buffer, Wherein, the graphics processor comprises: A graph manager that generates a first fragment graph indicating fragments to be shaded and generates a second fragment graph by adding or removing fragments to or from the first fragment graph based on a G-buffer; and A shader that colors the fragment according to the second fragment map, Wherein, the graph manager: generating an initial fragment graph; and The first fragment map is generated by adding fragments to or removing fragments from the initial fragment map based on intermediate rendering data obtained by rendering based on the initial fragment map.

2. The graphics processing system according to claim 1, wherein: The map manager generates an initial segment map based on edges and / or predefined patterns detected from the depth image.

3. The graphics processing system according to claim 1, wherein: The graph manager generates a first fragment graph based on edges detected from shaded fragments included in the intermediate rendering data.

4. The graphics processing system according to claim 1, wherein: The map manager adds fragments corresponding to edges detected from the image included in the G-buffer but not present in the first fragment map to the second fragment map.

5. The graphics processing system according to claim 1, wherein: The map manager removes, from the second fragment map, fragments in the first fragment map corresponding to regions in which no edge is detected in all images included in the G-buffer.

6. The graphics processing system of claim 1, wherein: memory, storing final rendering data including fragments colored using shaders, and The graphics processing system further includes an interpolation filter that obtains final rendering data by interpolating shading for unshaded fragments based on adjacent shaded fragments in the memory to generate a final image including the shaded fragments.

7. The graphics processing system according to claim 1, wherein: The graph manager further includes an edge detector that detects edges from the image.

8. A graphics processing system, comprising: A graphics processor, wherein the graphics processor generates a geometry buffer G-buffer by performing a first pass of deferred shading; and Memory, storage G buffer, Wherein, the graphics processor includes a shader and an interpolation filter, The shader generates intermediate rendering data by shading fragments according to an initial fragment map indicating fragments to be shaded, generates final rendering data according to a second fragment map, and provides the intermediate rendering data and the final rendering data to an interpolation filter, The interpolation filter generates a final image including colored pixels by interpolating for coloring of uncolored pixels based on neighboring colored pixels in the intermediate rendering data and the final rendering data, The graphics processor generates a first fragment map by adding or removing fragments from an initial fragment map based on the intermediate rendering data, and generates a second fragment map by adding or removing fragments from the first fragment map based on the G-buffer, The shader shades fragments according to the second fragment map.

9. The graphics processing system of claim 8, wherein: The initial segment map is generated based on edges and / or predefined patterns detected from the depth image.

10. The graphics processing system of claim 8, wherein: The shader shades fragments corresponding to edges detected from an image included in the G-buffer.

11. The graphics processing system according to claim 8, wherein: The shader does not shade fragments corresponding to areas where no edges are detected in all images included in the G-buffer.

12. The graphics processing system of claim 8, wherein: The shader selectively shades fragments via pixel discarding.

13. The graphics processing system of claim 8, wherein: The interpolation filter: Overlaying images included in the intermediate rendering data and the final rendering data; and Interpolation of the colored pixels based on neighboring colored pixels in the overlapping images is used for coloring of the uncolored pixels.

14. A graphics processing system comprising: A graphics processor, wherein the graphics processor generates a geometry buffer G-buffer by performing a first pass of deferred shading; and Memory, storage G buffer, The graphics processor includes a shader that generates intermediate rendering data by shading fragments according to a fragment map indicating the fragments to be shaded and generates a final image by providing the intermediate rendering data and a G-buffer stored in a memory to an artificial neural network, wherein the artificial neural network is trained to output the final image from an image of the G-buffer and an image of the intermediate rendering data.

15. The graphics processing system of claim 14, wherein: The shader includes an artificial neural network.

16. The graphics processing system of claim 14, wherein: The segment map is generated based on edges and / or predefined patterns detected from the depth image.

17. A graphics processing method, comprising: Generate a geometry buffer G-buffer by performing a first pass of deferred shading; generating intermediate rendering data by performing rendering according to an initial fragment map indicating fragments to be shaded; Determine the fragment to be interpolated as an unshaded fragment based on the G-buffer and the intermediate rendering data; and The shading of unshaded fragments is interpolated based on neighboring shaded fragments, The segments to be interpolated include: generating a first fragment map by adding fragments to or removing fragments from an initial fragment map based on the intermediate rendering data; generating a second fragment map by adding fragments to the first fragment map or removing fragments from the first fragment map based on the G-buffer; Coloring the fragments according to the second fragment map; and Uncolored fragments are identified according to the second fragment map.

18. The graphics processing method according to claim 17, wherein: Generating the intermediate rendering data includes generating an initial fragment map according to edges detected from the depth image and / or according to a predefined pattern.

19. The graphics processing method according to claim 17, wherein: Generating the first fragment map includes generating an initial fragment map based on edges detected from shaded fragments included in the intermediate rendering data.

20. The graphics processing method according to claim 17, wherein: Generating the second fragment graph includes: adding fragments corresponding to edges detected from an image included in the G-buffer to a second fragment map; and Segments in the first segment map corresponding to regions from which no edge is detected in all images included in the G-buffer are removed from the second segment map.

21. The graphics processing method according to claim 17, wherein: The segments to be interpolated are: generating final rendering data by shading fragments corresponding to edges detected from an image included in the G-buffer; and A fragment corresponding to a region from which no edge is detected in all images included in the G-buffer is determined as a fragment to be interpolated.

22. A graphics processing method, comprising: Generate a geometry buffer G-buffer by performing a first pass of deferred shading; generating intermediate rendering data by performing rendering according to an initial fragment map indicating fragments to be shaded; and Provide intermediate rendering data and G-buffer to the artificial neural network, The artificial neural network is trained to output a final image from an image of a G-buffer and an image of intermediate rendering data.

23. The graphics processing method according to claim 22, wherein: Generating the intermediate rendering data includes generating an initial fragment map according to edges detected from the depth image and / or according to a predefined pattern.

Images