Nerve coloring

Abstract

This disclosure provides systems, devices, apparatuses, and methods, including computer programs encoded on a storage medium, for nerve coloring. A processor may obtain a set of values indicative of surface attributes of a set of materials associated with graphical content. The processor may provide the set of values as input to a machine learning model, wherein the machine learning model includes learned parameters based on a plurality of values indicative of a plurality of surface properties of the plurality of materials. The processor may obtain, based on the input and the learned parameters, a value associated with the illumination corresponding to a surface attribute of a set of materials associated with the graphical content as an output of the machine learning model. The processor may output an indication of the value.

Description

Cross-reference to related applications

[0001] This application claims the benefit of Indian Patent Application Serial No. 202321063111 entitled “NEURAL SHADING”, filed on 20 September 2023, the entire contents of which are expressly incorporated herein by reference. Technical Field

[0002] This disclosure relates generally to processing systems, and more specifically, to one or more techniques for graphics processing. Background Technology

[0003] Computing devices typically perform graphics and / or display processing (e.g., utilizing a graphics processing unit (GPU), a central processing unit (CPU), a display processor, etc.) to render and display visual content. Such computing devices can include, for example, computer workstations, mobile phones (such as smartphones), embedded systems, personal computers, tablet computers, and video game consoles. A GPU is configured to execute a graphics processing pipeline that includes one or more processing stages that operate together to execute graphics processing commands and output frames. A CPU controls the operation of a GPU by issuing one or more graphics processing commands to it. Modern CPUs are typically capable of executing multiple applications concurrently, each of which may require the use of a GPU during execution. A display processor can be configured to convert digital information received from the CPU into analog values ​​and can issue commands to a display panel to display visual content. Devices that provide content for visual presentation on a display can utilize a CPU, GPU, and / or display processor.

[0004] Current shading techniques are based on evaluating the analytical bidirectional reflectance distribution (BRDF) function. Improved techniques for shading are needed. Summary of the Invention

[0005] The following is a simplified summary of one or more aspects to provide a basic understanding of these aspects. This summary is not a broad overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to describe the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0006] In one aspect of this disclosure, a method, a computer-readable medium, and an apparatus are provided. The apparatus includes: a memory; and a processor coupled to the memory, and configured, based on information stored in the memory, to: obtain a set of values ​​indicating a set of surface properties associated with a set of materials related to graphical content; provide the set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials; obtain, based on the input and the learned parameters, a value associated with lighting corresponding to the surface properties of the set of materials associated with the graphical content, as output of the machine learning model; and output an indication of the value.

[0007] To achieve the foregoing and related objectives, one or more aspects include the features fully described below and specifically pointed out in the claims. The following description and drawings set forth some exemplary features of one or more aspects in detail. However, these features indicate only some of the various ways in which the principles of the various aspects may be employed, and this description is intended to include all such aspects and their equivalents. Attached Figure Description

[0008] Figure 1 This is a block diagram illustrating an example of a system for generating content based on one or more techniques of this disclosure.

[0009] Figure 2 Example graphics processors (e.g., graphics processing units (GPUs)) according to one or more technologies according to this disclosure are illustrated.

[0010] Figure 3 Example images or surfaces are illustrated according to one or more techniques of this disclosure.

[0011] Figure 4 This is a diagram illustrating an example aspect of one or more techniques according to this disclosure related to the bidirectional reflection distribution function (BRDF).

[0012] Figure 5 This is a diagram illustrating examples of a first ray tracing pipeline and a second ray tracing pipeline according to one or more techniques of this disclosure.

[0013] Figure 6 This is a diagram illustrating an example aspect of a neural network (NN) according to one or more techniques of this disclosure.

[0014] Figure 7 The diagram illustrates a first example of a neural network based on a discrete data range and a second example of a neural network based on a continuous data range, according to one or more techniques of this disclosure.

[0015] Figure 8The diagram illustrates a first example of using different neural networks for different materials and a second example of using neural networks for different material properties, according to one or more techniques of this disclosure.

[0016] Figure 9 The diagram illustrates a first example of a neural network with latent space coding and a second example of a neural network without latent space coding, according to one or more techniques of this disclosure.

[0017] Figure 10 These are illustrations of a first example of a neural network with explicit layers and a second example of a neural network with implicit layers, based on one or more techniques according to this disclosure.

[0018] Figure 11 This is a call flowchart illustrating example communication between a first graphics processor component and a second graphics processor component according to one or more technologies of this disclosure.

[0019] Figure 12 This is a flowchart of an example method for graphical processing according to one or more techniques of this disclosure.

[0020] Figure 13 This is a flowchart of an example method for graphical processing according to one or more techniques of this disclosure. Detailed Implementation

[0021] Various aspects of the systems, apparatuses, computer program products, and methods will be described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided to make this disclosure comprehensive and complete, and to fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of the systems, apparatuses, computer program products, and methods disclosed herein, whether implemented independently of or in combination with other aspects of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover such apparatuses or methods implemented using structures, functionalities, or structures and functionalities other than or different from the various aspects of the disclosure set forth herein. Any aspect disclosed herein may be embodied by one or more elements of the claims.

[0022] Although various aspects are described herein, many variations and substitutions of these aspects fall within the scope of this disclosure. While some potential benefits and advantages of the aspects of this disclosure are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. Rather, the aspects of this disclosure are intended to be broadly applicable to different wireless technologies, system configurations, processing systems, networks, and transmission protocols, some of which are illustrated by way of example in the accompanying drawings and the description below. The detailed description and drawings are merely illustrative and not limiting of this disclosure, and the scope of this disclosure is defined by the appended claims and their equivalents.

[0023] Several aspects are presented with reference to various apparatuses and methods. These apparatuses and methods are described in detail and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.

[0024] For example, an element, any part of an element, or any combination of elements can be implemented as a “processing system” including one or more processors (which may also be referred to as processing units). Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, system-on-a-chip (SoCs), baseband processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic units, discrete hardware circuits, and other suitable hardware configured to perform the various functionalities described throughout this disclosure. One or more processors in the processing system can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description language, or other names, software is broadly understood to mean instructions, instruction sets, code, code segments, program code, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc.

[0025] The term "application" can refer to software. As described herein, one or more technologies can refer to an application (e.g., software) configured to perform one or more functions. In such examples, the application may be stored in memory (e.g., on-chip memory of a processor, system memory, or any other memory). Hardware described herein, such as a processor, may be configured to execute the application. For example, an application may be described as including code that, when executed by the hardware, causes the hardware to perform one or more technologies described herein. As an example, the hardware may access and execute code accessed from memory to perform one or more technologies described herein. In some examples, components are identified in this disclosure. In such examples, a component may be hardware, software, or a combination thereof. Each component may be a separate component or a subcomponent of a single component.

[0026] In one or more examples described herein, the described functionality can be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality can be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer storage media. Storage media can be any available medium accessible by a computer. By way of example, and not limitation, such computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disc storage devices, magnetic disk storage devices, other magnetic storage devices, combinations of computer-readable media of the types described above, or any other medium that can be used to store computer-executable code in the form of instructions or data structures accessible by a computer.

[0027] As used herein, instances of the term "content" may refer to "graphic content," "image," etc., regardless of whether the term is used as an adjective, noun, or other part of speech. In some examples, as used herein, the term "graphic content" may refer to content produced by one or more processes in a graphics processing pipeline. In other examples, as used herein, the term "graphic content" may refer to content produced by a processing unit configured to perform graphics processing. In yet another example, as used herein, the term "graphic content" may refer to content produced by a graphics processing unit.

[0028] A BRDF (Body-Based Function Definition) can refer to a function that defines how light interacts with a surface (e.g., a surface made of different surfaces such as glass, wood, metal, etc., and / or a surface comprising different surfaces). Accurate modeling of light-surface interactions allows for realistic rendering of materials and various surfaces (diffuse reflection, specular reflection, Fresnel effect, microfacet models, etc.). BRDFs (i.e., BRDF models) can be used in computer graphics rendering to produce physically based appearances in three-dimensional (3D) graphics workloads. For example, BRDFs can be used in applications that use ray tracing, rasterization, or hybrid rendering techniques (such as games), or in other applications that utilize physically based rendering to achieve realistic lighting effects (e.g., production rendering). Rasterization (i.e., the rasterization process) can refer to the process in which, for each pixel in each primitive in a scene, a pixel is shaded if a portion of that pixel is covered by a primitive. Ray tracing (i.e., the ray tracing process) can refer to the process in which rays are generated for each pixel corresponding to a primitive in a scene. If the generated rays are determined to hit or strike a primitive, the pixel is shaded.

[0029] BRDF models can be analytical BRDF models. Analytical BRDF models use several parameters to approximate surface reflectivity. They can also accurately represent a range of real-world materials. Analytical BRDF models can include diffuse and specular models, each with its own color. To determine realistic lighting for a scene, a device (e.g., a shader processor in a graphics processor) can evaluate the analytical BRDF model on a pixel-by-pixel basis. However, evaluating an analytical BRDF model can be computationally complex.

[0030] BRDF models can also be represented as neural networks. However, the neural network representation of BRDF can be trained on tabulated BRDF measurement data, which may include discrete values. Therefore, the ability of the neural network representation of BRDF to generate realistic lighting may be limited. Furthermore, the neural network representation of BRDF may include multiple different neural networks for different types of materials (e.g., a first neural network for glass, a second neural network for metals, etc.), which can lead to increased computational complexity when multiple different neural networks are executed. Additionally, the neural network representation of BRDF may generate latent space encodings of the input data before feeding the input data into the neural network, which may further increase computational complexity. Moreover, the neural network representation of BRDF may utilize explicit layering for different layers of the surface, which may also increase computational complexity.

[0031] This paper describes various techniques related to neural coloring. In an example, a device (e.g., a graphics processor) obtains a set of values ​​indicating a set of surface properties that are associated with a set of materials related to graphical content. Surface properties may refer to how light is reflected or refracted from a surface. Materials may refer to physical objects such as wood, glass, metal, etc. The graphics processor provides the set of values ​​as input to a machine learning model (e.g., a single neural network), where the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of various materials. The graphics processor, based on the input and learned parameters, obtains a value associated with lighting corresponding to the set of surface properties associated with the set of materials related to the graphical content as the output of the machine learning model. The graphics processor outputs an indication of that value.

[0032] Regarding the device that provides a set of surface properties indicating a set of materials associated with graphical content as input to a machine learning model and obtains values ​​associated with lighting corresponding to the surface properties of the set of materials associated with the graphical content as output to the machine learning model, this device can render a scene using realistic lighting in a less computationally complex manner compared to devices utilizing an analytical BRDF model. In the example, the machine learning model can be a single neural network trained on continuous values ​​and not utilizing latent space encoding. Therefore, the training and execution of this single neural network may be less computationally complex compared to other neural network representations of BRDF.

[0033] In one aspect, this paper describes a rendering framework in which the analytical model of a candidate BRDF is replaced by a (small) neural network capable of inferring BRDF values ​​provided by the analytical model. The neural network can be used to predict the D, G, and F components of the analytical BRDF model. G, where F can be calculated through analysis. Calculating F through analysis reduces the size of the training dataset and simplifies data generation. Since the components of F may be simple interpolations, they can be calculated through analysis.

[0034] The examples described herein may relate to the use and functionality of a graphics processing unit (GPU). As used herein, a GPU can be any type of graphics processor, and a graphics processor can be any type of processor designed or configured to process graphical content. For example, a graphics processor or GPU can be a dedicated circuit designed to process graphical content. As an additional example, a graphics processor or GPU can be a general-purpose processor configured to process graphical content.

[0035] Figure 1This is a block diagram illustrating an example content generation system 100 configured to implement one or more technologies of this disclosure. The content generation system 100 includes a device 104. Device 104 may include one or more components or circuitry for performing the various functions described herein. In some examples, one or more components of device 104 may be components of a System-on-a-Chip (SOC). Device 104 may include one or more components configured to perform one or more technologies of this disclosure. In the illustrated example, device 104 may include a processing unit 120, a content encoder / decoder 122, and a system memory 124. In some aspects, device 104 may include multiple components (e.g., a communication interface 126, a transceiver 132, a receiver 128, a transmitter 130, a display processor 127, and one or more displays 131). Display 131 may refer to one or more displays 131. For example, display 131 may include a single display or multiple displays, which may include a first display and a second display. The first display may be a left-eye display, and the second display may be a right-eye display. In some examples, the first and second displays may receive different frames for presentation on the first and second displays. In other examples, the first and second displays may receive the same frames used for rendering on both displays. In yet another example, the results of graphics processing may not be displayed on the device; for example, the first and second displays may not receive any frames used for rendering on either display. Instead, the frames or graphics processing results may be transferred to another device. In some respects, this can be referred to as split rendering.

[0036] Processing unit 120 may include internal memory 121. Processing unit 120 may be configured to perform graphics processing using graphics processing pipeline 107. Content encoder / decoder 122 may include internal memory 123. In some examples, device 104 may include a processor configured to perform one or more display processing techniques on one or more frames generated by processing unit 120, and then display those frames through one or more displays 131. Although the processor in example content generation system 100 is configured as display processor 127, it should be understood that display processor 127 is one example of a processor and other types of processors, controllers, etc., may be used instead of display processor 127. Display processor 127 may be configured to perform display processing. For example, display processor 127 may be configured to perform one or more display processing techniques on one or more frames generated by processing unit 120. One or more displays 131 may be configured to display or otherwise present the frames processed by display processor 127. In some examples, one or more displays 131 may include one or more of the following: liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, projection display device, augmented reality display device, virtual reality display device, head-mounted display, or any other type of display device.

[0037] Memory (such as system memory 124) external to processing unit 120 and content encoder / decoder 122 may be accessible to processing unit 120 and content encoder / decoder 122. For example, processing unit 120 and content encoder / decoder 122 may be configured to read from and / or write to external memory (such as system memory 124). Processing unit 120 may be communicatively coupled to system memory 124 via a bus. In some examples, processing unit 120 and content encoder / decoder 122 may be communicatively coupled to internal memory 121 via the bus or via a different connection.

[0038] Content encoder / decoder 122 can be configured to receive graphic content from any source, such as system memory 124 and / or communication interface 126. System memory 124 can be configured to store received encoded or decoded graphic content. Content encoder / decoder 122 can be configured to receive encoded or decoded graphic content from system memory 124 and / or communication interface 126, for example, in the form of encoded pixel data. Content encoder / decoder 122 can be configured to encode or decode any graphic content.

[0039] Internal memory 121 or system memory 124 may include one or more volatile or non-volatile memories or storage devices. In some examples, internal memory 121 or system memory 124 may include RAM, static random access memory (SRAM), dynamic random access memory (DRAM), erasable programmable ROM (EPROM), EEPROM, flash memory, magnetic data media or optical storage media, or any other type of memory. According to some examples, internal memory 121 or system memory 124 may be a non-transitory storage medium. The term "non-transitory" may indicate that the storage medium is not embodied in a carrier wave or propagating signal. However, the term "non-transitory" should not be construed as meaning that internal memory 121 or system memory 124 is not removable or that its contents are static. For example, system memory 124 may be removed from device 104 and moved to another device. Alternatively, system memory 124 may not be removable from device 104.

[0040] Processing unit 120 may be a CPU, GPU, GPGPU, or any other processing unit configured to perform graphics processing. In some examples, processing unit 120 may be integrated into the motherboard of device 104. In other examples, processing unit 120 may reside on a graphics card mounted in a port on the motherboard of device 104, or may otherwise be incorporated into a peripheral device configured to interoperate with device 104. Processing unit 120 may include one or more processors, such as one or more microprocessors, GPUs, ASICs, FPGAs, arithmetic logic units (ALUs), DSPs, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, processing unit 120 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 121) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors.

[0041] The content encoder / decoder 122 can be any processing unit configured to perform content decoding. In some examples, the content encoder / decoder 122 may be integrated into the motherboard of device 104. The content encoder / decoder 122 may include one or more processors, such as one or more microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), arithmetic logic units (ALUs), digital signal processors (DSPs), video processors, discrete logic components, software, hardware, firmware, other equivalent integrated or discrete logic circuits, or any combination thereof. If the technology is partially implemented in software, the content encoder / decoder 122 may store instructions for software in a suitable non-transitory computer-readable storage medium (e.g., internal memory 123) and may use one or more processors to execute instructions in hardware to perform the technology of this disclosure. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) can be considered as one or more processors.

[0042] In some aspects, the content generation system 100 may include a communication interface 126. The communication interface 126 may include a receiver 128 and a transmitter 130. The receiver 128 may be configured to perform any of the receiving functions described herein with respect to device 104. Additionally, the receiver 128 may be configured to receive information from another device, such as eye or head positioning information, rendering commands, and / or location information. The transmitter 130 may be configured to perform any of the transmitting functions described herein with respect to device 104. For example, the transmitter 130 may be configured to transmit information to another device, which may include a request for content. The receiver 128 and the transmitter 130 may be combined to form a transceiver 132. In such an example, the transceiver 132 may be configured to perform any of the receiving and / or transmitting functions described herein with respect to device 104.

[0043] Refer again Figure 1 In some aspects, processing unit 120 may include a neural shader 198 configured to: obtain a set of values ​​indicating a set of surface properties associated with a set of materials related to graphical content; provide the set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials; obtain, based on the input and the learned parameters, a value associated with lighting corresponding to the surface properties of the set of materials associated with the graphical content, as output of the machine learning model; and output an indication of that value. While the following description may focus on graphics processing, the concepts described herein are applicable to other similar processing techniques. Furthermore, while the following description may focus on neural network representations of BRDFs, the concepts described herein are also applicable to other types of machine learning (ML) model representations of BRDFs.

[0044] Devices such as device 104 can refer to any device, apparatus, or system configured to perform one or more of the technologies described herein. For example, a device can be a server, base station, user equipment, client device, station, access point, computer (such as a personal computer, desktop computer, laptop computer, tablet computer, computer workstation, or mainframe computer), end product, apparatus, telephone, smartphone, server, video game platform or console, handheld device (such as a portable video game device or personal digital assistant (PDA)), wearable computing device (such as a smartwatch, augmented reality device, or virtual reality device), non-wearable device, display or display device, television, set-top box, intermediate network device, digital media player, video streaming device, content streaming device, in-vehicle computer, any mobile device, any device configured to generate graphical content, or any device configured to perform one or more of the technologies described herein. The processes described herein may be described as being performed by a specific component (e.g., GPU), but in other embodiments, other components (e.g., CPU) consistent with the disclosed embodiments may be used to perform them.

[0045] A GPU can process various types of data or data packets within its pipeline. For example, in some aspects, a GPU can process two types of data or data packets, such as context register packets and draw call data. Context register packets can be a set of global state information, such as information about global registers, shaders, or constant data, which can adjust how the graphics context will be processed. For example, a context register packet may include information about the color format. In some aspects of a context register packet, there may be one or more bits indicating which workload belongs to the context register. Additionally, multiple functions or programs can run simultaneously and / or in parallel. For example, a function or program may describe an operation, such as a color mode or color format. Therefore, context registers can define various states of the GPU.

[0046] Context states can be used to determine how individual processing units (e.g., vertex extractors (VFDs), vertex shaders (VSs), shader processors, or geometry processors) operate and / or in which mode they operate. To do this, the GPU uses context registers and programming data. In some aspects, the GPU can generate workloads in the pipeline based on the context register definitions of modes or states, such as vertex or pixel workloads. Certain processing units (e.g., VFDs) can use these states to determine certain functions, such as how to aggregate vertices. Because these modes or states can change, the GPU may need to modify the corresponding context. Additionally, the workload corresponding to a mode or state may follow the changed mode or state.

[0047] Figure 2 Example GPU 200 is illustrated according to one or more technologies according to this disclosure. For example... Figure 2 As shown, GPU 200 includes a command processor (CP) 210, a draw call group 212, a VFD 220, a VS 222, a vertex cache (VPC) 224, a triangle setup engine (TSE) 226, a rasterizer (RAS) 228, a Z-process engine (ZPE) 230, a pixel interpolator (PI) 232, a fragment shader (FS) 234, a rendering backend (RB) 236, an L2 cache (UCHE) 238, and system memory 240. Although Figure 2 The GPU 200 includes processing units 220 to 238, but the GPU 200 may include multiple additional processing units. Additionally, processing units 220 to 238 are merely examples, and the GPU may use any combination or order of processing units in accordance with this disclosure. The GPU 200 also includes a command buffer 250, a context register group 260, and a context state 261.

[0048] like Figure 2 As shown, the GPU can use a CP (e.g., CP 210) or a hardware accelerator to resolve the command buffer into context register groups (e.g., context register group 260) and / or draw call data groups (e.g., draw call group 212). Subsequently, CP 210 can transfer the context register group 260 or the draw call group 212 to a processing unit or block in the GPU via a separate path. Furthermore, the command buffer 250 can alternate between different states of the context registers and draw calls. For example, the command buffer can simultaneously store the following information: the context register of context N, the draw call of context N, the context register of context N+1, and the draw call of context N+1.

[0049] GPUs can render images in a variety of different ways. In some cases, GPUs can render images using direct rendering and / or tiled rendering. In a tiled rendering GPU, an image can be divided or separated into different parts or tiles. After the image is divided, each part or tile can be rendered individually. A tiled rendering GPU can divide a computer graphics image into a grid format, so that each part of the grid (i.e., a tile) is rendered individually. In some aspects of tiled rendering, the image can be divided into different bins or tiles during binning passes. In some aspects, a visibility stream can be constructed during binning passes, where visible primitives or draw calls can be identified. A rendering pass can be performed after a binning pass. In contrast to tiled rendering, direct rendering does not divide a frame into smaller bins or tiles. Instead, in direct rendering, the entire frame is rendered at once (i.e., without binning passes). Additionally, some types of GPUs allow both tiled rendering and direct rendering (e.g., flex rendering).

[0050] In some respects, a GPU can apply the drawing or rendering process to different bins or tiles. For example, a GPU can render a bin and perform all drawing for the primitives or pixels within that bin. During the bin-based rendering process, the rendering target can be located in GPU Internal Memory (GMEM). In some instances, after rendering a bin, the contents of the rendering target can be moved to system memory, and GMEM can be freed to render the next bin. Additionally, a GPU can render another bin and perform drawing for the primitives or pixels within that bin. Thus, in some respects, there may be a small number of bins covering all the drawing on a surface, for example, four bins. Furthermore, a GPU can loop through all the drawing in a bin but perform drawing only for visible drawing calls, i.e., drawing calls that include visible geometry. In some respects, a visibility stream can be generated, for example, in binning passes, to determine the visibility information of each primitive in an image or scene. For example, such a visibility stream can identify whether a primitive is visible. In some respects, this information can be used to remove invisible primitives, such that, for example, invisible primitives are not rendered in a rendering pass. Additionally, at least some primitives that are marked as visible can be rendered in the rendering pass.

[0051] In some aspects of tile rendering, there can be multiple processing stages or passes. For example, rendering can be performed in two passes, such as a binning, visibility, or box visibility pass and a rendering or box rendering pass. During a visibility pass, the GPU can input a rendering workload, record the positions of primitives or triangles, and then determine which primitives or triangles fall into which bins or regions. In some aspects of a visibility pass, the GPU can also identify or mark the visibility of each primitive or triangle in the visibility stream. During a rendering pass, the GPU can input a visibility stream and process one bin or region at a time. In some aspects, the visibility stream can be analyzed to determine which primitives or primitive vertices are visible or invisible. Thus, visible primitives or primitive vertices can be processed. By doing so, the GPU can reduce the unnecessary workload of processing or rendering invisible primitives or triangles.

[0052] In some aspects, certain types of primitive geometry, such as localized geometry, can be processed during visibility passes. Additionally, primitives can be categorized into different bins or regions based on their localization or position. In some instances, categorizing primitives or triangles into different bins can be performed by determining visibility information for those primitives or triangles. For example, the GPU can determine the visibility information for each primitive in each bin or region or write it to, for example, system memory. This visibility information can be used to determine or generate a visibility stream. In a rendering pass, the primitives in each bin can be rendered individually. In these cases, the visibility stream can be retrieved from memory and used to remove primitives that are not visible to that bin.

[0053] Some aspects of a GPU or GPU architecture can provide multiple different options for rendering (e.g., software rendering and hardware rendering). In software rendering, the driver or CPU can process each view... Figure 1 The entire frame geometry is copied each time. Additionally, some different states can change depending on the viewpoint. Therefore, in software rendering, the software can copy the entire workload by changing some states that can be used for rendering for each viewpoint in the image. In some respects, this can lead to increased overhead because the GPU may submit the same workload multiple times for each viewpoint in the image. In hardware rendering, the hardware or GPU may be responsible for copying or processing the geometry for each viewpoint in the image. Therefore, the hardware can manage the copying or processing of primitives or triangles for each viewpoint in the image.

[0054] Figure 3 An image or surface 300 according to one or more techniques of this disclosure is illustrated, including multiple elements divided into multiple boxes. For example... Figure 3As shown, the image or surface 300 includes a region 302, which includes primitives 321, 322, 323, and 324. Primitives 321, 322, 323, and 324 are divided or placed into different bins, such as bins 310, 311, 312, 313, 314, and 315. Figure 3 This example illustrates tile rendering using multiple viewpoints for primitives 321-324. For instance, primitives 321-324 are in a first viewpoint 350 and a second viewpoint 351. Therefore, GPU processing or rendering of an image or surface 300 including region 302 can utilize multi-view or multi-view rendering.

[0055] As indicated in this article, GPUs or graphics processors can use tile rendering architectures to reduce power consumption or save memory bandwidth. As further stated above, this rendering method divides the scene into multiple bins, along with visibility paths that identify the visible triangles within each bin. Therefore, in tile rendering, the entire screen can be divided into multiple bins or tiles. The scene can then be rendered multiple times, for example, once or multiple times for each bin.

[0056] In various aspects of graphics rendering, some graphics applications may render a single target (i.e., the rendering target) once or multiple times. For example, in graphics rendering, the frame buffer on system memory can be updated multiple times. The frame buffer can be part of memory or random access memory (RAM) (e.g., containing bitmaps or storage devices) to help store display data for the GPU. The frame buffer can also be a memory buffer containing a complete frame of data. Additionally, the frame buffer can be a logical buffer. In some aspects, updating the frame buffer can be performed in bin or tile rendering, where, as discussed above, the surface is divided into multiple bins or tiles, and each bin or tile can then be rendered individually. Furthermore, in tile rendering, the frame buffer can be divided into multiple bins or tiles.

[0057] As this article points out, in some respects, such as in boxed or tiled rendering architectures, frame buffers allow data to be repeatedly stored or written to them, for example, when rendering from different types of memory. This can be referred to as unresolving the frame buffers or system memory. For example, when storing or writing to one frame buffer and then switching to another, the data or information on the frame buffer can be resolved from the GMEM at the GPU to system memory, i.e., memory in dual data rate (DDR) RAM or dynamic RAM (DRAM).

[0058] In some respects, system memory can also be system-on-chip (SoC) memory or another chip-based memory, such as on a device or smartphone, used for storing data or information. System memory can also be a physical data storage device shared by the CPU and / or GPU. In some respects, system memory can be, for example, a DRAM chip on a device or smartphone. Therefore, SoC memory can be a chip-based method for storing data.

[0059] In some respects, GMEM can be on-chip memory at the GPU, which can be implemented using static RAM (SRAM). Alternatively, GMEM can be stored on the device (e.g., a smartphone). As indicated herein, data or information can be transferred between system memory or DRAM and GMEM, for example, at the device. In some respects, system memory or DRAM can reside at the CPU or GPU. Furthermore, data can be stored in DDR or DRAM. In some respects, such as in bin or tiled rendering, a small portion of the memory can be stored at the GPU, for example, in GMEM. In some cases, storing data at GMEM may utilize a larger processing workload and / or consume more power compared to storing data at the frame buffer or system memory.

[0060] Figure 4 Figure 400 illustrates an example aspect of a bidirectional reflectance distribution function (BRDF) according to one or more techniques of this disclosure. A BRDF may refer to a function that defines how light is reflected at a surface. The surface may be made of and / or comprise different materials. For example, the material may be glass, wood, metal, etc. BRDFs can be used in computer graphics rendering to produce physically based appearances in three-dimensional (3D) graphics workloads. For example, BRDFs can be used in applications that use ray tracing, rasterization, or hybrid rendering techniques (such as games, e.g., high-end games running on consoles, personal computers (PCs), and mobile devices) or in other applications that employ physically based rendering to achieve realistic lighting effects (e.g., production rendering). As used herein, a BRDF may alternatively be referred to as a BRDF model.

[0061] BRDF models can be either measurement BRDF models or analytical BRDF models. In measurement BRDF models, the BRDF can be measured directly from a real object using a calibrated camera and light source. Such measurements can be performed over a wide range of wavelengths using broadband sources and bandpass filters. An example of a measurement BRDF model could be the Mitsubishi Electric Research Laboratory (MERL) dataset. Analytical BRDF models can use several parameters to approximate surface reflectivity. Analytical BRDF models can also accurately represent a range of real-world materials. Analytical BRDF models can include diffuse and specular models, each with color. The general form of an analytical BRDF model is provided in the following equation (I). (I)

[0062] In equation (I) above, It can represent the normal distribution function of various normals existing on the surface. It can be a geometric shading function. It can represent the change in reflectivity with the angle of incident light. It can refer to the direction vector of the incident light at a point, and The outgoing light direction vector can be pointed to at a point. As illustrated in equation (I), the analytical BRDF model can take the incident light direction vector and the outgoing light direction vector as inputs, and the analytical BRDF model can output the ratio of the reflected radiation emitted along the outgoing light direction vector to the irradiance incident on the surface from the incident light direction vector.

[0063] A measured BRDF model can more accurately represent light reflected from a surface than an analytical BRDF model; however, measuring a BRDF can involve complex measurements and storage, and may be difficult to adjust (i.e., edit, manipulate). An analytical BRDF model may be less accurate than a measured BRDF model; however, analytical models may be easier to store and adjust than measured BRDF models.

[0064] A graphics processor (e.g., a GPU) can evaluate an analytical BRDF on a per-pixel basis as part of the execution of a graphics shader in order to determine the appearance of light reflected from surfaces in a 3D scene. For example, a GPU's shader processor can use a complex combination of arithmetic logic units (ALUs), essential function units (EFUs), and flow control operations to evaluate the analytical BRDF. An EFU can be a hardware unit configured to execute specialized functions (e.g., sin, cos, tan, etc.). However, evaluating an analytical BRDF in this way can be computationally complex.

[0065] This paper describes various techniques related to the (generalized) neural network (NN) representation of analytical BRDF models. The NN representation described herein can be associated with several advantages. First, the NN representation described herein can represent a continuous range of material properties. In contrast, other representations of analytical BRDF models can represent a discrete range of material properties. Second, the NN representation described herein can represent multiple material types using a single NN. In contrast, other representations of analytical BRDF models can represent multiple material types using more than one NN. Third, the NN representation described herein can be generated and utilized without generating latent space encodings of the training data and / or input data. In contrast, other representations of analytical BRDF models can be generated and utilized by first generating latent space encodings of the training data and / or input data. Fourth, the NN representation described herein can support multiple layers (i.e., BRDF lobes) using a single NN. In contrast, other representations of analytical BRDF models can support multiple layers (i.e., BRDF lobes) by utilizing multiple NNs. Furthermore, since the NN representation described herein can be a single NN, the evaluation of analytical BRDF can be offloaded from the GPU shader to a dedicated hardware accelerator / unit (e.g., a neural processor) designed for the NN. Unloading the evaluation of analytical BRDF in this way can replace complex ALU / EFU / flow control operations (performed on the GPU) with simpler matrix multiplication / dot product operations (performed on the GPU), thereby freeing up GPU loops for other tasks.

[0066] In the first example 402, the graphics processing pipeline 107 of the processing unit 120 of device 104 may include an analytical BRDF model 404 or be associated with such an analytical BRDF model. For example, the memory (or other memory) of the graphics processing pipeline 107 may store the analytical BRDF model 404. In this example, the analytical BRDF model 404 may be or include the analytical BRDF model described above. In this example, the analytical BRDF model 404 may be associated with equation (I) above. The graphics processing pipeline 107 may include a shader processor 406. As described above, the shader processor 406 may perform ALU / EFU / flow control operations to evaluate the analytical BRDF model 404. Performing ALU / EFU / flow control operations may be computationally complex.

[0067] In the second example 408, the graphics processing pipeline 107 of the processing unit 120 of device 104 may include a BRDF machine learning (ML) model 410 or associated with such a BRDF ML model. For example, the memory (or other memory) of the graphics processing pipeline 107 may store the BRDF ML model 410. The BRDF ML model 410 may include learned parameters 412 or associated with such learned parameters, wherein the learned parameters 412 may be based on training data 414. More specifically, the learned parameters 412 may be values ​​influenced by the training data 414. The training data 414 may include analytical BRDF values ​​415. The analytical BRDF values ​​415 may include continuous values ​​representing material surface properties of different materials and values ​​corresponding to the continuous values. The graphics processing pipeline 107 may include a hardware accelerator 416 (e.g., a neural processor). The hardware accelerator 416 may execute the BRDF ML model 410.

[0068] In the example, the BRDF ML model 410 may be or include the NN representation described above, and the learned parameters 412 may be weights corresponding to the connections between neurons in different layers of the NN representation. The weights may be values ​​associated with the connections between neurons in different layers of the NN. The weights may be influenced by the analytical BRDF value 415. In the example, the BRDF ML model 410 may be a single ML model. In the example, the BRDF ML model 410 may represent a continuous range of material properties. More specifically, the BRDF ML model 410 may be trained / generated based on a first continuous value (rather than discrete values), and at inference, the BRDF ML model 410 may take a second continuous value (rather than discrete values) as input. In the example, the BRDF ML model 410 may use a single ML model to represent multiple types of materials. In the example, the BRDF ML model 410 may be generated and utilized without generating latent space encodings for the training data and / or input data. In the example, BRDF ML model 410 can support multiple layers (i.e., BRDF lobes) with a single ML model (i.e., BRDF ML model 410 can be a single BRDF ML model).

[0069] BRDF ML model 410 can be or includes artificial neural networks (ANN); decision tree learning; convolutional neural networks (CNN); deep learning architectures in which the output of a first layer of neurons becomes the input of a second layer of neurons, etc.; support vector machines (SVM), for example, which include a separating hyperplane (e.g., a decision boundary) for classifying data; regression analysis; Bayesian networks; genetic algorithms; deep convolutional networks (DCN) configured with additional pooling and normalization layers; and deep belief networks (DBN).

[0070] Machine learning models (such as artificial neural networks (ANNs)) may comprise a set of interconnected artificial neurons (e.g., neuron models) and may be computing devices or represent methods to be performed by computing devices. The connections in a neuron model can be modeled as weights. Machine learning models can be trained via datasets to provide predictive models, adaptive control, and other applications. The model can adapt based on external or internal information processed by the machine learning model. Machine learning can provide nonlinear statistical data models or decision-making and can model complex relationships between input data and output information.

[0071] Machine learning models can include multiple layers and / or operations, which can be formed by cascading one or more of the cited operations. Examples of operations that may be involved include: extraction of various features of the data, convolution operations, fully connected operations that can be activated or deactivated, compression, decompression, quantization, flattening, etc. The term "layer" in a machine learning model can be used to represent operations on the input data. For example, convolutional layers, fully connected layers, etc., can be used to refer to associated operations on the data input into the layer. Convolution A × B Operation refers to combining multiple input features A Converted into multiple output features B The operation involves combining neighboring coefficients in a dimension. "Kernel size" refers to the number of neighboring coefficients combined in that dimension. "Weights" can be used to represent one or more coefficients used in operations across various rows and / or columns of input data in each layer. For example, a fully connected layer operation might have an output... y The output is at least partially based on the input matrix. x and weight A The product of (which can be matrices) and the bias value B The weights are determined by the sum of (which can be matrices). The term "weight" in this text is generally used to refer to both weights and biases. Weights and biases are examples of parameters trained on a machine learning model. Different layers of a machine learning model can be trained individually.

[0072] Machine learning models can include various connectivity patterns, such as any of feedforward networks, hierarchical structures, recursive architectures, feedback connections, etc. Connections between layers of a neural network can be fully connected or locally connected. In a fully connected neural network, neurons in the first layer can pass their outputs to every neuron in the second layer, and every neuron in the second layer can receive inputs from every neuron in the first layer. In a locally connected network, neurons in the first layer can connect to a limited number of neurons in the second layer. In some aspects, convolutional networks can be locally connected and configured using shared connection strengths associated with the inputs of each neuron in the second layer. Locally connected layers of a network can be configured such that each neuron in the layer has the same or similar connectivity pattern but different connection strengths.

[0073] Machine learning models, or neural networks, can be trained. For example, a machine learning model can be trained based on supervised learning. During training, the machine learning model is presented with the inputs it uses to compute to produce the output. The actual output can be compared to the target output, and the difference can be used to adjust the parameters of the machine learning model, such as weights and biases, to provide an output that is closer to the target output. Before training, the output may be incorrect or less accurate, and the error or difference between the actual output and the target output can be calculated. The weights of the machine learning model can then be adjusted so that the output is more closely aligned with the target. To adjust the weights, the learning algorithm can compute the gradient vector of the weights. The gradient indicates the amount by which the error will increase or decrease with slight adjustments to the weights. At the top layers, the gradient corresponds directly to the value of the weights connecting the activated neurons in the penultimate layer to the neurons in the output layer. In lower layers, the gradient may depend on the values ​​of the weights and the error gradient computed in the higher layers. The weights can then be adjusted to reduce the error or move the output closer to the target. This method of adjusting the weights can be called backpropagation through the neural network. The process can continue until the achievable error rate stops decreasing, or until the error rate has reached the target level.

[0074] Figure 5 Figure 500 illustrates an example of a first ray tracing pipeline 502 and a second ray tracing pipeline 504 according to one or more techniques of this disclosure. In the example, the first ray tracing pipeline 502 may correspond to a first example 402, and the second ray tracing pipeline 504 may correspond to a second example 408.

[0075] In the first ray tracing pipeline 502, at 506, a device (e.g., GPU, device 104, etc.) can generate rays associated with the scene (i.e., ray-scene intersections), meaning the device can project rays from a camera viewing the scene. At 508, the device can determine rays that intersect with objects in the scene. At 510, the device can calculate direct lighting based on ray-scene intersections. At 512, the device can calculate indirect lighting based on ray-scene intersections. The device can generate additional rays based on the calculated indirect lighting (e.g., returning to 506). At 514, the device can evaluate an analytical BRDF model (e.g., BRDF=D) based on the calculated direct lighting and / or the calculated indirect lighting. G F). For example, the device can evaluate an analytical BRDF model 404. The device can perform additional processing on the evaluation output to obtain a final pixel value 516, where the final pixel value 516 may indicate the final pixel color. A pixel may refer to the smallest addressable element on a display device or the smallest addressable element in an image. A pixel value may refer to a value that determines the appearance of a pixel.

[0076] In the second ray tracing pipeline 504, at 518, a device (e.g., GPU, device 104, etc.) can generate rays associated with the scene (i.e., ray-scene intersections), meaning the device can project rays from a camera viewing the scene. At 520, the device can determine rays that intersect with objects in the scene. At 522, the device can calculate direct lighting based on ray-scene intersections. At 524, the device can calculate indirect lighting based on ray-scene intersections. The device can generate additional rays based on the calculated indirect lighting (e.g., returning to 518). At 526, the device can evaluate a BRDF ML model based on the calculated direct lighting and / or the calculated indirect lighting (e.g., BRDF = model.predict()). For example, the device can execute a BRDF ML model 410. The device can perform additional processing on the evaluated output to obtain a final pixel value 528, where the final pixel value 528 indicates the final pixel color.

[0077] Figure 6 This is a diagram 600 illustrating an example aspect of a neural network (NN) 602 according to one or more techniques of the present disclosure. The aspects described herein may correspond to... Figure 5 526 in the example. In the example, NN 602 can be or include BRDF ML model 410. NN 602 can be a single NN (i.e., a single NN). NN can refer to a network of neurons or nodes.

[0078] NN 602 can be a fully fused (i.e., fully connected) multilayer perceptron (MLP) ML model with three hidden layers (e.g., first hidden layer 604, second hidden layer 606, and third hidden layer 608). Each of the three hidden layers may include 64 neurons with rectified linear unit (ReLU) activation functions. Each neuron may have associated weights that are affected by the training process. Neurons may also be referred to as nodes. NN 602 may include an input layer of size 8 (… Figure 6 (Not depicted in the text). An input layer may refer to a layer in the NN that takes one or more input values. NN 602 may include an output layer 610 with two neurons having exponential activation functions. A device (e.g., device 104, another device (such as a server), etc.) may use a loss function (such as the mean absolute log loss function) to train NN 602. The device (e.g., device 104, another device (such as a server), etc.) may use the Adam optimizer or a stochastic gradient descent process to optimize NN 602.

[0079] NN 602 can take a set of values ​​for surface properties that indicate a set of materials associated with graphic content as input. In the example, the set of values ​​may include varnish coating roughness 612, roughness 614, D... i 616 and H 618. Based on the inputs and weights associated with the hidden layers, the output layer 610 can output the primary mirror value 620 (D). G) and clear coat mirror value 622 (D) G F). The output layer can refer to a layer in the NN that outputs one or more values, where the one or more values ​​are based on the input values ​​and weights of the NN. NN 602 can combine the primary specular value 620 and the varnish coating specular value 622 to obtain a final BRDF value 624. The final BRDF value 624 can be associated with lighting corresponding to one or more surface properties. In the example, a set of rendering equations can be evaluated based on the final BRDF value 624. The radiance value of a pixel on the display panel can be obtained based on the evaluated set of rendering equations. The radiance value can refer to a value indicating the radiant flux emitted, reflected, transmitted, or received per unit solid angle per unit projected area of ​​a given surface. The radiance value can be used to obtain the final value of a pixel.

[0080] Figure 7 Figure 700 illustrates a first example 702 of a neural network based on a discrete data range and a second example 704 of a neural network based on a continuous data range, according to one or more techniques of this disclosure. The second example 704 may correspond to the second example 408, the second ray tracing pipeline 504, and / or the neural network 602.

[0081] In the first example 702, the neural network (NN) can be trained based on a first discrete range (i.e., a first discrete value) of the material properties. Furthermore, during inference, a second discrete range (i.e., a second discrete value) of the material properties can be provided to the NN. A discrete value can refer to a variable that takes different values ​​within a given range, where a finite number of values ​​may exist within that range. In the first example 702, the device can obtain input data 706. Input data 706 may include tabulated BRDF measurement data 708. For example, tabulated BRDF measurement data 708 may be or include a MERL database of materials. Tabulated BRDF measurement data 708 may include different datasets 710 for different materials. In the example, roughness within tabulated BRDF measurement data 708 may be represented as "0.3", and metallicity may be represented as "0.5". The device can perform NN training 712 based on input data 706. The device can obtain a trained NN 714 based on NN training 712.

[0082] In the second example 704, the neural network (NN) can be trained based on a first continuous range of material properties (i.e., first continuous values). Furthermore, during inference, a second continuous range of material properties (i.e., second discrete values) can be provided to the NN. Continuous values ​​can refer to variables that can take any value within a given range, where an infinite number of values ​​may exist within that range. In the second example 704, a device (e.g., device 104, another device, etc.) can obtain input data 716. Input data 716 may include analytical BRDF values ​​718. Analytical BRDF values ​​718 may be associated with analytical BRDFs. In the example, analytical BRDF values ​​718 may be included in a dataset 720 (i.e., a single dataset) for a wide variety of materials (e.g., different types of material surfaces). In the example, input data 716 may include continuous roughness values ​​that can take any value ranging from zero to one. The device can perform NN training 722 based on input data 706. The device can obtain a trained NN 724 based on NN training 712. Compared to the NN 714 trained in the first example 702, the NN 724 trained in the second example 704 may be able to generate values ​​that more accurately depict photorealistic scenes (e.g., values ​​associated with lighting corresponding to surface properties).

[0083] Figure 8 Figure 800 illustrates a first example 802 using different neural networks (NNs) for different materials and a second example 804 using NNs for different material properties, according to one or more techniques of this disclosure. The second example 804 may correspond to the second example 408, the second ray tracing pipeline 504, and / or the NN 602.

[0084] In the first example 802, the device can obtain and have NThe values ​​corresponding to the input scenario for a unique material 806, among which N It is a positive integer greater than 2. The device can provide a subset of values ​​to different neural networks (NNs), such as the first NN 808, the second NN 810, and the Nth NN 812. Each of the different NNs can be trained to output a value that is consistent with... N The device outputs values ​​corresponding to different materials in a unique material 806. For example, a first neural network 808 can be trained to output values ​​for wood, and a second neural network 810 can be trained to output values ​​for glass. The device can combine the outputs of the different neural networks into an output 814 (i.e., the output value).

[0085] In the second example 804, a device (e.g., device 104, another device, etc.) can obtain and have N The device provides a value corresponding to the input scene of a unique material 806. This value can be fed to an NN 816, which can be trained to output a value corresponding to... N The unique material 806 corresponds to values ​​for different materials (e.g., values ​​associated with lighting corresponding to surface properties). For example, NN 816 can be trained to output values ​​for wood and values ​​for glass. NN 816 can be based on... N The output 818 is generated by using the values ​​corresponding to the input scene of a unique material 806 and the weights of NN 816. This is in contrast to the first NN 808, the second NN 810, and the third NN 816 of the first example 802. N Compared to NN 812, the execution of NN 816 in the second example 804 may be less computationally complex.

[0086] Figure 9 Figure 900 illustrates a first example 902 utilizing a neural network with latent space coding and a second example 904 utilizing a neural network without latent space coding, according to one or more techniques of this disclosure. The second example 904 may correspond to the second example 408, the second ray tracing pipeline 504, and / or the neural network 602.

[0087] In the first example 902, the device can obtain values ​​corresponding to input material 906 (e.g., wood, metal, glass, etc.). The device can use an encoder network to generate a latent space code 908 based on these values. The latent space can refer to an abstraction of a multidimensional space that encodes meaningful internal representations of externally observed events. The latent space code can refer to the encoding within the latent space. In the example, if the first and second materials are similar in the real world (e.g., glass and transparent plastic), the latent space code for the first material might be similar to the latent space code for the second material. The device can provide the latent space code 908 as input to a deep neural network 910. In the example, the deep neural network 910 may have more than three hidden layers. The deep neural network 910 may not be suitable for real-time rendering. The device can obtain a BRDF evaluation result 912 based on the weights of the deep neural network 910 and the latent space code 908.

[0088] In the second example 904, a device (e.g., device 104, another device, etc.) can obtain a value corresponding to an input material 906 (e.g., wood, metal, glass, etc.). This device can provide this value as input to a shallow neural network (NN) 914 (e.g., NN 602). In this example, the shallow NN 914 may have three or fewer hidden layers. The device can obtain a BRDF evaluation result 916 (e.g., a value associated with lighting corresponding to the surface properties of the material) based on the weights of the shallow NN 914 and the value corresponding to the input material 906. The execution of the shallow NN 914 in the second example 904 may be computationally less complex compared to the depth NN 910 of the first example 902.

[0089] Figure 10 Figure 1000 illustrates a first example of a neural network (NN) with explicit layers and a second example of a neural network (NN) with implicit layers, according to one or more techniques of this disclosure. The second example 1004 may correspond to the second example 408, the second ray tracing pipeline 504, and / or the NN 602.

[0090] In the first example 1002, the device may utilize different neural networks (NNs) for explicit layers. The device may obtain values ​​corresponding to input material 1006 (e.g., wood). In this example, the wood may have a vinyl coating (i.e., a first layer) and a facing coating (i.e., a second layer). The device may provide values ​​corresponding to input material 1006 as input to a first NN 1008. In this example, the first NN 1008 may be configured to output values ​​corresponding to the first layer. The device may provide values ​​corresponding to input material 1006 as input to a second NN 1010. In this example, the second NN 1010 may be configured to output values ​​corresponding to the second layer. The device may obtain a BRDF evaluation result 1012 based on the outputs of the first NN 1008 and the second NN 1010.

[0091] In the second example 1004, the device may utilize a single neural network (NN) with implicit layering. The device can obtain values ​​corresponding to an input material 1006 (e.g., wood). In this example, the wood may have a vinyl coating (i.e., a first layer) and a veneer coating (i.e., a second layer). The device can provide values ​​corresponding to the input material 1006 as input to an NN 1014 (e.g., NN 602) that includes implicit layering. NN 1014 may include additional input and output features to accommodate the first and second layers. NN 1014 can obtain a BRDF evaluation result 1016 (e.g., values ​​associated with lighting corresponding to surface properties of the material) based on its output. The execution of NN 1014 in the second example 1004 may be computationally less complex compared to the first NN 1008 and the second NN 1010 in the first example 1002.

[0092] Table 1 below illustrates the performance of the NNs described in this paper (e.g., NN 602, trained NN 724, NN 816, shallow NN 914, NN 1014) relative to the analytical BRDF shaders. Table 1: Computation time evaluation for analytical BRDF shaders and NN architectures

[0093] As illustrated in Table 1 above, the described NN can be up to six times faster than analytical BRDF shaders.

[0094] Figure 11 This is a call flowchart 1100 illustrating example communication between a first graphics processor component 1102 and a second graphics processor component 1104 according to one or more technologies of this disclosure. In the example, the first graphics processor component 1102 and the second graphics processor component 1104 may be included in device 104.

[0095] At 1110, the first graphics processing unit 1102 may obtain a set of values ​​indicating surface properties of a set of materials associated with the graphic content. At 1112, the first graphics processing unit 1102 may provide the set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of various materials. At 1114, the first graphics processing unit 1102 may obtain, based on the input and learned parameters, a value associated with lighting corresponding to the surface properties of the set of materials associated with the graphic content, as output of the machine learning model. At 1116, the first graphics processing unit 1102 may output an indication of that value. For example, at 1116A, the first graphics processing unit 1102 may send the indication of that value to the second graphics processing unit 1104.

[0096] In one aspect, at 1108, the first graphics processor component 1102 may train a machine learning model based on multiple consecutive values ​​prior to obtaining the set of values ​​at 1110. In one aspect, at 1106, the first graphics processor component 1102 may obtain the machine learning model after training. In one aspect, at 1118, the first graphics processor component 1102 may perform at least one of a ray tracing process or a rasterization process based on an indication of that value.

[0097] At 1120, the first graphics processor component 1102 may evaluate a set of rendering equations based on this value. At 1122, the first graphics processor component 1102 may obtain the radiance value of a pixel on the display panel based on the evaluated set of rendering equations. In one aspect, the second graphics processor component 1104 may perform 1120 and / or 1122. In the example, the first graphics processor component 1102 (or the second graphics processor component 1104) may utilize the radiance value to obtain the final pixel value of the pixel.

[0098] Figure 12 This is a flowchart 1200 illustrating an example method of graphic processing according to one or more techniques of this disclosure. The method can be performed by, for example, combining... Figures 1 to 11 The method is executed by various devices used in the process, such as devices for graphics processing, GPUs, first graphics processing unit 1102, CPUs, hardware accelerators, devices 104, wireless communication devices, etc. This method can be associated with various advantages, such as reduced rendering time for graphical content. In the example, the method can be executed by a neural shader 198.

[0099] At 1202, the device obtains a set of values ​​for surface properties that indicate a set of materials associated with the graphic content. For example, Figure 11 As shown at 1110, the first graphics processor component 1102 can obtain a set of values ​​indicating a set of surface properties associated with a set of materials in relation to graphic content. In one example, the set of values ​​may include varnish coating roughness 612, roughness 614, D... i 616 and H 618. In another example, the set of values ​​may correspond to having N The input scenario is a unique material 806. In another example, the set of values ​​could correspond to input material 906 or input material 1006. In this example, the material set could include wood, glass, metal, etc. In this example, 1202 could be executed by neural shader 198.

[0100] At 1204, the device provides a set of values ​​as input to a machine learning model, which includes learned parameters based on multiple values ​​indicative of multiple surface properties of various materials. For example, Figure 11As shown at 1108, the first graphics processor component 1102 can provide a set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of various materials. In one example, the machine learning model may be a BRDF ML model 410, the learned parameters may be learned parameters 412, and the multiple values ​​may be included in training data 414. In another example, the machine learning model may be an NN 602, and the learned parameters may be associated with a first hidden layer 604, a second hidden layer 606, and a third hidden layer 608. In another example, the multiple values ​​may be input data 716. In yet another example, the machine learning model may be a trained NN 724, NN 816, a shallow NN 914, or NN 1014. In the example, 1204 may be performed by a neural shader 198.

[0101] At 1206, the device obtains values ​​associated with lighting that correspond to surface properties of a set of materials associated with the graphic content, based on input and learned parameters, as the output of a machine learning model. For example, Figure 11 As shown at 1114, the first graphics processor component 1102 can obtain a value associated with lighting corresponding to a set of surface properties associated with a set of materials associated with the graphics content, based on input and learned parameters, as the output of a machine learning model. In the example, this value can be or include a final BRDF value 624, output 818, BRDF evaluation result 916, or BRDF evaluation result 1016. In the example, this value can be associated with a final pixel value 528. In the example, 1206 can be performed by a neural shader 198.

[0102] At position 1208, the device outputs an indication of that value. For example, Figure 11 As shown at 1116, the first graphics processor component 1102 can output an indication of this value. In the example, 1208 can be executed by the neural shader 198.

[0103] Figure 13 This is a flowchart 1300 illustrating an example method of graphic processing according to one or more techniques of this disclosure. The method can be performed by, for example, combining... Figures 1 to 11 The method is executed by various devices used in the process, such as devices for graphics processing, GPUs, first graphics processor components 1102, CPUs, hardware accelerators, devices 104, wireless communication devices, etc. This method can be associated with various advantages, such as reduced rendering time of graphical content. In the example, the method (including the various aspects detailed below) can be executed by a neural shader 198.

[0104] At 1306, the device obtains a set of values ​​for surface properties that indicate a set of materials associated with the graphic content. For example, Figure 11As shown at 1110, the first graphics processor component 1102 can obtain a set of values ​​indicating a set of surface properties associated with a set of materials in relation to graphic content. In one example, the set of values ​​may include varnish coating roughness 612, roughness 614, D... i 616 and H 618. In another example, the set of values ​​may correspond to having N The input scenario is a unique material 806. In another example, the set of values ​​could correspond to input material 906 or input material 1006. In this example, the material set could include wood, glass, metal, etc. In this example, 1306 could be executed by neural shader 198.

[0105] At 1308, the device provides a set of values ​​as input to a machine learning model, which includes learned parameters based on multiple values ​​indicative of multiple surface properties of various materials. For example, Figure 11 As shown at 1108, the first graphics processor component 1102 can provide a set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of various materials. In one example, the machine learning model may be a BRDF ML model 410, the learned parameters may be learned parameters 412, and the multiple values ​​may be included in training data 414. In another example, the machine learning model may be an NN 602, and the learned parameters may be associated with a first hidden layer 604, a second hidden layer 606, and a third hidden layer 608. In another example, the multiple values ​​may be input data 716. In yet another example, the machine learning model may be a trained NN 724, NN 816, a shallow NN 914, or NN 1014. In this example, 1308 may be executed by a neural shader 198.

[0106] At 1310, the device obtains values ​​associated with lighting that correspond to surface properties of a set of materials associated with the graphic content, based on input and learned parameters, as the output of a machine learning model. For example, Figure 11 As shown at 1114, the first graphics processor component 1102 can obtain a value associated with lighting corresponding to a set of surface properties associated with a set of materials associated with the graphics content, based on input and learned parameters, as the output of a machine learning model. In the example, this value can be or include a final BRDF value 624, output 818, BRDF evaluation result 916, or BRDF evaluation result 1016. In the example, this value can be associated with a final pixel value 528. In the example, 1310 can be performed by a neural shader 198.

[0107] At position 1312, the device outputs an indication of that value. For example, Figure 11As shown at 1116, the first graphics processor component 1102 can output an indication of this value. In the example, 1312 can be executed by the neural shader 198.

[0108] In one aspect, outputting an indication of the value may include: sending an indication of the value to a display panel; or storing the indication of the value in at least one of memory, a buffer, or a cache. For example, outputting an indication of the value at 1116 may include: sending an indication of the value to a display panel; or storing the indication of the value in at least one of memory, a buffer, or a cache. In the example, the display panel may be or include display 131. In the example, the memory, buffer, or cache may be included in the graphics processing pipeline 107.

[0109] In one respect, at 1316, the device can evaluate the set of rendering equations based on this value. For example, Figure 11 As shown at 1120, the first graphics processor component 1102 can evaluate a set of rendering equations based on this value. In the example, 1316 can be performed by the neural shader 198.

[0110] In one aspect, at 1318, the device can obtain the radiance values ​​of pixels on the display panel based on the evaluated set of rendering equations. For example, Figure 11 As shown at 1122, the first graphics processor component 1102 can obtain the radiance values ​​of pixels on the display panel based on the evaluated set of rendering equations. In the example, 1318 can be performed by the neural shader 198.

[0111] In one aspect, the set of values ​​may include a continuous set of values ​​indicating surface properties of a set of materials, and the multiple values ​​may include multiple continuous values ​​indicating multiple surface properties of multiple materials. For example, the foregoing aspect may correspond to the second example 704.

[0112] In one respect, at 1304, the device can train a machine learning model based on multiple consecutive values ​​before the set of values ​​is obtained. For example, Figure 11 As shown at 1108, the first graphics processor component 1102 can train a machine learning model based on multiple consecutive values ​​prior to obtaining the set of values. In the example, the foregoing aspect may correspond to NN training 722. In the example, 1304 may be performed by a neural shader 198.

[0113] In one respect, multiple continuous values ​​may be associated with multiple analytical two-way reflection distribution functions (BRDFs). For example, multiple continuous values ​​may be or include analytical BRDF values ​​415 and / or 718.

[0114] In one aspect, at position 1302, the device can obtain a machine learning model after the machine learning model has been trained. For example, Figure 11 As shown at 1106, the first graphics processor component 1102 can obtain a machine learning model after training the machine learning model. In the example, 1302 can be executed by a neural shader 198.

[0115] In one aspect, at 1314, the device can perform at least one of a ray tracing process or a rasterization process based on an indication of that value. For example, Figure 11 As shown at 1118, the first graphics processor component 1102 may perform at least one of a ray tracing process or a rasterization process based on an instruction to that value. In the example, the foregoing aspect may correspond to a second ray tracing pipeline 504. In the example, 1314 may be performed by a neural shader 198.

[0116] In one aspect, the machine learning model may include a single neural network, and the learned parameters may include multiple weights of the single neural network. For example, the foregoing aspect may correspond to the second example 804 or the second example 1004.

[0117] In one aspect, a single neural network may include a single input layer and a single output layer. For example, the foregoing aspect may correspond to the second example 804 or the second example 1004.

[0118] In one aspect, surface properties may include at least one of metallicity, reflectivity, or roughness. For example, surface properties indicated in a set of values ​​may include at least one of metallicity, reflectivity, or roughness. Metallicity may refer to the degree to which a surface exhibits metallic properties relative to illumination. Reflectivity may refer to the degree to which a surface reflects light. Roughness may refer to the unevenness of the surface's quality.

[0119] In one aspect, the value set may include (1) a first set of values ​​indicating a first surface attribute of a first material associated with graphic content, and (2) a second set of values ​​indicating a second surface attribute of a second material associated with graphic content, wherein providing the value set as input to a machine learning model may include providing the first set of values ​​and the second set of values ​​as input to the machine learning model, and wherein obtaining the value as output of the machine learning model may include obtaining a first value associated with a first illumination corresponding to the first surface attribute and a second value associated with a second illumination corresponding to the second surface attribute. For example, the foregoing aspect may correspond to a second example 804.

[0120] In one aspect, providing a set of values ​​as input to a machine learning model may include providing a set of values ​​as input to a machine learning model without generating a latent space encoding of the set of values. For example, the foregoing aspect may correspond to the second example 904.

[0121] In one respect, the set of values ​​can be associated with a two-way reflectance distribution function (BRDF). For example, the set of values ​​can be associated with an analytical BRDF value of 415 or an analytical BRDF value of 718.

[0122] In one respect, this value can be associated with the primary mirror value and the clear coat mirror value. For example, the primary mirror value could be 620, and the clear coat mirror value could be 622.

[0123] In one aspect, surface properties may correspond to layers of a set of materials associated with graphic content. For example, the aforementioned aspect may correspond to the second example 1004.

[0124] In the configuration, a method or apparatus for graphics processing is provided. The apparatus may be a GPU, a CPU, or some other processor capable of performing graphics processing. In various aspects, the apparatus may be a processing unit 120 within device 104, or some other hardware within device 104 or another device. The apparatus may include components for obtaining a set of values ​​indicating a set of surface properties associated with a set of materials related to graphical content. The apparatus may also include components for providing the set of values ​​as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials. The apparatus may include components for obtaining, based on the input and learned parameters, values ​​associated with lighting corresponding to the set of surface properties associated with the set of materials related to the graphical content, as output of the machine learning model. The apparatus may also include components for outputting an indication of such values. The apparatus may include components for training the machine learning model based on multiple consecutive values ​​prior to obtaining the set of values. The apparatus may include components for obtaining the machine learning model after training. The apparatus may include components for performing at least one of a ray tracing process or a rasterization process based on the indication of such values. The apparatus may also include components for evaluating a set of rendering equations based on such values. The device may also include components for obtaining the radiance values ​​of pixels on the display panel based on the evaluated set of rendering equations.

[0125] It should be understood that the specific order or hierarchy of boxes / steps in the processes, flowcharts, and / or call flowcharts disclosed herein are merely illustrative of example methods. It should be understood that the specific order or hierarchy of boxes / steps in these processes, flowcharts, and / or call flowcharts may be rearranged based on design preferences. Furthermore, some boxes / steps may be combined and / or omitted. Other boxes / steps may also be added. The appended method claims provide the elements of various boxes / steps in an exemplary order, but are not intended to limit one to the given specific order or hierarchy.

[0126] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims, wherein, unless specifically stated otherwise, references to elements in the singular are not intended to mean “one and only one,” but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0127] Unless otherwise specified, the term "some" refers to one or more, and unless otherwise specified in the context, the term "or" may be interpreted as "and / or". Combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiple A, multiple B, or multiple C. Specifically, combinations such as "at least one of A, B, or C", "one or more of A, B, or C", "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, C, or any combination thereof" may be only A, only B, only C, A and B, A and C, B and C, or A and B and C, wherein any such combination may include one or more members of A, B, or C. The various aspects described throughout this disclosure are all structural and functional equivalents known now or hereafter to those skilled in the art, and are expressly incorporated herein by reference and intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly recited in the claims. The terms “module,” “mechanism,” “element,” “device,” etc., cannot replace the word “component.” Therefore, no claim element will be interpreted as a functional component unless the element is explicitly described using the phrase “component for…”. Unless otherwise stated, the phrase “processor” may refer to “any processor in one or more processors” (e.g., one processor in one or more processors, multiple (more than one) processors in one or more processors, or all processors in one or more processors), and the phrase “memory” may refer to “any memory in one or more memories” (e.g., one memory in one or more memories, multiple (more than one) memories in one or more memories, or all memories in one or more memories).

[0128] In one or more examples, the functionality described herein may be implemented in hardware, software, firmware, or any combination thereof. For example, although the term "processing unit" is used throughout this disclosure, such a processing unit may be implemented in hardware, software, firmware, or any combination thereof. If any functionality, processing unit, technique, or other module described herein is implemented in software, then such functionality, processing unit, technique, or other module may be stored on or transmitted on a computer-readable medium as one or more instructions or code.

[0129] Computer-readable media may include computer data storage media and communication media, including any media that facilitates the transfer of computer programs from one place to another. In this way, computer-readable media may generally correspond to: (1) a tangible computer-readable storage medium that is non-transitory; or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to extract instructions, code, and / or data structures for implementing the techniques described in this disclosure. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, compressed optical disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices, or other magnetic storage devices. As used herein, magnetic disks and optical discs include compressed optical discs (CD), laser optical discs, optical discs, digital versatile optical discs (DVD), floppy disks, and Blu-ray discs, wherein magnetic disks typically magnetically copy data, while optical discs optically copy data using lasers. Combinations of the above should also be included within the scope of computer-readable media. Computer program products may include computer-readable media.

[0130] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented by different hardware units. Rather, as described above, various units can be combined in any hardware unit or provided by a collection of interoperable hardware units (including one or more processors as described above) combined with suitable software and / or firmware. Therefore, the term "processor" as used herein can refer to any of the above-described structures or any other structure suitable for implementing the techniques described herein. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0131] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.

[0132] Aspect 1 is a method for graphics processing, the method comprising: obtaining a set of values of surface attributes indicating a set of materials associated with graphic content; providing the set of values as an input to a machine learning model, wherein the machine learning model comprises learned parameters based on a plurality of values of a plurality of surface attributes indicating a plurality of materials; obtaining, based on the input and the learned parameters, a value associated with illumination corresponding to the surface attributes of the set of materials associated with the graphic content, as an output of the machine learning model; and outputting an indication of the value.

[0133] Aspect 2 can be combined with aspect 1, wherein outputting the indication of the value comprises: sending the indication of the value to a display panel; or storing the indication of the value in at least one of a memory, a buffer or a cache.

[0134] Aspect 3 can be combined with any one of aspects 1 to 2, the method further comprising: evaluating a set of rendering equations based on the value; and obtaining a radiance value of pixels on a display panel based on the evaluated set of rendering equations.

[0135] Aspect 4 can be combined with any one of aspects 1 to 3, wherein the set of values comprises a set of continuous values indicating the surface attributes of the set of materials, and wherein the plurality of values comprises a plurality of continuous values indicating the plurality of surface attributes of the plurality of materials.

[0136] Aspect 5 can be combined with aspect 4, the method further comprising: training the machine learning model based on the plurality of continuous values before the obtaining of the set of values.

[0137] Aspect 6 can be combined with aspect 5, wherein the plurality of continuous values are associated with a plurality of analytical bidirectional reflectance distribution functions (BRDFs).

[0138] Aspect 7 can be combined with any one of aspects 5 to 6, the method further comprising: obtaining the machine learning model after the training of the machine learning model.

[0139] Aspect 8 can be combined with any one of aspects 1 to 7, the method further comprising: performing at least one of a ray tracing process or a rasterization process based on the indication of the value.

[0140] Aspect 9 can be combined with any one of aspects 1 to 8, wherein the machine learning model comprises a single neural network, and wherein the learned parameters comprise a plurality of weights of the single neural network.

[0141] Aspect 10 can be combined with aspect 9, wherein the single neural network comprises a single input layer and a single output layer.

[0142] Aspect 11 may be combined with any one of aspects 1 to 10, wherein the surface property includes at least one of metallicity, reflectivity, or roughness.

[0143] Aspect 12 may be combined with any one of aspects 1 to 11, wherein the set of values ​​includes (1) a first set of values ​​indicating a first surface property of a first material associated with the graphic content, and (2) a second set of values ​​indicating a second surface property of a second material associated with the graphic content, wherein providing the set of values ​​as input to the machine learning model includes providing the first set of values ​​and the second set of values ​​as input to the machine learning model, and wherein obtaining the values ​​as output of the machine learning model includes obtaining a first value associated with a first illumination corresponding to the first surface property and a second value associated with a second illumination corresponding to the second surface property.

[0144] Aspect 13 may be combined with any of aspects 1 to 12, wherein providing the set of values ​​as input to the machine learning model includes providing the set of values ​​as input to the machine learning model without generating a latent space encoding of the set of values.

[0145] Aspect 14 may be combined with any of aspects 1 to 13, wherein the set of values ​​is associated with a two-way reflectance distribution function (BRDF).

[0146] Aspect 15 may be combined with any of aspects 1 to 14, wherein the value is associated with the primary mirror value and the clear coat mirror value.

[0147] Aspect 16 may be combined with any one of aspects 1 to 15, wherein the surface properties correspond to a layer of the set of materials associated with the graphic content.

[0148] Aspect 17 is an apparatus for image processing, the apparatus including a memory and a processor coupled to the memory, and the processor being configured to implement the method according to any one of aspects 1 to 16 based on information stored in the memory.

[0149] Aspect 18 may be combined with aspect 17 and includes: the device is a wireless communication device, the wireless communication device including at least one of a transceiver or an antenna coupled to the processor.

[0150] Aspect 19 is an apparatus for graphics processing, the apparatus including components for implementing the method according to any one of aspects 1 to 16.

[0151] Aspect 20 is a computer-readable medium (e.g., a non-transitory computer-readable medium) storing computer-executable code that, when executed by a processor, causes the processor to implement the method according to any one of aspects 1 to 16.

[0152] Various aspects have been described herein. These and other aspects are within the scope of the following claims.

Claims

1. An apparatus for graphics processing, the apparatus comprising: Memory; and A processor, coupled to the memory, and configured based on information stored in the memory, to: Obtain the set of surface property values ​​that indicate a set of materials associated with graphic content; The set of values ​​is provided as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials; The machine learning model outputs values ​​associated with lighting, corresponding to the surface properties of the material set associated with the graphic content, based on the input and the learned parameters; and Output an indication of the value.

2. The apparatus of claim 1, wherein, in order to output the indication of the value, the processor is configured to: Send the instruction for the value to the display panel; or The indication of the value is stored in at least one of the memory, buffer, or cache.

3. The apparatus of claim 1, wherein the processor is further configured to: The set of rendering equations is evaluated based on the values; and The radiance values ​​of pixels on the display panel are obtained based on the evaluated set of rendering equations.

4. The apparatus of claim 1, wherein the set of values ​​comprises a continuous set of values ​​indicating the surface properties of the set of materials, and wherein the plurality of values ​​comprises a plurality of continuous values ​​indicating the plurality of surface properties of the plurality of materials.

5. The apparatus of claim 4, wherein the processor is further configured to: Prior to obtaining the set of values, the machine learning model is trained based on the plurality of consecutive values.

6. The apparatus of claim 5, wherein the plurality of continuous values ​​are associated with a plurality of analytical bidirectional reflectance distribution functions (BRDFs).

7. The apparatus of claim 5, wherein the processor is further configured to: The machine learning model is obtained after the training of the machine learning model.

8. The apparatus of claim 1, wherein the processor is further configured to: At least one of the ray tracing process or the rasterization process is performed based on the indication of the value.

9. The apparatus of claim 1, wherein the machine learning model comprises a single neural network, and wherein the learned parameters comprise a plurality of weights of the single neural network.

10. The apparatus of claim 9, wherein the single neural network comprises a single input layer and a single output layer.

11. The apparatus of claim 1, wherein the surface property includes at least one of metallicity, reflectivity, or roughness.

12. The apparatus of claim 1, wherein the set of values ​​comprises (1) a first set of values ​​indicating a first surface property of a first material associated with the graphic content, and (2) a second set of values ​​indicating a second surface property of a second material associated with the graphic content, wherein, in order to provide the set of values ​​as input to the machine learning model, the processor is configured to provide the first set of values ​​and the second set of values ​​as input to the machine learning model, and wherein, in order to obtain the value associated with the lighting corresponding to the surface property as the output of the machine learning model, the processor is configured to obtain a first value associated with the first lighting corresponding to the first surface property and a second value associated with the second lighting corresponding to the second surface property.

13. The apparatus of claim 1, wherein, in order to provide the set of values ​​as input to the machine learning model, the processor is configured to provide the set of values ​​as input to the machine learning model without configuring the processor to generate latent space encodings of the set of values.

14. The apparatus of claim 1, wherein the set of values ​​is associated with a bidirectional reflectance distribution function (BRDF).

15. The apparatus of claim 1, wherein the value is associated with the primary mirror value and the varnish coating mirror value.

16. The apparatus of claim 1, wherein the surface properties correspond to a layer of the material set associated with the graphic content.

17. The apparatus of claim 1, wherein the apparatus is a wireless communication device, the wireless communication device comprising at least one of a transceiver or an antenna coupled to the processor.

18. A method for image processing, the method comprising: Obtain the set of surface property values ​​that indicate a set of materials associated with graphic content; The set of values ​​is provided as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials; The machine learning model outputs values ​​associated with lighting, corresponding to the surface properties of the material set associated with the graphic content, based on the input and the learned parameters; and Output an indication of the value.

19. The method of claim 18, wherein outputting the indication of the value comprises: Send the instruction for the value to the display panel; or The indication of the value is stored in at least one of a memory, a buffer, or a cache.

20. The method of claim 18, further comprising: The set of rendering equations is evaluated based on these values; as well as The radiance values ​​of pixels on the display panel are obtained based on the evaluated set of rendering equations.

21. The method of claim 18, wherein the set of values ​​comprises a continuous set of values ​​indicating the surface properties of the set of materials, and wherein the plurality of values ​​comprises a plurality of continuous values ​​indicating the plurality of surface properties of the plurality of materials.

22. The method according to claim 21, further comprising: Prior to obtaining the set of values, the machine learning model is trained based on the plurality of consecutive values.

23. The method of claim 22, wherein the plurality of continuous values ​​are associated with a plurality of analytical bidirectional reflection distribution functions (BRDFs).

24. The method according to claim 22, further comprising: The machine learning model is obtained after the training of the machine learning model.

25. The method according to claim 18, further comprising: At least one of the ray tracing process or the rasterization process is performed based on the indication of the value.

26. The method of claim 18, wherein the machine learning model comprises a single neural network, and wherein the learned parameters comprise a plurality of weights of the single neural network.

27. The method of claim 26, wherein the single neural network comprises a single input layer and a single output layer.

28. The method of claim 18, wherein the surface property includes at least one of metallicity, reflectivity, or roughness.

29. The method of claim 18, wherein the set of values ​​comprises (1) a first set of values ​​indicating a first surface property of a first material associated with the graphic content, and (2) a second set of values ​​indicating a second surface property of a second material associated with the graphic content, wherein providing the set of values ​​as input to the machine learning model comprises providing the first set of values ​​and the second set of values ​​as input to the machine learning model, and wherein obtaining the value associated with the lighting corresponding to the surface property as the output of the machine learning model comprises obtaining a first value associated with the first lighting corresponding to the first surface property and a second value associated with the second lighting corresponding to the second surface property.

30. A computer-readable medium storing computer-executable code, which, when executed by a processor, causes the processor to: Obtain the set of surface property values ​​that indicate a set of materials associated with graphic content; The set of values ​​is provided as input to a machine learning model, wherein the machine learning model includes learned parameters based on multiple values ​​indicating multiple surface properties of multiple materials; The machine learning model outputs values ​​associated with lighting, corresponding to the surface properties of the material set associated with the graphic content, based on the input and the learned parameters; and Output an indication of the value.

Images