Precision modulated coloring

Abstract

A method and apparatus for performing precision modulated shading (PMS) with a graphics processing unit (GPU) includes obtaining a shading instruction corresponding to a floating point operand; determining a precision mode applied to the shading instruction among a plurality of precision modes for processing the shading instruction; and truncating the floating-point operand based on the determined precision pattern, and executing the shading instruction using the truncated floating-point operand.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based upon and claims the benefit of priority of U.S. Provisional Patent Application No. 63 / 526,067 filed in the U.S. Patent and Trademark Office on July 11, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates to performing graphics processing, and more particularly, to precision modulated shading (PMS) in a graphics processing pipeline. Background Art

[0004] The floating point format is a computer number format that can be used in computer processing tasks such as graphics processing and artificial intelligence (AI). The single-precision floating point format (also referred to as the FP32 format or float 32 format) is a 32-bit floating point format defined in the Institute of Electrical and Electronics Engineers (IEEE) 754 standard. The FP32 format can be useful because it can allow a relatively large dynamic range to be represented with relatively high precision, but because each floating point number is represented with 32 bits, FP32 operations can require a large amount of processing and memory capacity.

[0005] Recently, other floating point formats (e.g., 16-bit floating point formats called BF16 format or bfloat16 format) have been used to reduce processor and memory requirements in graphics processing and AI operations. However, the precision that can be represented by floating point formats such as BF16 format may be significantly lower than the precision of FP32 format. This reduced precision may be suitable for some operations, but may cause obvious processing errors when used for other operations. Therefore, when a low-precision floating point format such as BF16 format is applied to all instructions in the processing pipeline, the results provided by the processing pipeline (e.g., graphics rendering results) may be significantly damaged compared to the results provided by a more accurate number format such as FP32 format.

[0006] Therefore, there is a need for methods and apparatus that can selectively apply different floating point formats to instructions in a processing pipeline to reduce processing and memory requirements while reducing errors and corruption in processing results. Summary of the invention

[0007] Apparatus and methods for performing precision modulated shading (PMS) are provided.

[0008] Also provided are apparatus and methods for inserting floating point operations with different precisions into appropriate locations in a processing pipeline by identifying code segments (or, for example, instructions) that are sensitive to precision that will affect processing results (e.g., rendering quality).

[0009] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the present embodiments.

[0010] According to one aspect of the present disclosure, a method for performing precision modulated shading (PMS) using a graphics processing unit (GPU) includes: obtaining a shading instruction corresponding to a floating-point operand; determining a precision mode applied to the shading instruction from among multiple precision modes for processing the shading instruction; and based on the determined precision mode, truncating the floating-point operand, and executing the shading instruction using the truncated floating-point operand.

[0011] According to one aspect of the present disclosure, a method for performing precision modulated shading (PMS) using a graphics processing unit (GPU) includes: obtaining a control flow graph including multiple shading instructions; setting the precision mode for the multiple shading instructions to a default precision mode, wherein the default precision mode corresponds to a first precision level; evaluating each of the multiple shading instructions to determine whether to apply a modified precision mode, wherein the modified precision mode corresponds to a second precision level different from the first precision level; and based on the default precision mode being applied to a first shading instruction among the multiple shading instructions, controlling at least one shader processor to perform the following steps: setting a mode register included in the at least one shader processor to a first value corresponding to the default precision mode, truncating a first floating-point operand corresponding to the first shading instruction, and executing the first shading instruction using a computing module included in the at least one shader processor based on the truncated first floating-point operand.

[0012] According to one aspect of the present disclosure, a graphics processing unit (GPU) for performing precision modulated shading (PMS) includes: at least one shader processor, which is configured to: obtain a shading instruction corresponding to a floating-point operand; determine a precision mode applied to the shading instruction from among multiple precision modes for processing the shading instruction; and based on the determined precision mode, truncate the floating-point operand, and execute the shading instruction using the truncated floating-point operand.

[0013] According to one aspect of the present disclosure, a device for performing precision modulated shading (PMS) includes: a graphics processing unit (GPU), which includes at least one shader processor, wherein the at least one shader processor includes a mode register and a computing module; and at least one controller, which is configured to: obtain a control flow graph including multiple shading instructions; set the precision mode for the multiple shading instructions to a default precision mode, wherein the default precision mode corresponds to a first precision level; and evaluate each of the multiple shading instructions to determine whether to apply a modified precision mode, wherein the modified precision mode corresponds to a second precision level different from the first precision level, wherein based on the default precision mode being applied to a first shading instruction among the multiple shading instructions, the at least one controller is also configured to control the at least one shader processor to perform the following operations: set the mode register to a first value corresponding to the default precision mode; truncate a first floating-point operand corresponding to the first shading instruction; and execute the first shading instruction using the computing module based on the truncated first floating-point operand. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings, in which:

[0015] Figure 1 is a block diagram showing a display device according to an embodiment;

[0016] Figure 2 is a block diagram showing a system on chip (SOC) according to an embodiment;

[0017] Figure 3A is a block diagram illustrating a graphics processing unit (GPU) pipeline according to an embodiment;

[0018] Figure 3B is a block diagram illustrating a shader array pipeline according to an embodiment;

[0019] Figure 4 is a diagram showing an example of a floating point number format according to an embodiment;

[0020] Figure 5 is a block diagram illustrating an image processing pipeline according to an embodiment;

[0021] FIG. 6A to FIG. 6C is a flow chart illustrating an example heuristic for performing precision modulated shading (PMS) according to an embodiment;

[0022] Figure 7 is a diagram illustrating a programming language for performing an image processing task according to an embodiment; and

[0023] Figure 8is a flow chart illustrating an example process for executing a PMS according to an embodiment. DETAILED DESCRIPTION

[0024] The advantages and features of the embodiments of the present disclosure and the methods for realizing them will become clearer with reference to the following description in conjunction with the accompanying drawings. However, the embodiments are not limited thereto. In addition, the specific configurations described only in the specific embodiments can be used in other embodiments. Throughout the following description, the same reference numerals can generally refer to the same elements.

[0025] The terms used herein are to describe example embodiments and are not intended to limit the scope of the present disclosure. In this specification, unless otherwise specifically stated in a phrase, the singular also includes the plural. As used herein, "include" and / or "comprise" may refer to the existence of stated elements, steps, operations and / or equipment, and also do not exclude the existence of one or more other elements, steps, operations and / or equipment or the addition of one or more other elements, steps, operations and / or equipment.

[0026] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein may be used with the meanings commonly understood by those of ordinary skill in the art to which the present disclosure belongs. In addition, unless specifically clearly defined, terms defined in commonly used dictionaries should not be overly idealized or over-interpreted.

[0027] In addition, before the following detailed description, the definitions of certain words and phrases used in this article are set forth. The terms "include" and "comprise" and the derivatives of the terms "include" and "comprise" represent including but not limited to. The term "connect" and the derivatives of the term "connect" refer to any direct or indirect communication between two or more components, regardless of whether the two or more components are in physical contact with each other. The terms "send", "receive" and "communication" and the derivatives of the terms "send", "receive" and "communication" include both direct communication and indirect communication. The term "or" is a non-exclusive term meaning "and / or". The term "about" and the derivatives of the term "about" refer to including, being included in, being interconnected with, suggesting, implied in, being connected with, being combined with, communicating with, cooperating with, intervening, being placed side by side, being approximately, being constrained by, having the characteristics of, being related to, etc. The term "controller" refers to any device, system or part thereof that controls at least one operation. Such a controller can be implemented as a combination of hardware or hardware and software and / or firmware. The functionality associated with any particular controller may be centralized or distributed, or local or remote. When the phrase "at least one" is used with a list of items, it means that different combinations of one or more of the listed items may be used, and only one item in the list may be required. For example, "at least one of A, B, and C" includes any of A, B, C, A and B, A and C, B and C, and a combination of A, B, and C.

[0028] In addition, the various functions described below can be implemented or supported by artificial intelligence technology or one or more computer programs, and each of the programs can include computer-readable program code and can be implemented in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, related data or their parts suitable for implementing suitable computer-readable program codes. The term "computer-readable program code" includes any type of computer code, including source code, object code and executable code. The term "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disc (CD), a digital video disc (DVD) or any other type of memory. "Non-transitory" computer-readable media excludes wired, wireless, optical or other communication links that transmit temporary electrical signals or other signals. Non-transitory computer-readable media include media in which data can be permanently stored, and media in which data is stored and can be rewritten later, for example, rewritable optical discs or removable storage devices.

[0029] In various exemplary embodiments described below, a hardware method is described as an example. However, since various exemplary embodiments include techniques using both hardware and software, various exemplary embodiments do not exclude software-based methods.

[0030] In addition, the terms used in the following description are examples for convenience of description. Therefore, the exemplary embodiments are not limited to the terms described below, and other terms with equivalent technical meanings may also be used.

[0031] Figure 1 is a block diagram showing a display device according to an embodiment. Specifically, Figure 1 An embodiment of a display device 100 in which any method or apparatus described in the present disclosure may be integrated is shown. The display device 100 may have any form factor (such as a flat-panel display for a personal computer (PC), a notebook computer, a mobile device, a projector, VR glasses, etc.), and may be based on any imaging technology for displaying a rasterized image 101 using pixels 102 (such as a cathode ray tube (CRT), a digital light projector (DLP), a light emitting diode (LED), a liquid crystal display (LCD), an organic LED (OLED), quantum dots, etc.). An image processor 104 such as a graphics processing unit (GPU) and / or a display driver 103 may process the image into a form that can be displayed on or by the display device 100, and / or convert the image into a form that can be displayed on or by the display device 100. A portion of the image 101 is displayed magnified so that the pixel 102 is visible. Any method or apparatus described herein may be integrated in the display device 100, the image processor 104, and / or the display driver 103 to generate Figure 1 Pixel 102 and / or groups of pixels 102 are shown. In some embodiments, image processor 104 may include a pipeline that may perform processing operations implemented, for example, on integrated circuit 105, which may implement one or more precision modulated shading (PMS) operations and any other embodiments described herein. In some embodiments, integrated circuit 105 may also include display driver 103 and / or any other components that may implement any other functions of display device 100.

[0032] Figure 2 is a block diagram showing a system on chip (SOC) device according to an embodiment. Specifically, Figure 2An embodiment of a SOC 200 including a graphics processing unit (GPU) 203 that can implement PMS operations according to the present disclosure is shown. The SOC 200 may include a central processing unit (CPU) 201, a main memory 202, a GPU 203, and a display driver 206. The GPU 203 may include a pipeline 204 and a memory subsystem 205, which may be, for example, described in the above and below references. Figure 1 3, and the pipeline 204 and the memory subsystem 205 may implement any of the PMS operations described herein, including, for example, those described below with reference to Figures 4 to 8 Those described. Figure 2 The SOC 200 shown may be integrated into a Figure 1 The image display device 100 shown is an image display device.

[0033] Figure 3A is a block diagram showing a GPU pipeline according to an embodiment. Specifically, Figure 3A The pipeline 204 that may be included in the GPU 203 is shown, and the pipeline 204 may include components or stages. The pipeline 204 may perform processing operations such as image processing or graphics processing operations. The pipeline 204 may include one or more clients such as a command processor 2041, a geometry module 2042, a rasterization module 2043, one or more shader arrays 2044 that may include one or more shader modules 321, and a texture module 2045 that may access the memory interface 312 through a memory request 313, however, the embodiment is not limited thereto, and other embodiments may have any other number and / or type of clients. In some embodiments, the memory interface 312 may include one or more buses, arbiters, etc. A software driver that may be included in one or more of the GPU 203 and the CPU 201 or a software driver that may be executed by one or more of the GPU 203 and the CPU 201 may provide input 311 such as commands, drawing, vertices, primitives, etc. to the pipeline 204.

[0034] Figure 3B is a block diagram showing a shader array pipeline according to an embodiment. Figure 3B As shown, the shader array 2044 may include a shader pipeline input module 323, one or more shader modules 321, and one or more shader export modules 322. In an embodiment, the GPU 203 may include a plurality of shader arrays 2044, each of which may be connected to the shader pipeline input module 323. Each of the one or more shader modules 321 may include a single instruction multiple data (SIMD) module 3211 and a mode register 3212.

[0035] In an embodiment, a thread may refer to a minimum sequence of instructions that can be managed independently, and a thread block may refer to a group of threads that can be executed in series or in parallel. A wave or a thread bundle may refer to a group of thread blocks that run simultaneously. In an embodiment, a shader pipeline input module 323 may allocate resources and allocate waves to available wave slots in one or more shader modules 321 for execution. Shader modules 321 may schedule waves to interleave their instructions and control how to execute instructions. In an embodiment, a SIMD module 3211 may process a single instruction on multiple pieces of data (e.g., data corresponding to multiple threads). In an embodiment, a SIMD module 3211 may be a SIMD32 module capable of processing data corresponding to thirty-two threads, but embodiments are not limited thereto. A SIMD module 3211 may execute instructions according to a precision level corresponding to a precision mode indicated by a value stored in a mode register 3212. In an embodiment, a SIMD module 3211 may be referred to as a computing module. In an embodiment, a wave may correspond to at least one of a vertex, a pixel, a primitive, or any other element processed by a GPU 203. When the wave has finished processing, the results of the processing may be exported to the shader export module 322 .

[0036] As described above, floating point formats such as the FP32 format can allow a relatively large dynamic range to be represented with relatively high precision, but can also impose relatively large processing and memory costs. In order to reduce such processing and memory costs, other floating point formats such as the BF16 format can be used. However, these floating point formats can result in reduced precision compared to the FP32 format, which can result in processing errors and failures when used with some operations or instructions performed by the processing pipeline.

[0037] Thus, embodiments are directed to a process for performing precision modulated shading (PMS) that may enable multiple floating point formats to be used by a processing pipeline (eg, pipeline 204 including shader module 321 discussed above).

[0038] Figure 4 is a diagram showing an example of a floating point number format according to an embodiment. A floating point number may be represented using a sign bit S, one or more exponent bits E, and one or more fraction bits F. In an embodiment, the number represented by the fraction bits F may be referred to as a mantissa.

[0039] For example, the binary number "101011.101" can be expressed in scientific notation as "1.01011101×2 5 To represent this example binary number using a floating point format, the sign bit can be used to indicate that the number is positive, the exponent bit E can be used to indicate the bits "101" to represent the exponent "5", and the fraction bit F can be used to indicate the bits "01011101" to the right of the binary point (with an implicit bit "1" to the left of the binary point) to represent the mantissa.

[0040] like Figure 4 As shown, the FP32 format may include one sign bit S, eight exponent bits E, and 23 fraction bits F. The BF16 format may be understood as an FP32 format in which the lower sixteen least significant bits (LSBs) of the mantissa are truncated. Therefore, the BF16 format may include one sign bit and eight exponent bits E, but may only include seven fraction bits F. Because the FP32 format and the BF16 format can express floating-point numbers using the same number of exponent bits E, the dynamic ranges of the two floating-point formats may be the same or similar. However, because the BF16 format uses fewer fraction bits F than the FP32 format, the precision level of the floating-point numbers that can be represented by the BF16 format may be significantly lower than the precision level of the floating-point numbers that can be represented by the FP32 format.

[0041] As described above, embodiments may involve processing for performing PMS that may enable a pipeline, such as pipeline 204, to perform operations using different floating point formats based on the level of precision required for those operations. In embodiments, these other floating point formats may have the same number of sign bits S and exponent bits E as the FP32 format, but may vary the number of fraction bits F used to represent the mantissa. For example, Figure 4 As shown, each of the BF20 format, the BF24 format, and the BF28 format may include a sign bit and eight exponent bits E. Therefore, all of these floating point formats may have the same or similar dynamic range as the FP32 format. However, the BF20 format may include eleven fraction bits F, the BF24 format may include fifteen fraction bits F, and the BF28 format may include nineteen fraction bits F. Therefore, the BF20 format, the BF24 format, and the BF28 format may provide a range of precision levels higher than the precision level provided by the BF16 format, while still providing a reduction in processing and memory requirements relative to the FP32 format. According to an embodiment, the PMS may enable the GPU 203 to apply different precision modes corresponding to different precision levels to various instructions. When executing an instruction corresponding to a specific precision mode, the GPU 203 may truncate the mantissa of the operand corresponding to the instruction so that the operand has a floating point format corresponding to the precision mode.

[0042] Although the embodiments described herein utilize floating point number formats such as the BF16 format, the BF20 format, the BF24 format, the BF28 format, and the FP32 format, the embodiments are not limited thereto and the embodiments may be applied to any other floating point number formats.

[0043] Figure 5 is a block diagram showing an image processing pipeline according to an embodiment. Specifically, Figure 55 shows a pipeline 500 that can represent a logical process flow for performing a processing task such as image processing or graphics processing. Figure 5 As shown, pipeline 500 may include input component 501 , vertex shading 502 , tessellation 503 , geometry shading 504 , rasterization 505 , fragment shading 506 , and color blending 507 .

[0044] In an embodiment, pipeline 500 may correspond to operations performed by elements of GPU 203 (e.g., pipeline 204 and elements included therein). For example, input assembly 501 and tessellation 503 may be performed by geometry module 2042, rasterization 505 may be performed by rasterization module 2043, color blending may be performed by texture module 2045, and vertex shading 502, geometry shading 504, and fragment shading 506 may be performed by one or more shader modules 321.

[0045] According to an embodiment, the PMS may allow floating point operations to be performed at different precision levels by distinguishing between precision-sensitive instructions and precision-insensitive instructions when processing instructions in the pipeline 204 (e.g., by the shader module 321 performing functions such as vertex shading 502, geometry shading 504, and fragment shading 506). The PMS may allow different precision modes to be selected for precision-sensitive instructions and precision-insensitive instructions, thereby allowing precision-sensitive instructions or instruction blocks to be executed using a precision mode corresponding to a higher precision level (e.g., a precision level corresponding to a relatively high precision floating point format such as the FP32 format), and allowing precision-insensitive instructions or instruction blocks to be executed using a precision mode corresponding to a lower precision level (e.g., a precision level corresponding to a relatively low precision floating point format such as the BF16 format).

[0046] In embodiments, by allowing the pipeline 204 to switch between different precision modes corresponding to different floating point formats, the PMS may allow the number of fraction bits F used to represent the mantissa of FP32 operands in instructions executed by the shader module 321 to be dynamically tailored or truncated based on the particular precision mode used by the shader module 321. As a result, embodiments may provide a pipeline 204 capable of performing operations using different floating point formats having different precision levels (e.g., a range of precision levels between a BF16 format and an FP32 format).

[0047] In some embodiments, the precision mode may be selected based on hints that can be applied using different levels of programming languages, which may allow the PMS to be used without modifying user-level applications.

[0048] In embodiments, one or more shader modules 321 may be responsible for a significant portion of the power consumption of the GPU 203. By reducing the amount of computation performed by one or more shader modules 321, embodiments may allow power consumption to be reduced while adding only a small amount of overhead operations to switch precision modes based on the PMS. Thus, embodiments may allow rendering quality to be preserved while reducing processing and memory requirements, thereby saving power by reducing the total amount of computation.

[0049] In an embodiment, such a precision mode to be used by a particular shader module 321 to perform a particular operation may be determined or set based on a value stored in a mode register corresponding to the shader module 321. In an embodiment, the mode register may be included in the shader module 321, however the embodiment is not limited thereto. For example, in some embodiments, the mode register may be included in the memory subsystem 205 or in another part on the GPU 203 or SOC 200. In an embodiment, the mode register may indicate the number of LSB bits of the FP32 operand to be masked. In an embodiment, the mode register value stored in the mode register may be read and written by the shader module 321 using scalar instructions.

[0050] In an embodiment, when PMS is enabled, the mode register value may be set and updated per wave. For example, when an instruction or command is received to initiate a new wave, the mode register corresponding to the wave may be initialized by setting the mode register value to a default value corresponding to a default precision mode. Then, based on receiving an instruction or command to change the precision mode for the wave to a new precision mode, the mode register value may be updated to indicate the new precision mode.

[0051] For example, in some embodiments, the mode register value may be a 3-bit value. Therefore, the default value may be "000" which may indicate a precision mode corresponding to the FP32 format. After the mode register is initialized to store the default value, an instruction or command may be received to change the precision mode according to the operation to be performed in the wave. Based on the changed precision mode, the mode register value may be updated to "001" which may indicate a precision mode corresponding to the BF28 format, "010" which may indicate a precision mode corresponding to the BF24 format, "100" which may indicate a precision mode corresponding to the BF20 format, or "010" which may indicate a precision mode corresponding to the BF16 format. However, the embodiment is not limited thereto, and in an embodiment, any mode register value may be used to represent any precision mode corresponding to any number format. In an embodiment, the updated mode register value may be visible to instructions after the instruction or command for setting the mode register value in programming order, and may not be visible to instructions before the instruction or command for setting the mode register value in programming order.

[0052] In some embodiments, the command or instruction for updating the mode register value may be a special register setting instruction for changing the mode register value only. For example, the special register setting instruction may include an explicit scalar value that can be stored in the mode register. As another example, the special register setting instruction may indicate a scalar register storing the mode register value, and based on receiving the mode register instruction, the mode register value may be retrieved from the scalar register and stored in the mode register. This may allow programmatic determination of the precision mode. For example, the desired precision mode may depend on the calculation result. After performing the calculation, the calculation result may be stored in the scalar register and then retrieved to be stored in the mode register to set the appropriate precision mode. In an embodiment, the special register setting instruction may be referred to as a mode instruction.

[0053] In some embodiments, the command or instruction for updating the mode register value may be an instruction that is sometimes used to change the mode register value or is sometimes used to perform a modification of a different operation. For example, the special register setting instruction may not be executed until all instructions sent to the shader module 321 are completed, and therefore, a set of dependency counters may be required to reach zero before the special register setting instruction can be executed. This can enhance the delay of the shader module 321 and reduce the locality of the cache used to store the operands of the shader module 321, which may cause power and performance issues. However, because only vector instructions may be affected by the precision mode, and other instructions such as scalar instructions and branch instructions may not depend on the precision mode, it is not necessary to wait until all instructions sent to the shader module are completed. Therefore, different instructions included in a given instruction set architecture (ISA) (which are not subject to the wait state penalty of the special register setting instruction) may be conditionally modified to allow it to be used to set the mode register value. For example, unused bits in the modified instruction may be designated as control bits. Based on the control bit having the first value, the modified instruction can be interpreted as a register setting instruction, and based on the control bit having the second value, the modified instruction can be interpreted as the original instruction before modification. For example, the original instruction may include a denormalization instruction or a rounding instruction, but the embodiment is not limited thereto.

[0054] In some embodiments, the control bit may be checked based on determining whether the PMS is enabled or disabled. For example, a configuration file or configuration bits of GPU 203 may include a chicken bit corresponding to the PMS. In an embodiment, the chicken bit may be included in a configuration file corresponding to GPU 203. Based on the chicken bit having a first value, the PMS may be enabled, and GPU 203 may check the control bit before executing the modified instruction. Based on the chicken bit having a second value, the control bit of the modified instruction may be ignored.

[0055] According to embodiments, the behavior of the shader module 321 may be modified in other ways based on the mode register value stored in the mode register. For example, when the mode register value is set to any value other than the default value, the floating point rounding mode may be assumed to be round towards zero, and the denormal portion (denormal) may always be flushed to zero. In embodiments, operations such as fetching and deriving operands (e.g., reading operands from and writing operands to an operand cache) may be performed at a normal level specified by the instruction and may not be affected or modified based on the precision mode.

[0056] In some embodiments, when the precision mode is any mode other than the default mode, the operand may be clipped or truncated and rounded toward zero according to the precision mode inserted into the pipeline 204. However, the embodiments are not limited thereto, and in some embodiments, truncation may be applied elsewhere. Truncation may be applied to all FP32 operands, regardless of the source of the operands (e.g., read from a cache, forwarded directly from a destination buffer, etc.). Truncation may be applied before performing input exception checks to ensure functionally correct behavior. Truncated operands may be rounded toward zero after an input denormalization partial exception check but before other exception checks (e.g., other IEEE exception checks).

[0057] In some embodiments, truncation may be applied only to the inputs of the shader module 321, and not to the outputs of the shader module 321. For example, for an operation corresponding to the expression a×b=c, the values ​​of a and b may be input to the shader module 321, and the value of c may be the output of the shader module 321. Thus, the values ​​of a and b may be truncated based on the precision mode, and the value of c output by the shader module 321 may not be truncated. The operation result (e.g., the value of c) may be zero-filled to isolate changes in the pipeline 204.

[0058] In some embodiments, the mode register value may be propagated with instructions in pipeline 204 so that instructions with different precision modes may exist simultaneously in different stages of pipeline 204 .

[0059] In some embodiments, one or more instructions may be excluded from PMS processing because changing the precision mode may affect the output of these instructions or because changing the precision mode will not result in sufficient power savings. For example, a particular instruction may be included in a whitelist that may indicate that PMS is to be applied. Based on determining that the instruction is not included in the whitelist, GPU 203 may execute the instruction without applying a different precision mode according to PMS.

[0060] In some embodiments, implementation of PMS may cause changes in the abnormal behavior (e.g., IEEE numerical exceptions) of GPU 203. In an embodiment, some IEEE numerical exceptions may be affected while other IEEE numerical exceptions may not be affected. For example, because the denormalized portion may be refreshed to zero when the precision mode is any mode other than the default mode, more underflow exceptions cannot be received. However, instructions that can generate a larger result than the original non-truncated operand from the truncated operand receive new overflow exceptions. In addition, implementation of PMS may cause more input denormalized partial exceptions, and because more denormalized partial results may be generated, implementation of PMS may cause higher input denormalized partial counts. Due to truncated operands, additional division by zero exceptions should not occur, so implementation of PMS may not cause division by zero exceptions to change. However, truncating operands according to PMS may cause additional inaccurate exceptions, and when executing instructions based on truncated operands, inaccurate exceptions that may occur earlier in instructions when executed based on non-truncated operands may not occur. For example, an instruction may generate an exact output (m / (n+e)) based on non-truncated operands m and (n+e). However, when the operands are truncated according to a specific precision mode, the truncated operands may be m and n. Therefore, the instruction may generate an output (m / n) based on the truncated operands that may be inexact. Implementing PMS may not cause any changes to invalid exceptions. In addition, because integer operations may not be affected, implementing PMS may not cause any changes to integer divide by zero exceptions.

[0061] According to an embodiment, when PMS is enabled, GPU 203 may perform PMS by applying different precision modes to various instructions. GPU 203 may determine whether to apply different precision modes to instructions based on a heuristic. FIG. 6A to FIG. 6C is a flow chart illustrating an example heuristic according to an embodiment.

[0062] Fig. 6A 6 is a flow chart of a process 600A for executing PMS. In an embodiment, the process 600A may correspond to a heuristic for determining whether to apply different precision modes while executing PMS. In an embodiment, the process 600A may be performed by any of the above elements (e.g., GPU 203 and at least one of the elements included therein) or any element of the execution pipeline 500 as described above.

[0063] like Fig. 6A As shown, in operation S611, the process 600A may include obtaining a control flow graph including a plurality of shading instructions. In an embodiment, the shading instructions may be instructions to be executed by a shading processor (e.g., the shader module 321 described above) or any element that performs the vertex shading 502, geometry shading 504, and fragment shading 506 described above.

[0064] like Fig. 6A As further shown in FIG. 6 , at operation S612 , process 600A may include setting the precision mode for the plurality of shading instructions to a default precision mode. In an embodiment, the default precision mode may correspond to a relatively low precision level, for example, a precision level corresponding to a BF16 format.

[0065] like Fig. 6A As further shown in FIG. 6 , at operation S613, process 600A may include obtaining a block included in the control flow graph, and at operation S614, process 600A may include obtaining a shading instruction included in the block. In an embodiment, a block may refer to a set of instructions that receives a single input and produces a single output, for example, a set of instructions that may be executed without any intermediate exit points.

[0066] like Fig. 6A As further shown in , at operation S615, the process 600A may include evaluating the instruction to determine whether to apply a modified precision mode when the instruction is executed. For example, based on a default precision mode corresponding to a relatively low precision level, the operation S615 may include evaluating the instruction to determine whether to apply a modified precision mode corresponding to a higher precision level, for example, a precision mode corresponding to any one of the BF16 format, the BF20 format, the BF24 format, the BF28 format, and the FP32 format. As another example, based on a default precision mode corresponding to a relatively high precision level, the operation S615 may include evaluating the instruction to determine whether to apply a modified precision mode corresponding to a lower precision level, for example, a precision mode corresponding to any one of the BF16 format, the BF20 format, the BF24 format, the BF28 format, and the FP32 format. Examples of evaluation are provided below for processes 600B and 600C.

[0067] like Fig. 6A As further shown in FIG. 6 , at operation S616, process 600A may include determining whether the last instruction of the block has been reached. Based on determining that additional instructions remain in the block (no in operation S616), process 600A may return to operation S614. Based on determining that no additional instructions remain in the block (yes in operation S616), process 600A may proceed to operation S617.

[0068] like Fig. 6A As further shown in FIG. 6 , at operation S617, process 600A may include determining whether the last block of the control flow graph has been reached. Based on determining that there are additional blocks remaining in the control flow graph (no in operation S617), process 600A may return to operation S613. Based on determining that there are no additional blocks remaining in the control flow graph (yes in operation S617), process 600A may proceed to operation S618, and the heuristic algorithm terminates.

[0069] Figure 6B 600B for executing PMS. In an embodiment, the process 600B may correspond to the above-mentioned operation S615. In an embodiment, the process 600B may be performed by any of the above-mentioned elements (e.g., GPU 203 and at least one of the elements included therein) or any element of the execution pipeline 500 as described above.

[0070] like Figure 6B As shown, in operation S621, processing 600B may include determining whether the shading instruction is a predetermined shading instruction. Based on determining that the shading instruction is a predetermined shading instruction (yes in operation S621), processing 600B may proceed to operation S622, and change the precision mode to a modified precision mode for the shading instruction. For example, operation S622 may include setting a high-precision mode for the shading instruction, but the embodiment is not limited thereto. For example, in some embodiments, operation S622 may include setting the precision mode to a low-precision mode corresponding to a low-precision level lower than the default precision level. Based on determining that the shading instruction is not a predetermined shading instruction (no in operation S621), processing 600B may proceed to operation S623, and maintain the default precision mode for the shading instruction.

[0071] For example, the predetermined shading instruction may be a shading instruction that is known to be precision sensitive. This may mean that if the operands of the shading instruction are truncated based on the default precision mode, the shading instruction will produce incorrect results or corrupted results. Therefore, based on determining that the shading instruction is a precision sensitive instruction, a modified precision mode (e.g., a high precision mode) is set for the instruction.

[0072] Figure 6C 600C for executing PMS. In an embodiment, the process 600C may correspond to the above-mentioned operation S615. In an embodiment, the process 600C may be performed by any of the above-mentioned elements (e.g., GPU 203 and at least one of the elements included therein) or any element of the execution pipeline 500 as described above.

[0073] like Figure 6C As shown, at operation S631 , process 600C may include obtaining a shading instruction.

[0074] like Figure 6CAs further shown in, at operation S632, processing 600C may include determining whether the shading instruction is precision sensitive. Based on determining that the shading instruction is a predetermined shading instruction (yes in operation S632), processing 600C may proceed to operation S633, and change the precision mode to a modified precision mode for the shading instruction. For example, operation S633 may include setting a high-precision mode for the shading instruction, but the embodiments are not limited thereto. For example, in some embodiments, operation S633 may include setting the precision mode to a low-precision mode corresponding to a low-precision level lower than the default precision level. Based on determining that the shading instruction is not a predetermined shading instruction (no in operation S632), processing 600B may proceed to operation S634, and maintain the default precision mode for the shading instruction.

[0075] like Figure 6C As further shown in FIG. 6 , at operation S634 , process 600C may include obtaining operands for the shading instruction.

[0076] like Figure 6C As further shown in , at operation S635, the process 600C may include determining whether the operand is a vector operand. Based on determining that the operand is a vector operand (yes in operation S635), the process 600C may increase the depth count by one in operation S636. In an embodiment, the depth count may indicate the depth in the upward direction in the usage definition chain of the instruction. As described above, in some embodiments, PMS may only be applied to vector instructions that operate on vector operands. Therefore, the process 600C may be used to determine whether the previous shading instruction in the usage definition chain (e.g., a shading instruction that generates an operand) is also subject to PMS to determine whether the modified precision mode (e.g., high precision mode) should be applied to the previous shading instruction.

[0077] like Figure 6C As further shown in , at operation S637, the process 600C may include determining whether a maximum recursion depth has been reached, for example, by comparing a depth counter to a threshold recursion depth. Based on determining that the maximum recursion depth has been reached (yes in operation S637), the process 600C may proceed to operation S638, at which the process 600C terminates (e.g., by proceeding to operation S616 in process S600A). Based on determining that the maximum recursion depth has not been reached (no in operation S637), the process 600C may proceed to operation S639, and determine whether shading instructions are available at the current depth. Based on determining that shading instructions are available at the current depth (yes in operation S639), the process 600C may return to operation S631. Based on determining that shading instructions are not available at the current depth, the process 600C may proceed to operation S638.

[0078] As further shown in operation S640, based on determining that the operand is not a vector operand (no in operation S635), process 600C may proceed to operation S640 and determine whether the last operand has been reached. Based on determining that the last operand has been reached (yes in operation S640), process 600C may proceed to operation S638. Based on determining that the last operand has not been reached, process 600C may return to operation S634. In some embodiments, if the last operand for the current depth has been reached, process 600C may include decrementing the depth count by one before returning to operation S634 so that all operands at each depth can be evaluated.

[0079] Thus, process 600C may be used to evaluate additional instructions in an upward direction along a use-definition chain of shading instructions to ensure that appropriate levels of precision are maintained in operands as they are provided or generated in a downward direction along the use-definition chain.

[0080] According to an embodiment, the above-mentioned FIG. 6A to FIG. 6C The heuristics described herein may be referred to as heuristic knobs. These parameters may be modified to change the manner in which the PMS is applied. In an embodiment, the parameters or heuristic knobs may include a default precision mode, a modified precision mode applied to predetermined shading instructions (e.g., the modified precision mode in process 600B), a modified precision mode applied to shading instructions in a usage definition chain (e.g., the modified precision mode applied in process 600C), a list of precision sensitive instructions, and a maximum recursion depth. By modifying one or more of these heuristic knobs, the behavior of the PMS operation may be modified as desired.

[0081] Figure 7 is a diagram illustrating a programming language for performing processing tasks such as image and graphics processing tasks according to an embodiment. Specifically, Figure 7 A high-level programming language 701, an intermediate level representation 702, and assembly code 703 that can be used to directly control one or more of the above-mentioned elements (e.g., GPU 203, pipeline 204, and shader module 321) are shown. In an embodiment, the high-level programming language 701 can be a programming language used by a user to control the operation of the GPU 703. The intermediate level representation 702 can be generated based on the code written in the high-level programming language 701, and the assembly code 703 can be generated based on the intermediate level representation 702. In an embodiment, the assembly code 703 may include or correspond to the above-mentioned instructions, such as register setting instructions, modified instructions, and shader instructions. As described above, when PMS is enabled, the precision mode for a particular instruction can be set by the user in the assembly code 703. Figure 7For example, a user may insert a hint into a program written in a programming language 701, or may insert a hint into an intermediate level representation 702 or assembly code 703 generated based on a program. Thus, a user may be able to set the precision mode without directly accessing instructions in assembly code 703.

[0082] Figure 8 is a flow chart illustrating an example process for executing a PMS according to an embodiment. In an embodiment, process 800 may be performed by any of the above elements (eg, GPU 203 and at least one of the elements included therein) or any element of execution pipeline 500 as described above.

[0083] like Figure 8 As shown, at operation S801, the process 800 may include obtaining a shading instruction corresponding to a floating point operand. In an embodiment, the shading instruction may be an instruction to be executed by a shading processor (e.g., the shader module 321 described above) or any element that performs vertex shading 502, geometry shading 504, and fragment shading 506 described above.

[0084] like Figure 8 As further shown in FIG. 8 , at operation S802 , process 800 may include determining a precision mode to apply to a shading instruction. In an embodiment, the precision mode may be determined based on a value stored in a mode register.

[0085] like Figure 8 As further shown in operation S803, process 800 may include truncating floating point operands based on the determined precision mode, and executing shading instructions using the truncated floating point operands.

[0086] The truncation of the floating point operand may include truncating the mantissa of the floating point operand while retaining the exponent of the floating point operand. In an embodiment, the precision of the truncated floating point operand may be lower than the precision of the floating point operand, and the dynamic range of the truncated floating point operand may be the same as the dynamic range of the floating point operand.

[0087] In an embodiment, process 800 may further include determining whether the shading instruction is included in a whitelist, wherein based on determining that the shading instruction is not included in the whitelist, the shading instruction may be processed using floating point operands without determining a precision mode.

[0088] In an embodiment, determining the precision mode may include determining whether PMS is enabled based on the value of a configuration bit included in a configuration file corresponding to the GPU, wherein, based on PMS being enabled, multiple precision modes may be used to process shading instructions, wherein, based on PMS not being enabled, multiple precision modes may not be used to process shading instructions.

[0089] In an embodiment, process 800 may also include receiving a mode instruction for setting a new mode; and changing a value stored in a mode register to a new value corresponding to the new mode, wherein the new value may be visible to instructions after the mode instruction in the programming order, and may not be visible to instructions before the mode instruction in the programming order. In an embodiment, the mode instruction may correspond to at least one of the register setting instruction and the modified instruction described above.

[0090] In an embodiment, the mode instruction may include a control bit and an instruction bit, wherein the instruction bit may be interpreted as a mode instruction based on PMS being enabled and the control bit being set, wherein the instruction bit may be interpreted as a different instruction based on PMS being enabled and the control bit not being set, and wherein the control bit may be ignored and the instruction bit may be interpreted as a different instruction based on PMS not being enabled. In an embodiment, the mode instruction may correspond to the modified instruction described above, and the different instruction may correspond to the original instruction described above.

[0091] Although FIG. 6A to FIG. 6C and Figure 8 Example blocks of processes 600A, 600B, 600C, and 800 are shown, but in some implementations, one or more of processes 600A, 600B, 600C, and 800 may include additional blocks, fewer blocks, different blocks, or different FIG. 6A to FIG. 6C and Figure 8 Blocks in different arrangements shown. Additionally or alternatively, two or more of the blocks of processes 600A, 600B, 600C, and 800 may be combined in any order or performed in parallel.

[0092] Although various example embodiments have been particularly shown and described with reference to the accompanying drawings, it should be understood that various changes in form and details may be made therein without departing from the spirit and scope of the appended claims.

Claims

1. A method for performing precision-modulated shading using a graphics processing unit, the method comprising: Obtaining a shading instruction corresponding to a floating point operand; determining a precision mode from among a plurality of precision modes for processing a shading instruction to apply to the shading instruction; as well as Based on the determined precision mode, the floating point operand is truncated, and the shading instruction is executed using the truncated floating point operand.

2. The method according to claim 1, wherein: The shading instructions are executed by a shader processor included in the graphics processing unit.

3. The method according to claim 1, wherein: The step of truncating the floating-point operand comprises: truncating the mantissa of the floating-point operand while retaining the exponent of the floating-point operand, wherein the precision of the truncated floating-point operand is lower than the precision of the floating-point operand, and The dynamic range of the truncated floating-point operand is the same as the dynamic range of the floating-point operand.

4. The method according to claim 1, further comprising: determining whether the shading instruction is included in a whitelist, Wherein, based on determining that the shading instruction is not included in the whitelist, the shading instruction is processed using a default precision mode.

5. The method according to claim 1, wherein: The step of determining the precision mode includes: determining whether to enable the precision modulated shading based on a value of a configuration bit included in a configuration file corresponding to the graphics processing unit, wherein shading based on the precision modulation is enabled, the plurality of precision modes are used to process the shading instructions, and Wherein, shading based on the precision modulation is not enabled, and the multiple precision modes are not used to process the shading instruction.

6. The method according to claim 5, wherein: The precision mode is determined based on a value stored in a mode register corresponding to the shading instruction.

7. The method according to claim 6, further comprising: receiving a mode instruction for setting a new mode; as well as changing the value stored in the mode register to a new value corresponding to the new mode, The new value is visible to instructions that follow the mode instruction in the programming order and is not visible to instructions that precede the mode instruction in the programming order.

8. The method according to claim 7, wherein: The mode instruction includes a control bit and an instruction bit, wherein shading based on the precision modulation is enabled and the control bit is set, the instruction bits are interpreted as the mode instruction, wherein, based on the precision modulated shading being enabled and the control bit being unset, the instruction bits are interpreted as different instructions, and Wherein, shading based on the precision modulation is not enabled, the control bits are ignored, and the instruction bits are interpreted as the different instruction.

9. The method according to claim 1, further comprising: Obtaining a control flow graph including a plurality of shading instructions; setting a precision mode for the plurality of shading instructions to a default precision mode, wherein the default precision mode corresponds to a first precision level; and Each of the plurality of shading instructions is evaluated to determine whether to apply a modified precision mode, wherein the modified precision mode corresponds to a second precision level different than the first precision level.

10. The method according to claim 9, wherein: The step of evaluating includes: determining whether a first shading instruction of the plurality of shading instructions is a predetermined shading instruction, and Wherein, based on determining that the first shading instruction is the predetermined shading instruction, the method further includes setting the precision mode for the first shading instruction to the modified precision mode.

11. The method according to claim 10, wherein: The step of evaluating further comprises: determining whether the first shading instruction is precision sensitive, and Wherein, based on determining that the first shading instruction is precision sensitive, the method further includes: determining a second instruction that provides the first shading instruction, and The precision mode for the second instruction is set to the modified precision mode.

12. A method for performing precision modulated shading using a graphics processing unit, the method comprising: Obtaining a control flow graph including a plurality of shading instructions; setting a precision mode for the plurality of shading instructions to a default precision mode, wherein the default precision mode corresponds to a first precision level; and evaluating each of the plurality of shading instructions to determine whether to apply a modified precision mode, wherein the modified precision mode corresponds to a second precision level different than the first precision level; and Based on the default precision mode being applied to a first shading instruction of the plurality of shading instructions, controlling at least one shader processor: setting a mode register included in the at least one shader processor to a first value corresponding to the default precision mode, truncating a first floating-point operand corresponding to the first shading instruction, and The first shading instruction is executed using a computation module included in the at least one shader processor based on the truncated first floating-point operand.

13. The method according to claim 12, wherein: The step of evaluating includes: determining whether the first shading instruction is a predetermined shading instruction, and Wherein, based on determining that the first shading instruction is the predetermined shading instruction, the method further includes setting the precision mode for the first shading instruction to the modified precision mode.

14. The method according to claim 13, wherein: The step of evaluating further comprises: determining whether the first shading instruction is precision sensitive, and Wherein, based on determining that the first shading instruction is precision sensitive, the method further includes: determining a second shading instruction that provides the first shading instruction, and The precision mode for the second shading instruction is set to the modified precision mode.

15. The method according to claim 14, wherein: The first shading instruction and the second shading instruction are included in a usage definition chain, wherein the step of evaluating comprises evaluating a plurality of additional instructions in the usage definition chain, and The number of the plurality of additional instructions is determined based on a predetermined maximum recursion depth.

16. The method according to claim 12, wherein: The plurality of shading instructions include a second shading instruction corresponding to a second floating point operand, and Wherein, based on the default precision mode being applied to the second shading instruction, the method further includes controlling the shader processor to execute the second shading instruction using the second floating-point operand.

17. The method according to claim 16, wherein: The step of truncating the first floating-point operand includes: truncating the mantissa of the first floating-point operand while retaining the exponent of the first floating-point operand, wherein the precision of the truncated first floating-point operand is lower than the precision of the first floating-point operand, and The dynamic range of the truncated first floating-point operand is the same as the dynamic range of the first floating-point operand.

18. A graphics processing unit for performing precision modulated shading, the graphics processing unit comprising: At least one shader processor configured to: Obtaining a shading instruction corresponding to a floating point operand; determining a precision mode from among a plurality of precision modes for processing a shading instruction that applies to the shading instruction; and Based on the determined precision mode, the floating point operand is truncated, and the shading instruction is executed using the truncated floating point operand.

19. The graphics processing unit according to claim 18, wherein: To truncate the floating point operand, the at least one shader processor is further configured to truncate a mantissa of the floating point operand while preserving an exponent of the floating point operand, wherein the precision of the truncated floating-point operand is lower than the precision of the floating-point operand, and The dynamic range of the truncated floating-point operand is the same as the dynamic range of the floating-point operand.

20. An apparatus for performing precision-modulated shading, the apparatus comprising: A graphics processing unit comprising at least one shader processor, wherein the at least one shader processor comprises a mode register and a compute module; and At least one controller configured to: obtaining a control flow graph including a plurality of shading instructions, setting a precision mode for the plurality of shading instructions to a default precision mode, wherein the default precision mode corresponds to a first precision level, and evaluating each of the plurality of shading instructions to determine whether to apply a modified precision mode, wherein the modified precision mode corresponds to a second precision level different than the first precision level, Wherein, based on the default precision mode being applied to a first shading instruction among the plurality of shading instructions, the at least one controller is further configured to control the at least one shader processor: setting the mode register to a first value corresponding to the default precision mode, truncating a first floating-point operand corresponding to the first shading instruction, and The first shading instruction is executed using the computation module based on the truncated first floating point operand.

Images