Random rounding of values

Abstract

A method, computer readable medium, and system for rounding a numerical value are disclosed. A set of bits from an input value is identified as a rounded value. A second set of bits representing a second value is extracted from the input value and added to the rounded value to produce a sum. The sum is truncated to produce a rounded output value. Therefore, the present invention provides a random rounding technique that rounds an input value as a function of a second value and a rounded value, both of which are obtained from the input value. When the second value and the rounded value are obtained from consistent bit positions of the input value, the resulting output value is deterministic. Random rounding is deterministic and is advantageously suitable for deep learning applications.

Description

Technical Field

[0001] The present invention relates to rounding values ​​and, more particularly, to circuits for performing random rounding. Background Art

[0002] In some cases, it may be beneficial to reduce the precision of numerical values ​​representing mathematical calculations. For example, 32-bit floating-point values ​​can be replaced with less precise 16-bit floating-point values ​​to reduce circuitry, power, and bandwidth used for neural network training. Reduced-precision values ​​require only half the bandwidth during data transmission, as well as less than half the chip area and power consumption, compared to using full-precision values. However, the range of values ​​that can be represented using the 16-bit floating-point format is much smaller than the range of values ​​that can be represented using the 32-bit floating-point format. Numbers that are smaller than the minimum value that can be represented in the 16-bit floating-point format are lost (i.e., become zero).

[0003] Random rounding is a method commonly used to extend the range of values. With traditional rounding, numbers are deterministically rounded up or down, e.g. values ​​between 0.5 and 1 are rounded up to 1, and values ​​below 0.5 are always rounded down to 0. With random rounding, the rounding is instead probabilistic. Specifically, 0.5% has a 50% chance of being rounded up to 1 and a 50% chance of being rounded down to 0, while 0.1 has a 10% chance of being rounded up to 1 and a 90% chance of being rounded down to 0.

[0004] With random rounding, individual rounding events can actually introduce more error, but on average over a long accumulation sequence, the result will have less error. For example, when accumulating 1000 numbers with a value of 0.1 using traditional rounding, the result will be zero after accumulating each number, however, with random rounding the result should be closer to the correct answer of 100.

[0005] Therefore, when applying accumulation and rounding operations on long sequences of values, random rounding is preferred, and the challenge is to efficiently implement the random rounding behavior of random rounding in software or hardware. One approach may be to generate random numbers for each rounding operation, but this is very expensive in terms of additional circuitry and / or power consumption. There is a need to address these and / or other problems associated with the prior art. Summary of the invention

[0006] A method, computer readable medium and system for rounding off a numerical value are disclosed. An input value represented by a first digit is received, and a portion of the first digit of the input value is identified as a rounded value represented by a second digit. A second value is extracted from the input value, and the rounded value is aligned with a rounded position within the second value, wherein the rounded position corresponds to the least significant bit of an output value represented by a third digit. The aligned rounded value and the second value are added to generate a sum, and the sum is truncated to generate an output value. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A flow chart of a method for rounding a numerical value according to an embodiment is shown.

[0008] Figure 2A A block diagram of a rounding unit according to an embodiment is shown.

[0009] Figure 2B The rounding of a 23-bit mantissa to produce a 10-bit reduced-bit-width mantissa is shown according to one embodiment.

[0010] Figure 2C Another block diagram of a rounding unit according to an embodiment is shown.

[0011] Figure 2D Another flow chart of a method for rounding a numerical value according to an embodiment is shown.

[0012] Figure 3 A parallel processing unit according to an embodiment is shown.

[0013] Figure 4A According to an embodiment, Figure 3 A general-purpose processing cluster within a parallel processing unit.

[0014] Figure 4B According to an embodiment, Figure 3 A memory partitioning unit for parallel processing units.

[0015] Figure 5A According to an embodiment, Figure 4A Streaming multiprocessor.

[0016] Figure 5B According to one embodiment, the use Figure 3 Conceptual diagram of the processing system implemented by the PPU.

[0017] Figure 5C An exemplary system is shown that may implement the various architecture and / or functionality of various previous embodiments. DETAILED DESCRIPTION

[0018] A new rounding mechanism is described for reducing the bit width of a value from its original full bit width (N bits) to the reduced bit width (M bits). In one embodiment, a portion of the N bits of a floating-point mantissa value is used as a rounded value. The rounded position in the full bit width value is the position of the least significant bit of the reduced bit width value. In other words, the position on the right side of the rounded position is truncated to perform bit width reduction. The number of decimal places on the right side of the rounded position to be truncated can be fixed, calculated or programmable. Similarly, the number of digits (M) in the reduced bit width value can be fixed, calculated or programmable. M determines the rounded position in the N-bit input value. All or less than the N-bit of the original input value is used to generate a second value summed with the rounded value.

[0019] The rounded value is aligned to the right of the rounded position and added to the second value to calculate the sum. In contrast, when performing conventional rounding to the nearest, the fixed value is summed with the full bit width value to calculate the sum. In both cases, the decimal places to the right of the rounded position in the sum are truncated to produce a reduced bit width value.

[0020] Figure 1 A flow chart of a method 100 for rounding off a numerical value according to an embodiment is shown. Although the method 100 is described in the context of a processing unit, the method 100 may also be performed by a program, a custom circuit, or a combination of a custom circuit and a program. For example, the method 100 may be performed by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing arithmetic operations. In addition, it will be appreciated by those of ordinary skill in the art that any system that performs the method 100 is within the scope and spirit of embodiments of the present invention.

[0021] At step 110, an input value represented by a first digit is received. In one embodiment, the input value is represented in a floating point format including an exponent and a mantissa. In another embodiment, the input value is represented in a fixed point format.

[0022] In step 120, a portion of the first digit of the input value is identified as a rounded value represented by the second digit. In one embodiment, the portion of the first digit of the input value is identified within a mantissa. In one embodiment, the portion of the first digit of the input value identified is the least significant bit of the mantissa. In one embodiment, the portion of the first digit of the input value is identified within a fixed-point format value. In one embodiment, the portion of the first digit of the input value identified is the least significant bit of the fixed-point format value. In another embodiment, the portion of the first digit of the input value identified is a group of consecutive bits of the input value. In yet another embodiment, the portion of the first digit of the input value identified is a group of bits of the input value from a predetermined bit position of the input value.

[0023] At step 130, a second value is extracted from the input value. In one embodiment, the second value is the same value as the input value. In another embodiment, the second value is a less precise value of the input value. In yet another embodiment, the second value is represented by fewer bits than the input value. At step 140, the rounded value is aligned with the rounded position within the second value, wherein the rounded position corresponds to the least significant bit of the output value represented by the third bit number. Figure 2B The rounding position is described in more detail. In step 150, the aligned rounding value and the second value are added to produce a sum. In step 160, the fourth digit is intercepted from the sum to produce an output value. In one embodiment, the fourth digit is determined so that the output value is a reduced bit width including the third digit. In one embodiment, the fourth digit is equal to the second digit. In one embodiment, the input value is the mantissa of a floating point format number, and the output value is an integer.

[0024] Now will set forth more illustrative information about various optional architectures and features that can implement the aforementioned framework according to the user's desire. It should be particularly noted that the following information is set forth for illustrative purposes and should not be interpreted as limiting in any way. Any of the following features may be optionally incorporated into or not exclude other features described.

[0025] Figure 2A The block diagram of rounding unit 200 according to an embodiment is shown. Rounding unit 200 comprises threshold comparison unit 210, multiplexer 215 and accumulator 220. Rounding unit 200 receives the floating point format value of the mantissa that comprises exponent and N position. In one embodiment, IEEE (Institute of Electrical and Electronics Engineers) floating point format is used to represent the floating point format value. In one embodiment, mantissa is rounded according to the present invention.

[0026] The threshold comparison unit 210 compares the floating point input with the threshold. When the floating point input is greater than or equal to the threshold, the fractional value can be used as the rounding value to round off the mantissa. The threshold can be fixed, calculated or programmed. In one embodiment, the threshold is equal to the minimum value that can be represented using a floating point format with a reduced bit width. In another embodiment, the threshold is equal to the maximum value that can be represented using a floating point format with a reduced bit width.

[0027] Threshold comparison unit 210 outputs a selection signal to multiplexer 215, which is used to select a portion of the fractional value or input value as a rounded value. In one embodiment, a portion of the mantissa is provided as an input to multiplexer 215, wherein the mantissa is a first digit (N), and a portion of the mantissa includes less than N bits. Rounding unit 200 outputs a reduced mantissa with a third digit (M), wherein M is less than N.

[0028] The fractional value can be used to perform traditional IEEE 754-2008 rounding. The fractional value can be fixed, calculated or programmed. Otherwise, when the floating-point input is less than (that is, not greater than or not equal to) the threshold value, the multiplexer 215 selects a part of the mantissa to round off the mantissa as the rounding value. The threshold comparison unit 210 extracts the second value from the floating-point input. In one embodiment, the second value is a part of the mantissa with less than N bits.

[0029] Accumulator 220 receives second value and rounded value and produces the mantissa that reduces.The rounded position in the rounded value and the second value is aligned.The rounded position corresponds to the least significant bit of the mantissa that reduces.The rounded value of this alignment and the second value are summed to produce and.Accumulator 220 intercepts a plurality of positions from and to produce the mantissa that has the reduction of less number of digits than the mantissa of input value.

[0030] In one embodiment, the input value includes a 23-bit mantissa of a floating point format number, and the output value generated by the rounding unit 200 is a 10-bit reduced mantissa. In another embodiment, the input value includes a 23-bit mantissa of a floating point format number, and the output value generated by the rounding unit 200 is a 7-bit mantissa. In yet another embodiment, the input value includes a 52-bit mantissa of a floating point format number, and the output value generated by the rounding unit 200 is a 23-bit mantissa.

[0031] Figure 2B The rounding of a 23-bit mantissa to produce a 10-bit reduced bit width mantissa according to one embodiment is shown. In one embodiment, the 23-bit mantissa includes an implicit leading one. In one embodiment, the 23-bit mantissa is reduced to a 10-bit reduced mantissa by summing the 8 lsbs (least significant bits) of the 23-bit mantissa that are aligned with the bits of the 23-bit mantissa, which are located to the right of the lsb of the 10-bit reduced mantissa. Figure 2B As shown, the 10 most significant bits (MS) correspond to the reduced bit width mantissa, where M=10. The 8 least significant bits (LS) of the 23-bit mantissa (bits 16 to 23) are used as an 8-bit rounding value, which is aligned with the rounding position and added to the 23-bit mantissa. The carry bit generated by adding bits 11 to 18 to bits 16 to 23 increases the 10 MS bits when the carry bit is high. The resulting 23-bit sum is truncated to produce a 10-bit reduced mantissa.

[0032] In one embodiment, if during rounding and overflow, then and shift right one position and then be truncated.In one embodiment, the mantissa that reduces is 7.In other embodiments, the mantissa that reduces is less or more positions.In certain embodiments, the digit number of the representation input mantissa is greater than or less than 23.In one embodiment, before calculating and, optionally revise the rounded value or the LS position of mantissa.For example, fractional value can be XORed with the LS position of mantissa, fractional value can be used as the position rotation count of the LS position of mantissa, or fractional value can be used as mask to select the specific position on the right side of the rounded position in the mantissa to generate the rounded value.

[0033] Figure 2C 2 shows another block diagram of a rounding unit 250 according to an embodiment. The rounding unit 250 includes a threshold comparison unit 212, a rounded value generation unit 225, and an accumulator 220. The rounding unit 250 receives a floating point format value including a mantissa of N bits.

[0034] One or more scopes and / or threshold values ​​are defined corresponding to different rounding-off modes, and threshold comparison unit 212 receives the rounding-off mode, which threshold range or value of this rounding-off mode control is compared with the floating-point input. In one embodiment, when the floating-point input is greater than or equal to the threshold value, the fractional value can be used to round off the mantissa as the rounding-off value. In another embodiment, when the floating-point input is within or outside the threshold range, the fractional value can be used as the rounding-off value to round off the mantissa. Threshold value and / or the rounding-off mode can be fixed, calculated or programming.

[0035] Threshold value comparison unit 212 outputs selection signal to rounded value generation unit 225, and this rounded value generation unit 225 is used to select a part of fractional value or input value as rounded value.Rounding mode can be used for modifying rounded value.In one embodiment, threshold value comparison unit 212 extracts and provides a part of mantissa as the input of rounded value generation unit 225, wherein mantissa is represented by the first digit (N), and the described part of mantissa comprises less than N position.In one embodiment, the rounded value extracted by threshold value comparison unit 212 is the LS position of mantissa.Rounding unit 250 output has the minimization mantissa of the third digit (M), and wherein M is less than N.In one embodiment, the rounded value selected by rounded value generation unit 225 is the LS position of mantissa.

[0036] In one embodiment, the rounding value generation unit 225 randomly modifies the rounding value or modifies the LS position of mantissa before calculating and.For example, based on the rounding pattern, the fractional value can be XORed with the LS position of mantissa, and the fractional value can be used as the position rotation count of the LS position of mantissa, or the fractional value can be to be used as a mask to select the specific position on the right side of the rounding position in the mantissa to generate the rounding value.Rounding pattern and / or fractional value can be fixed, calculated or programmed.

[0037] As previously mentioned, accumulator 220 receives second value and rounded value and produces the mantissa that reduces.The rounded value and the second value are aligned in the rounded position.The rounded position corresponds to the least significant bit of the mantissa that reduces.The rounded value of alignment and second value are summed to produce and.Accumulator 220 is from and intercepts a plurality of positions to produce the mantissa that reduces with the third digit.

[0038] Special instructions can be used to perform rounding, for example, the value with the first bit width is converted to the instruction with the value of the second bit width, wherein the second bit width is less than the first bit width. In one embodiment, one or more of a fractional value, a threshold range, a threshold value and a rounding mode can be provided as the operand of the instruction. In one embodiment, the fractional value can be generated by an incrementer. For performing arithmetic operations (for example, multiplication accumulation, summation, etc.) and generating the instruction of the bit width output that reduces, the rounding operation can be selectively enabled. In one embodiment, when executing the instruction, the rounded value and the bit width value that reduces are returned.

[0039] Figure 2D Another flow chart of a method 260 for rounding off a numerical value according to an embodiment is shown. Although the method 260 is described in the context of a processing unit, the method 260 may also be performed by a program, a custom circuit, or a combination of a custom circuit and a program. For example, the method 260 may be performed by a GPU (graphics processing unit), a CPU (central processing unit), or any processor capable of performing arithmetic operations. In addition, it will be appreciated by those of ordinary skill in the art that any system that performs the method 260 is within the scope and spirit of embodiments of the present invention.

[0040] At step 110, an input value represented by a first digit is received. In one embodiment, the input value is represented in a floating point format including an exponent and a mantissa. At step 265, the threshold comparison unit 212 generates a selection signal for the input value and a threshold value or a threshold range based on a rounding mode. In one embodiment, the threshold comparison unit 212 compares the input value with a threshold value, and when the input value is greater than the threshold value, generates a selection signal to select a fractional value. In one embodiment, the threshold comparison unit 212 compares the input value with a threshold value, and when the input value is greater than or equal to the threshold value, generates a selection signal to select a fractional value.

[0041] In step 270, the rounded value generation unit 225 determines whether the selection signal selects the fractional value. If not, then before proceeding to step 275, complete step 120 to identify a part of the input value to be selected as the rounded value. In one embodiment, the part of the input value is identified by connecting the subset of the line used for the mantissa to the rounded value generation unit 225. If in step 270, the rounded value generation unit 225 determines that the selection signal selects the fractional value, the rounded value generation unit 225 directly enters step 275. In step 275, the rounded value generation unit 225 selects the rounded value according to the selection signal. In step 280, the rounded value generation unit 225 modifies the rounded value. The rounded value can be modified based on the fractional value and / or the rounding mode. Complete steps 130 to 160 as described above.

[0042] The advantage of using a part of input value to perform rounding operation is that the result of given input value is deterministic. Specifically, whenever rounding operation is performed, the same output value will be produced for a specific input value. On the contrary, when using random value to perform rounding operation, the result is not necessarily deterministic. Compared with generating random value, using a part of input value also requires less circuit. Therefore, the benefit of random rounding (reducing the error of continuous accumulation) can be realized without generating random value.

[0043] Parallel processing architecture

[0044] Figure 3 A parallel processing unit (PPU) 300 is shown according to one embodiment. In one embodiment, the PPU 300 is a multi-threaded processor implemented on one or more integrated circuit devices. The PPU 300 is a latency-hiding architecture designed for processing many threads in parallel. A thread (i.e., an execution thread) is an instance of an instruction set configured to be executed by the PPU 300. In one embodiment, the PPU 300 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device (such as a liquid crystal display (LCD) device). In other embodiments, the PPU 300 can be used to perform general-purpose computations. Although an exemplary parallel processor is provided herein for illustrative purposes, it should be specifically noted that the processor is described only for illustrative purposes and any processor can be used to supplement and / or replace the processor.

[0045] One or more PPUs 300 can be configured to accelerate thousands of high-performance computing (HPC), data center, and machine learning applications. PPU 300 can be configured to accelerate numerous deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analysis, molecular simulation, drug development, disease diagnosis, weather forecasting, big data analysis, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations.

[0046] like Figure 3 As shown, the PPU 300 includes an input / output (I / O) unit 305, a front-end unit 315, a scheduler unit 320, a work distribution unit 325, a hub 330, a crossbar switch (Xbar) 370, one or more general processing clusters (GPCs) 350, and one or more partition units 380. The PPU 300 can be connected to a host processor or other PPUs 300 via one or more high-speed NVLink 310 interconnects. The PPU 300 can be connected to a host processor or other peripheral devices via an interconnect 302. The PPU 300 can also be connected to a local memory including a plurality of memory devices 304. In one embodiment, the local memory may include a plurality of dynamic random access memory (DRAM) devices. The DRAM device can be configured as a high bandwidth memory (HBM) subsystem with a plurality of DRAM dies stacked within each device.

[0047] The NVLink 310 interconnect enables the system to scale and include one or more PPUs 300 in conjunction with one or more CPUs, supporting cache coherency between the PPU 300 and the CPU, and CPU mastering. Data and / or commands may be sent by the NVLink 310 through the hub 330 to or from other units of the PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). Figure 5B NVLink 310 is described in more detail.

[0048] I / O unit 305 is configured to send and receive communications (i.e., commands, data, etc.) from a host processor (not shown) via interconnect 302. I / O unit 305 may communicate with the host processor directly via interconnect 302, or through one or more intermediate devices (such as a memory bridge). In one embodiment, I / O unit 305 may communicate with one or more other processors (e.g., one or more PPUs 300) via interconnect 302. In one embodiment, I / O unit 305 implements a peripheral component interconnect express (PCIe) interface for communicating over a PCIe bus, and interconnect 302 is a PCIe bus. In alternative embodiments, I / O unit 305 may implement other types of well-known interfaces for communicating with external devices.

[0049] The I / O unit 305 decodes packets received via the interconnect 302. In one embodiment, the packets represent commands configured to cause the PPU 300 to perform various operations. The I / O unit 305 sends the decoded commands to various other units of the PPU 300 as specified by the commands. For example, some commands may be sent to the front end unit 315. Other commands may be sent to the hub 330 or other units of the PPU 300, such as one or more replication engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, the I / O unit 305 is configured to route communications between and among the various logical units of the PPU 300.

[0050] In one embodiment, a program executed by a host processor encodes a command stream in a buffer that provides a workload to the PPU 300 for processing. The workload may include many instructions and data to be processed by those instructions. A buffer is an area in memory that is accessible (i.e., read / write) by both the host processor and the PPU 300. For example, the I / O unit 305 may be configured to access a buffer in a system memory connected to the interconnect 302 via a memory request transmitted through the interconnect 302. In one embodiment, the host processor writes a command stream to the buffer and then sends a pointer to the start of the command stream to the PPU 300. The front end unit 315 receives a pointer to one or more command streams. The front end unit 315 manages one or more streams, reads commands from the streams, and forwards the commands to the various units of the PPU 300.

[0051] The front end unit 315 is coupled to a scheduler unit 320, which configures the various GPCs 350 to process the tasks defined by one or more streams. The scheduler unit 320 is configured to track state information related to the various tasks managed by the scheduler unit 320. The state may indicate which GPC 350 the task is assigned to, whether the task is active or inactive, a priority associated with the task, etc. The scheduler unit 320 manages the execution of multiple tasks on one or more GPCs 350.

[0052] The scheduler unit 320 is coupled to a work distribution unit 325 that is configured to dispatch tasks for execution on the GPCs 350. The work distribution unit 325 may track a plurality of scheduled tasks received from the scheduler unit 320. In one embodiment, the work distribution unit 325 manages a pending task pool and an active task pool for each GPC 350. The pending task pool may include a plurality of slots (e.g., 32 slots) containing tasks assigned to be processed by a particular GPC 350. The active task pool may include a plurality of slots (e.g., 4 slots) for tasks that are being actively processed by the GPC 350. When a GPC 350 completes execution of a task, the task is evicted from the active task pool of the GPC 350, and one of the other tasks from the pending task pool is selected and scheduled for execution on the GPC 350. If an active task on a GPC 350 has become idle, such as while waiting for a pending data dependency to be resolved, then the active task may be evicted from the GPC 350 and returned to the pending task pool while another task in the pending task pool is selected and scheduled for execution on the GPC 350 .

[0053] Work distribution unit 325 communicates with one or more GPCs 350 via XBar (crossbar) 370. XBar 370 is an interconnect network that couples many units of PPU 300 to other units of PPU 300. For example, XBar 370 can be configured to couple work distribution unit 325 to a particular GPC 350. Although not explicitly shown, one or more other units of PPU 300 can also be connected to XBar 370 via hub 330.

[0054] Tasks are managed by a scheduler unit 320 and dispatched to GPCs 350 by a work distribution unit 325. GPCs 350 are configured to process tasks and generate results. Results can be consumed by other tasks within the GPC 350, routed to a different GPC 350 via XBar 370, or stored in memory 304. Results can be written to memory 304 via a partition unit 380, which implements a memory interface for reading data from and writing data to memory 304. Results can be sent to another PPU 304 or CPU via NVLink 310. In one embodiment, a PPU 300 includes a number U of partition units 380, where U is equal to the number of separate and distinct memory devices 304 coupled to the PPU 300. This will be described below in conjunction with Figure 4B The partition unit 380 is described in more detail.

[0055] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications to be executed on the host processor to schedule operations for execution on the PPU 300. In one embodiment, multiple computer applications are executed simultaneously by the PPU 300, and the PPU 300 provides isolation, quality of service (QoS), and independent address spaces for the multiple computer applications. The application can generate instructions (i.e., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 300. The driver kernel outputs the tasks to one or more streams being processed by the PPU 300. Each task can include one or more related groups of threads, referred to herein as warps. In one embodiment, a warp includes 32 related threads that can execute in parallel. Collaborating threads can refer to multiple threads that include instructions to execute tasks and can exchange data through shared memory. In combination Figure 5A Describes threads and cooperative threads in more detail.

[0056] Figure 4A According to an embodiment, Figure 3 PPU 300 GPC 350. Figure 4A As shown, each GPC 350 includes multiple hardware units for processing tasks. In one embodiment, each GPC 350 includes a pipeline manager 410, a pre-raster operation unit (PROP) 415, a raster engine 425, a work distribution crossbar (WDX) 480, a memory management unit (MMU) 490, and one or more data processing clusters (DPCs) 420. It should be understood that Figure 4A The GPC 350 may include instead Figure 4A Other hardware units or units other than those shown in Figure 4AOther hardware units than those shown in .

[0057] In one embodiment, the operation of the GPC 350 is controlled by a pipeline manager 410. The pipeline manager 410 manages the configuration of one or more DPCs 420 for processing tasks assigned to the GPC 350. In one embodiment, the pipeline manager 410 may configure at least one of the one or more DPCs 420 to implement at least a portion of a graphics rendering pipeline. For example, the DPC 420 may be configured to execute a vertex shading program on a programmable streaming multiprocessor (SM) 440. The pipeline manager 410 may also be configured to route packets received from the work distribution unit 325 to appropriate logic units in the GPC 350. For example, some packets may be routed to fixed-function hardware units in the PROP 415 and / or the raster engine 425, while other packets may be routed to the DPC 420 for processing by the primitive engine 435 or the SM 440. In one embodiment, the pipeline manager 410 may configure at least one of the one or more DPCs 420 to implement a neural network model and / or a computational pipeline.

[0058] PROP unit 415 is configured to route data generated by raster engine 425 and DPC 420 to a raster operations (ROP) unit, in conjunction with Figure 4B The PROP unit 415 may also be configured to perform optimization of color blending, organize pixel data, perform address translation, etc.

[0059] The raster engine 425 includes a plurality of fixed-function hardware units configured to perform various raster operations. In one embodiment, the raster engine 425 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a tile aggregation engine. The setup engine receives the transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices. The plane equations are sent to the coarse raster engine to generate coverage information (e.g., x, y coverage masks of tiles) of the primitives. The output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and the unculled fragments are sent to the clipping engine, where the fragments outside the viewing cone are clipped. Those fragments remaining after clipping and culling can be passed to the fine raster engine to generate attributes of pixel fragments based on the plane equations generated by the setup engine. The output of the raster engine 425 includes, for example, fragments to be processed by a fragment shader implemented in the DPC 420.

[0060] Each DPC 420 included in the GPC 350 includes an M pipeline controller (MPC) 430, a primitive engine 435, and one or more SMs 440. The MPC 430 controls the operation of the DPC 420, routing packets received from the pipeline manager 410 to appropriate units in the DPC 420. For example, packets associated with vertices may be routed to the primitive engine 435, which is configured to fetch vertex attributes associated with the vertices from the memory 304. Conversely, packets associated with shading programs may be sent to the SM 440.

[0061] SM 440 includes a programmable stream processor configured to process tasks represented by multiple threads. Each SM440 is multithreaded and is configured to execute multiple threads (e.g., 32 threads) from a specific thread group simultaneously. In one embodiment, SM 440 implements a SIMD (single instruction, multiple data) architecture, wherein each thread in a thread group (i.e., a warp) is configured to process different data sets based on the same instruction set. All threads in a thread group execute the same instruction. In another embodiment, SM 440 implements a SIMT (single instruction, multiple thread) architecture, wherein each thread in a thread group is configured to process different data sets based on the same instruction set, but wherein each thread in a thread group is allowed to diverge during execution. In one embodiment, a program counter, a call stack, and an execution state are maintained for each warp, and when the threads in the warp diverge, concurrency between the warp and the serial execution in the warp is made possible. In another embodiment, a program counter, a call stack, and an execution state are maintained for each individual thread, thereby achieving equal concurrency between all threads in the warp and between the warps. When execution state is maintained for each individual thread, threads executing the same instructions can be converged and executed in parallel for maximum efficiency. Figure 5A Describe SM440 in more detail.

[0062] MMU 490 provides an interface between GPC 350 and partition unit 380. MMU 490 can provide virtual address to physical address translation, memory protection, and arbitration of memory requests. In one embodiment, MMU 490 provides one or more translation lookaside buffers (TLBs) for performing translations from virtual addresses to physical addresses in memory 304.

[0063] Figure 4B According to an embodiment, Figure 3 The memory partition unit 380 of the PPU 300. Figure 4BAs shown, the memory partition unit 380 includes a raster operation (ROP) unit 450, a second level (L2) cache 460, and a memory interface 470. The memory interface 470 is coupled to the memory 304. The memory interface 470 can implement a 32, 64, 128, 1024 bit data bus, etc. for high-speed data transmission. In one embodiment, the PPU 300 incorporates U memory interfaces 470, one for each pair of partition units 380, wherein each pair of partition units 380 is connected to a corresponding memory device 304. For example, the PPU 300 can be connected to up to Y memory devices 304, such as a high bandwidth memory stack or a graphics double data rate version 5 synchronous dynamic random access memory or other types of persistent memory.

[0064] In one embodiment, memory interface 470 implements an HBM2 memory interface and Y is equal to half of U. In one embodiment, the HBM2 memory stack is located on the same physical package as PPU 300, providing significant power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack includes four memory dies and Y is equal to 4, where

[0065] The HBM2 stack includes two 128-bit channels per die, for a total of eight channels and a data bus width of 1024 bits.

[0066] In one embodiment, memory 304 supports single error correction double error detection (SECDED) error correction code (ECC) to protect data. ECC provides higher reliability for computing applications that are sensitive to data corruption. Reliability is particularly important in large cluster computing environments where PPU 300 processes very large data sets and / or long-running applications.

[0067] In one embodiment, the PPU 300 implements a multi-level memory hierarchy. In one embodiment, the memory partition unit 380 supports unified memory to provide a single unified virtual address space for the CPU and PPU 300 memory, enabling data sharing between virtual memory systems. In one embodiment, the frequency of accesses by the PPU 300 to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 300 where the pages are accessed more frequently. In one embodiment, the NVLink 310 supports address translation services that allow the PPU 300 to directly access the CPU's page tables and provide full access to the CPU memory by the PPU 300.

[0068] In one embodiment, the copy engine transfers data between multiple PPUs 300 or between a PPU 300 and a CPU. The copy engine can generate a page fault for an address that is not mapped to a page table. The memory partition unit 380 can then service the page fault, map the address into a page table, and then the copy engine can perform the transfer. In conventional systems, fixed memory (i.e., non-pageable) is operated for multiple copy engines between multiple processors, which significantly reduces the available memory. Due to hardware page faults, addresses can be passed to the copy engine without worrying about whether the memory page is resident, and whether the copy process is transparent.

[0069] Data from memory 304 or other system memory may be retrieved by memory partition unit 380 and stored in L2 cache 460, which is located on-chip and shared between various GPCs 350. As shown, each memory partition unit 380 includes a portion of L2 cache 460 associated with a corresponding memory device 304. Lower level caches may then be implemented in multiple units within a GPC 350. For example, each SM 440 may implement a level 1 (L1) cache. An L1 cache is a dedicated memory dedicated to a particular SM 440. Data from L2 cache 460 may be retrieved and stored in each L1 cache for processing in a functional unit of the SM 440. L2 cache 460 is coupled to a memory interface 470 and an XBar 370.

[0070] The ROP unit 450 performs graphics raster operations related to pixel color, such as color compression, pixel blending, etc. The ROP unit 450 also implements depth testing in conjunction with the raster engine 425, receiving the depth of a sample position associated with a pixel fragment from the culling engine of the raster engine 425. The depth is tested against the corresponding depth in the depth buffer of the sample position associated with the fragment. If the fragment passes the depth test for the sample position, the ROP unit 450 updates the depth buffer and sends the result of the depth test to the raster engine 425. It will be understood that the number of partition units 380 may be different than the number of GPCs 350, and thus each ROP unit 450 may be coupled to each GPC 350. The ROP unit 450 tracks packets received from different GPCs 350 and determines to which GPC 350 the results generated by the ROP unit 450 are routed via the Xbar 370. Although the ROP unit 450 is included in Figure 4B 380 in GPC 350, but in other embodiments, ROP unit 450 may be external to memory partition unit 380. For example, ROP unit 450 may reside in GPC 350 or another unit.

[0071] Figure 5A According to an embodiment, Figure 4A The streaming multiprocessor 440. Figure 5A As shown, SM 440 includes an instruction cache 505, one or more scheduler units 510, a register file 520, one or more processing cores 550, one or more special function units (SFU) 552, one or more load / store units (LSU) 554, an interconnection network 580, and a shared memory / L1 cache 570.

[0072] As described above, the work distribution unit 325 dispatches tasks to be executed on the GPCs 350 of the PPU 300. Tasks are assigned to specific DPCs 420 within the GPCs 350, and if the task is associated with a shader program, the task may be assigned to the SM 440. The scheduler unit 510 receives tasks from the work distribution unit 325 and manages the scheduling of instructions assigned to one or more thread blocks assigned to the SM 440. The scheduler unit 510 schedules the thread blocks to execute as warps of parallel threads, where each thread block is assigned at least one warp. In one embodiment, each warp executes 32 threads. The scheduler unit 510 can manage a plurality of different thread blocks, assign warps to different thread blocks, and then dispatch instructions from a plurality of different cooperative groups to various functional units (i.e., cores 550, SFUs 552, and LSUs 554) during each clock cycle.

[0073] Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, enabling the expression of richer and more efficient decompositions of parallelism. The cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. Conventional programming models provide a single simple structure for synchronizing cooperative threads: a barrier across all threads of a thread block (i.e., the syncthreads() function). However, programmers often want to define thread groups at a granularity smaller than the thread block granularity and synchronize within the defined group, enabling higher performance, design flexibility, and software reuse in the form of a collective group-wide function interface.

[0074] Cooperative Groups enables programmers to explicitly define thread groups at sub-block (i.e., as small as a single thread) and multi-block granularity and perform collective operations such as synchronization on threads in a cooperative group. The programming model supports clean composition across software boundaries so that libraries and utility functions can safely synchronize in local environments without making assumptions about convergence. Cooperative Groups primitives enable new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across an entire grid of thread blocks.

[0075] The dispatch unit 515 is configured to send instructions to one or more functional units. In this embodiment, the scheduler unit 510 includes two dispatch units 515, which enables dispatching two different instructions from the same thread warp during each clock cycle. In alternative embodiments, each scheduler unit 510 may include a single dispatch unit 515 or additional dispatch units 515.

[0076] Each SM 440 includes a register file 520 that provides a set of registers for the functional units of the SM 440. In one embodiment, the register file 520 is divided between each functional unit such that each functional unit is assigned a dedicated portion of the register file 520. In another embodiment, the register file 520 is divided between different warps executed by the SM 440. The register file 520 provides temporary storage for operands for data paths connected to the functional units.

[0077] Each SM 440 includes L processing cores 550. In one embodiment, the SM 440 includes a large number (e.g., 128, etc.) of different processing cores 550. Each core 550 may include a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, which includes a floating-point arithmetic logic unit and an integer arithmetic logic unit. In one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point operations. In one embodiment, the core 550 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores. In one embodiment, the core 550 is configured to perform rounding operations using method 100 or 260.

[0078] The tensor core is configured to perform matrix operations, and in one embodiment, one or more tensor cores are included in core 550. Specifically, the tensor core is configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In one embodiment, each tensor core operates on a 4×4 matrix and performs a matrix multiplication and accumulation operation D=A×B+C, where A, B, C, and D are 4×4 matrices. In one embodiment, the tensor core is configured to perform rounding operations using method 100 or 260.

[0079] In one embodiment, the matrix multiplication inputs A and B are 16-bit floating point matrices, while the accumulation matrices C and D can be 16-bit floating point or 32-bit floating point matrices. The tensor cores operate on 16-bit floating point input data as well as 32-bit floating point accumulations. The 16-bit floating point multiplication requires 64 operations, producing full-precision products, which are then accumulated using 32-bit floating point additions with other intermediate products of the 4×4×4 matrix multiplication. In practice, tensor cores are used to perform larger two-dimensional or higher dimensional matrix operations built from these smaller elements. APIs (such as the CUDA 9 C++ API) expose specialized matrix loads, matrix multiplications and accumulations, and matrix storage operations to efficiently use tensor cores from CUDA-C++ programs. At the CUDA level, the warp-level interface assumes that a 16×16 size matrix spans all 32 threads of the warp.

[0080] Each SM 440 also includes M SFUs 552 that perform special functions (e.g., attribute evaluation, reciprocal square root, etc.). In one embodiment, the SFUs 552 are configured to perform rounding operations using method 100 or 260. In one embodiment, the SFUs 552 may include a tree traversal unit that is configured to traverse a hierarchical tree data structure. In one embodiment, the SFUs 552 may include a texture unit that is configured to perform texture map filtering operations. In one embodiment, the texture unit is configured to load a texture map (e.g., a 2D array of texture pixels) from memory 304 and sample the texture map to produce sampled texture values ​​for use in a shader program executed by the SM 440. In one embodiment, the texture map is stored in a shared memory / L1 cache 470. The texture unit implements texture operations, such as filtering operations using mip maps (i.e., texture maps of different levels of detail). In one embodiment, each SM 440 includes two texture units.

[0081] Each SM 440 also includes N LSUs 554 that implement load and store operations between the shared memory / L1 cache 570 and the register file 520. Each SM 440 includes an interconnect network 580 that connects each functional unit to the register file 520 and connects the LSUs 554 to the register file 520 and the shared memory / L1 cache 570. In one embodiment, the interconnect network 580 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 520 and to connect the LSUs 554 to memory locations in the register file and the shared memory / L1 cache 570.

[0082] The shared memory / L1 cache 570 is an on-chip memory array that allows data storage and communication between the SM 440 and the primitive engine 435 and between threads in the SM 440. In one embodiment, the shared memory / L1 cache 570 includes 128KB of storage capacity and is in the path from the SM 440 to the partition unit 380. The shared memory / L1 cache 570 can be used to cache reads and writes. One or more of the shared memory / L1 cache 570, the L2 cache 460, and the memory 304 are backing stores.

[0083] Combining the data cache and shared memory functions into a single memory block provides the best overall performance for both types of memory accesses. This capacity can be used as a cache for programs that do not use the shared memory. For example, if the shared memory is configured to use half of the capacity, texture and load / store operations can use the remaining capacity. The integration within the shared memory / L1 cache 570 enables the shared memory / L1 cache 570 to function as a high throughput pipeline for streaming data, while providing high bandwidth and low latency access to frequently multiplexed data.

[0084] When configured for general parallel computing, a simpler configuration can be used compared to graphics processing. Specifically, Figure 3 The fixed function graphics processing unit is shown bypassed, creating a simpler programming model. In a general purpose parallel computing configuration, the work distribution unit 325 assigns and distributes thread blocks directly to the DPC 420. The threads in the block execute the same program, use unique thread IDs in computations to ensure that each thread produces unique results, use SM 440 to execute the program and perform computations, use shared memory / L1 cache 570 to communicate between threads, and use LSU 554 to read and write global memory through shared memory / L1 cache 570 and memory partition unit 380. When configured for general purpose parallel computing, SM 440 can also write commands that scheduler unit 320 can use to start new work on DPC 420.

[0085] The PPU 300 may be included in a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, etc. In one embodiment, the PPU 300 is embodied on a single semiconductor substrate. In another embodiment, the PPU 300 is included on a system on a chip (SoC) along with one or more other devices (such as an additional PPU 300, the memory 204, a reduced instruction set computer (RISC) CPU, a memory management unit (MMU), a digital-to-analog converter (DAC), etc.).

[0086] In one embodiment, PPU 300 may be included on a graphics card that includes one or more memory devices 304. The graphics card may be configured to interface with a PCIe slot on a motherboard of a desktop computer. In yet another embodiment, PPU 300 may be an integrated graphics processing unit (iGPU) or parallel processor included in a chipset of a motherboard.

[0087] Exemplary Computing System

[0088] Systems with multiple GPUs and CPUs are used in a variety of industries as developers expose and exploit more parallelism in applications such as artificial intelligence computing. High-performance GPU-accelerated systems with tens to thousands of computing nodes are deployed in data centers, research institutions, and supercomputers to solve larger problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to scale to support this increased bandwidth.

[0089] Figure 5B According to one embodiment, the use Figure 3 300. The exemplary system 565 may be configured to implement Figure 1 The processing system 500 includes a CPU 530, a switch 510, and each of the plurality of PPUs 300 and respective memories 304. The NVLink 310 provides a high-speed communication link between each PPU 300. Figure 5B 302 connections, but the number of connections connected to each PPU 300 and CPU 530 may vary. Switch 510 interfaces between interconnect 302 and CPU 530. PPU 300, memory 304, and NVLink 310 may be located on a single semiconductor platform to form a parallel processing module 525. In an embodiment, switch 510 supports two or more protocols that interface between various different connections and / or links.

[0090] In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between each PPU 300 and CPU 530, and switch 510 interfaces between interconnect 302 and each PPU 300. PPU 300, memory 304, and interconnect 302 may be located on a single semiconductor platform to form a parallel processing module 525. In yet another embodiment (not shown), interconnect 302 provides one or more communication links between each PPU 300 and CPU 530, and switch 510 interfaces between each PPU 300 using NVLink 310 to provide one or more high-speed communication links between PPUs 300. In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between PPU 300 and CPU 530 through switch 510. In yet another embodiment (not shown), interconnect 302 provides one or more communication links directly between each PPU 300. One or more NVLink 310 high-speed communication links may be implemented as a physical NVLink interconnect or as an on-chip or on-die interconnect using the same protocol as NVLink 310.

[0091] In the context of this specification, a single semiconductor platform may refer to a unique single semiconductor-based integrated circuit manufactured on a bare die or chip. It should be noted that the term single semiconductor platform may also refer to a multi-chip module with increased connectivity that simulates on-chip operations and is substantially improved by utilizing conventional bus implementations. Of course, various circuits or devices may also be placed separately or in various combinations of semiconductor platforms, depending on the user's expectations. Optionally, the parallel processing module 525 may be implemented as a circuit board substrate, and each of the PPU 300 and / or memory 304 may be a packaged device. In one embodiment, the CPU 530, the switch 510, and the parallel processing module 525 are located on a single semiconductor platform.

[0092] In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 Gbit / s, and each PPU 300 includes six NVLink 310 interfaces (e.g., Figure 5B As shown, each PPU 300 includes five NVLink 310 interfaces. Each NVLink 310 provides a data transfer rate of 25 Gbit / s in each direction, with six links providing 300 Gbit / s. When the CPU 530 also includes one or more NVLink 310 interfaces, the NVLink 310 can be used exclusively for example Figure 5B PPU to PPU communication as shown, or some combination of PPU to PPU and PPU to CPU.

[0093] In one embodiment, NVLink 310 allows direct load / store / atomic access from CPU 530 to memory 304 of each PPU 300. In one embodiment, NVLink 310 supports coherency operations, allowing data read from memory 304 to be stored in the cache hierarchy of CPU 530, reducing cache access latency of CPU 530. In one embodiment, NVLink 310 includes support for address translation services (ATS), allowing PPU 300 to directly access page tables within CPU 530. One or more NVLink 310 can also be configured to operate in a low power mode.

[0094] Figure 5C An exemplary system 565 is shown in which various architectures and / or functions of various previous embodiments may be implemented. The exemplary system 565 may be configured to implement Figure 1 The method 100 or Figure 2D The method 260 shown in .

[0095] As shown, a system 565 is provided that includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or one or more point-to-point communication protocols. The system 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of a random access memory (RAM).

[0096] The system 565 also includes an input device 560, a parallel processing system 525, and a display device 545, i.e., a conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light emitting diode), plasma display, etc. User input can be received from the input device 560 (e.g., keyboard, mouse, touch pad, microphone, etc.). Each of the aforementioned modules and / or devices can even be located on a single semiconductor platform to form the system 565. Alternatively, the various modules can also be placed separately or in various combinations of semiconductor platforms according to the user's desire.

[0097] Furthermore, system 565 may be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) through network interface 535 for communication purposes.

[0098] The system 565 may also include secondary storage (not shown). The secondary storage 610 includes, for example, a hard disk drive and / or a removable storage drive, representative of a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk (DVD) drive, a recording device, a universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to a removable storage unit in a well-known manner.

[0099] Computer programs or computer control logic algorithms may be stored in the main memory 540 and / or secondary storage. These computer programs, when executed, enable the system 565 to perform various functions. The memory 540, storage and / or any other storage are possible examples of computer-readable media.

[0100] The architecture and / or functionality of the various prior figures may be implemented in the context of a general purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, a dedicated system, and / or any other desired system. For example, system 565 may take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head mounted display, a handheld electronic device, a mobile telephone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic.

[0101] Machine Learning

[0102] Deep Neural Networks (DNNs) developed on processors such as the PPU 300 have been used in a variety of use cases: from self-driving cars to faster drug development, from automatic image captioning in online image databases to intelligent real-time language translation in video chat applications. Deep learning is a technology that mimics the neural learning process of the human brain, constantly learning, getting smarter, and delivering more accurate results faster over time. A child is initially taught by an adult to correctly identify and classify various shapes, and eventually is able to recognize shapes without any coaching. Similarly, deep learning or neural learning systems need to be trained in object recognition and classification in order to become smarter and more efficient in identifying basic objects, occluded objects, etc., while also assigning context to objects.

[0103] At the simplest level, neurons in the human brain look at the various inputs they receive, assign a level of importance to each of these inputs, and pass outputs to other neurons for processing. An artificial neuron, or perceptron, is the most basic model of a neural network. In one example, a perceptron can receive one or more inputs that represent various features of the object that the perceptron is being trained to recognize and classify, and each of these features is given a certain weight based on the importance of that feature when defining the shape of the object.

[0104] A deep neural network (DNN) model consists of multiple layers of many connected perceptrons (e.g., nodes) that can be trained with large amounts of input data to solve complex problems quickly and accurately. In one example, the first layer of a DLL model breaks down an input image of a car into its parts and looks for basic patterns (such as lines and angles). The second layer assembles lines to look for higher-level patterns, such as wheels, windshields, and mirrors. The next layer identifies the type of vehicle, and the last few layers generate labels for the input image, identifying the model of a specific car brand.

[0105] Once a DNN is trained, it can be deployed and used to recognize and classify objects or patterns in a process called inference. Examples of inference (the process by which a DNN extracts useful information from a given input) include recognizing handwritten numbers on checks deposited in ATM machines, identifying images of friends in photos, providing movie recommendations to over 50 million users, recognizing and classifying different types of cars, pedestrians, and road hazards in self-driving cars, or translating human speech in real time.

[0106] During training, data flows through the DNN in a forward propagation phase until a prediction is produced that indicates a label corresponding to the input. If the neural network does not correctly label an input, the error between the correct label and the predicted label is analyzed, and the weights are adjusted for each feature during a backward propagation phase until the DNN correctly labels that input and other inputs in the training data set. Training complex neural networks requires a large amount of parallel computing performance, including floating-point multiplications and additions supported by the PPU 300. Inference is less computationally intensive than training and is a latency-sensitive process in which a trained neural network is applied to new inputs it has not seen before to classify images, translate speech, and generally reason about new information.

[0107] Neural networks rely heavily on matrix math operations, and complex multi-layer networks require a lot of floating point performance and bandwidth for efficiency and speed. With thousands of processing cores optimized for matrix math operations and delivering tens to hundreds of TFLOPS of performance, the PPU 300 is a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications.

[0108] Although various embodiments have been described above, it should be understood that they are presented by way of example only, and not limitation. Thus, the breadth and scope of the preferred embodiment should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following and subsequently filed claims and their equivalents.

Claims

1. A circuit for rounding off an input value, configured as follows: Receives an input value represented by a first digit; identifying a first portion of the first digit of the input value as a rounded value represented by a second digit; extracting a second value from the input value, wherein the second value includes a second portion of the first digit; aligning the rounded value with a rounded position within the second value, wherein the rounded position corresponds to a least significant bit of an output value represented by a third bit number; adding the aligned rounded value to the second value to produce a sum; and The fourth digit is truncated from the sum to produce the output value.

2. The circuit of claim 1, wherein the third bit number is smaller than the first bit number.

3. The circuit of claim 1, wherein when the input value is greater than a threshold, a fractional value replaces the rounded value.

4. The circuit of claim 1, wherein the input value is a mantissa of a number in a floating point format.

5. The circuit of claim 1, wherein the second bit number and the fourth bit number are the same.

6. The circuit of claim 3, wherein the fractional value is one of a fixed value, a programmed value, and a calculated value.

7. The circuit of claim 1, wherein when the input value is within a threshold range, a fractional value replaces the rounded value.

8. The circuit of claim 1, wherein a fractional value replaces the rounded value based on a rounding mode.

9. The circuit of claim 1, wherein: Prior to aligning the rounded value, the rounded value is modified based on a fractional value that is one of a fixed value, a programmed value, or a calculated value.

10. The circuit of claim 9, wherein the rounded value is bitwise XORed with the fractional value.

11. The circuit of claim 9, wherein: The rounded value is rotated bit by bit according to the fractional value.

12. The circuit of claim 9, wherein the rounded value and the fractional value are ANDed bit by bit.

13. The circuit of claim 1, wherein the rounded value comprises a second number of least significant bits of the input value.

14. The circuit of claim 13, wherein the second bit number is eight.

15. The circuit of claim 1, wherein the input value is a mantissa of a floating point format number and the output value is an integer.

16. The circuit of claim 1, wherein at least one of the input value, the first bit number, the second bit number, and the third bit number is included as an operand of a conversion instruction.

17. The circuit of claim 16, wherein the instruction is a format conversion instruction.

18. The circuit of claim 16, wherein the instruction is an arithmetic operation instruction.

19. A method for rounding off an input value, comprising: Receives an input value represented by a first digit; identifying a first portion of the first digit of the input value as a rounded value represented by a second digit; extracting a second value from the input value, the second value comprising a second portion of the first digit; aligning the rounded value with a rounded position within the second value, wherein the rounded position corresponds to a least significant bit of an output value represented by a third bit number; adding the aligned rounded value to the second value to produce a sum; and The fourth digit is truncated from the sum to produce the output value.

20. A non-transitory computer-readable storage medium storing instructions that, when executed by a processing unit, cause the processing unit to: Receives an input value represented by a first digit; identifying a first portion of the first digit of the input value as a rounded value represented by a second digit; extracting a second value from the input value, the second value comprising a second portion of the first digit; aligning the rounded value with a rounded position within the second value, wherein the rounded position corresponds to a least significant bit of an output value represented by a third bit number; adding the aligned rounded value to the second value to produce a sum; and The fourth digit is truncated from the sum to produce the output value.

Images