Techniques for configuring processors to execute instructions efficiently

Abstract

The present invention provides a technology for configuring a processor to efficiently execute instructions. Specifically, systems and techniques are provided for improving circuit performance while adapting to dynamic voltage drops caused by executing noisy instructions (e.g., high power consumption instructions). Performance is improved by selectively slowing down the operating frequency of each type of noisy instruction. The exemplary technology controls the clock by detecting instructions of a predetermined noise type predicted to have predefined noise characteristics (e.g., a high level of noise generated on the voltage rail of the circuit due to a larger amount of current consumed by the instruction), and in response to the detection, reduces the frequency of the clock. The detection occurs before the instruction is executed. Even if some workloads include instructions that slow down the operating frequency, changing the frequency according to the instruction type enables the circuit to operate at a high frequency.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 716,538, filed on August 9, 2018, entitled “Fast Clock Slowdown Based on Instruction Type,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to power management in processors and, more particularly, to changing clock speed and / or power instructions based on the type of instructions being executed. More particularly, the present disclosure relates to dynamically controlling clock frequency and the subsequent voltage drop associated with executing certain types of instructions in a manner that adapts to noise. Background Art

[0004] Modem processors are very complex, containing millions of transistors on a single substrate. While each individual transistor consumes only a small amount of power, the millions of transistors on the substrate together consume a large amount of power. This power consumption generates heat and also uses a large amount of electricity from the power supply. Therefore, power delivery is an important aspect of modem processor design.

[0005] The wiring used to deliver power on the processor substrate forms a power delivery network. The power delivery network includes, for example, power rails for supplying current at appropriate voltages to different components on the processor chip. The power delivery network affects many performance aspects, such as the amount of noise on the voltage rails. The term "noise" used in this document refers to transient changes in voltage that occur on the voltage rails. Although power supplies can be designed to be almost noise-free, the processor itself generates noise on its internal power delivery network. For example, executing high-volume instructions (which switch many transistors on the chip simultaneously or nearly simultaneously) can cause momentary voltage fluctuations on the voltage rails. Excessive noise on the voltage rails can cause various circuit operation failures, such as the processor not being able to obtain the power (current and / or voltage) required for its operating frequency, resulting in timing errors, incorrect switching, etc.

[0006] In more detail, one or more voltage rails are powered from a power source (such as a power supply to a processor). Voltage regulators typically control the voltage on each voltage rail to one or more pads of the processor (or an integrated circuit containing the processor) according to the power requirements of the various components of the processor. Metal vias and traces distribute the received power at the pads throughout the processor (and / or integrated circuit). One or more voltage regulators can dynamically convert the supply voltage up or down according to changes in the workload and therefore the power requirements of the processor. The nominal constant voltage of the rails leading to the processing core is called "Vdd ”.

[0007] As processors become faster and smaller, the design of the power delivery network within the chip becomes more complex. In general, the faster a processor runs, the more power it consumes. Conversely, slowing down a processor reduces the amount of power required by the various functional units of that processor. The characteristic representation of processor power consumption P makes this relationship between power and frequency clear: P = CV 2 f, where C, V, and f represent the switching capacitance of the processor, the voltage seen by the processor, and the operating frequency of the processor, respectively.

[0008] The speed at which a circuit operates is affected by the voltage driving the circuit. Voltage fluctuations (noise) on the voltage rail can cause the voltage of one or more processor circuits to be different from the nominal voltage. The main factor affecting the noise amplitude is the workload that the circuit (or other circuits on the chip) is processing. In particular, the dynamic resistance drop (also called "IR drop") component of the noise is affected by the current I consumed (drawn) by a specific instruction when it is executed. There are two main components of noise: Di / dt and IR drop. Di / dt noise depends on the rate of change of current over time, while IR drop depends only on the magnitude of the current. Noise-aware frequency-locked loop (NAFLL) technology can be used to address the Di / dt component of noise. The technology described in this disclosure is mainly aimed at IR drop.

[0009] Different instructions cause different numbers of transistors to switch, draw different amounts of current, and therefore cause different amounts of IR drop on the voltage rail. By way of a crude analogy, consider a muggy day in the middle of a summer workday. The thermometer reaches 100 degrees Fahrenheit (38 degrees Celsius). When everyone in a city gets home from get off work, the first thing they do is turn on the air conditioner to cool down their living spaces. The grid may not be designed to handle such high peak current draws, resulting in "browns" - voltage fluctuations that deviate from the nominal 110VAC that the grid is designed to deliver. Now consider this situation in microscopic detail on a modern processor chip, where millions of transistors operate thousands of times per second. This is like turning hundreds of a city's air conditioners on and off synchronously many times per second. The resulting power fluctuations will generate a lot of noise on the city's power delivery network.

[0010] As processors become smaller and more complex, increases in power density between circuits can further exacerbate problems associated with noise generated on voltage rails by executing certain instructions.

[0011] In some processors, such as Nvidia graphics processing units (GPUs) from the past few years, the voltage V ddThe operating frequency (i.e., speed) under is conservatively set so that even the noisiest expected workloads will produce appropriate operation. In addition, features (such as NAFLL) have been implemented to address the effects of noise on the power distribution network by tracking the noise on the voltage rail and dynamically changing the clock frequency in response. In particular, when a voltage drop is detected on the voltage rail, the NAFLL mechanism reduces the clock frequency (i.e., the rate at which the processor transistors switch) to give the transistors more time to operate correctly and ensure that there are no circuit failures. This is effective because transistors require higher voltages to switch faster. Reducing the switching rate enables the transistors to operate at lower, declining voltage levels. When the drop disappears, the clock frequency can be increased. For example, U.S. Patent Publication No. 2018 / 0123604 describes an example NAFLL that follows this principle.

[0012] However, as processors continue to become smaller, faster, and more complex, technologies such as NAFLL may not be able to fully address the problems and inefficiencies associated with noise on the voltage rails. Therefore, further improvements are desired to adapt processors to the effects of noise for different instruction types. Summary of the invention

[0013] Exemplary embodiments remedy some of the deficiencies of the above-described techniques for addressing noise-related voltages introduced in a processor. Systems and methods are disclosed for controlling the operation of a processor in a manner that is adaptive to noise on a voltage rail.

[0014] An exemplary embodiment provides a method of controlling a clock of a processor. The method includes detecting an instruction predicted to have a predefined noise characteristic, and in response to the detection, changing the frequency of the clock. The detection may occur before the processor executes the instruction.

[0015] According to another exemplary embodiment, a system is provided that includes a processing circuit and a clock generator that controls the operating frequency of the processing circuit. The processing circuit is configured to detect an instruction that is predicted to have a predefined noise characteristic and, in response to the detection, signal the clock generator to change the frequency of the clock generator. The detection may occur before the instruction is executed in the processing circuit. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A A block diagram of a processing system including a fast clock slow down circuit is shown, according to certain exemplary embodiments.

[0017] Figure 1B A block diagram of a processor implementing fast clock slowdown is shown, according to certain exemplary embodiments.

[0018] Figure 1C is another example processor that implements fast clock slowing according to some example embodiments.

[0019] Figure 2A A process for performing a fast clock slowdown is shown in accordance with some example embodiments.

[0020] Figure 2B is a timing diagram illustrating clock slowdown signals and corresponding acknowledgement signals that may be used in a processor incorporating rapid clock slowdown, according to some exemplary embodiments.

[0021] Figure 2C The change of clock frequency over time when performing a fast clock slowdown is shown according to some example embodiments.

[0022] Figure 2D A timing diagram illustrating halting execution of instructions during a fast clock slowdown is shown in accordance with some exemplary embodiments.

[0023] Figure 3 A parallel processing unit according to one embodiment is shown.

[0024] Figure 4A According to one embodiment, Figure 3 A general-purpose processing cluster within a parallel processing unit.

[0025] Figure 4B According to one embodiment, Figure 3 A memory partitioning unit for parallel processing units.

[0026] Figure 5A According to one embodiment, Figure 4A Streaming multiprocessor.

[0027] Figure 5B According to one embodiment, using Figure 3 Conceptual diagram of a processing system implemented with a parallel processing unit (PPU).

[0028] Figure 5C An exemplary system is shown that may implement the various architecture and / or functionality of various previous embodiments.

[0029] Figure 6 According to one embodiment, Figure 3 Conceptual diagram of the graphics processing pipeline implemented by the PPU. DETAILED DESCRIPTION

[0030] Certain exemplary embodiments of the present invention dynamically control the operating frequency of a processor and / or integrated circuit to reduce inefficiencies built into conventional techniques in order to handle dynamic voltage drops caused by noise generated by certain instructions on a power distribution network. For example, some embodiments provide for detecting noisy workloads based on instructions in an instruction pipeline, and quickly slowing down the clock frequency before executing detected noisy instructions. Moreover, more exemplary embodiments provide for detecting and responding to such noisy workloads in local processing areas that have not been detected by techniques such as NAFLL.

[0031] As mentioned above in the Background section, in some current commercial processors, the operating frequency is set so that even the noisiest expected workload can be executed at a given nominal voltage or set voltage (i.e., the frequency is reduced to compensate for the worst possible drop from the nominal voltage). However, this approach is suboptimal because it requires maintaining a frequency margin for all other workloads that produce less noise than the worst-case (e.g., noisiest) workload. That is, by setting the frequency based on the noisiest workload, which requires a lower frequency for stable operation, even less noisy instructions are forced to execute at the reduced frequency. For example, in the example of a commercially available processor, at V dd When the voltage drop is 900mV, the operating frequency for all instructions is set to 1871MHz based on the voltage drop predicted to be caused by the noisiest instruction type, while all other instruction types can safely operate at a faster frequency of 1886MHz, which means that when instructions other than the noisiest instruction types are being executed, about 15% of the frequency margin remains unused. dd , similar margins ranging from 2-26% have been observed. This slows down the effective processing rate, which will affect how quickly a high-performance processor can complete complex tasks.

[0032] Furthermore, in NAFLL implementations in existing processors, voltage / noise sensing is performed only at locations on the chip where the NAFLL logic is physically located. Therefore, the NAFLL may miss noise / voltage drops in other parts of the chip that are localized and do not propagate to the NAFLL sensing location. In particular, IR drop can be a highly localized phenomenon. Circuits on chips with high current density have higher IR drops than detected at the NAFLL. Noise that is not detected by the NAFLL and therefore not compensated for needs to be margined and has been shown to result in a frequency loss of approximately 3-4% for workloads that produce lower noise. Furthermore, the response of the NAFLL occurs after the noise / voltage has dropped.

[0033] The exemplary embodiment proactively slows the clock frequency based on the instruction type to accommodate the expected or predicted noise (primarily consisting of IR drops) on the voltage rail. Compared to NAFLL, which responds after the noise / voltage has dropped, the exemplary embodiment is predictive and responds to the predicted noise / voltage drop before the drop occurs. In some embodiments, the clock frequency slowdown compensates for the predicted noise that is not compensated by NAFLL. By selectively slowing down one or more clocks only for instructions of a type that is expected to generate a high level of noise, the exemplary embodiment provides instructions that generate lower noise to be clocked at a faster rate than noisy instructions, thereby improving the performance of at least the lower noise instructions. Therefore, in a typical workload, most instructions are of the lower noise type, and the exemplary embodiment can also improve the overall performance of the processor and / or an integrated circuit having the processor.

[0034] Some exemplary embodiments take advantage of the property that a large inrush current that causes increased noise on the voltage rails occurs during instruction execution after the instruction has been decoded. This allows one or more clocks to be modulated based on the decoded instruction, but before execution of the decoded instruction begins, the clocks are actively offset to account for frequency loss due to noise generated during instruction execution. By quickly modulating one or more clocks to reduce the frequency before executing certain noisy instructions, exemplary embodiments can provide for the processor to operate at a reduced frequency only to execute those noisy instructions. base Fast clocks slow down the system depending on the type of instruction

[0035] Figure 1A is a block diagram of a system 100 for fast clock slowing based on instruction type according to some exemplary embodiments. The system 100 is configured to actively slow down the clock frequency based on instruction type in order to accommodate noise (primarily consisting of IR drop) on the voltage rail. The fast clock slowing in the exemplary embodiments is achieved through coordination between detection and signaling circuitry, actuation circuitry for clock slowing, and clock generation circuitry. In an exemplary embodiment, the processor 102 may include a detection and signaling component, and the actuation component and the fast clock slowing circuitry may be in the clock logic 114 associated with the processor. In the fast clock slowing, a detection and signaling component may be provided to the processor 102 for operation in a manner such as a clock source. Figure 3 In an exemplary embodiment implemented in a parallel processing unit PPU (e.g., a graphics processing unit (GPU)) as shown in FIG, the detection and signaling components may be implemented in a streaming multiprocessor (SM, such as a GPU) of the PPU. Figure 4A However, embodiments are not limited to GPUs and other parallel processing units, and may include any instruction processing circuit that relies on a controllable clock signal.

[0036] The processor 102 receives power from one or more voltage regulators 110 via one or more power rails 112. The one or more voltage regulators 110 control the voltage from a power supply unit (not shown) according to the requirements of the processor 102. In some exemplary embodiments, the one or more voltage regulators 110 can control the voltage supplied to the processor 102 so that one or more voltage rails to the processor have a different voltage than the voltage on another rail to the processor. The power received via the voltage rails 112 is distributed to various power domains of the processor, including various functional units (e.g., arithmetic and logic unit (ALU), floating point unit (FPU), load / store unit, branch unit, dedicated execution path, SIMD execution unit, etc.) 106 to the data path 104 of the processing circuit. The data path 104 (and other components of the processor 102 or an integrated circuit including the processor 102) receives a clock signal 116 from one or more clock generators in the clock logic 114.

[0037] If the reduced frequency required by the noisy instruction is set as the clock frequency of all instructions executed on the processor 102, as described above, the voltage and frequency margin that is not used during the execution of the less noisy instruction will result in inefficiency. The exemplary embodiment avoids setting the clock frequency based on the worst-case noisy instruction by utilizing the characteristics of the increase in noise that occurs during instruction execution and after the instruction has been decoded in the instruction pipeline 108 as described above. This allows one or more clock signals 116 to be modulated based on the decoded noisy instruction before the execution of the decoded instruction begins, actively offsetting the clock frequency to account for the voltage loss suffered due to noise during instruction execution. When the instruction reaches the execution stage in the instruction pipeline, the execution of the instruction begins and is dispatched to be executed on the data path 104. More specifically, when the various functional units of the data path are activated according to the noisy instruction, a surge in current occurs and therefore an increase in noise occurs, so the clock frequency in the exemplary embodiment is slowed down before the various functional units in the data path begin to execute the noisy instruction.

[0038] The instruction pipeline 108 can be a pipeline of any number of pipeline stages, including separate pipeline stages for instruction decoding and instruction execution. An exemplary pipeline can have separate pipeline stages for instruction fetch, instruction decoding, instruction execution, memory access, and writing. The pipeline enables different portions of the processing and control circuitry to be used simultaneously by different instructions in the pipeline.

[0039] The instruction detection and signaling block 118 includes logic for monitoring the instruction pipeline 108. More specifically, the block 118 includes logic for detecting when an instruction that can cause a certain predetermined level of noise on the voltage rail is decoded in the pipeline 108. The detection can be based on an instruction type code (i.e., an opcode) and / or an instruction type code and operand combination. The instruction type to be detected can be hard-coded and / or can be specified in a programmable table. When hard-coded, according to some embodiments, the classification can be done using the current (I) level as a representative of the noise level. The classification can be a relative ranking, rather than an absolute ranking (e.g., "Instruction A: 1 amp"; "Instruction B: 2 amps"; "Instruction C: 2.1 amps"; "Instruction D: 3 amps"). Assuming a constant resistance (R), I can be considered a good representative of IR. Thus, for example, instruction A can be classified in a low power group, instructions B and C can be classified in a medium power group, and instruction D can be classified in a high power group. The specific clock slowdown amount required for each group can be characterized on silicon and programmed in the slowdown table. Therefore, as is clear from the above description, in certain embodiments, actual noise per instruction is not necessary for grouping.

[0040] Note that although the exemplary embodiment detects instructions that may cause a certain level of noise, the embodiment compensates for frequency loss when running high power instructions. Frequency loss may also be due to jitter and / or other second order components, but most frequency loss may be due to noise.

[0041] The instruction detection and signaling block 118 may also include logic for signaling with the clock logic 114 in response to detecting a predetermined instruction type in the pipeline 108. For example, the signaling to the clock circuit 114 (also referred to as the clock logic) may include asserting a clock slow signal 120 in response to detecting a noisy instruction in the pipeline 108.

[0042] Additionally, in at least some embodiments, the instruction detection and signaling block 118 may also include interlock logic for stalling the pipeline 108 until the clock is slowed down in response to detecting a noisy instruction in the pipeline.

[0043] although Figure 1A The clock logic 114 is shown as being external to the processor 102, but the embodiments are not limited thereto. Exemplary embodiments may include various combinations of processing circuitry and clock logic, for example, processing circuitry and clock logic on the same die, a single clock per processor, a single clock shared by multiple processors, etc. wherein the communication between the processing circuitry and the clock logic is fast and efficient enough that a clock slowdown can be requested and validated during the time interval between instruction decoding and the start of its execution.

[0044] Figure 1B1 shows a block diagram of a processor 130 according to some exemplary embodiments. According to some embodiments, the processor 130 may be identical to the processor 102, but the clock logic is located on the same die as the processor 102. Figure 1A As shown, processor 130 also receives power via one or more voltage rails controlled by one or more voltage regulators. Neither the voltage rails nor the voltage regulators are shown separately. Figure 1B middle.

[0045] Processor 130 includes one or more processing cores 132 (eg, core A and core B), and clock logic 133. Clock logic 133 provides clock signal 158 to control the operating frequency of one or more processing cores 132.

[0046] The noisy instruction detection and stopping block 136 (also referred to as a noisy instruction detection block) is configured to detect certain predetermined instruction types 141 in the instructions 140 in the instruction pipeline 134 prior to the execution stage of the pipeline, where the instructions are executed by one or more processor cores 132. The predetermined instruction types 141 (also referred to as "noisy instruction types") to be detected can be hard-coded and / or specified in a programmable table. According to some embodiments, the instructions detected for clock slowing include certain high power instructions. That is, the detected instructions are instructions that are expected to consume an amount of power above a certain threshold. For example, certain matrix multiplication instructions utilize more functional units in the data path and / or use data paths with very high gate density, and therefore are more prolific power consumers than many other instruction types. For example, according to one embodiment, a half-precision matrix multiply accumulate (HMMA) instruction can utilize an SM (e.g., with respect to Figure 4A The dedicated high gate density HMMA data path in the described) is more power consuming than many instructions, and it is also prone to generating more noise on the voltage rail than many other instructions. The noise of the HMMA instruction and the resulting voltage drop may be confined to the vicinity of the HMMA data path. For example, HMMA instructions are described in U.S. application No. 15 / 826,435 filed on November 29, 2017, the entire contents of which are incorporated herein by reference. Therefore, block 136 can monitor the instruction pipeline to detect when a high-power instruction is decoded (or is being decoded). However, some embodiments may be configured to detect one or more noisy instructions, which are not necessarily high-power instructions, but which generate a lot of noise on the voltage rail (e.g., due to the density of simultaneously activated portions of the data path, etc.).

[0047] Clock control signaling block 138 is configured to assert a clock slow signal 154 (also referred to as a clock slow request signal) in response to detection by noisy instruction detection block 136. Clock slow signal 154 is a request to clock logic 133 to slow down a clock signal 158 provided to processor core 132.

[0048] In some embodiments, clock logic 133 may provide a clock slow down confirmation signal 156 in response to receiving clock slow down request signal 154 , indicating that it has implemented the requested clock slow down.

[0049] In some embodiments, the noisy instruction detection and stop block 136 also includes logic for stopping the instruction pipeline 134 in response to activating the clock slowdown request signal 154 until it is determined that the clock frequency has been slowed as requested. Upon receiving the confirmation signal 156, the pipeline may not be stopped so that the detected noisy instruction can be dispatched for execution. The stop helps ensure that the detected noisy instruction is not executed before the clock is slowed down in response to the detection. In some embodiments, where the delay between instruction decoding and execution is too small for the clock slowdown to start, block 136 may also include logic for stopping.

[0050] exist Figure 1B In the embodiment shown in , clock logic 133 is provided on the same die as one or more processor cores 132. However, in some embodiments, clock logic 133 may not be located on the same die as the processor cores that detect and / or execute noisy instructions.

[0051] The signaling paths of signals 154 and 156 between one or more processor cores 132 and clock logic 133 are configured for low latency. In an embodiment, it is highly desirable to maintain the delay on the signaling path and the timing of the clock to achieve slowdown so as not to delay the instruction pipeline. In some embodiments, request / acknowledgement messages (e.g., REQ / ACK bits) can be moved between the SM and the clock through a retiming stage. For example, such a retiming stage can be custom selected and / or custom placed. Between retiming flops, data can be routed in a manner similar to the main line clock, for example, through a large clock buffer to drive a wide upper metal layer shielding mesh, using custom routing rules that eliminate expensive routing. In some embodiments, the delay level between decoding and execution can be 12 cycles or the like, and the delay between request and clock change can be 10 cycles or the like. Therefore, in an exemplary embodiment, the clock slowdown occurs before the decoded instruction reaches execution.

[0052] The clock logic 133 may include a clock generator block 142 and a clock update block 148. In some embodiments, the clock generator block 142 may include a clock source, such as a digital voltage controlled oscillator (DVCO) clock source, a NAFLL mechanism, such as described in U.S. Patent Application No. 15 / 340,901 filed on November 1, 2016 (U.S. Patent Publication No. 2018 / 0123604), the entire contents of which are incorporated herein by reference.

[0053] However, the clock frequency changing techniques described in U.S. Patent Application No. 15 / 340,901 are not sufficient to meet the time constraints of the rapid slowdown required in certain exemplary embodiments in order to achieve slowdown in a short time interval between the decode and execution pipeline stages of instructions. In some embodiments, the clock frequency change is achieved by performing a lookup in a lookup table such as any of Tables 144 and 146. In embodiments where the clock frequency is configured to be either "full speed" (e.g., the maximum possible operating frequency with no margin for voltage noise) or "reduced speed" (e.g., the maximum possible operating frequency with margin for noise), lookup table 144 may provide a voltage-frequency curve (VF curve) for full speed operation, and table 146 may provide a VF curve for reduced speed operation. In response to a clock slowdown request 154, some embodiments may change the frequency at the current voltage to the VF curve in Table 146 (reduced speed operation).

[0054] In another embodiment, there may be no lookup table and the clock source or other clock generator may be set to slow the clock to the requested frequency.

[0055] Incoming clock slowdown requests may be aggregated at block 152. For example, block 152 determines the clock slowdown request that requests the maximum slowdown among the slowdown requests 154 received from the various processing cores 132, and then forwards it to the clock generator block 142. This ensures that the adjusted clock frequency is equal to or lower than the clock frequency required to safely execute all instructions at any time.

[0056] The hysteresis block 150 may be located in a signal path 154 between the aggregation block 152 and the clock generator block 142. In some exemplary embodiments, although the clock slow down request is immediately forwarded to the clock generator block 142 (i.e., sent to the clock generator without being subject to hysteresis), the clock slow down de-assert may be subject to a hysteresis process before being forwarded to the clock generator block 142. Subjecting the de-assert to hysteresis may minimize or entirely avoid switching the clock frequency back and forth between a nominal normal operating frequency and one or more reduced frequencies.

[0057] The clock update block 148 controls the update clock and may also generate a clock slow down confirmation signal 158 when clock slow down has been achieved. The clock update block 148 may initiate an update of the clock frequency by the clock generator 142 in response to receiving the clock slow down signal 154 asserted or deasserted.

[0058] Figure 1C A block diagram of another exemplary processor implementing fast clock slowing is shown in accordance with some exemplary embodiments. Figure 1C The embodiment shown in implements a finer level of slowdown, thereby achieving better performance in the case of workloads with instructions that consume significantly varying amounts of power (e.g., when compared to an implementation with only a single frequency slowdown level).

[0059] It has been observed that various computational instructions used during certain applications vary significantly in the amount of power consumed when executed in the data path. This variation in the power consumed by various instruction types when executed in the data path is expected to result in variations in noise and IR drop for different instruction types. On some example processors, the variation can be in the range of 10-12%, depending on the instruction type. Based on estimates that every 1% of noise can result in approximately a 2% slowdown, this variation in power levels can cause the processor to slow down more than necessary. Figure 1C Implementing multiple levels of slowing down as shown in the embodiments of can achieve better performance when there are more than two different instruction types in the workload, which consume corresponding significantly varying amounts of power and thus produce corresponding significantly varying amounts of noise on the voltage rails.

[0060] The parallel processor 161, for example, Figure 3 The PPU shown in FIG. 1 may include one or more processors (e.g., processors 160a and 160b). Each processor, such as processor 160a, may include one or more processing cores 162 (also referred to as processor cores). For example, regarding Figure 3 In the embodiment shown in FIG. 1 , processor 160 a may correspond to GPC 350, and each processing core 162 may correspond to a corresponding streaming multiprocessor (SM) 440 (see Figure 4A ).

[0061] In each processing core 162, instructions 170 enter the pipeline 164 where they are decoded and subsequently dispatched for execution. A noisy instruction detection and stop block 166 monitors the pipeline 164 to detect when any instruction in a predetermined instruction set 171 is decoded. For example, block 166 may monitor the decoding of any instruction (e.g., a high power instruction) from a predetermined noisy instruction set 171 (also referred to as a noisy instruction type). When one or more instructions in the predetermined set are detected, a clock slow down request signal is generated.

[0062] In About Figure 1C In the described embodiment, the clock slow down request signal 182 (e.g., signals 182a and 183b) can depend on the type of instruction detected (e.g., different types of instruction types detected can result in different clock slow down signals). The detected instructions are classified according to their estimated noise generation. The classification can be performed according to the programmable classification in the instruction noise category block 167. The programmability enables the classification of pre-silicon to post-silicon based on simulation modifications on the actual chip. The noise caused by the instruction can vary between chips and / or between workloads, and the programmability enables the classification to be adjusted to suit the specific environment.

[0063] Can be classified according to multiple groups of instruction types, wherein each group in one or more groups of instruction types is mapped to corresponding clock offset.Can determine each group of instruction types according to the noise amount of expected instruction generation of every instruction type.For example, all instruction types expected to generate noise level in the first scope can be in different groups with all instruction types expected to generate noise level in the second scope.The clock offset specified in the mapping of the group can be the amount of slowing down of the current frequency required for compensating the voltage drop caused by the expected noise level generated by the instruction of the instruction type in the group.

[0064] In some embodiments, the mapping is programmable, so that post-silicon reprogramming or reconfiguration of the mapping of instruction types to specific groups can be performed. In this way, the initial mapping made based on pre-silicon design considerations and noise estimates can be subsequently overwritten based on more accurate noise levels of instruction types observed in post-silicon simulations. The pre-silicon and post-silicon terms used here refer to the chip design stage before sending it to manufacturing (tapeout) and the chip after sending it to manufacturing, respectively. Because the noise level generated by each type of instruction in the actual circuit (i.e., post-silicon) can and usually does differ greatly from the pre-silicon estimate, the programmability of the mapping provided in some embodiments enables instruction types to be reclassified into groups based on post-silicon simulations, which can significantly improve the performance of the processor. In some embodiments, the clock offset associated with each group may also be programmable.

[0065] Based on the detection and classification, the clock control signaling logic 168 is configured to generate one of the clock slowing request signals 182a and 182b for each processor core 162. In an exemplary embodiment, the clock slowing request signals 182a and 182b can be mapped to a first class of instruction types for high power consumers and a second class of instruction types for medium power consumers, respectively. Therefore, the clock slowing request signal 182a can request a greater slowing than the signal 182b.

[0066] In an exemplary embodiment, for each slowdown level added, a separate wire may need to be routed from the processor core 162 to the clock block 163 (also referred to as clock logic). In the illustrated embodiment, separate wires (i.e., clock slowdown request signal 182a and clock slowdown request signal 182b) are provided to each of the two slowdown levels. Figure 1A and Figure 1B Note that the conductors used to send the clock slow signal must be formed so as to minimize delays in order to meet the strict timing constraints required to send the clock slow signal and receive the slowed clock signal in the time interval between the instruction decode and execution pipeline stages.

[0067] For each deceleration level, signals from all processor cores 162 can be aggregated with corresponding aggregation logic (e.g., aggregate clock request blocks 183a and 183b). In an exemplary embodiment, aggregation can include subjecting the input signals to a logical or operation. In some embodiments, aggregation can include subjecting groups of processor cores 162 to a preset threshold of high noise that determines whether any corresponding group exceeds a high noise that triggers a clock slowdown request. These groups can be predetermined based on the physical layout of the cores, adjacencies, and / or other considerations of the power distribution network. In some embodiments, weights can be assigned to clock slowdown requests based on the characteristics of the triggering cores and / or groups. The aggregate clock slowdown request signal from each aggregation block is provided to the clock logic 163.

[0068] The clock logic 163 may include a priority encoder 190 that receives clock slowdown requests for each of a plurality of slowdown levels and ensures that the resulting clock frequency 188 is at the lowest requested level. Thus, between the priority encoder 190 and the clock generator 172 (also referred to as a clock generation block), only one of 182a and 182b is active. This is important to ensure that there are no failures in the data path that executes the noisiest (e.g., most power-hungry) instructions. When clock generators are shared between cores, this priority encoding may be performed to ensure that there are no failures in the data path that executes the noisiest instructions. Such priority encoding is not required between different clock generators. Thus, for example, at least some Nvidia GPUs have a GPC that has a collection of TPCs and NAFLLs. In some embodiments, priority encoding may be across TPCs in a GPC, but not across GPCs, since each GPC has its own NAFLL.

[0069] In some embodiments, hysteresis block 180 may be triggered (e.g., a hysteresis counter may be started) any time a clock change occurs, even within the slowdown level. Hysteresis is only triggered when the clock slowdown signal is de-asserted to prevent the frequency from switching back and forth between frequencies.

[0070] The lookup table 174 may include one or more frequency entries for each voltage. In accordance with an embodiment in which either of the two deceleration levels 192 and 194 may be signaled to the clock generation block 172, the clock update logic 178 and / or the clock generation block 172 may access frequency settings for immediate transitions and long-term locks. For example, three frequency entries for each voltage may be accessed in the lookup table 174. For each voltage level, ndiv, ndiv_offset, and dvco_offset may be obtained from the lookup table, which are used by the NAFLL logic 196 to set the clock frequency. ndiv provides a long-term frequency target for low noise instructions, while (ndiv-ndiv_offset) represents a long-term frequency target for high noise instructions. dvco_offset provides an immediate pull-down code to quickly change the clock frequency from ndiv to a low (ndiv-ndiv_offset). The following is combined with Figure 2C The change of the clock frequency is described in more detail.

[0071] The acknowledge signal 186 may also need more than one bit to achieve a smooth process of slowing down the switching. The multi-bit acknowledge signal 186 enables the stall logic in the processor to properly handle the acknowledgement.

[0072] Fast clock slowing method based on instruction type

[0073] Figure 2A A process 200 for performing fast clock slowdown according to some exemplary embodiments is shown. The process 200 may be performed by Figure 1A , Figure 1B and Figure 1C In some exemplary embodiments, process 200 is implemented entirely in hardware. According to exemplary embodiments, process 200 is at least partially implemented by a SM (e.g., Figure 4A ) as shown in .

[0074] At operation 202, when an instruction pipeline (e.g., Figure 1A , Figure 1B and Figure 1C When one or more noisy instructions are detected in any processor (as shown in any processor shown in ), process 200 is initiated. The types of noisy instructions that the instruction pipeline is monitored for can be predetermined and either hard-coded, specified in one or more registers, or specified in a programmable table. In an exemplary embodiment, each instruction entering the instruction pipeline is tested to determine whether its instruction type matches any of the one or more predetermined instruction types.

[0075] At operation 204, the detected one or more instructions are grouped according to the noise level expected to be generated by the detected instructions. After decoding the instructions in the instruction pipeline, the programmable grouping logic enables the decoded instructions to be grouped into a bucket (e.g., a logical grouping) with other instructions of similar noise characteristics. Each instruction bucket is configured to assert a clock slow signal, which enters the clock logic.

[0076] The programmability of the grouping enables instructions to be moved from one bucket to another based on post-silicon learning, and allows any errors in the pre-silicon power characterization of instructions (e.g., inaccurate estimates of the noise levels associated with various instruction types) to be corrected on silicon. For example, simulations performed on a post-silicon die may show that the noise level of a particular instruction type is different from the noise level corresponding to the grouping / bucket into which the particular instruction was grouped based on pre-silicon estimates. In this case, the programmability of the grouping enables a particular instruction type to be remapped to a group different from the group to which it was originally mapped based on pre-silicon estimates of the noise generation characteristics of the particular instruction.

[0077] At operation 206, in response to detecting a noisy instruction at operation 202, a clock slow down request signal is asserted. In some embodiments, the clock slow down signal may include an associated clock modulation or slow down value specified as a frequency offset or frequency. The value of the frequency offset or frequency may be obtained from a mapping of a group associated with the clock slow down signal. In some embodiments, the clock slow down signal may include an associated voltage or voltage offset instead of a frequency or frequency offset.

[0078] At operation 208, a dispatch stop signal is asserted to stop the detected instruction before execution. Dispatch stop is optional, and some embodiments may proceed from operation 206 to operation 210 without performing operation 208. For example, in some embodiments, the pipeline may not be stalled where it can be reliably expected that the clock frequency can be adjusted to a reduced frequency between the time when the clock slow request is asserted during or at the end of the execution phase of the instruction pipeline and the time when the clock slow request is asserted. However, in embodiments where it cannot be reliably expected that the clock frequency will be adjusted in response to the clock slow signal at a time before execution begins, the pipeline may be stalled until the requested clock slow is asserted.

[0079] In order not to significantly degrade performance, signaling from the processor core (e.g., SM) to the clock logic should have very low latency. The physical design and the load on the signal must be carefully controlled to ensure very low latency. Ideally, the latency is low enough to slow down the clock before the instruction reaches the execution pipeline after the signal is valid, thereby allowing uninterrupted instruction progress. However, given the size of the chip, this may not always be possible, and the need for interlock logic is not always necessary. In some embodiments, the interlock logic stops the execution of one or more instructions in the processor core until the clock logic validates the signal that the clock has been slowed down. After the acknowledgment signal, all stopped processor cores are unstopped in an interleaved manner (e.g., according to a software configurable delay for each core) to ensure that the current gradually climbs and avoids di / dt noise problems.

[0080] In response to asserting the clock slow request signal, the clock frequency is reduced at operation 210. When multiple processor cores are clocked by the same clock source, the clock slow signals asserted by the various processor cores are aggregated (eg, "ORed") per group before being interfaced with the clock logic.

[0081] The clock logic includes fast clock slowdown logic and hysteresis logic, and may also include a priority encoder. In addition, since in some embodiments, the fast clock slowdown works with the NAFLL clock logic, the clock logic includes logic for ensuring that the frequency locked loop function of the NAFLL can be maintained with the fast clock slowdown.

[0082] In some embodiments where multiple clock slowdown levels may be requested, a priority encoder may be used to ensure that the clock is set to honor the largest slowdown request corresponding to the highest current instruction that will be executed on any one processing core clocked by the NAFLL. In embodiments where there is only one slowdown level (in addition to the normal or full speed operating frequency), a priority encoder may not be needed.

[0083] In some embodiments, each clock slowdown signal has a programmable slowdown value associated with it. When the signal is active, the frequency is reduced by the associated slowdown amount (after the priority encoder) within a predetermined number of clock cycles by a fast slowdown mechanism within the NAFLL. According to one embodiment, the predetermined number of cycles is 1.

[0084] In some exemplary embodiments, clock generation inside the NAFLL is through a DVCO made of tunable parallel drivers. In normal operation, multiple parallel drivers of the DVCO are tuned until the frequency matches the desired frequency of the NAFLL. However, a typical frequency transition from one state to another is done in a small tuning change and requires multiple microseconds. In an exemplary embodiment, a fast tuning change mechanism controlled by a side-band is used to change the frequency within one clock cycle. In some embodiments, the DVCO may include an input (e.g., offset_enable) to immediately reduce the number of parallel drivers (to reduce the frequency) by a preset amount (e.g., offset) for a short duration (e.g., one or two clock cycles). Decoder logic may be included in the DVCO, which applies a specified offset only when indicated by an input to the DVCO (e.g., offset_enable=1). This fast offset mechanism is different from the main tuning mechanism of the DVCO. The preset offset amount can be a function of the operating voltage. This low-latency frequency change mechanism helps to meet the strict timing requirements of exemplary embodiments.

[0085] Additionally, in contrast to clock slowing techniques (such as LDIV or clock skipping), the fast tuning change mechanism employed in some exemplary embodiments can slow the clock frequency by a small percentage (e.g., 2%, 4%, etc.) while maintaining the duty cycle of all cycles.

[0086] Hysteresis logic is used to ensure that the NAFLL does not switch between slowdown modes too frequently, which could cause instability in the frequency lock of the NAFLL. Switching from a smaller slowdown mode to a larger slowdown mode occurs quickly. Hysteresis may only apply from a larger slowdown to a smaller slowdown, or when a slowdown is released.

[0087] Additional logic may also be required to maintain the frequency-locked loop functionality of the NAFLL while engaging and disengaging the various slowdowns. In some embodiments, two VF curves, "full speed" and "slowed down," are separately characterized in silicon, each representing the maximum frequency (F) of the workload that its instructions accommodate. max ). These frequency targets are provided to the NAFLL's frequency locked loop for the current voltage. A tunable error accumulator can be used to keep the NAFLL's tunable DVCO locked to the target frequency - usually the "full speed" frequency.

[0088] When entering the slowdown state, the NAFLL's control loop can temporarily stop tracking the target "full speed" frequency and immediately apply its fast slowdown mechanism to reduce the frequency and allow the slowdown instruction to execute. It will then try to slowly re-lock to the "slowdown" frequency, assuming that the fast slowdown mechanism may have additional margin to account for PVT (process voltage temperature) variations or reduce characterization efforts.

[0089] When leaving the slowdown state after the hysteresis window has expired, the NAFLL can again suspend target frequency tracking. The internal sequencer can optionally apply a static, transient, negative frequency offset to reduce the risk of overshoot later (e.g., due to PVT variations or characterization uncertainties). It can then disengage the fast slowdown mechanism to restore the frequency to less than or equal to the original "full speed" frequency. Finally, it moves its long-term lock target to the "full speed" frequency, which will gradually restore the frequency to the pre-slowdown frequency. Figure 2C The behavior of the clock frequency when the fast slow down mechanism is disengaged is shown.

[0090] At operation 212, in some embodiments, the clock slow down confirmation signal is asserted by the clock logic. Once the slow down is initiated, the confirmation can be sent to the processor core, which can be used to release the stall of instruction execution in the case where the loop delay of the clock slow down is large.

[0091] With the processor core and clock logic working together, the clock frequency can be quickly modulated based on the instruction type, allowing instructions of different power to execute at their respective optimal frequencies.

[0092] At operation 214, in response to the clock slow down confirmation signal, instruction execution is not stopped, and the instruction is dispatched for execution. In the case where the instruction pipeline has been stopped, upon receiving the confirmation signal, the detected instruction can be dispatched for execution in the data path. As described above, instruction execution includes activating the data path to execute the instruction.

[0093] At operation 216, one or more noisy instructions are executed at a reduced clock frequency. In other words, one or more noisy instructions are executed by a processor resource (eg, in a data path) with the clock frequency set to a maximum frequency for the noisy instructions.

[0094] At operation 218, the clock slow down request signal is deasserted. The deassertion may be performed before or after the instruction pipeline is not stopped. In some embodiments, the deassertion may be based on the expiration of a timer that was set when the clock slow down request signal was asserted. Figure 2D A description of two exemplary techniques for invalidation of a clock-based slow-down request signal is provided.

[0095] After completing execution of one or more instructions at the reduced frequency, at operation 220, the clock frequency is restored. In other words, the clock frequency is restored to the maximum frequency of normal instructions (i.e., instructions that are not noisy instructions). An example of clock frequency recovery is as follows: Figure 2C shown.

[0096] Signaling of fast clock slowdown based on instruction type

[0097] Figure 2B 2 is a timing diagram illustrating a clock slowdown request signal 222 and a corresponding confirmation signal 224 that may be used in fast clock slowdown according to some exemplary embodiments. The timing diagram illustrates the validity and invalidation of each signal from left to right over time. The rising edge of the representative curve indicates the validity of the signal, and the falling edge of the representative curve indicates invalidation.

[0098] As described above, signal 222 is asserted by an instruction pipeline associated with processing circuitry (e.g., instruction pipelines 108, 134, and 164) when it is detected during decoding of the instruction that the instruction belongs to one or more of the monitored instruction types. One example of a monitored instruction that may cause a lot of noise in the power rail is a matrix multiplication instruction.

[0099] In response to the request signal 202, the clock logic asserts the signal 224. In some embodiments, the confirmation signal 224 is asserted after the clock logic has set the clock to the target reduced frequency. In some embodiments, the confirmation signal 224 may be asserted immediately before or during the change of the clock to the target reduced frequency.

[0100] As shown, in some embodiments, the clock slow down request signal 222 may remain valid even after the instruction pipeline and / or processing logic receives the confirmation signal 224 and dispatches the detected high power instruction for execution. The instruction pipeline and / or processing logic may deactivate the request signal 222 after the detected high power instruction has completed execution.

[0101] In some embodiments, the confirmation signal 224 may remain asserted until the request signal is deasserted. For example, the clock logic may deassert the confirmation signal 224 after determining that the request signal 222 has been deasserted.

[0102] Figure 2C 2 shows the change of clock frequency 228 over time when performing fast clock slowdown according to some exemplary embodiments. The y-axis represents the clock frequency and the x-axis represents time. The figure shows the change of clock frequency 228 over time when performing fast clock slowdown according to some exemplary embodiments. Figure 1A , Figure 1B or Figure 1C An exemplary variation of clock frequency 228 associated with clock slow request signal 226 in the system is shown.

[0103] In the exemplary embodiment shown, the clock can be configured to operate at either of two frequencies: a maximum frequency for non-noisy (eg, non-high power) instructions (eg, in the case of Figure 1B The examples described involve F 非高功率指令 230 or "full speed"), or the maximum frequency for noisy (e.g., high power) instructions (e.g., in Figure 1B The examples described involve F 高功率指令 232 or "slow down"). As shown, the maximum frequency for non-noisy instructions can be faster than the maximum frequency for high power instructions.

[0104] A primary advantage of the exemplary embodiment is that it reduces the time during which a processor must operate at a lower frequency (e.g., a "slowed down" frequency) due to noise-related margin requirements. More specifically, the exemplary embodiment provides for operating at a reduced frequency only for the duration of executing certain instructions that may result in noise effects that are large enough to cause voltage drops on the circuit.

[0105] Thus, it is expected that the processor will operate at frequency 230 most of the time. When the instruction pipeline and / or processing logic detects the decoding of the monitored instruction, then, as described above, next, the clock slow down request signal 226 is asserted. The clock slow down request signal 226 may remain asserted until the detected instruction has completed execution and / or until a predetermined interval expires. The clock logic is configured to reduce the clock frequency 228 when it receives the clock slow down request signal 226.

[0106] As shown (see the leftmost sharp drop in clock frequency 228), when it receives the clock slow down request signal 226, the clock logic can quickly slow down the clock to the target reduced frequency 232. In some embodiments, the clock logic can set the initial drop to be a larger drop than the target frequency. The drop is shown by the drop of the clock frequency 228 advancing below the target reduced frequency 232. After the rapid drop, the clock frequency 228 gradually advances to the target reduced frequency 232.

[0107] A fast slow down may be performed by referencing a lookup table to determine the new frequency and then immediately setting the clock source to the new operating frequency. In some embodiments, the new frequency becomes effective in one clock cycle after the clock logic receives the clock slow down request signal 226 .

[0108] After the clock slow down request signal 226 is deasserted, the clock logic may restore the clock frequency 228, now at a reduced frequency 232, back to a higher frequency 230. The restoration may be initiated by temporarily reducing the frequency to a lower frequency 234 before resuming the higher frequency. In some exemplary embodiments, the reduction to a lower frequency 234 may be necessary in order to accommodate overshoot.

[0109] As can be noted in the clock frequency 228 curve, the drop to the target lower frequency 232 is a sharp drop compared to the gradual increase in frequency during the recovery process. As described above, the clock slowdown needs to be implemented quickly between decoding and dispatching instructions to the data path, and the slowdown must be immediately effective as soon as possible. However, the recovery is subject to a hysteresis process, so that oscillations between the frequencies 230 and 232 can be avoided.

[0110] Figure 2D A timing diagram illustrating stalling execution of instructions during fast clock slowdown and stall release according to some exemplary embodiments is shown. The diagram illustrates the timing associated with the detection of a noisy instruction and the assertion of a clock slowdown request signal 240, a corresponding slowdown acknowledgement signal 242, a stall signal 244 according to some embodiments, and an alternative stall signal 246 according to some other embodiments.

[0111] With respect to the two stop signals 244 and 246, as shown, they are valid at the rising edge of the slow down request signal 240 (or shortly after the start of the slow down request signal 240). However, while the stop signal 244 in one embodiment is invalidated when the confirmation signal 242 is valid, the stop signal 246 controlled by the programmable timer in another embodiment can be invalidated without waiting for the confirmation signal. For example, the programmable timer can be invalidated after a predetermined duration. The duration can be programmed according to post-silicon simulation.

[0112] Although it may be described in the context of processing units and certain hardware components Figure 1A , Figure 1B and Figure 1C The rapid clock slowdown shown in the system and the Figure 2A The processes described herein may be performed by custom circuits, or by a combination of CPUs, GPUs, custom circuits, and / or programs. A person skilled in the art will appreciate that any system that performs, for example, method 200 is within the scope and spirit of embodiments of the present invention.

[0113] Parallel processing architecture slowed down using fast clock

[0114] Now, more illustrative information about various optional architectures and features that can implement the aforementioned framework will be described, depending on the user's desire. It should be particularly noted that the following information is described for illustrative purposes and should not be interpreted as limiting in any way. The optional following features may be optionally incorporated with or without excluding the other features described.

[0115] Figure 3A parallel processing unit (PPU) 300 is shown according to one embodiment. In one embodiment, the PPU 300 is a multi-threaded processor implemented on one or more integrated circuit devices. The PPU 300 is a latency-hiding architecture designed for processing many threads in parallel. A thread (i.e., an execution thread) is an instance of an instruction set configured to be executed by the PPU 300. In one embodiment, the PPU 300 is a graphics processing unit (GPU) configured to implement a graphics rendering pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device (such as a liquid crystal display (LCD) device). In other embodiments, the PPU 300 can be used to perform general-purpose calculations. Although an exemplary parallel processor is provided herein for illustrative purposes, it should be specifically noted that the processor is described only for illustrative purposes and any processor can be used to supplement and / or replace the processor.

[0116] One or more PPUs 300 can be configured to accelerate thousands of high-performance computing (HPC), data center, and machine learning applications. PPU 300 can be configured to accelerate numerous deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image, and text recognition systems, intelligent video analysis, molecular simulation, drug development, disease diagnosis, weather forecasting, big data analysis, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization, and personalized user recommendations, among others.

[0117] like Figure 3 As shown, the PPU 300 includes an input / output (I / O) unit 305, a front-end unit 315, a scheduler unit 320, a work distribution unit 325, a hub 330, a crossbar switch (Xbar) 370, one or more general processing clusters (GPCs) 350, and one or more partition units 380. The PPU 300 can be connected to a host processor or other PPUs 300 via one or more high-speed NVLink 310 interconnects. The PPU 300 can be connected to a host processor or other peripheral devices via an interconnect 302. The PPU 300 can also be connected to a local memory including multiple memory devices 304. In one embodiment, the local memory can include multiple dynamic random access memory (DRAM) devices. The DRAM device can be configured as a high bandwidth memory (HBM) subsystem, in which multiple DRAM dies are stacked within each device.

[0118] The NVLink 310 interconnect enables the system to scale and include one or more PPUs 300 in conjunction with one or more CPUs, supporting cache coherency between the PPU 300 and the CPU, and CPU mastering. Data and / or commands may be sent by the NVLink 310 through the hub 330 to or from other units of the PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). Figure 5B NVLink 310 is described in more detail.

[0119] I / O unit 305 is configured to send and receive communications (e.g., commands, data, etc.) from a host processor (not shown) via interconnect 302. I / O unit 305 may communicate with the host processor directly via interconnect 302, or through one or more intermediate devices (such as a memory bridge). In one embodiment, I / O unit 305 may communicate with one or more other processors (e.g., one or more PPUs 300) via interconnect 302. In one embodiment, I / O unit 305 implements a peripheral component interconnect express (PCIe) interface for communicating over a PCIe bus, and interconnect 302 is a PCIe bus. In alternative embodiments, I / O unit 305 may implement other types of known interfaces for communicating with external devices.

[0120] The I / O unit 305 decodes packets received via the interconnect 302. In one embodiment, the packets represent commands configured to cause the PPU 300 to perform various operations. The I / O unit 305 sends the decoded commands to various other units of the PPU 300 as specified by the commands. For example, some commands may be sent to the front end unit 315. Other commands may be sent to the hub 330 or other units of the PPU 300, such as one or more copy engines, video encoders, video decoders, power management units, etc. (not explicitly shown). In other words, the I / O unit 305 is configured to route communications between and among the various logical units of the PPU 300.

[0121] In one embodiment, a program executed by a host processor encodes a command stream in a buffer that provides a workload to the PPU 300 for processing. The workload may include many instructions and data to be processed by those instructions. A buffer is an area in memory that is accessible (e.g., read / write) by both the host processor and the PPU 300. For example, the I / O unit 305 may be configured to access a buffer in a system memory connected to the interconnect 302 via a memory request transmitted through the interconnect 302. In one embodiment, the host processor writes a command stream to the buffer and then sends a pointer to the start of the command stream to the PPU 300. The front end unit 315 receives a pointer to one or more command streams. The front end unit 315 manages one or more streams, reads commands from the streams, and forwards the commands to the various units of the PPU 300.

[0122] The front end unit 315 is coupled to a scheduler unit 320, which configures the various GPCs 350 to process the tasks defined by one or more streams. The scheduler unit 320 is configured to track state information related to the various tasks managed by the scheduler unit 320. The state may indicate which GPC 350 the task is assigned to, whether the task is active or inactive, a priority associated with the task, etc. The scheduler unit 320 manages the execution of multiple tasks on one or more GPCs 350.

[0123] Scheduler unit 320 is coupled to work distribution unit 325, which is configured to dispatch tasks for execution on GPC 350. Work distribution unit 325 may keep track of a number of scheduled tasks received from scheduler unit 320. In one embodiment, work distribution unit 325 manages a pending task pool and an active task pool for each GPC 350. The pending task pool may include a number of time slots (e.g., 32 time slots) containing tasks assigned to be processed by a particular GPC 350. The active task pool may include a number of time slots (e.g., 4 time slots) for tasks that are being actively processed by GPC 350. When a GPC 350 completes execution of a task, the task is evicted from the active task pool of GPC 350, and one of the other tasks from the pending task pool is selected and scheduled for execution on GPC 350. If an active task on a GPC 350 has become idle, such as while waiting for a data dependency to be resolved, then the active task may be evicted from the GPC 350 and returned to the pending task pool, while another task in the pending task pool is selected and scheduled for execution on the GPC 350 .

[0124] Work distribution unit 325 communicates with one or more GPCs 350 via XBar (crossbar) 370. XBar 370 is an interconnect network that couples many units of PPU 300 to other units of PPU 300. For example, XBar 370 can be configured to couple work distribution unit 325 to a particular GPC 350. Although not explicitly shown, one or more other units of PPU 300 can also be connected to XBar 370 via hub 330.

[0125] Tasks are managed by the scheduler unit 320 and dispatched to the GPCs 350 by the work distribution unit 325. The GPCs 350 are configured to process tasks and generate results. The results may be consumed by other tasks within the GPC 350, routed to a different GPC 350 via the XBar 370, or stored in the memory 304. The results may be written to the memory 304 via the partition unit 380, which implements a memory interface for reading data from and writing data to the memory 304. The results may be sent to another PPU 300 or CPU via the NVLink 310. In one embodiment, the PPU 300 includes a number U of partition units 380, which is equal to the number of independent and distinct memory devices 304 coupled to the PPU 300. This will be described below in conjunction with Figure 4B The partition unit 380 is described in more detail.

[0126] In one embodiment, the host processor executes a driver kernel that implements an application programming interface (API) that enables one or more applications to be executed on the host processor to schedule operations for execution on the PPU 300. In one embodiment, multiple computing applications are executed simultaneously by the PPU 300, and the PPU 300 provides isolation, quality of service (QoS), and independent address spaces for multiple computing applications. The application can generate instructions (e.g., API calls) that cause the driver kernel to generate one or more tasks to be executed by the PPU 300. The driver kernel outputs the tasks to one or more streams being processed by the PPU 300. Each task can include one or more related thread groups, referred to herein as warps. In one embodiment, a warp includes 32 related threads that can execute in parallel. Collaborative threads can refer to multiple threads that include instructions to perform tasks and can exchange data through shared memory. In combination Figure 5A Describes threads and cooperative threads in more detail.

[0127] Figure 4A According to an embodiment Figure 3 PPU 300 GPC 350. Figure 4AAs shown, each GPC 350 includes multiple hardware units for processing tasks. In one embodiment, each GPC 350 includes a pipeline manager 410, a pre-raster operation unit (PROP) 415, a raster engine 425, a work distribution crossbar switch (WDX) 480, a memory management unit (MMU) 490, and one or more data processing clusters (DPCs) 420. It should be understood that Figure 4A The GPC 350 may include instead Figure 4A Other hardware units or units other than those shown in Figure 4A Other hardware units than those shown in .

[0128] In one embodiment, the operation of the GPC 350 is controlled by a pipeline manager 410. The pipeline manager 410 manages the configuration of one or more DPCs 420 for processing tasks assigned to the GPC 350. In one embodiment, the pipeline manager 410 may configure at least one of the one or more DPCs 420 to implement at least a portion of a graphics rendering pipeline. For example, a DPC 420 may be configured to execute a vertex shading program on a programmable streaming multiprocessor (SM) 440. The pipeline manager 410 may also be configured to route packets received from the work distribution unit 325 to appropriate logic units in the GPC 350. For example, some packets may be routed to fixed-function hardware units in the PROP 415 and / or the raster engine 425, while other packets may be routed to the DPC 420 for processing by the primitive engine 435 or the SM 440. In one embodiment, the pipeline manager 410 may configure at least one of the one or more DPCs 420 to implement a neural network model and / or a computational pipeline.

[0129] PROP unit 415 is configured to route data generated by raster engine 425 and DPC 420 to a raster operations (ROP) unit, in conjunction with Figure 4B The PROP unit 415 may also be configured to perform optimization of color blending, organize pixel data, perform address translation, etc.

[0130] Raster engine 425 includes several fixed function hardware units configured to perform various raster operations. In one embodiment, raster engine 425 includes a setup engine, a coarse raster engine, a culling engine, a clipping engine, a fine raster engine, and a tile aggregation engine. The setup engine receives the transformed vertices and generates plane equations associated with the geometric primitives defined by the vertices. The plane equations are sent to the coarse raster engine to generate coverage information (e.g., x, y coverage masks of tiles) of the primitives. The output of the coarse raster engine is sent to the culling engine, where fragments associated with primitives that fail the z-test are culled, and unculled fragments are sent to the clipping engine, where fragments outside the viewing cone are clipped. Those fragments that remain after clipping and culling can be passed to the fine raster engine to generate attributes of pixel fragments based on the plane equations generated by the setup engine. The output of raster engine 425 includes, for example, fragments to be processed by a fragment shader implemented in DPC 420.

[0131] Each DPC 420 included in the GPC 350 includes an M pipeline controller (MPC) 430, a primitive engine 435, and one or more SMs 440. The MPC 430 controls the operation of the DPC 420, routing packets received from the pipeline manager 410 to appropriate units in the DPC 420. For example, packets associated with vertices may be routed to the primitive engine 435, which is configured to fetch vertex attributes associated with the vertices from the memory 304. Conversely, packets associated with shading programs may be sent to the SM 440.

[0132] SM 440 includes a programmable stream processor configured to process tasks represented by multiple threads. Each SM440 is multithreaded and is configured to execute multiple threads (e.g., 32 threads) from a specific thread group simultaneously. In one embodiment, SM 440 implements a SIMD (single instruction, multiple data) architecture, wherein each thread in a thread group (e.g., warp) is configured to process different data sets based on the same instruction set. All threads in a thread group execute the same instruction. In another embodiment, SM 440 implements a SIMT (single instruction, multiple thread) architecture, wherein each thread in a thread group is configured to process different data sets based on the same instruction set, but wherein each thread in a thread group is allowed to diverge during execution. In one embodiment, a program counter, a call stack, and an execution state are maintained for each thread bundle, and when threads within a thread bundle diverge, concurrency between serial executions in the thread bundle and the thread bundle is made possible. In another embodiment, a program counter, a call stack, and an execution state are maintained for each individual thread, thereby achieving equal concurrency between all threads within and between thread bundles. When execution state is maintained for each individual thread, threads executing the same instructions can be converged and executed in parallel for maximum efficiency. Figure 5A Describe SM440 in more detail.

[0133] MMU 490 provides an interface between GPC 350 and partition unit 380. MMU 490 can provide virtual address to physical address translation, memory protection, and arbitration of memory requests. In one embodiment, MMU 490 provides one or more translation lookaside buffers (TLBs) for performing translations from virtual addresses to physical addresses in memory 304.

[0134] Figure 4B According to an embodiment Figure 3 The memory partition unit 380 of the PPU 300. Figure 4B As shown, the memory partition unit 380 includes a raster operation (ROP) unit 450, a second level (L2) cache 460, and a memory interface 470. The memory interface 470 is coupled to the memory 304. The memory interface 470 can implement a 32, 64, 128, 1024 bit data bus, etc. for high-speed data transmission. In one embodiment, the PPU 300 incorporates U memory interfaces 470, one for each pair of partition units 380, wherein each pair of partition units 380 is connected to a corresponding memory device 304. For example, the PPU 300 can be connected to up to Y memory devices 304, such as a high bandwidth memory stack or a graphics double data rate version 5 synchronous dynamic random access memory or other types of persistent memory.

[0135] In one embodiment, memory interface 470 implements an HBM2 memory interface, and Y is equal to half of U. In one embodiment, the HBM2 memory stack is located on the same physical package as PPU 300, providing significant power and area savings compared to conventional GDDR5 SDRAM systems. In one embodiment, each HBM2 stack includes four memory dies and Y is equal to 4, where the HBM2 stack includes two 128-bit channels per die, a total of 8 channels and a data bus width of 1024 bits.

[0136] In one embodiment, memory 304 supports single error correction double error detection (SECDED) error correction code (ECC) to protect data. ECC provides higher reliability for computing applications that are sensitive to data corruption. Reliability is particularly important in large cluster computing environments where PPU 300 processes very large data sets and / or long-running applications.

[0137] In one embodiment, the PPU 300 implements a multi-level memory hierarchy. In one embodiment, the memory partition unit 380 supports unified memory to provide a single unified virtual address space for the CPU and PPU 300 memory, enabling data sharing between virtual memory systems. In one embodiment, the frequency of PPU 300 accesses to memory located on other processors is tracked to ensure that memory pages are moved to the physical memory of the PPU 300 that accesses the page more frequently. In one embodiment, the NVLink 310 supports address translation services that allow the PPU 300 to directly access the CPU's page tables and provide full access to the CPU memory by the PPU 300.

[0138] In one embodiment, the copy engine transfers data between multiple PPUs 300 or between a PPU 300 and a CPU. The copy engine can generate a page fault for an address that is not mapped to a page table. The memory partition unit 380 can then service the page fault, map the address into a page table, and then the copy engine can perform the transfer. In conventional systems, fixed memory (e.g., non-pageable) is operated for multiple copy engines between multiple processors, which significantly reduces the available memory. Due to hardware paging faults, addresses can be passed to the copy engine without worrying about whether the memory page is resident, and the copy process is transparent.

[0139] Data from memory 304 or other system memory may be retrieved by memory partition unit 380 and stored in L2 cache 460, which is located on-chip and shared between various GPCs 350. As shown, each memory partition unit 380 includes a portion of L2 cache 460 associated with a corresponding memory device 304. Lower level caches may then be implemented in multiple units within a GPC 350. For example, each SM 440 may implement a level 1 (L1) cache. An L1 cache is a dedicated memory dedicated to a particular SM 440. Data from L2 cache 460 may be retrieved and stored in each L1 cache for processing in a functional unit of the SM 440. L2 cache 460 is coupled to a memory interface 470 and XBar 370.

[0140] The ROP unit 450 performs graphics raster operations related to pixel color such as color compression, pixel blending, etc. The ROP unit 450 also implements depth testing in conjunction with the raster engine 425, receiving the depth of a sample position associated with a pixel fragment from the culling engine of the raster engine 425. The depth of the sample position associated with the fragment is tested relative to the corresponding depth in the depth buffer. If the fragment passes the depth test for the sample position, the ROP unit 450 updates the depth buffer and sends the result of the depth test to the raster engine 425. It will be understood that the number of partition units 380 may be different than the number of GPCs 350, and thus each ROP unit 450 may be coupled to each GPC 350. The ROP unit 450 tracks packets received from different GPCs 350 and determines to which GPC 350 the results generated by the ROP unit 450 are routed via the Xbar 370. Although in Figure 4B In the embodiment shown, ROP unit 450 is included within memory partition unit 380, but in other embodiments, ROP unit 450 may be external to memory partition unit 380. For example, ROP unit 450 may reside in GPC 350 or another unit.

[0141] Figure 5A According to an embodiment Figure 4A The streaming multiprocessor 440. Figure 5A As shown, SM 440 includes an instruction cache 505, one or more scheduler units 510, a register file 520, one or more processing cores 550, one or more special function units (SFU) 552, one or more load / store units (LSU) 554, an interconnection network 580, and a shared memory / L1 cache 570.

[0142] As described above, the work distribution unit 325 schedules tasks to be executed on the GPC 350 of the PPU 300. Tasks are assigned to specific DPCs 420 within the GPC 350, and if the task is associated with a shader program, the task may be assigned to the SM 440. The scheduler unit 510 receives tasks from the work distribution unit 325 and manages the scheduling of instructions assigned to one or more thread blocks assigned to the SM 440. The scheduler unit 510 schedules the thread blocks to execute as warps of parallel threads, where each thread block is assigned at least one warp. In one embodiment, each warp executes 32 threads. The scheduler unit 510 can manage a plurality of different thread blocks, assign warps to different thread blocks, and then dispatch instructions from a plurality of different cooperative groups to various functional units (i.e., core 550, SFU 552, and LSU 554) during each clock cycle.

[0143] Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity at which threads are communicating, enabling the expression of richer and more efficient decompositions of parallelism. The cooperative launch API supports synchronization between thread blocks to execute parallel algorithms. Conventional programming models provide a single simple structure for synchronizing cooperative threads: a barrier across all threads of a thread block (e.g., the syncthreads() function). However, programmers often want to define thread groups at a granularity smaller than the thread block granularity and synchronize within the defined group, enabling higher performance, design flexibility, and software reuse in the form of a collective group-wide function interface.

[0144] Cooperative Groups enables programmers to explicitly define thread groups at sub-block (e.g., as small as a single thread) and multi-block granularity and perform collective operations such as synchronization on threads in a cooperative group. The programming model supports clean composition across software boundaries so that libraries and utility functions can safely synchronize in their local environment without making assumptions about convergence. Cooperative Groups primitives enable new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across the entire grid of thread blocks.

[0145] The dispatch unit 515 is configured to transmit instructions to one or more functional units. In this embodiment, the scheduler unit 510 includes two dispatch units 515, which enables two different instructions from the same thread warp to be scheduled during each clock cycle. In alternative embodiments, each scheduler unit 510 may include a single dispatch unit 515 or additional dispatch units 515.

[0146] Each SM 440 includes a register file 520 that provides a set of registers for the functional units of the SM 440. In one embodiment, the register file 520 is divided between each functional unit so that each functional unit is assigned a dedicated portion of the register file 520. In another embodiment, the register file 520 is divided between different warps executed by the SM 440. The register file 520 provides temporary storage for operands connected to the data paths of the functional units.

[0147] Each SM 440 includes L processing cores 550. In one embodiment, SM 440 includes a large number (e.g., 128, etc.) of different processing cores 550. Each core 550 may include a fully pipelined, single-precision, double-precision, and / or mixed-precision processing unit, including a floating-point arithmetic logic unit and an integer arithmetic logic unit. In one embodiment, the floating-point arithmetic logic unit implements the IEEE 754-2008 standard for floating-point operations. In one embodiment, core 550 includes 64 single-precision (32-bit) floating-point cores, 64 integer cores, 32 double-precision (64-bit) floating-point cores, and 8 tensor cores.

[0148] The tensor cores are configured to perform matrix operations, and in one embodiment, one or more tensor cores are included in core 550. Specifically, the tensor cores are configured to perform deep learning matrix operations, such as convolution operations for neural network training and inference. In one embodiment, each tensor core operates on a 4×4 matrix and performs a matrix multiplication and accumulation operation D=A×B+C, where A, B, C, and D are 4×4 matrices.

[0149] In one embodiment, the matrix multiplication inputs A and B are 16-bit floating point matrices, while the accumulation matrices C and D can be 16-bit floating point or 32-bit floating point matrices. The tensor cores operate on 16-bit floating point input data as well as 32-bit floating point accumulations. The 16-bit floating point multiplication requires 64 operations, producing full-precision products, which are then accumulated using 32-bit floating point additions with other intermediate products of the 4×4×4 matrix multiplication. In practice, tensor cores are used to perform larger two-dimensional or higher dimensional matrix operations built from these smaller elements. APIs (such as the CUDA 9 C++ API) expose specialized matrix loads, matrix multiplications and accumulations, and matrix storage operations to efficiently use tensor cores from CUDA-C++ programs. At the CUDA level, the warp-level interface assumes that the 16×16 size matrix spans all 32 threads of the warp.

[0150] Each SM 440 also includes M SFUs 552 that perform special functions (e.g., attribute evaluation, reciprocal square root, etc.). In one embodiment, the SFUs 552 may include a tree traversal unit configured to traverse a hierarchical tree data structure. In one embodiment, the SFUs 552 may include a texture unit configured to perform texture map filtering operations. In one embodiment, the texture unit is configured to load a texture map (e.g., a 2D array of texture pixels) from memory 304 and sample the texture map to produce sampled texture values ​​for use in a shader program executed by the SM 440. In one embodiment, the texture map is stored in a shared memory / L1 cache 570. The texture unit implements texture operations, such as filtering operations using mip maps (i.e., texture maps of different levels of detail). In one embodiment, each SM 440 includes two texture units.

[0151] Each SM 440 also includes N LSUs 554 that implement load and store operations between the shared memory / L1 cache 570 and the register file 520. Each SM 440 includes an interconnect network 580 that connects each functional unit to the register file 520 and connects the LSUs 554 to the register file 520, the shared memory / L1 cache 570. In one embodiment, the interconnect network 580 is a crossbar switch that can be configured to connect any functional unit to any register in the register file 520 and to connect the LSUs 554 to memory locations in the register file and the shared memory / L1 cache 570.

[0152] The shared memory / L1 cache 570 is an on-chip memory array that allows data storage and communication between the SM 440 and the primitive engine 435, as well as between threads in the SM 440. In one embodiment, the shared memory / L1 cache 570 includes 128KB of storage capacity and is in the path from the SM 440 to the partition unit 380. The shared memory / L1 cache 570 can be used to cache reads and writes. One or more of the shared memory / L1 cache 570, the L2 cache 460, and the memory 304 is a backing store.

[0153] Combining the data cache and shared memory functions into a single memory block provides the best overall performance for both types of memory accesses. This capacity can be used by programs as a cache that does not use the shared memory. For example, if the shared memory is configured to use half of the capacity, texture and load / store operations can use the remaining capacity. Integration within the shared memory / L1 cache 570 enables the shared memory / L1 cache 570 to function as a high throughput pipeline for streaming data, while providing high bandwidth and low latency access to frequently reused data.

[0154] When configured for general parallel computing, a simpler configuration can be used compared to graphics processing. Specifically, Figure 3 The fixed function graphics processing unit is shown bypassed, creating a simpler programming model. In a general purpose parallel computing configuration, the work distribution unit 325 assigns and distributes thread blocks directly to the DPC 420. The threads in the block execute the same program, use unique thread IDs in computations to ensure that each thread produces unique results, use SM 440 to execute the program and perform computations, use shared memory / L1 cache 570 to communicate between threads, and use LSU 554 to read and write global memory through shared memory / L1 cache 570 and memory partition unit 380. When configured for general purpose parallel computing, SM 440 can also write commands that scheduler unit 320 can use to start new work on DPC 420.

[0155] The PPU 300 may be included in a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a handheld electronic device, etc. In one embodiment, the PPU 300 is included on a single semiconductor substrate. In another embodiment, the PPU 300 is included on a system on a chip (SoC) with one or more other devices (such as an additional PPU 300, a memory 304, a reduced instruction set computer (RISC) CPU, a memory management unit (MMU), a digital-to-analog converter (DAC), etc.).

[0156] In one embodiment, PPU 300 may be included on a graphics card that includes one or more memory devices 304. The graphics card may be configured to interface with a PCI e slot on a motherboard of a desktop computer. In yet another embodiment, PPU 300 may be an integrated graphics processing unit (iGPU) or parallel processor included in a chipset of a motherboard.

[0157] Exemplary Computing System

[0158] Systems with multiple GPUs and CPUs are used in a variety of industries as developers expose and exploit more parallelism in applications such as artificial intelligence computing. High-performance GPU-accelerated systems with tens to thousands of computing nodes are deployed in data centers, research institutions, and supercomputers to solve larger problems. As the number of processing devices within high-performance systems increases, communication and data transmission mechanisms need to scale to support this increased bandwidth.

[0159] Figure 5BAccording to one embodiment, the use Figure 3 A conceptual diagram of a processing system 500 implemented by a PPU 300 of FIG. 5 . The exemplary system 500 may be configured to implement Figure 2A The processing system 500 includes a CPU 530, a switch 555, and each of the plurality of PPUs 300 and a corresponding memory 304. The NVLink 310 provides a high-speed communication link between each PPU 300. Figure 5B 302 connections, but the number of connections connected to each PPU 300 and CPU 530 may vary. Switch 555 interfaces between interconnect 302 and CPU 530. PPU 300, memory 304, and NVLink 310 may be located on a single semiconductor platform to form parallel processing module 525. In one embodiment, switch 555 supports two or more protocols that interface between various different connections and / or links.

[0160] In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between each PPU 300 and CPU 530, and switch 555 interfaces between interconnect 302 and each PPU 300. PPU 300, memory 304, and interconnect 302 may be located on a single semiconductor platform to form parallel processing module 525. In yet another embodiment (not shown), interconnect 302 provides one or more communication links between each PPU 300 and CPU 530, and switch 555 interfaces between each PPU 300 using NVLink 310 to provide one or more high-speed communication links between PPU 300. In another embodiment (not shown), NVLink 310 provides one or more high-speed communication links between PPU 300 and CPU 530 through switch 555. In yet another embodiment (not shown), interconnect 302 provides one or more communication links directly between each PPU 300. One or more NVLink 310 high-speed communication links may be implemented as a physical NVLink interconnect or as an on-chip or on-die interconnect using the same protocol as NVLink 310.

[0161] In the context of this specification, a single semiconductor platform may refer to a unique single semiconductor-based integrated circuit manufactured on a bare die or chip. It should be noted that the term single semiconductor platform may also refer to a multi-chip module with increased connectivity that simulates on-chip operations and is substantially improved by utilizing conventional bus implementations. Of course, various circuits or devices may also be placed separately or in various combinations of semiconductor platforms, depending on the needs of the user. Optionally, the parallel processing module 525 may be implemented as a circuit board substrate, and each of the PPU 300 and / or memory 304 may be a packaged device. In one embodiment, the CPU 530, the switch 555, and the parallel processing module 525 are located on a single semiconductor platform.

[0162] In one embodiment, the signaling rate of each NVLink 310 is 20 to 25 Gbit / s, and each PPU 300 includes six NVLink 310 interfaces (e.g., Figure 5B Each NVLink 310 provides a data transfer rate of 25 Gbit / s in each direction, with six links providing 300 Gbit / s. When the CPU 530 also includes one or more NVLink 310 interfaces, the NVLink 310 can be used exclusively for, for example, Figure 5B PPU to PPU communication as shown, or some combination of PPU to PPU and PPU to CPU.

[0163] In one embodiment, NVLink 310 allows direct load / store / atomic access from CPU 530 to memory 304 of each PPU 300. In one embodiment, NVLink 310 supports coherency operations, allowing data read from memory 304 to be stored in the cache hierarchy of CPU 530, reducing cache access latency of CPU 530. In one embodiment, NVLink 310 includes support for address translation services (ATS), allowing PPU 300 to directly access page tables within CPU 530. One or more NVLink 310 can also be configured to operate in a low power mode.

[0164] Figure 5C An exemplary system 565 is shown in which various architectures and / or functions of various previous embodiments may be implemented. The exemplary system 565 may be configured to implement Figure 2A The method 200 shown in FIG.

[0165] As shown, a system 565 is provided that includes at least one central processing unit 530 connected to a communication bus 575. The communication bus 575 can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport, or any other bus or one or more point-to-point communication protocols. The system 565 also includes a main memory 540. Control logic (software) and data are stored in the main memory 540, which can take the form of a random access memory (RAM).

[0166] The system 565 also includes an input device 560, a parallel processing module 525, and a display device 545, such as a conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light emitting diode), plasma display, etc. User input can be received from the input device 560 (e.g., keyboard, mouse, touch pad, microphone, etc.). Each of the aforementioned modules and / or devices can even be located on a single semiconductor platform to form the system 565. Optionally, the modules can also be placed separately or in various combinations of semiconductor platforms according to the needs of the user.

[0167] Furthermore, system 565 may be coupled to a network (e.g., a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, a peer-to-peer network, a cable network, etc.) through network interface 535 for communication purposes.

[0168] The system 565 may also include auxiliary storage (not shown). The auxiliary storage includes, for example, a hard disk drive and / or a removable storage drive, representative of a floppy disk drive, a tape drive, an optical disk drive, a digital versatile disk (DVD) drive, a recording device, a universal serial bus (USB) flash memory. The removable storage drive reads from and / or writes to a removable storage unit in a well-known manner.

[0169] Computer programs or computer control logic algorithms may be stored in the main memory 540 and / or the secondary storage. These computer programs, when executed, enable the system 565 to perform various functions. The memory 540, storage and / or any other storage are possible examples of computer-readable media.

[0170] The architecture and / or functionality of the various prior figures may be implemented in the context of a general purpose computer system, a circuit board system, a game console system dedicated to entertainment purposes, a dedicated system, and / or any other desired system. For example, system 565 may take the form of a desktop computer, a laptop computer, a tablet computer, a server, a supercomputer, a smartphone (e.g., a wireless, handheld device), a personal digital assistant (PDA), a digital camera, a vehicle, a head mounted display, a handheld electronic device, a mobile telephone device, a television, a workstation, a game console, an embedded system, and / or any other type of logic.

[0171] Although various embodiments have been described above, it should be understood that they are presented by way of example only and not limitation. Thus, the breadth and scope of a preferred embodiment should not be limited by any of the above-described exemplary embodiments, but should be limited only in accordance with the following claims and their equivalents.

[0172] Graphics Processing Pipeline

[0173] In one embodiment, the PPU 300 includes a graphics processing unit (GPU). The PPU 300 is configured to receive commands specifying a shader for processing graphics data. The graphics data may be defined as a set of primitives, such as points, lines, triangles, quadrilaterals, triangle strips, etc. Typically, a primitive includes data specifying a plurality of vertices of the primitive (e.g., in a model space coordinate system) and attributes associated with each vertex of the primitive. The PPU 300 may be configured to process the primitives to generate a frame buffer (e.g., pixel data for each of the pixels of a display).

[0174] The application writes the model data (e.g., a collection of vertices and attributes) of the scene to a memory (such as system memory or memory 304). The model data defines each of the objects that may be visible on the display. The application then makes an API call to the driver kernel, which requests the model data to be rendered and displayed. The driver kernel reads the model data and writes commands to one or more streams to perform operations to process the model data. These commands may refer to different shading programs to be implemented on the SM 440 of the PPU 300, including one or more of vertex shading, hull shading, domain shading, geometry shading, and pixel shading. For example, one or more of the SM 440 may be configured to execute a vertex shading program that processes multiple vertices defined by the model data. In one embodiment, different SMs 440 may be configured to execute different shading programs simultaneously. For example, a first subset of SMs 440 may be configured to execute a vertex shading program, and a second subset of SMs 440 may be configured to execute a pixel shading program. The first subset of SMs 440 processes the vertex data to generate processed vertex data, and writes the processed vertex data to the L2 cache 460 and / or the memory 304. After the processed vertex data is rasterized (e.g., converted from three-dimensional data to two-dimensional data in screen space) to generate fragment data, a second subset of SMs 440 performs pixel shading to generate processed fragment data, which is then blended with other processed fragment data and written to a frame buffer in memory 304. Vertex shading programs and pixel shading programs can be executed simultaneously, processing different data from the same scene in a pipelined manner until all model data for the scene has been rendered to the frame buffer. The contents of the frame buffer are then transmitted to a display controller for display on a display device.

[0175] Figure 6 According to one embodiment, Figure 3 300 of FIG. 1 . Graphics processing pipeline 600 is an abstract flow chart of processing steps implemented to generate a 2D computer-generated image from 3D geometric data. As is well known, pipeline architectures can perform long latency operations more efficiently by breaking the operations into multiple stages, where the output of each stage is coupled to the input of the next consecutive stage. Thus, graphics processing pipeline 600 receives input data 601 that is passed from one stage of graphics processing pipeline 600 to the next stage to generate output data 602. In one embodiment, graphics processing pipeline 600 may represent a graphics processing pipeline composed of API-defined graphics processing pipeline. Alternatively, graphics processing pipeline 600 can be implemented in the functional and architectural context of the previous figures and / or one or more of any subsequent figures.

[0176] like Figure 6As shown, the graphics processing pipeline 600 includes a pipeline architecture including multiple stages. These stages include, but are not limited to, a data assembly stage 610, a vertex shading stage 620, a primitive assembly stage 630, a geometry shading stage 640, a viewport scale, cull, and clip (VSCC) stage 650, a rasterization stage 660, a fragment shading stage 670, and a raster operation stage 680. In one embodiment, input data 601 includes commands that configure a processing unit to implement the stages of the graphics processing pipeline 600 and configure geometric primitives (e.g., points, lines, triangles, quadrilaterals, triangle strips or fans, etc.) to be processed by these stages. Output data 602 may include pixel data (i.e., color data), which is copied to a frame buffer or other type of surface data structure in memory.

[0177] The data assembly stage 610 receives input data 601, which specifies vertex data for high-order surfaces, primitives, etc. The data assembly stage 610 collects the vertex data in temporary storage or queues, such as by receiving a command from a host processor including a pointer to a buffer in memory and reading the vertex data from the buffer. The vertex data is then passed to the vertex shading stage 620 for processing.

[0178] The vertex shading stage 620 processes vertex data by executing a set of operations (e.g., a vertex shader or program) once for each vertex. A vertex may be specified, for example, as a 4-coordinate vector (e.g.,<x,y,z,w> ). The vertex shading stage 620 can manipulate various vertex attributes, such as position, color, texture coordinates, etc. In other words, the vertex shading stage 620 performs operations on vertex coordinates or other vertex attributes associated with the vertex. These operations typically include lighting operations (e.g., modifying the color attribute of the vertex) and transformation operations (e.g., modifying the coordinate space of the vertex). For example, a vertex can be specified using coordinates in an object coordinate space, which is transformed by multiplying the coordinates by a matrix that converts the coordinates from the object coordinate space to world space or normalized-device-coordinate (NCD) space. The vertex shading stage 620 generates transformed vertex data that is transmitted to the primitive assembly stage 630.

[0179] The primitive assembly stage 630 collects the vertices output by the vertex shading stage 620 and groups the vertices into geometric primitives for processing by the geometry shading stage 640. For example, the primitive assembly stage 630 may be configured to group every three consecutive vertices into geometric primitives (e.g., triangles) for transmission to the geometry shading stage 640. In some embodiments, particular vertices may be reused for consecutive geometric primitives (e.g., two consecutive triangles in a triangle strip may share two vertices). The primitive assembly stage 630 transmits the geometric primitives (e.g., a collection of associated vertices) to the geometry shading stage 640.

[0180] The geometry shading stage 640 processes geometric primitives by performing a set of operations (e.g., geometry shaders or programs) on the geometric primitives. A tessellation operation can generate one or more geometric primitives from each geometric primitive. In other words, the geometry shading stage 640 can subdivide each geometric primitive into a finer grid of two or more geometric primitives for processing by the rest of the graphics processing pipeline 600. The geometry shading stage 640 transmits the geometric primitives to the viewport SCC stage 650.

[0181] In one embodiment, the graphics processing pipeline 600 can operate within a streaming multiprocessor and vertex shading stage 620, primitive assembly stage 630, geometry shading stage 640, fragment shading stage 670 and / or hardware / software associated therewith, and can perform processing operations sequentially. Once the sequential processing operations are completed, in one embodiment, the viewport SCC stage 650 can utilize the data. In one embodiment, primitive data processed by one or more stages in the graphics processing pipeline 600 can be written to a cache (e.g., an L1 cache, a vertex cache, etc.). In this case, in one embodiment, the viewport SCC stage 650 can access the data in the cache. In one embodiment, the viewport SCC stage 650 and the rasterization stage 660 are implemented as fixed function circuits.

[0182] The viewport SCC stage 650 performs viewport scaling, culling, and clipping of geometric primitives. Each surface being rendered is associated with an abstract camera position. The camera position represents the position of the viewer who is viewing the scene and defines a view cone that surrounds the objects of the scene. The view cone may include a viewing plane, a back plane, and four clipping planes. Any geometric primitives that are completely outside the view cone may be culled (e.g., discarded) because they will not contribute to the final rendered scene. Any geometric primitives that are partially within the view cone and partially outside the view cone may be clipped (e.g., converted to new geometric primitives that are enclosed within the view cone). In addition, each geometric primitive may be scaled based on the depth of the view cone. All potentially visible geometric primitives are then transferred to the rasterization stage 660.

[0183] The rasterization stage 660 converts 3D geometric primitives into 2D fragments (e.g., capable of being used for display, etc.). The rasterization stage 660 can be configured to use the vertices of the geometric primitives to set a set of plane equations from which various attributes can be interpolated. The rasterization stage 660 can also calculate a coverage mask for multiple pixels, which indicates whether one or more sample positions of a pixel intercept the geometric primitive. In one embodiment, a z test can also be performed to determine whether the geometric primitive is occluded by other geometric primitives that have been rasterized. The rasterization stage 660 generates fragment data (e.g., interpolated vertex attributes associated with a specific sample position of each covered pixel), which is transmitted to the fragment shading stage 670.

[0184] The fragment shading stage 670 processes the fragment data by executing a set of operations (e.g., a fragment shader or program) on each of the fragments. The fragment shading stage 670 may generate pixel data (e.g., color values) for the fragments, such as by performing lighting operations or sampling a texture map using interpolated texture coordinates of the fragments. The fragment shading stage 670 generates pixel data, which is sent to the raster operations stage 680.

[0185] Raster operations stage 680 may perform various operations on the pixel data, such as performing alpha tests, stencil tests, and blending the pixel data with other pixel data corresponding to other fragments associated with the pixel. When raster operations stage 680 has completed processing the pixel data (e.g., output data 602), the pixel data may be written to a render target, such as a frame buffer, a color buffer, etc.

[0186] It should be appreciated that one or more additional stages may be included in the graphics processing pipeline 600 in addition to or in place of one or more of the above-described stages. Various implementations of the abstract graphics processing pipeline may implement different stages. Furthermore, in some embodiments, one or more of the above-described stages may be excluded from the graphics processing pipeline (such as the geometry shading stage 640). Other types of graphics processing pipelines are considered to be contemplated within the scope of the present disclosure. Furthermore, any stage of the graphics processing pipeline 600 may be implemented by one or more dedicated hardware units within a graphics processor (such as PPU 300). Other stages of the graphics processing pipeline 600 may be implemented by programmable hardware units (such as SM 440 of PPU 300).

[0187] The graphics processing pipeline 600 may be implemented via an application program executed by a host processor (such as a CPU). In one embodiment, a device driver may implement an application programming interface (API) that defines various functions that can be utilized by an application program to generate graphics data for display. A device driver is a software program that includes a plurality of instructions that control the operation of the PPU 300. The API provides an abstraction for programmers that allows programmers to utilize dedicated graphics hardware (such as the PPU 300) to generate graphics data without requiring the programmer to utilize a specific instruction set of the PPU 300. An application program may include an API call that is routed to a device driver of the PPU 300. The device driver interprets the API call and performs various operations in response to the API call. In some cases, the device driver may perform operations by executing instructions on the CPU. In other cases, the device driver may perform operations at least in part by initiating operations on the PPU 300 using an input / output interface between the CPU and the PPU 300. In one embodiment, the device driver is configured to implement the graphics processing pipeline 600 using the hardware of the PPU 300.

[0188] Various programs may be executed within the PPU 300 to implement the various stages of the graphics processing pipeline 600. For example, a device driver may launch a kernel on the PPU 300 to execute the vertex shading stage 620 on one SM 440 (or multiple SMs 440). The device driver (or the initial kernel executed by the PPU 400) may also launch other kernels on the PPU 400 to execute other stages of the graphics processing pipeline 600, such as the geometry shading stage 640 and the fragment shading stage 670. In addition, some of the stages of the graphics processing pipeline 600 may be implemented on fixed unit hardware, such as a rasterizer or data assembler implemented within the PPU 400. It should be appreciated that the results from one kernel may be processed by one or more intermediate fixed function hardware units before being processed by subsequent kernels on the SM 440.

[0189] Machine Learning

[0190] Deep Neural Networks (DNNs) developed on processors such as the PPU 300 have been used in a variety of use cases: from self-driving cars to faster drug development, from automatic image captioning in online image databases to intelligent real-time language translation in video chat applications. Deep learning is a technology that models the neural learning process of the human brain, constantly learning, constantly getting smarter, and delivering more accurate results faster over time. A child is initially taught by an adult to correctly identify and classify various shapes, and eventually is able to recognize shapes without any coaching. Similarly, deep learning or neural learning systems need to be trained in object recognition and classification in order to become smarter and more efficient in recognizing basic objects, occluded objects, etc. while also assigning context to objects.

[0191] At the simplest level, neurons in the human brain look at the various inputs they receive, assign importance levels to each of these inputs, and pass outputs to other neurons for processing. An artificial neuron, or perceptron, is the most basic model of a neural network. In one example, a perceptron can receive one or more inputs that represent various features of an object that the perceptron is being trained to recognize and classify, and each of these features is given a certain weight based on the importance of that feature when defining the shape of the object.

[0192] Deep neural network (DNN) models include multiple layers of many connected nodes (e.g., perceptrons, Boltzmann machines, radial basis functions, convolutional layers, etc.), which can be trained with large amounts of input data to solve complex problems quickly and accurately. In one example, the first layer of a DNN model breaks down an input image of a car into its parts and looks for basic patterns (such as lines and angles). The second layer assembles lines to look for higher-level patterns, such as wheels, windshields, and mirrors. The next layer identifies the type of vehicle, and the last few layers generate labels for the input image, identifying the model of a specific car brand.

[0193] Once a DNN is trained, it can be deployed and used to recognize and classify objects or patterns in a process called inference. Examples of inference (the process by which a DNN extracts useful information from a given input) include recognizing handwritten numbers on check deposits in ATM machines, identifying images of friends in photos, providing movie recommendations to over 50 million users, recognizing and classifying different types of cars, pedestrians, and road hazards in self-driving cars, or translating human speech in real time.

[0194] During training, data flows through the DNN in a forward propagation phase until a prediction is produced, which indicates a label corresponding to the input. If the neural network does not correctly label an input, the error between the correct label and the predicted label is analyzed, and the weights are adjusted for each feature during a backward propagation phase until the DNN correctly labels that input and other inputs in the training data set. Training complex neural networks requires a large amount of parallel computing performance, including floating-point multiplication and addition supported by the PPU 300. Inference is less computationally intensive than training and is a latency-sensitive process in which a trained neural network is applied to new inputs it has not seen before to classify images, translate speech, and generally reason about new information.

[0195] Neural networks rely heavily on matrix math operations, and complex multi-layer networks require a lot of floating point performance and bandwidth for efficiency and speed. With thousands of processing cores optimized for matrix math operations and delivering tens to hundreds of TFLOPS of performance, the PPU 300 is a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications.

[0196] Exemplary Technical Advantages of Some Embodiments

[0197] Certain exemplary embodiments provide for rapid slowing of the clock frequency so that the processor can effectively reduce the operating frequency for executing only noisy instructions while enabling noiseless instructions to execute at or near the highest nominal frequency of the voltage level.

[0198] Compared to solutions such as NAFLL, which are designed to address the frequency effects of noise but address noise seen over a large area, the exemplary embodiments also accommodate very localized noise. Additionally, the exemplary embodiments allow flexibility in IR drop closure, as known power-dense cells can be closed to a looser IR drop budget in the expectation that the fast clock will slow down to a lower frequency when instructions are executed on the power-dense cells. Relaxing the IR drop budget can ease routing congestion and prevent or reduce die area growth.

[0199] Many modifications and variations of the present invention are possible in light of the above teachings.It is therefore to be understood that within the scope of the appended claims, the present invention may be practiced otherwise than as specifically described herein.

Claims

1. A method for controlling a clock of a processor, comprising: detecting an instruction predicted to have a predefined noise characteristic, said detecting occurring before said processor executes said instruction; as well as In response to the detecting, changing the frequency of the clock, wherein said changing said frequency comprises: reducing said frequency by an offset determined to compensate for a predicted voltage drop corresponding to said predefined noise characteristic, and, The offset is determined by a programmable mapping of commands to frequency offsets.

2. The method according to claim 1, further comprising: executing the instructions at least partially when the clock operates at the reduced frequency; as well as The frequency of the clock is increased after the executing. The method of claim 2 , further comprising subjecting the increasing to a hysteresis process. The method of claim 3 , wherein the lowering is performed without undergoing a hysteresis process.

5. The method of claim 1, wherein the detecting comprises: It is determined that the type of the instruction matches a type in a predetermined set of instruction types. 6 . The method of claim 5 , wherein the predetermined set of instruction types includes at least one high-power instruction type predicted to consume more than a threshold power level.

7. The method of claim 5, wherein the predetermined set of instruction types comprises at least two groups of instruction types, each of the two groups having a mapping to a corresponding frequency offset, wherein the detecting further comprises determining to which of the at least two groups the instruction belongs, and The changing further comprises adjusting the frequency according to a corresponding frequency offset mapped to a determined one of the at least two groups.

8. The method of claim 7, wherein at least one of the composition of the at least two groups or the corresponding frequency amounts is programmable.

9. The method according to claim 1, further comprising: configure the clock to operate at a default frequency, and Wherein the detecting occurs when the clock operates at the default frequency, wherein the default frequency is determined based on other instructions predicted to have a lower noise level than the detected instruction.

10. The method of claim 1, wherein the detecting occurs during decoding of the instruction and the changing occurs before the processor executes the decoded instruction.

11. The method according to claim 10, further comprising: responsive to said detecting, stopping said instructions prior to execution; as well as In response to the frequency being changed or the timer signal, the instruction is not stopped.

12. The method of claim 1 , wherein changing the frequency of the clock comprises: After receiving a clock change signal at a clock source, the frequency of the clock source is changed within a predetermined fixed number of clock cycles.

13. The method of claim 12, wherein changing the frequency of the clock further comprises: A target frequency is identified from a lookup table, and the clock source is set to the target frequency.

14. The method of claim 13, wherein changing the frequency of the clock further comprises: The frequency is adjusted according to a noise-aware frequency locked loop technique.

15. A system comprising a processing circuit and a clock generator for controlling an operating frequency of the processing circuit, the processing circuit being configured to: detecting an instruction predicted to have a predefined noise characteristic, the detecting occurring prior to execution of the instruction in the processing circuit; and responsive to the detecting, signaling the clock generator to change a frequency of the clock generator, in, The system further includes a mapping from the type of the instruction to a frequency offset amount determined to compensate for a predicted voltage drop corresponding to the predefined noise characteristic, and wherein the signaling includes including the frequency amount in the signaling.

16. The system of claim 15, wherein said detecting is performed during decoding of said instruction, and said processing circuit is further configured to execute said instruction after said changing said frequency.

17. The system of claim 15, wherein the detecting comprises: It is determined that the type of the instruction matches a type in a predetermined set of instruction types.

18. The system of claim 17 , further comprising at least a first signaling path and a second signaling path between the processing circuit and the clock generator, wherein the predetermined set of instruction types comprises at least a first group of instruction types and a second group of instruction types associated with the first signaling path and the second signaling path, respectively, each of the two groups having a mapping to a corresponding frequency offset, wherein the detecting further comprises determining to which of at least two groups the instruction belongs, and The changing further comprises adjusting the frequency according to a corresponding frequency offset mapped to a determined one of the at least two groups.

19. The system of claim 15, wherein changing the frequency comprises: After receiving a clock change signal at the clock generator, the frequency of the clock source is changed within a predetermined fixed number of clock cycles.

20. The system of claim 19, wherein changing the frequency further comprises: A target frequency is identified from a lookup table, and the clock source is set to the target frequency.

21. The system of claim 20, wherein changing the frequency further comprises: The frequency is adjusted according to a Noise Aware Frequency Locked Loop (NAFLL) technique.

22. The system of claim 15, wherein the processing circuit comprises an instruction pipeline configured to prevent execution of the instruction until a predetermined signal is received from the clock generator or timer.

23. The system of claim 15, wherein the system further comprises a plurality of processor cores including the processing circuit, wherein each of the processor cores receives the frequency from the clock generator, and wherein the clock generator is configured to: receiving a signal from one or more of the plurality of processor cores, and The frequency is changed based on the selected subset of the received signals.

24. The system of claim 15, wherein the processing circuit and the clock generator are formed on an integrated circuit.

Images