Method and device for actively screening hardware errors of computer processing system

Abstract

The invention relates to a method and a device for actively screening hardware errors of a computer processing system. In one embodiment, a computer processing system includes: a plurality of computing units to execute tasks of one or more applications; the plurality of sensors are used for collecting measurement data of the plurality of calculation units; a data structure indicative of the hardware health status of the plurality of computing units determined based on the measurement data is stored in the storage device; and the plurality of computing units are scheduled for task execution on the computer processing system for the one or more applications based on the hardware health states of the plurality of computing units indicated in the data structure, where when a respective first hardware health state of a first computing unit indicates an impending hardware failure, the first computing unit is scheduled for task execution on the computer processing system for the one or more applications. The first computing unit is excluded from task execution.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computing; more specifically, the embodiments relate to an apparatus and method for actively screening hardware errors of a computer processing system. Background Art

[0002] Modern processing units are collectively referred to as xPUs, and they refer to processing units such as central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), or infrastructure processing units (IPUs). xPUs are experiencing significant increases in complexity as a direct result of increased core counts, dedicated hardware accelerator units, increased cache sizes, reduced transistor / wire sizes, reduced power (Vmin), increased frequencies (Fmax), and increasing performance characteristics. These increased complexities lead to increased defects per million (DPM) at time 0 (defects at manufacturing, also known as T0 errors) and during use due to aging.

[0003] In xPU manufacturing, test coverage is critical for lower predictable DPM. Higher coverage is achieved by obtaining a longer and exhaustive combination of functional, logical, and performance coverage. Additionally, test data such as silent data errors (SDE), T0 errors, usage-related errors (early life / aging) are difficult to simulate / detect / reproduce in manufacturing. Parts shipped to system manufacturers undergo further test screening to test the integrity of the parts that make up the complete system. These tests are critical to capture SDE-related errors to eliminate them from production, where the impact of failures may be even higher.

[0004] Once a computing system enters operation, the system may experience early life failures and aging-related failures, such as electromigration, time-dependent dielectric breakdown, and result in a large number of observable non-SDE / SDE. In order to minimize this, the computing system needs to be continuously screened for failures. For example, the computing units of the system may be taken offline from the operating system (OS) and / or applications of the computing system to run screening tests, such offline screening is disruptive to the OS / applications and reduces the quality-of-service (QoS) and availability of the computing system. Summary of the invention

[0005] One aspect of the present disclosure provides a computer processing system, comprising: multiple computing units for executing tasks of one or more applications; multiple sensors, which are coupled to the multiple computing units and are used to collect measurement data of the multiple computing units; and a storage device for storing a hardware health status corresponding to each of the multiple computing units, wherein the hardware health status is based on the measurement data; the multiple computing units are scheduled to perform tasks on the computer processing system for the one or more applications based on the hardware health status of the multiple computing units, wherein, in response to the corresponding hardware health status of a first computing unit among the multiple computing units indicating that a hardware failure is about to occur, the first computing unit is excluded from the task execution.

[0006] One aspect of the present disclosure provides a method, comprising: executing tasks of one or more applications by multiple computing units of a computer processing system; collecting measurement data of the multiple computing units by multiple sensors coupled to the multiple computing units; storing a hardware health status corresponding to each of the multiple computing units in a storage device, wherein the hardware health status is based on the measurement data; and scheduling the multiple computing units to perform tasks on the computer processing system for the one or more applications based on the hardware health status of the multiple computing units, wherein, in response to a corresponding hardware health status of a first computing unit among the multiple computing units indicating an impending hardware failure, the first computing unit is excluded from the task execution.

[0007] One aspect of the present disclosure provides a non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a computing system, enable the computing system to perform the following items: executing tasks of one or more applications by multiple computing units of the computer processing system; collecting measurement data of the multiple computing units by multiple sensors coupled to the multiple computing units; storing a hardware health state corresponding to each of the multiple computing units in a storage device, wherein the hardware health state is based on the measurement data; and scheduling the multiple computing units to perform tasks on the computer processing system for the one or more applications based on the hardware health state of the multiple computing units, wherein, in response to a corresponding hardware health state of a first computing unit among the multiple computing units indicating that a hardware failure is about to occur, the first computing unit is excluded from the task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present disclosure may be best understood by referring to the following description and accompanying drawings which are used to illustrate embodiments of the disclosure.

[0009] Figure 1 An architecture for proactively screening a computer processing system for hardware errors is shown in accordance with some embodiments.

[0010] Figure 2 Operations / entities for determining the health status of a computing unit according to some embodiments are shown.

[0011] Figure 3 A software hierarchy for proactively screening a computer processing system for hardware errors is illustrated in accordance with some embodiments.

[0012] Figure 4 The operational sequence of non-intrusive screening and intrusive content interleaving according to some embodiments is illustrated.

[0013] Figure 5 is a block diagram illustrating hardware fault filtering in an in-order architecture core and a register renaming, out-of-order issue / execution architecture core to be included in a processor according to some embodiments.

[0014] Figure 6 An operational flow for implementing proactive hardware error filtering according to some embodiments is illustrated.

[0015] Figure 7 Examples of execution unit circuit(s) such as execution unit circuit(s) are illustrated.

[0016] Figure 8 An example of a computing system is illustrated.

[0017] Fig. 9 A block diagram of an example processor and / or system on a chip (SoC) that may have one or more cores and an integrated memory controller is illustrated.

[0018] Fig.10 is a block diagram illustrating a computing system 1000 configured to implement one or more aspects of the examples described herein.

[0019] Fig.11A An example of a parallel processor is illustrated.

[0020] Fig. 11B An example of a block diagram of a partition unit is illustrated.

[0021] Fig. 11C An example of a block diagram of a processing cluster within a parallel processing unit is illustrated.

[0022] Fig.11D An example of a graphics multiprocessor is illustrated, wherein the graphics multiprocessor is coupled with a pipeline manager of a processing cluster.

[0023] FIG. 12A to FIG. 12C Additional graphics multiprocessors according to examples are illustrated.

[0024] Fig.13 A parallel computing system 1300 is illustrated according to some examples.

[0025] FIG. 14A to FIG. 14B A hybrid logical / physical view of a factorized parallel processor according to examples described herein is illustrated.

[0026] FIG. 15A to FIG. 15B Thread execution logic including an array of processing elements employed in a graphics processor core according to examples described herein is illustrated.

[0027] Fig.16 Additional execution units according to examples are illustrated.

[0028] Fig.17 is a block diagram of another example of a graphics processor.

[0029] Fig.18 is a block diagram illustrating converting binary instructions in a source ISA to binary instructions in a target ISA using a software instruction converter according to an example.

[0030] Fig.19 is a block diagram illustrating an IP core development system 1900 that may be used to fabricate an integrated circuit to perform operations according to some examples. DETAILED DESCRIPTION

[0031] In the following description, many specific details are set forth. However, it should be understood that embodiments of the present disclosure may be practiced without these specific details. In other cases, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the understanding of this specification.

[0032] Text in brackets and boxes with dashed borders (such as large dashes, small dashes, dot-dash lines, and dots) may be used to illustrate optional operations that add additional features to embodiments of the present disclosure. However, such notation should not be understood to mean that these are the only options or optional operations and / or that blocks with solid borders are not optional in some embodiments of the present disclosure.

[0033] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered to be within the knowledge of those skilled in the art to implement such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0034] The term "connected" means a direct electrical or magnetic connection between the connected things without any intermediate devices, while the term "coupled" means a direct electrical or magnetic connection between the connected things or an indirect connection through one or more passive or active intermediate devices. The term "circuit" means one or more passive and / or active components that are arranged to cooperate with each other to provide a desired function. As used herein, "set" refers to any positive integer number of items, including one item.

[0035] The embodiments herein describe a computer processing system in an example. The computer processing system includes one or more central processing units (CPUs), graphics processing units (GPUs), tensor processing units (TPUs), or infrastructure processing units (IPUs), each of which may include one or more cores. The computer processing system may be implemented in a system on a chip (SoC) integrated in an integrated circuit ("IC") die, or in some embodiments in a chip. In alternative embodiments (e.g., see FIG. 14A to FIG. 14B ), a computer processing system can be broken down into various dies, blocks, and / or chiplets that are physically and electrically coupled together by a packaging structure.

[0036] Architecture for filtering hardware failures

[0037] Currently, processors (xPUs) are typically tested offline for hardware failures. Offline testing is performed using error screening content (ESC) with exhaustive capabilities and part-level defect coverage pressure, and the silicon logic is screened and the test results are compared with expected results. A static ESC approach with static configuration is used. ESC is run and rerun in a high-volume manufacturing (HVM) operating environment, and offline testing looks for errors to manifest within a predetermined time as per manufacturing cycle / time budget constraints.

[0038] This offline testing is primarily used for HVM screening, and it removes the compute unit within the xPU from use by the operating system (OS) and / or applications (consumers of the compute unit). This intrusion can have quality of service (QoS) implications and affect customer satisfaction, not to mention the loss of usage / revenue from taking the compute unit offline for screening.

[0039] Furthermore, offline testing is not suitable for continuous proactive hardware error screening. Workloads may run for long periods of time (e.g., hours or even days), and moving the workload from a first computing unit to a second computing unit to perform screening on the first computing unit is cumbersome and inefficient. Additionally, offline testing cannot provide fine-grained opportunistic screening (e.g., when a computing unit is running at full capacity).

[0040] A set of on-die silicon methods have also been proposed to test and prevent hardware failures. For example, dedicated test circuits can be integrated directly onto the die itself. This allows comprehensive testing of die functionality and performance at the silicon level. In addition, hardware screening techniques such as core lock-step can be deployed at runtime to ensure the reliability and fault tolerance of the processor or processor cores. In core lock-step, two or more processor cores receive the same input, execute the same instructions, and produce the same output. The outputs of the individual cores are then compared to check for any differences. If a difference is detected, it indicates a potential error or failure in one of the cores.

[0041] However, these hardware-based screening approaches consume valuable silicon real estate that could otherwise be used for more performance and result in a significant degradation in performance / cost metrics.

[0042] Embodiments of the present disclosure proactively screen for hardware errors by scheduling continuous screening to identify unhealthy computing units and remove unhealthy computing units from the resource pool. Proactive screening does not remove computing units from the execution task of the OS / application, thereby being superior to offline testing.

[0043] Additionally, active screening in an embodiment does not require the addition of dedicated hardware for testing, thereby having less impact on the die itself and being easier to implement than silicon-on-die approaches.

[0044] Figure 1 An architecture for actively screening hardware errors of a computer processing system according to some embodiments is shown. Architecture 100 receives an application at reference numeral 102 or a task to be performed for an application. Tasks can come from internal operations of the OS and the system (e.g., virtual machines, containers). Some or all of the applications / tasks 102 may be mission-critical or time-sensitive, and in this case, they have stricter QoS requirements and lower tolerance for hardware failures, which will cause significant impacts. Tasks are queued in one or more task queues 106, waiting to be scheduled for execution by a scheduler 150. Scheduler 150 may also be referred to as a hardware error resilient scheduler (HERS), indicating that it provides features for hardware error resilient scheduling. Note that the terms "task" and "workload" are used interchangeably herein.

[0045] Scheduler 150 is implemented as a software module in some embodiments and as hardware logic / circuitry in alternative embodiments. Task scheduling in a computing processing system is typically based on the resource capacity of the computing units therein, where the resource capacity is based on the availability of resources in the computing units and new tasks are assigned resources that (1) have not yet been assigned to existing tasks or (2) have been assigned but can be shared with the new task. Such task scheduling does not take into account whether the computing unit may suffer an impending hardware failure that prevents the application / task from successfully completing. Instead, scheduler 150 takes into account both the resource capacity 108 and the hardware health status 166 of the computing unit.

[0046] The scheduler 150 interacts with a telemetry analyzer 162, which determines and provides the health status of the computing unit of the computer processing system at reference numeral 166. The telemetry analyzer 162 collects and analyzes measurement data from sensors 120 coupled to the computing unit to determine the hardware health status and stores the health status to a storage device at reference numeral 164. Although the scheduler 150 and the telemetry analyzer 162 are shown as separate entities, in some embodiments they may be integrated into a single software module or hardware logic / circuitry.

[0047] The health status of a computing unit indicates the health of the computing unit. A computing unit may be a hardware entity involved in performing a task in a computer processing system. For example, a hardware entity may be an xPU or xPU core, a memory system (e.g., a cache / storage device), a transmission unit (e.g., an interconnect such as a peripheral component interconnect (PCI)), an input / output (I / O) device, a communication device, a control circuit (e.g., power / clock control). In addition, components within an xPU or xPU core may also be referred to as computing units. Figure 6 Some examples of these components are shown.A computing unit herein refers to a hardware entity in a computing processing system that may experience a hardware failure, although a resource in a computing processing system may also refer to a more abstract / virtual unit (eg, bandwidth).

[0048] Sensors 120 measure errors and aging issues of the corresponding computing units. These sensors measure and monitor the effects of hardware component aging (e.g., transistor aging) and potential errors that may occur on the processor over time. Although these sensors can be added specifically to detect impending hardware failures, existing sensors can be repurposed to perform such detection. For example, existing sensors are already used to measure temperature, voltage, current, latency, power consumption, usage patterns, and other performance metrics of computing units, and they can be used to detect impending hardware failures.

[0049] Telemetry analyzer 162 analyzes the measurement data collected from sensors 120 and constructs screening data for different computing units including memory, xPUs, and other hardware entities (and / or components within these hardware entities). The screening data may include ongoing measurements based on which telemetry analyzer 162 can determine the health status of the computing unit.

[0050] The health state of a computing unit can be one of three categories: (1) healthy, in which the computing unit is operating normally with no detected errors; (2) faulted, in which the computing unit is not operating normally and cannot be assigned to be executed for application / task 102; and (3) available but considered to have impending failure, in which the computing unit is still functional but may soon fail.

[0051] In some embodiments, more categories may be included to further specify the health status of a computing unit. For example, a determination of an impending failure may be mapped to a confidence level of such determination, an impending failure may be mapped to a severity of the failure and / or an impact of the failure, and an impending failure may be mapped to an expected time frame for the failure to occur.

[0052] All health status information may be stored in a storage device as hardware health status 164. The information may be stored in a table, map, dictionary, list, array, file, or another data structure, and the data structure may be saved in a database. In some embodiments, the data structure may be indexed by an identifier (ID) of a computing unit, and each entry in the data structure may include one or more of the following: an identifier, a corresponding health status of a corresponding computing unit, a confidence level of a health status determination, and / or a severity / impact / ID of (one or more) impending failures.

[0053] Scheduler 150 schedules task execution by assigning compute units 170 to applications / tasks 102. Compute units 170 are selected based not only on their resource capacity 108 but also on the health status of the compute units at reference numeral 166.

[0054] Since the scheduler 150 takes into account the health of the computing units, scheduling can be performed opportunistically to utilize healthy and reliable computing units to complete the assigned tasks. Computing units that are available but considered to be about to fail can be excluded from being assigned to task execution. By actively filtering out computing units that are about to fail by the scheduler 150, critical tasks will not suffer performance degradation. The scheduler 150 can allocate computing units based on the confidence level determined by the health state, for example, computing units that are considered to be about to fail and have lower confidence can still be allocated for non-critical tasks, while computing units with higher confidence can be completely excluded. The scheduler 150 can allocate computing units based on the severity, impact and timing of the impending failure. For example, when the computer processing system does not have enough healthy computing units, computing units with impending failures determined as follows can still be allocated for task execution, and the failure has low severity, small impact, and occurs later / less frequently.

[0055] Scheduling based on the health of a computing unit is non-intrusive because normal execution of an application / task 102 is not interrupted by hardware filtering. Scheduling can also provide better QoS for normal execution because a computing unit can be detected as having an impending failure and an application / task 102 running on the computing unit can be migrated to a healthy alternative computing unit.

[0056] In some embodiments, in addition to the application / task 102, error screening content (ESC) may be included in the task queue 106 for the scheduler 150 to allocate computing units. The injected ESC allows the scheduler 150 to interleave the execution of the ESC into the natural execution of the OS and application tasks without taking the computing unit completely offline.

[0057] Figure 2 Operations / entities for determining the health status of a computing unit according to some embodiments are shown. Sensors 120, telemetry analyzer 162 are shown in more detail in this figure. Various sensors can be used to determine hardware and aging issues of the corresponding computing unit, including the sensors shown in the figure.

[0058] For example, sensor 120 may include an in-die variation (IDV) oscillator (also referred to as an IDV probe or sensor) 202. IDV oscillator 202 may include a bank of oscillators at distributed locations in a processor die and provide a stable and reliable timing reference for various digital circuits on the die by continuously monitoring and compensating for variations occurring within a single die.

[0059] Sensors 120 may include reference voltage / current sink sensors 204 for monitoring and controlling voltage or current levels within the processor to maintain stable and accurate operation of the processor. Reference voltage / current sink sensors 204 may measure the actual voltage or current level in a particular domain and compare it to a predetermined reference value. If the measured value deviates from the reference value, the sensor may initiate corrective action to maintain the desired level.

[0060] Sensors 120 may include a reference critical path delay drift sensor (RCPDDS) 206 that monitors the delay of a critical path in a processor. The critical path includes the longest path through the processor, and variations in its delay affect the overall performance of the processor. RCPDDS 206 measures the delay and may adjust the processor's clock speed to compensate for variations in the delay.

[0061] Sensors 120 may also include replica path delay drift sensors (also referred to as replica delay lines) 208 that measure the delay experienced by a signal as it propagates through a particular critical path within the processor and compensate for variations in the path delay.

[0062] Sensors 120 may additionally include other error / aging sensors 210 that measure temperature, power consumption, usage patterns, and other performance metrics of computing units on the processor / die.

[0063] In addition, the sensor 120 may include component-level BIST (built-in self-test) logic 212 integrated directly into various components or modules of the processor. BIST allows self-testing of various functional units or components within the processor at a circuit or logic level. The BIST logic 212 operates during the manufacturing process or during system initialization and helps ensure that the internal components of the processor meet the required specifications and performance standards.

[0064] Note that the described sensors 120 measure various performance metrics on various computing units on the processor / die. They have known functions and are already implemented on the processor / die. Thus, no additional hardware is needed to measure the health of the computing units to enable active screening of hardware errors. Existing sensors can be repurposed to determine impending failures of corresponding computing units.

[0065] Various sensors within the sensors 120 provide measurement data to the telemetry analyzer 162, which includes a telemetry collection module 220 (sometimes referred to as a telemetry collector). The telemetry collection module 220 monitors and collects measurement data from the sensors 120. The collection may be performed periodically, or triggered by an event of the computer processing system, such as when the scheduler 150 is required to allocate a computing unit for an application / task 102. The telemetry collection module 220 may poll the sensors 120 for measurement data, and the sensors 120 may also push the measurement data to the telemetry collection module 220, or the two entities may operate using a combination of pushing and polling.

[0066] Analyzer 222 predicts whether a computing unit will cause an imminent failure based on the measurement data collected by telemetry collection module 220 and / or the historical hardware health status stored in the data structure discussed herein. Analyzer 222 may perform prediction based on rules or heuristic algorithms to analyze various measurement data patterns that may precede a failure.

[0067] Additionally or alternatively, the analyzer 222 can predict impending failures based on machine learning, where the machine learning takes the measurement data and / or historical hardware health states collected by the telemetry collection module 220 and uses one or more machine learning models to determine impending failures. The machine learning model can use supervised learning, unsupervised learning, semi-supervised learning, or other types of learning. It can use artificial neural networks, decision trees, support vector machines, regression analysis, Bayesian networks, genetic algorithms, or any other framework. The machine learning model can be trained with historical hardware health states of the computing unit and verified subsequent failures (or none) to calibrate the machine learning model.

[0068] Once a prediction of the health of a computing unit is made, the prediction can be saved in hardware health status 164 for comparison with later performance and analysis of the corresponding computing unit. Such data comparison can be used to adjust rules, heuristic algorithms, and / or machine learning models.

[0069] Software hierarchy for filtering hardware failures

[0070] Figure 3 Illustrated is a software hierarchy for proactively screening a computer processing system for hardware errors according to some embodiments.Computer processing system 300 includes software and hardware, the latter including a computing unit for which telemetry analyzer 162 collects and analyzes health status, as discussed.

[0071] The computing units include cores #0 to n at reference numerals 352 to 354 (where a core may be a core of any xPU or other processing unit discussed above) and other computing units at reference numeral 360, including one or more of a memory system, a transport unit, an I / O device, a communication device, and / or a control circuit. Each computing unit may be coupled to one or more sensors (e.g., sensor 120) to collect measurement data from the respective computing unit, and the measurement data is analyzed by telemetry analyzer 162, the operation of which is discussed above.

[0072] Scheduler 150 is implemented as a software module in computer processing system 300. In some embodiments, scheduler 150 is part of a software package (e.g., Windows Server Hyper-V) that provides virtualization capabilities for OS systems and allows users to create and manage virtual machines (VMs) on a single physical server, thereby enabling the consolidation of multiple operating systems and applications.

[0073] Scheduler 150 allocates computing units for natural execution of OS and application tasks. In addition, error screening content (ESC) can be interleaved with OS and application tasks, so computing units can execute tasks while screening for hardware failures, and executing simultaneously (e.g., queuing and executing application / task 102 and ESC 104) is superior to existing methods of taking computing units offline for screening.

[0074] The interleaving of tasks can be implemented at several levels in software. The scheduler 150 may include a service or library at reference numeral 342 so that the ESC can be inserted directly into the schedule in the scheduler 150. Alternatively or additionally, the service or library may be included at the OS or VM level, where services or libraries 332 and 334 are included in OS / VM 336 and 338, respectively. In addition, such a service or library may be included as an application / task at the workload level, as shown by service or library 322 at workload 302. For example, the error screening content (ESC) at reference numeral 104 as an application / task may be a service or library 322 at the application level, which is interleaved with workload #0 at reference numeral 302. Note that the scheduler 150 may schedule interleaved workloads (e.g., workloads #0 and #3 at reference numerals 302 and 306, respectively) and non-interleaved workloads (e.g., workloads #1 and #4 at reference numerals 304 and 308) based on the hardware health status provided by the telemetry analyzer 162.

[0075] The scheduler 150 implemented as a software module provides advantages over existing approaches such as silicon-on-die approaches, in which dedicated hardware is used on the die itself. The scheduler 150 schedules computing resources based on measurement data collected from sensors 120, which are repurposed to determine hardware health status; thus, in some embodiments, no additional dedicated hardware is required to implement active screening. Thus, a hybrid approach using software (scheduler as software) and hardware (sensors) can actively screen for hardware faults not only during the processor manufacturing phase when building new processors, but also during processor maintenance cycles when testing existing processors to identify defects (because no new dedicated hardware is required); and can even be used for ongoing screening while the processor is executing OS / application tasks (because interleaving of tasks is supported).

[0076] The services or libraries required for interleaving filtering can be accessed through an application programming interface (API), a software development kit (SDK), a command line interface (CLI), or other means for interleaving ESCs. The services or libraries can be created by a third party that does not produce the processor or OS. For example, a processor vendor can provide APIs, SDKs, and / or CLIs to other parties to build ESCs or applications, thereby interleaving ESCs with OS / application tasks. By doing so, the vendor provides application providers or cloud service providers with the opportunity to innovate continuous trickle filtering of computing units allocated to tenants of cloud service providers.

[0077] A hardware error-aware scheduler such as scheduler 150 may use the same computing unit for screening as well as OS / application tasks, and may perform such screening continuously while executing OS / application tasks. Continuous screening accumulates health information of the computing unit from sensors coupled to the computing unit, and the stored health information allows the scheduler to create health trends for the computing unit and adjust scheduling accordingly. In addition, the scheduler may schedule hardware screening content on computing units that are running below capacity during execution of OS / application tasks. By scheduling hardware screening content on computing units that exhibit additional computing capacity relative to the current task, the scheduler thereby strives to minimize the impact on normal execution in the computer processing system.

[0078] Example Hardware Fault Screening

[0079] Figure 4The operation sequence of non-intrusive screening and intrusive content interleaving according to some embodiments is illustrated. In addition to the telemetry analyzer 162, sensor 120, hardware health status 164 and scheduler 150 discussed above, the figure also includes a resource consumer 402. The resource consumer 402 consumes resources of the computer processing system, including computing units therein, and it can include application / task 102 and / or error screening content (ESC) at reference numeral 104 as an application / task.

[0080] The scheduler 150 sets the telemetry analyzer 162 so that the telemetry analyzer 162 can continuously scan the sensor 120 in the continuous scanner period 450 to obtain the measurement data of the corresponding computing unit. The measurement data allows the telemetry analyzer 162 to determine the hardware health status of the computing unit and update the database storing the hardware health status. Such operation is non-intrusive and does not interrupt the execution of the application / task of the resource consumer 402.

[0081] The scheduler 150 can schedule intrusive silent data errors (SDE) and / or non-SDE workloads on the computing units that need to be screened, as well as workloads from resource consumers 402. The SDE / non-SDE workloads can be interleaved as applications / tasks at reference numeral 104. SDE refers to errors that occur within a computer processing system that are not noticed or detected by the usual error checking mechanisms. SDEs primarily affect data that is processed or stored, and they can occur in various computing units. The selection of SDE and / or non-SDE workloads can be based on many requirements, including resource type, manufacturing information such as characteristics, physical location within a wafer, die, and / or packaging.

[0082] Telemetry analyzer 162 collects and analyzes measurement data collected from executing workloads (and applications / tasks 102 in some embodiments) to determine the health of computing units involved in the execution and update the database accordingly.

[0083] Once the scanner cycle 450 is completed, another scanner cycle 452 may be started, wherein resources are requested to execute a new workload (e.g., a new application / task 102) of the resource consumer 402. The scheduler 150 may schedule computing units to execute the new workload. The scheduler 150 may assign healthy computing units, and / or relocate / migrate applications / tasks away from computing units that are about to fail. The telemetry analyzer 162 then collects and analyzes the measurement data collected from executing the new workload, and updates the database with the updated health status obtained from the telemetry analyzer 162. The scheduling process continues for the next scheduling cycle.

[0084] Example Compute Unit with Hardware Fault Filtering

[0085] Figure 5 5 is a block diagram illustrating hardware fault screening in an in-order architecture core and a register renaming, out-of-order issue / execution architecture core to be included in a processor according to some embodiments. The figure shows a processor core 590, which includes a front end unit circuit 530 coupled to an execution engine unit circuit 550, and both are coupled to a memory unit circuit 570. The core 590 can be a reduced instruction set architecture computing (RISC) core, a complex instruction set architecture computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As another option, the core 590 can be a dedicated core, such as a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0086] The front end unit circuit 530 may include a branch prediction circuit 532 coupled to an instruction cache circuit 534, which is coupled to an instruction translation lookaside buffer (TLB) 536, which is coupled to an instruction fetch circuit 538, which is coupled to a decoding circuit 540. In some examples, the instruction cache circuit 534 is included in the memory unit circuit 570 instead of the front end circuit 530. The decoding circuit 540 (or decoder) can decode the instruction and generate one or more micro-operations, microcode entry points, microinstructions, other instructions or other control signals as outputs, which are decoded from the original instruction or otherwise reflect the original instruction or are derived from the original instruction. The decoding circuit 540 may also include an address generation unit (AGU, not shown) circuit. In some examples, the AGU generates an LSU address using a forwarded register port, and may further perform branch forwarding (e.g., immediate offset branch forwarding, LR register branch forwarding, etc.). The decoding circuit 540 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to, lookup tables, hardware implementations, programmable logic arrays (PLAs), microcode read-only memories (ROMs), etc. In some examples, core 590 includes a microcode ROM (not shown) or other medium that stores microcode for certain macroinstructions (e.g., in decode circuitry 540 or otherwise within front-end circuitry 530). In some examples, decode circuitry 540 includes a micro-op or operation cache (not shown) to hold / cache decode operations, micro-tags, or micro-ops generated during decode or other stages of the processor pipeline. Decode circuitry 540 may be coupled to rename / allocator unit circuitry 552 in execution engine circuitry 550.

[0087] The execution engine circuit 550 includes a rename / allocator unit circuit 552 coupled to a retirement unit circuit 554 and a set of one or more scheduler circuits 556. The scheduler circuit(s) 556 represent any number of different schedulers, including reservation stations, central instruction windows, and the like. In some examples, the scheduler circuit(s) 556 may include an arithmetic logic unit (ALU) scheduler / scheduling circuit, an ALU queue, an address generation unit (AGU) scheduler / scheduling circuit, an AGU queue, and the like. The scheduler circuit(s) 556 are coupled to physical register file(s) circuit(s) 558. Each of the physical register file(s) circuit(s) 558 represents one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer that is the address of the next instruction to be executed), and the like. In some examples, the physical register file(s) circuit(s) 558 include vector register unit circuits, write mask register unit circuits, and scalar register unit circuits. These register units may provide architectural vector registers, vector mask registers, general purpose registers, etc. Physical register file(s) circuitry 558 is coupled to retirement unit circuitry 554 (also referred to as retirement queues) to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using reorder buffer(s) (ROBs) and retirement register file(s); using future stack(s), history buffer(s) and retirement register file(s); using register maps and register pools, etc.). Retirement unit circuitry 554 and physical register file(s) circuitry 558 are coupled to execution cluster(s) 560. Execution cluster(s) 560 includes a set of one or more execution unit circuitry 562 and a set of one or more memory access circuitry 564. Execution unit circuitry 562 may perform various arithmetic, logical, floating point, or other types of operations (e.g., shifts, additions, subtractions, multiplications) on various types of data (e.g., scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some examples may include multiple execution units or execution unit circuits dedicated to a particular function or set of functions, other examples may include only one execution unit circuit or multiple execution units / execution unit circuits that all perform all functions.Scheduler circuit(s) 556, physical register file circuit(s) 558, and execution cluster(s) 560 are shown as possibly multiple because some examples create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines, each with its own scheduler circuit(s), physical register file circuit(s), and / or execution cluster—and in the case of a separate memory access pipeline, in some examples, the execution cluster implementing only that pipeline has memory access unit circuit(s) 564). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue / execution, while the rest are in-order.

[0088] In some examples, execution engine unit circuit 550 may perform load store unit (LSU) address / data pipelining, as well as address stage and writeback, data stage loads, stores, and branches to an advanced microcontroller bus (AMB) interface (not shown).

[0089] The set of memory access circuits 564 is coupled to a memory cell circuit 570, which includes a data TLB circuit 572 coupled to a data cache circuit 574, which is coupled to a level 2 (L2) cache circuit 576. In some examples, the memory access circuit 564 may include a load unit circuit, a store address unit circuit, and a store data unit circuit, each of which is coupled to the data TLB circuit 572 in the memory cell circuit 570. The instruction cache circuit 534 is also coupled to the level 2 (L2) cache circuit 576 in the memory cell circuit 570. In some examples, the instruction cache 534 and the data cache 574 are combined into a single instruction and data cache (not shown) in the L2 cache circuit 576, the level 3 (L3) cache circuit (not shown), and / or the main memory. The L2 cache circuit 576 is coupled to one or more other levels of cache and ultimately to the main memory.

[0090] Core 590 may support one or more instruction sets (e.g., the x86 instruction set architecture (optionally with some extensions that have been added to newer versions); the MIPS instruction set architecture; the ARM instruction set architecture (optionally with optional additional extensions such as NEON)), including the instruction(s) described herein. In some examples, core 590 includes logic to support packed data instruction set architecture extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using packed data.

[0091] Sensor 120 may be implemented in front end unit 530 or execution unit 550. In some embodiments, one or more sensors may be implemented in components of front end unit 530 or execution unit 550. For example, a sensor may be implemented in each of decode circuit 540, instruction fetch unit 538, and / or instruction cache 534.

[0092] Note that some of these components already have sensors coupled to measure their performance data, and no additional sensors need to be implemented to make the measurements. In addition, scheduler 150 can be implemented as a software module, for example, in an existing hardware / software module such as a performance monitoring unit (PMU) of a computer processing system, so in some embodiments no dedicated new hardware is required to actively screen for hardware errors.

[0093] Although processor core 590 is used as an example of a computer processing system that can implement active screening for hardware faults, other computer processing systems (e.g., systems that implement xPUs) can also implement active hardware fault screening, and several exemplary computer processing systems and their computing units are described below. Figures 7 to 17 In these computer processing systems, existing sensors can be repurposed and existing hardware can implement the scheduler discussed herein, or the scheduler can be implemented as a software module, thereby enabling active hardware fault screening in these and other computer processing systems without adding additional dedicated hardware.

[0094] Operation in Some Embodiments

[0095] Figure 6 An operational flow for implementing proactive hardware error screening according to some embodiments is illustrated. The operations in method 600 are performed in a computer processing system as discussed herein.

[0096] At reference numeral 602, a plurality of computing units of a computer processing system execute tasks of one or more applications. At reference numeral 604, a plurality of sensors coupled to the plurality of computing units collect measurement data of the plurality of computing units. At reference numeral 606, a hardware health status corresponding to each of the plurality of computing units is stored in a storage device, wherein the hardware health status is based on the measurement data. At reference numeral 608, the plurality of computing units are scheduled to perform task execution for one or more applications on the computer processing system based on the hardware health status of the plurality of computing units, wherein, in response to a corresponding hardware health status of a first computing unit of the plurality of computing units indicating an impending hardware failure, the first computing unit is excluded from the task execution.

[0097] In some embodiments, the error screening content is scheduled to be executed with tasks of one or more applications to further determine the hardware health of the plurality of computing units. In some embodiments, the error screening content includes a test package for silent data errors (SDEs). In some embodiments, the error screening content is executed as an application. In some embodiments, the error screening content is executed on a computing unit in response to the computing unit running below capacity for tasks of the one or more applications.

[0098] In some embodiments, states of the plurality of computing units are updated based on measurement data periodically obtained from the plurality of sensors.

[0099] In some embodiments, the plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component-level built-in self-test (BIST) logic.

[0100] In some embodiments, the plurality of computing units include one or more of the following entities in a computer processing system: a central processing unit (CPU) core set, a graphics processing unit (GPU) core set, a tensor processing unit (TPU) core set, an intelligence processing unit (IPU) core set, and a memory unit set. In some embodiments, the plurality of computing units include one or more components of a CPU core set, a GPU core set, a TPU core set, an IPU core set, and a memory unit set.

[0101] In some embodiments, machine learning or heuristic methods are used to determine the states of the plurality of computing units.

[0102] In some embodiments, in response to the hardware health status of the first group of computing units being no hardware health issues and the hardware health status of the second group of computing units being no failures but about to have hardware failures, the second group of computing units are excluded from task execution.

[0103] In some embodiments, in response to a corresponding second hardware health state of a second computing unit among the plurality of computing units being an impending hardware failure and a corresponding third hardware health state of a third computing unit among the plurality of computing units being no hardware health issues, a task of the second computing unit is moved to the third computing unit.

[0104] In some embodiments, a microinstruction waste rate (eg, waste) for a single thread is determined based on a ratio of allocated micro-ops and retired micro-ops.

[0105] In some embodiments, the cache miss rate of a single thread is determined based on a number of cache misses per number of instructions.

[0106] In some embodiments, the method further comprises removing branch prediction throttling once thread execution of the single thread is no longer in a low confidence branch.

[0107] The active hardware error screening disclosed herein can be used in various computer processing systems and execution circuit processor architectures, where sensors are repurposed and scheduling is implemented to take hardware health into account, and some examples are provided herein below.

[0108] Exemplary Execution Unit Circuit(s)

[0109] Figure 7 Examples include Figure 5 709. As shown in the figure, the execution unit circuit 562 may include one or more ALU circuits 701, optional vector / single instruction multiple data (SIMD) circuits 703, load / store circuits 705, branch / jump circuits 707, and / or floating point unit (FPU) circuits 709. The ALU circuit 701 performs integer operations and / or Boolean operations. The vector / SIMD circuit 703 performs vector / SIMD operations on packed data (such as SIMD / vector registers). The load / store circuit 705 executes load and store instructions to load data from memory into registers or store data from registers to memory. The load / store circuit 705 may also generate addresses. The branch / jump circuit 707 causes a branch or jump to a memory address according to the instruction. The FPU circuit 709 performs floating point operations. The width of the execution unit circuit 562 varies according to the example and can be in the range of 16 bits to 1024 bits, for example. In some examples, two or more smaller execution units are logically combined to form a larger execution unit (eg, two 128-bit execution units are logically combined to form a 256-bit execution unit).

[0110] Example System

[0111] Figure 8 An example computing system is illustrated. Multiprocessor system 800 is an interface system and includes multiple processors or cores, including a first processor 870 and a second processor 880 coupled via an interface 850 such as a point-to-point (PP) interconnect, structure and / or bus. In some examples, the first processor 870 and the second processor 880 are isomorphic. In some examples, the first processor 870 and the second processor 880 are heterogeneous. Although example system 800 is shown as having two processors, the system can have three or more processors, or can be a single processor system. In some examples, the computing system is a system on chip (SoC).

[0112] Processors 870 and 880 are shown to include integrated memory controller (IMC) circuits 872 and 882, respectively. Processor 870 also includes interface circuits 876 and 878; similarly, second processor 880 includes interface circuits 886 and 888. Processors 870, 880 may exchange information via interface 850 using interface circuits 878, 888. IMCs 872 and 882 couple processors 870, 880 to respective memories, namely memory 832 and memory 834, which may be portions of main memory locally attached to the respective processors.

[0113] Processors 870, 880 can each exchange information with a network interface (NW I / F) 890 via separate interfaces 852, 854 using interface circuits 876, 894, 886, 898. Network interface 890 (e.g., one or more of an interconnect, bus, and / or fabric, and in some examples a chipset) can optionally exchange information with a coprocessor 838 via interface circuit 892. In some examples, coprocessor 838 is a special-purpose processor, such as, for example, a high-throughput processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose graphics processing unit (GPGPU), a neural network processing unit (NPU), an embedded processor, etc.

[0114] A shared cache (not shown) may be included in either processor 870, 880 or external to both processors but connected to the processors via an interface such as a PP interconnect so that if the processors are placed in a low power mode, local cache information of either or both processors may be stored in the shared cache.

[0115] The network interface 890 may be coupled to the first interface 816 via the interface circuit 896. In some examples, the first interface 816 may be an interface such as a peripheral component interconnect (PCI), a PCI Express interconnect, or another I / O interconnect. In some examples, the first interface 816 is coupled to a power control unit (PCU) 817, which may include circuits, software, and / or firmware that perform power management operations with respect to the processors 870, 880, and / or the coprocessor 838. The PCU 817 provides control information to a voltage regulator (not shown) so that the voltage regulator generates an appropriate regulated voltage. The PCU 817 also provides control information that controls the generated operating voltage. In various examples, the PCU 817 may include various power management logic units (circuits) that perform hardware-based power management. Such power management may be fully processor-controlled (e.g., through various processor hardware, and it may be triggered by workload and / or power, thermal, or other processor constraints) and / or power management may be performed in response to an external source (such as a platform or power management source or system software).

[0116] PCU 817 is illustrated as existing as logic separate from processor 870 and / or processor 880. In other cases, PCU 817 may execute on a given one or more of the cores (not shown) of processor 870 or 880. In some cases, PCU 817 may be implemented as a microcontroller (dedicated or general purpose) or other control logic that is configured to execute its own dedicated power management code, sometimes referred to as P-code. In other examples, the power management operations to be performed by PCU 817 may be implemented external to the processor, such as by a separate power management integrated circuit (PMIC) or another component external to the processor. In other examples, the power management operations to be performed by PCU 817 may be implemented in BIOS or other system software.

[0117] Various I / O devices 814 may be coupled to the first interface 816 along with a bus bridge 818 coupling the first interface 816 to the second interface 820. In some examples, one or more additional processors 815 such as a coprocessor, a high throughput integrated many core (MIC) processor, a GPGPU, an accelerator (such as a graphics accelerator or a digital signal processing (DSP) unit), a field programmable gate array (FPGA), or any other processor are coupled to the first interface 816. In some examples, the second interface 820 may be a low pin count (LPC) interface. Various devices may be coupled to the second interface 820, including, for example, a keyboard and / or mouse 822, a communication device 827, and a storage circuit 828. The storage circuit 828 may be one or more non-transitory machine-readable storage media as described below, such as a disk drive or other mass storage device, which may include instructions / code and data 830 and may implement a storage device storing instructions in some examples. Further, an audio I / O 824 may be coupled to the second interface 820. It should be noted that other architectures besides the point-to-point architecture described above are also possible. For example, instead of a point-to-point architecture, a system such as multiprocessor system 800 may implement a multi-point interface or other such architecture.

[0118] Example core architectures, processors, and computer architectures.

[0119] Processor cores may be implemented in different ways, for different purposes, and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance general-purpose out-of-order core intended for general-purpose computing; 3) a special-purpose core intended primarily for graphics and / or scientific (throughput) computing. Different processor implementations may include: 1) a CPU including one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor including one or more special-purpose cores intended primarily for graphics and / or scientific (throughput) computing. Such different processors lead to different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor on a separate die in the same package as the CPU; 3) a coprocessor on the same die as the CPU (in which case such coprocessors are sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic, or as dedicated cores); and 4) a system on a chip (SoC), which may be included on the same die as the CPU (sometimes referred to as application core(s) or application processor(s), the coprocessors described above, and additional functionality. An example core architecture is described next, followed by a description of an example processor and computer architecture.

[0120] Fig. 9 A block diagram of an example processor and / or SoC 900 that may have one or more cores and an integrated memory controller is illustrated. The solid lined box illustrates a processor 900 having a single core 902(A), a system agent unit circuit 910, and a set of one or more interface controller unit circuits 916, while the optional addition of the dashed lined box illustrates an alternative processor 900 having multiple cores 902(A)-(N), a set of one or more integrated memory controller unit circuits 914 and dedicated logic 908 in the system agent unit circuit 910, and a set of one or more interface controller unit circuits 916. It should be noted that the processor 900 may be one of the processors 870 or 880, or may be a processor 870 or 880. Figure 8 coprocessor 838 or 815.

[0121] Thus, different implementations of processor 900 may include: 1) a CPU having: dedicated logic 908, which is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown); and cores 902(A)-(N), which are one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, or a combination of both); 2) a coprocessor having cores 902(A)-(N), which are a large number of dedicated cores primarily intended for graphics and / or scientific (throughput); and 3) a coprocessor having cores 902(A)-(N), which are a large number of general purpose in-order cores. Thus, processor 900 may be a general purpose processor, a coprocessor, or a dedicated processor, such as, for example, a network or communications processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput many integrated core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 900 may be part of and / or implemented on one or more substrates using any of a variety of process technologies, such as, for example, complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0122] The memory hierarchy includes one or more levels of cache unit circuits 904(A)-(N) within the cores 902(A)-(N), a set of one or more shared cache unit circuits 906, and external memory (not shown) coupled to the set of (one or more) integrated memory controller unit circuits 914. The set of one or more shared cache unit circuits 906 may include one or more mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache (such as a last level cache (LLC)), and / or combinations thereof. While in some examples, an interface network circuit 912 (e.g., a ring interconnect) interfaces with dedicated logic 908 (e.g., integrated graphics logic), the set of (one or more) shared cache unit circuits 906, and the system agent unit circuit 910, alternative examples use any number of well-known techniques to interface with such units. In some examples, coherency is maintained between the (one or more) shared cache unit circuits 906 and one or more of the cores 902(A)-(N). In some examples, interface controller unit circuit 916 couples core 902 to one or more other devices 918, such as one or more I / O devices, storage, one or more communication devices (e.g., wireless networking, wired networking, etc.), and the like.

[0123] In some examples, one or more of the cores 902(A)-(N) are capable of multithreading. System agent unit circuitry 910 includes those components that coordinate and operate the cores 902(A)-(N). System agent unit circuitry 910 may include, for example, power control unit (PCU) circuitry and / or display unit circuitry (not shown). The PCU may be or may include logic and components required to regulate the power state of the cores 902(A)-(N) and / or the dedicated logic 908 (e.g., integrated graphics logic). The display unit circuitry is used to drive one or more externally connected displays.

[0124] The cores 902(A)-(N) may be homogeneous in terms of instruction set architecture (ISA). Alternatively, the cores 902(A)-(N) may be heterogeneous in terms of ISA; that is, a subset of the cores 902(A)-(N) may be capable of executing an ISA, while other cores may be capable of executing only a subset of that ISA or another ISA.

[0125] Fig.101 is a block diagram illustrating a computing system 1000 configured to implement one or more aspects of the examples described herein. The computing system 1000 includes a processing subsystem 1001 having one or more processors 1002 and a system memory 1004 communicating via an interconnect path that may include a memory hub 1005. The memory hub 1005 may be a separate component within a chipset component, or may be integrated within the one or more processors 1002. The memory hub 1005 is coupled to an I / O subsystem 1011 via a communication link 1006. The I / O subsystem 1011 includes an I / O hub 1007, which may enable the computing system 1000 to receive input from one or more input devices 1008. In addition, the I / O hub 1007 may enable a display controller that may be included in the one or more processors 1002 to provide output to one or more display devices 1010A. In some examples, the one or more display devices 1010A coupled to the I / O hub 1007 may include local, internal, or embedded display devices.

[0126] The processing subsystem 1001 includes, for example, one or more parallel processors 1012, which are coupled to the memory hub 1005 via a bus or other communication link 1013. The communication link 1013 can be one of any number of standard-based communication link technologies or protocols, such as but not limited to PCI Express, or can be a vendor-specific communication interface or communication structure. The one or more parallel processors 1012 can form a parallel or vector processing system in a computational concentration, which can include a large number of processing cores and / or processing clusters, such as an integrated many-core (MIC) processor. For example, the one or more parallel processors 1012 form a graphics processing subsystem that can output pixels to one of the one or more display devices 1010A coupled via the I / O hub 1007. The one or more parallel processors 1012 can also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 1010B.

[0127] Within the I / O subsystem 1011, a system storage unit 1014 may be connected to the I / O hub 1007 to provide a storage mechanism for the computing system 1000. An I / O switch 1016 may be used to provide an interface mechanism that enables connection between the I / O hub 1007 and other components such as a network adapter 1018 and / or a wireless network adapter 1019 that may be integrated into the platform and various other devices that may be added via one or more additional devices 1020. The additional device(s) 1020 may also include, for example, one or more external graphics processor devices, graphics cards, and / or computing accelerators. The network adapter 1018 may be an Ethernet adapter or another wired network adapter. The wireless network adapter 1019 may include one or more of Wi-Fi, Bluetooth, near field communication (NFC), or other network devices including one or more wireless radios.

[0128] Computing system 1000 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices, etc., which may also be connected to I / O hub 1007. Fig.10 The communication paths of the various components in the system may be implemented using any suitable protocol, such as a PCI (Peripheral Component Interconnect) based protocol (e.g., PCI Express) or any other bus or point-to-point communication interface and / or protocol(s), such as NVLink high-speed interconnect, Compute Express Link™ (CXL), ... TM ) )(e.g., CXL.mem), Infinity Fabric (IF), Ethernet (IEEE 802.3), Remote Direct Memory Access (RDMA), InfiniBand, Internet Wide Area RDMA Protocol (iWARP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Quick UDP Internet Connection (QUIC), RDMA over Converged Ethernet (RoCE), Intel Quick Path Interconnect (QPI), Intel Ultra Path Interconnect (UPI), Intel System on Chip Fabric (IOSF), Omnichannel, HyperTransport, Advanced Microcontroller Bus Architecture (AMBA) Interconnect, OpenCAPI, Gen-Z, Cache Coherent Interconnect for Accelerators (CCIX), 3GPP Long Term Evolution (LTE) (4G), 3GPP 5G, and variations thereof, or a wired or wireless interconnect protocol known in the art. In some examples, data may be copied or stored to a virtualized storage node using a protocol such as Non-Volatile Memory Express (NVMe) (NVMe-oF) or NVMe over Fabric.

[0129] One or more parallel processors 1012 may be incorporated with circuits optimized for graphics and video processing, including, for example, video output circuits, and constitute a graphics processing unit (GPU). Alternatively or additionally, one or more parallel processors 1012 may be incorporated with circuits optimized for general processing while retaining the underlying computing architecture, as described in more detail herein. The components of the computing system 1000 may be integrated with one or more other system elements on a single integrated circuit. For example, one or more parallel processors 1012, a memory hub 1005, (one or more) processors 1002, and an I / O hub 1007 may be integrated into a system on chip (SoC) integrated circuit. Alternatively, the components of the computing system 1000 may be integrated into a single package to form a system in package (SIP) configuration. In some examples, at least a portion of the components of the computing system 1000 may be integrated into a multi-chip module (MCM), which may be interconnected with other multi-chip modules into a modular computing system.

[0130] It should be understood that the computing system 1000 shown herein is illustrative, and variations and modifications are possible. The connection topology, including the number and arrangement of bridges, the number of (one or more) processors 1002, and the number of (one or more) parallel processors 1012, can be modified as desired. For example, the system memory 1004 can be directly connected to the (one or more) processors 1002 instead of through a bridge, while other devices communicate with the system memory 1004 via the memory hub 1005 and the (one or more) processors 1002. In other alternative topologies, (one or more) parallel processors 1012 are connected to the I / O hub 1007 or directly connected to one of the one or more processors 1002, rather than being connected to the memory hub 1005. In other examples, the I / O hub 1007 and the memory hub 1005 can be integrated into a single chip. It is also possible to attach two or more groups of (one or more) processors 1002 via multiple sockets, which can be coupled to two or more instances of (one or more) parallel processors 1012.

[0131] Some of the specific components shown herein are optional and may not be included in all implementations of computing system 1000. For example, any number of add-in cards or peripherals may be supported, or some components may be removed. In addition, some architectures may be specific to the Fig.10 Different terminology is used for similar components as illustrated. For example, in some architectures, the memory hub 1005 may be referred to as a north bridge, while the I / O hub 1007 may be referred to as a south bridge.

[0132] Fig.11AAn example of a parallel processor 1100 is illustrated. Parallel processor 1100 may be a GPU, a GPGPU, etc., as described herein. Various components of parallel processor 1100 may be implemented using one or more integrated circuit devices, such as a programmable processor, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). The illustrated parallel processor 1100 may be Fig.10 One or more of the parallel processor(s) 1012 are shown.

[0133] The parallel processor 1100 includes a parallel processing unit 1102. The parallel processing unit includes an I / O unit 1104 that enables communication with other devices including other instances of the parallel processing unit 1102. The I / O unit 1104 can be directly connected to other devices. For example, the I / O unit 1104 is connected to other devices by using a hub or switch interface (such as a memory hub 1005). The connection between the memory hub 1005 and the I / O unit 1104 forms a communication link 1013. Within the parallel processing unit 1102, the I / O unit 1104 is connected to a host interface 1106 and a memory cross switch 1116, wherein the host interface 1106 receives commands dedicated to performing processing operations, and the memory cross switch 1116 receives commands dedicated to performing memory operations.

[0134] When the host interface 1106 receives the command buffer via the I / O unit 1104, the host interface 1106 may direct the work operations to execute the commands to the front end 1108. In some examples, the front end 1108 is coupled with a scheduler 1110, which is configured to distribute commands or other work items to the processing cluster array 1112. The scheduler 1110 ensures that the processing cluster array 1112 is properly configured and is in a valid state before the task is distributed to the processing cluster of the processing cluster array 1112. The scheduler 1110 may be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1110 may be configured to perform complex scheduling and work distribution operations at coarse and fine granularity, thereby achieving fast preemption and context switching of threads executed on the processing cluster array 1112. Preferably, the host software may prove the workload for scheduling on the processing cluster array 1112 via one of a plurality of graphics processing doorbells. In other examples, polling for new workloads or interrupts may be used to identify or indicate the availability of work to be executed. The scheduler 1110 logic within the scheduler microcontroller can then automatically distribute the workload across the array of processing clusters 1112 .

[0135] The processing cluster array 1112 may include up to "N" processing clusters (e.g., cluster 1114A, cluster 1114B to cluster 1114N). Each cluster 1114A-1114N of the processing cluster array 1112 may execute a large number of concurrent threads. The scheduler 1110 may use various scheduling and / or work distribution algorithms to distribute work to the clusters 1114A-1114N of the processing cluster array 1112, and the algorithms may vary depending on the workload generated by various types of programs or calculations. Scheduling may be handled dynamically by the scheduler 1110, or may be partially assisted by compiler logic during the compilation of program logic configured for execution by the processing cluster array 1112. Optionally, different clusters 1114A-1114N of the processing cluster array 1112 may be allocated to process different types of programs or to perform different types of calculations.

[0136] Processing cluster array 1112 may be configured to perform various types of parallel processing operations. For example, processing cluster array 1112 is configured to perform general-purpose parallel computing operations. For example, processing cluster array 1112 may include logic to perform processing tasks including filtering of video and / or audio data, performing modeling operations (including physics operations), and performing data transformations.

[0137] The processing cluster array 1112 is configured to perform parallel graphics processing operations. In such examples where the parallel processor 1100 is configured to perform graphics processing operations, the processing cluster array 1112 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic to perform texture operations, and tessellation logic and other vertex processing logic. In addition, the processing cluster array 1112 may be configured to execute graphics processing related shader programs, such as but not limited to vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. The parallel processing unit 1102 may transfer data from the system memory via the I / O unit 1104 for processing. During processing, the transferred data may be stored in an on-chip memory (e.g., parallel processor memory 1122) during processing and then written back to the system memory.

[0138] In examples where parallel processing unit 1102 is used to perform graphics processing, scheduler 1110 may be configured to divide the processing workload into tasks of approximately equal size to better achieve the allocation of graphics processing operations to multiple clusters 1114A-1114N of processing cluster array 1112. In some of these examples, portions of processing cluster array 1112 may be configured to perform different types of processing. For example, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen space operations to produce a rendered image for display. Intermediate data generated by one or more clusters 1114A-1114N may be stored in a buffer to allow the intermediate data to be transferred between clusters 1114A-1114N for further processing.

[0139] During operation, the processing cluster array 1112 may receive processing tasks to be performed via the scheduler 1110, which receives commands defining the processing tasks from the front end 1108. For graphics processing operations, the processing tasks may include indices of data to be processed (e.g., surface (tile) data, primitive data, vertex data, and / or pixel data), and state parameters and commands defining how the data is to be processed (e.g., what program to execute). The scheduler 1110 may be configured to obtain an index corresponding to a task, or may receive an index from the front end 1108. The front end 1108 may be configured to ensure that the processing cluster array 1112 is configured to a valid state before initiating a workload specified by an incoming command buffer (e.g., a batch buffer, a push buffer, etc.).

[0140] Each of the one or more instances of the parallel processing unit 1102 may be coupled to a parallel processor memory 1122. The parallel processor memory 1122 may be accessed via a memory crossbar 1116, which may receive memory requests from the processing cluster array 1112 and the I / O unit 1104. The memory crossbar 1116 may access the parallel processor memory 1122 via a memory interface 1118. The memory interface 1118 may include a plurality of partition units (e.g., partition unit 1120A, partition unit 1120B, through partition unit 1120N), which may each be coupled to a portion (e.g., memory cells) of the parallel processor memory 1122. The number of partition units 1120A-1120N may be configured to be equal to the number of memory cells, such that a first partition unit 1120A has a corresponding first memory cell 1124A, a second partition unit 1120B has a corresponding second memory cell 1124B, and an Nth partition unit 1120N has a corresponding Nth memory cell 1124N. In other examples, the number of partition units 1120A-1120N may not be equal to the number of memory devices.

[0141] The memory units 1124A-1124N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Optionally, the memory units 1124A-1124N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). Those skilled in the art will appreciate that the specific implementation of the memory units 1124A-1124N may vary and may be selected from one of a variety of conventional designs. Rendering targets such as frame buffers or texture maps may be stored across the memory units 1124A-1124N, allowing the partition units 1120A-1120N to write portions of the respective rendering targets in parallel to efficiently use the available bandwidth of the parallel processor memory 1122. In some examples, the local instance of the parallel processor memory 1122 may be excluded to support a unified memory design that uses system memory in conjunction with a local cache memory.

[0142] Optionally, any of the clusters 1114A-1114N of the processing cluster array 1112 has the ability to process data to be written to any of the memory units 1124A-1124N within the parallel processor memory 1122. The memory crossbar 1116 can be configured to transmit the output of each cluster 1114A-1114N to any partition unit 1120A-1120N or to another cluster 1114A-1114N, which can perform additional processing operations on the output. Each cluster 1114A-1114N can communicate with the memory interface 1118 through the memory crossbar 1116 to read from or write to various external memory devices. In one of the examples having a memory crossbar 1116, the memory crossbar 1116 has connections to a memory interface 1118 in communication with the I / O unit 1104 and to a local instance of a parallel processor memory 1122, thereby enabling processing units within different processing clusters 1114A-1114N to communicate with system memory or other memory that is not local to the parallel processing unit 1102. In general, the memory crossbar 1116 may be able to separate traffic flows between the clusters 1114A-1114N and the partition units 1120A-1120N, for example, using virtual channels.

[0143] Although a single instance of PPU 1102 is illustrated within PPU 1100, any number of instances of PPU 1102 may be included. For example, multiple instances of PPU 1102 may be provided on a single add-in card, or multiple add-in cards may be interconnected. For example, PPU 1102 may be a processor such as a processor 1100. Fig.10 The additional device 1020 of the additional device 1020 can be a graphics card such as an independent graphics card, which includes one or more GPUs, one or more memory devices, and a device-to-device or network or structure interface. Different instances of the parallel processing unit 1102 can be configured to interoperate, even if different instances have different numbers of processing cores, different amounts of local parallel processor memory and / or other configuration differences. Optionally, some instances of the parallel processing unit 1102 may include a floating point unit with higher precision relative to other instances. Systems incorporating one or more instances of the parallel processing unit 1102 or parallel processor 1100 can be implemented in various configurations and form factors, including but not limited to desktop, laptop or handheld personal computers, servers, workstations, game consoles and / or embedded systems. The coordinator can use one or more of the following to form a composite node for workload execution: decomposed processor resources, cache resources, memory resources, storage resources and networking resources.

[0144] In some examples, the parallel processing unit 1102 may be divided into multiple instances. The multiple instances may be configured to execute workloads associated with different clients in an isolated manner, thereby enabling a predetermined quality of service to be provided to each client. For example, each cluster 1114A-1114N may be divided and isolated from other clusters, thereby allowing the processing cluster array 1112 to be divided into multiple computing partitions or instances. In this configuration, the workloads executed on the isolated partitions are protected from the impact of failures or errors associated with different workloads executed on different partitions. The partition units 1120A-1120N may be configured to implement dedicated and / or isolated paths to memory for the clusters 1114A-1114N associated with the corresponding computing partitions. This data path isolation enables the computing resources within the partition to communicate with one or more allocated memory units 1124A-1124N without interference from the activities of other partitions.

[0145] Fig. 11B is a block diagram of the partition unit 1120. The partition unit 1120 may be Fig.11A1120N. As shown, partition unit 1120 includes L2 cache 1121, frame buffer interface 1125, and ROP 1126 (raster operation unit). L2 cache 1121 is a read / write cache that is configured to perform load and store operations received from memory crossbar switch 1116 and ROP 1126. Read misses and urgent write-back requests are output by L2 cache 1121 to frame buffer interface 1125 for processing. Updates can also be sent to the frame buffer via frame buffer interface 1125 for processing. In some examples, frame buffer interface 1125 is connected to a memory unit (such as a parallel processor memory) in parallel processor memory. Fig.11A Partition unit 1120 may additionally or alternatively interface with a memory unit in the parallel processor memory via a memory controller (not shown).

[0146] In graphics applications, ROP 1126 is a processing unit that performs raster operations such as stencils, z-tests, blending, etc. ROP 1126 then outputs processed graphics data stored in graphics memory. In some examples, ROP 1126 includes or is coupled to CODEC 1127, which includes compression logic that compresses depth or color data written to memory or L2 cache 1121 and decompresses depth or color data read from memory or L2 cache 1121. The compression logic can be lossless compression logic that utilizes one or more of a variety of compression algorithms. The type of compression performed by CODEC 1127 may vary based on the statistical characteristics of the data to be compressed. For example, in some examples, incremental color compression is performed on depth and color data on a per-tile basis. In some examples, CODEC 1127 includes compression and decompression logic that can compress and decompress computational data associated with machine learning operations. CODEC 1127 can, for example, compress sparse matrix data for sparse machine learning operations. CODEC 1127 may also compress sparse matrix data encoded in a sparse matrix format (e.g., coordinate list encoding (COO), compressed sparse row (CSR), compressed sparse column (CSC), etc.) to generate compressed and encoded sparse matrix data. The compressed and encoded sparse matrix data may be decompressed and / or decoded before being processed by the processing element, or the processing element may be configured to consume compressed, encoded, or compressed and encoded data for processing.

[0147] ROP 1126 may be included in each processing cluster (e.g., Fig.11AThe processed graphics data may be displayed on a display device (e.g., a display device such as a display unit 1120). Fig.10 1010B), is routed for further processing by (one or more) processors 1002, or is routed for processing by Fig.11A The processing is further processed by one of the processing entities within parallel processor 1100.

[0148] Fig. 11C is a block diagram of a processing cluster 1114 within a parallel processing unit. For example, a processing cluster is Fig.11A An instance of one of the processing clusters 1114A-1114N. Processing cluster 1114 can be configured to execute many threads in parallel, wherein the term "thread" refers to an instance of a specific program executed on a specific input data set. Optionally, single instruction multiple data (SIMD) instruction issuance technology can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Alternatively, single instruction multiple thread (SIMT) technology can be used to support the parallel execution of a large number of generally synchronized threads, using a common instruction unit configured to issue instructions to a group of processing engines within each processing cluster. Unlike the SIMD execution mode (where all processing engines typically execute the same instructions), SIMT execution allows different threads to more easily follow the divergent execution path through a given thread program. Those skilled in the art will understand that the SIMD processing mode represents a functional subset of the SIMT processing mode.

[0149] The operation of the processing cluster 1114 can be controlled via a pipeline manager 1132 that distributes processing tasks to SIMT parallel processors. The pipeline manager 1132 Fig.11A The scheduler 1110 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1134 and / or the texture unit 1136. The illustrated graphics multiprocessor 1134 is an exemplary instance of a SIMT parallel processor. However, various types of SIMT parallel processors of different architectures may be included in the processing cluster 1114. One or more instances of the graphics multiprocessor 1134 may be included in the processing cluster 1114. The graphics multiprocessor 1134 may process data, and the data crossbar 1140 may be used to distribute the processed data to one of a plurality of possible destinations, including other shader units. The pipeline manager 1132 may facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data crossbar 1140.

[0150] Each graphics multiprocessor 1134 within a processing cluster 1114 may include a set of identical function execution logic (e.g., arithmetic logic units, load-store units, etc.). The function execution logic may be configured in a pipelined manner, where a new instruction may be issued before a previous instruction is completed. The function execution logic supports a variety of operations, including integer and floating point arithmetic, comparison operations, Boolean operations, bit shifts, and calculation of various algebraic functions. The same functional unit hardware may be utilized to perform different operations, and any combination of functional units may be present.

[0151] The instructions transmitted to the processing cluster 1114 constitute threads. A group of threads executed across the group of parallel processing engines is a thread group. The thread group executes the same program on different input data. Individual threads within a thread group can be assigned to different processing engines within the graphics multiprocessor 1134. A thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 1134. When a thread group includes fewer threads than the number of processing engines, one or more of the processing engines may be idle during the cycle of processing the thread group. A thread group may also include more threads than the number of processing engines within the graphics multiprocessor 1134. When a thread group includes more threads than the number of processing engines within the graphics multiprocessor 1134, processing may be performed in consecutive clock cycles. Optionally, multiple thread groups may be executed concurrently on the graphics multiprocessor 1134.

[0152] Graphics multiprocessor 1134 may include an internal cache memory that performs load and store operations. Alternatively, graphics multiprocessor 1134 may abandon the internal cache and use a cache memory (e.g., level 1 (L1) cache 1148) within processing cluster 1114. Each graphics multiprocessor 1134 also has access to a partition unit (e.g., L1 cache 1148) that is shared among all processing clusters 1114 and can be used to transfer data between threads. Fig.11A 1120A-1120N). Graphics multiprocessor 1134 may also access off-chip global memory, which may include one or more of local PPU memory and / or system memory. Any memory external to PPU 1102 may be used as global memory. Embodiments where processing cluster 1114 includes multiple instances of graphics multiprocessor 1134 may share common instructions and data that may be stored in L1 cache 1148.

[0153] Each processing cluster 1114 may include an MMU 1145 (memory management unit) configured to map virtual addresses to physical addresses. In other examples, one or more instances of MMU 1145 may reside in Fig.11A1148. The MMU 1145 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles and optionally cache line indexes. The MMU 1145 may include an address translation lookaside buffer (TLB) or cache that may reside within the graphics multiprocessor 1134 or L1 cache 1148 of the processing cluster 1114. Physical addresses are processed to assign surface data access locality, allowing efficient request interleaving between partition units. The cache line index may be used to determine whether a request for a cache line is a hit or a miss.

[0154] In graphics and compute applications, the processing clusters 1114 may be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data is read from an internal texture L1 cache (not shown) or in some examples from an L1 cache within the graphics multiprocessor 1134, and retrieved from an L2 cache, local parallel processor memory, or system memory when needed. Each graphics multiprocessor 1134 outputs a processed task to a data crossbar 1140 to provide the processed task to another processing cluster 1114 for further processing, or stores the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar 1116. A preROP 1142 (pre-raster operations unit) is configured to receive data from the graphics multiprocessor 1134, direct the data to a ROP unit, which may communicate with a partition unit (e.g., Fig.11A The preROP 1142 unit may perform color blending optimizations, organize pixel color data, and perform address translation.

[0155] It should be understood that the core architecture described herein is illustrative and that variations and modifications are possible. Any number of processing units may be included within a processing cluster 1114, such as a graphics multiprocessor 1134, a texture unit 1136, a preROP 1142, etc. Further, while only one processing cluster 1114 is shown, a parallel processing unit as described herein may include any number of instances of a processing cluster 1114. Optionally, each processing cluster 1114 may be configured to operate independently of other processing clusters 1114 using separate and distinct processing units, L1 caches, L2 caches, etc.

[0156] Fig.11DAn example of a graphics multiprocessor 1134 is shown, where the graphics multiprocessor 1134 is coupled to a pipeline manager 1132 of a processing cluster 1114. The graphics multiprocessor 1134 has an execution pipeline that includes, but is not limited to, an instruction cache 1152, an instruction unit 1154, an address mapping unit 1156, a register file 1158, one or more general purpose graphics processing unit (GPGPU) cores 1162, and one or more load / store units 1166. The GPGPU cores 1162 and the load / store units 1166 are coupled to a cache memory 1172 and a shared memory 1170 via a memory and cache interconnect 1168. The graphics multiprocessor 1134 may additionally include a tensor and / or ray tracing core 1163 that includes hardware logic to accelerate matrix and / or ray tracing operations.

[0157] The instruction cache 1152 may receive an instruction stream to be executed from the pipeline manager 1132. The instructions are cached in the instruction cache 1152 and dispatched for execution by the instruction unit 1154. The instruction unit 1154 may dispatch the instructions as thread groups (e.g., warps), where the threads of the thread groups are assigned to different execution units within the GPGPU core 1162. The instructions may access any of the local, shared, or global address spaces by specifying an address within the unified address space. The address mapping unit 1156 may be used to translate addresses in the unified address space into different memory addresses accessible by the load / store unit 1166.

[0158] The register file 1158 provides a set of registers for the functional units of the graphics multiprocessor 1134. The register file 1158 provides temporary storage for operands of the data paths connected to the functional units of the graphics multiprocessor 1134 (e.g., GPGPU core 1162, load / store unit 1166). The register file 1158 may be divided between the various functional units so that each functional unit is allocated a dedicated portion of the register file 1158. For example, the register file 1158 may be divided between different warps executed by the graphics multiprocessor 1134.

[0159] The GPGPU cores 1162 may each include a floating point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1134. In some implementations, the GPGPU core 1162 may include hardware logic that may otherwise reside within the tensor and / or ray tracing core 1163. The GPGPU cores 1162 may be architecturally similar or architecturally different. For example, in some examples, the first portion of the GPGPU core 1162 includes a single-precision FPU and an integer ALU, while the second portion of the GPGPU core includes a double-precision FPU. Optionally, the FPU may implement the IEEE 754-2008 standard for floating-point operations, or enable variable-precision floating-point operations. The graphics multiprocessor 1134 may additionally include one or more fixed-function or special-function units that perform specific functions such as copying rectangles or pixel blending operations. One or more GPGPU cores may also include fixed or special-function logic.

[0160] GPGPU core 1162 may include SIMD logic capable of executing a single instruction to multiple data sets. Optionally, GPGPU core 1162 may physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. SIMD instructions for GPGPU cores may be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a single program multiple data (SPMD) or SIMT architecture. Multiple threads of a program configured for a SIMT execution model may be executed via a single SIMD instruction. For example, in some examples, eight SIMT threads performing the same or similar operation may be executed in parallel via a single SIMD8 logic unit.

[0161] The memory and cache interconnect 1168 is an interconnect network that connects the various functional units of the graphics multiprocessor 1134 to the register file 1158 and the shared memory 1170. For example, the memory and cache interconnect 1168 is a crossbar interconnect that allows the load / store unit 1166 to perform load and store operations between the shared memory 1170 and the register file 1158. The register file 1158 can operate at the same frequency as the GPGPU core 1162, so that data transfers between the GPGPU core 1162 and the register file 1158 are very low latency. The shared memory 1170 can be used to implement communication between threads executing on the functional units within the graphics multiprocessor 1134. The cache memory 1172 can be used, for example, as a data cache that caches texture data transferred between the functional units and the texture unit 1136. The shared memory 1170 can also be used as a program management cache. Shared memory 1170 and cache memory 1172 may be coupled to data crossbar switch 1140 to enable communication with other components of the processing cluster. In addition to automatically cached data stored in cache memory 1172, threads executing on GPGPU core 1162 may also programmatically store data in shared memory.

[0162] FIG. 12A to FIG. 12C Additional graphics multiprocessors according to examples are illustrated. FIG. 12A to FIG. 12B Graphics multiprocessors 1225, 1250 are illustrated, which are Fig. 11C The graphics multiprocessor 1134 of FIG. 1134 is related to and can be used to replace one of them. Therefore, the disclosure of any feature in this article in combination with the graphics multiprocessor 1134 also discloses the corresponding combination with (one or more) graphics multiprocessors 1225, 1250, but is not limited to this. Fig. 12C A graphics processing unit (GPU) 1280 is illustrated, which includes multiple groups of dedicated graphics processing resources arranged into multi-core groups 1265A-1265N, which correspond to graphics multiprocessors 1225, 1250. The illustrated graphics multiprocessors 1225, 1250 and multi-core groups 1265A-1265N may be streaming multiprocessors (SMs) capable of executing a large number of execution threads simultaneously.

[0163] Fig. 12A The graphics multiprocessor 1225 includes relative Fig.11DThe graphics multiprocessor 1225 may include multiple additional instances of execution resource units of the graphics multiprocessor 1134. For example, the graphics multiprocessor 1225 may include multiple instances of instruction units 1232A-1232B, register files 1234A-1234B, and (one or more) texture units 1244A-1244B. The graphics multiprocessor 1225 also includes multiple groups of graphics or compute execution units (e.g., GPGPU cores 1236A-1236B, tensor cores 1237A-1237B, ray tracing cores 1238A-1238B) and multiple groups of load / store units 1240A-1240B. The execution resource units have a common instruction cache 1230, texture and / or data cache memory 1242, and shared memory 1246.

[0164] The various components may communicate via the interconnect structure 1227. The interconnect structure 1227 may include one or more crossbar switches that enable communication between the various components of the graphics multiprocessor 1225. The interconnect structure 1227 may be a separate high-speed network fabric layer on which the various components of the graphics multiprocessor 1225 are stacked. The components of the graphics multiprocessor 1225 communicate with remote components via the interconnect structure 1227. For example, the cores 1236A-1236B, 1237A-1237B, and 1238A-1238B may each communicate with the shared memory 1246 via the interconnect structure 1227. The interconnect structure 1227 may arbitrate communications within the graphics multiprocessor 1225 to ensure fair bandwidth allocation between components.

[0165] Fig. 12B The graphics multiprocessor 1250 includes multiple groups of execution resources 1256A-1256D, wherein each group of execution resources includes multiple instruction units, register files, GPGPU cores, and load storage units, such as Fig.11D and Fig. 12A 1256A-1256D may coordinate with texture unit(s) 1260A-1260D to perform texture operations while sharing instruction cache 1254 and shared memory 1253. For example, execution resources 1256A-1256D may share instruction cache 1254 and shared memory 1253, as well as multiple instances of texture and / or data caches 1258A-1258B. The various components may be communicated via a communication protocol similar to Fig. 12A The interconnect structure 1227 communicates with the interconnect structure 1252.

[0166] Those skilled in the art should understand that Figure 1 , FIG. 11A to FIG. 11D and FIG. 12A to FIG. 12BThe architecture described in is illustrative and not limiting of the scope of the examples. Thus, without departing from the scope of the examples described herein, the techniques described herein may be implemented on any appropriately configured processing unit, including but not limited to one or more mobile application processors, one or more desktop or server central processing units (CPUs) including multi-core CPUs, such as Fig.11A One or more parallel processing units of parallel processing unit 1102, and one or more graphics processors or special processing units.

[0167] A parallel processor or GPGPU as described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., a high-speed interconnect such as PCIe, NVLink, or other known protocols, standardized protocols, or proprietary protocols). In other examples, the GPU can be integrated on the same package or chip as the core and communicatively coupled to the core via an internal processor bus / interconnect (i.e., inside the package or chip). Regardless of the manner in which the GPU is connected, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a work descriptor. The GPU then uses dedicated circuits / logic to efficiently process these commands / instructions.

[0168] Fig. 12C A graphics processing unit (GPU) 1280 is illustrated, which includes dedicated multiple groups of graphics processing resources arranged into multi-core groups 1265A-1265N. Although only details of a single multi-core group 1265A are provided, it should be understood that other multi-core groups 1265B-1265N can be equipped with the same or similar groups of graphics processing resources. The details described about multi-core groups 1265A-1265N can also be applied to any graphics multiprocessor 1134, 1225, 1250 described herein.

[0169] As shown, multi-core group 1265A may include a group of graphics cores 1270, a group of tensor cores 1271, and a group of ray tracing cores 1272. Scheduler / dispatcher 1268 schedules and dispatches graphics threads for execution on the respective cores 1270, 1271, 1272. A group of register files 1269 stores operand values ​​used by cores 1270, 1271, 1272 when executing graphics threads. These register files may include, for example, integer registers for storing integer values, floating point registers for storing floating point values, vector registers for storing packed data elements (integer and / or floating point data elements), and tile registers for storing tensor / matrix values. Tile registers may be implemented as a combination of multiple groups of vector registers.

[0170] One or more combined level 1 (L1) caches and shared memory units 1273 store graphics data locally within each multi-core group 1265A, such as texture data, vertex data, pixel data, ray data, bounding box data, etc. One or more texture units 1274 can also be used to perform texture operations, such as texture mapping and sampling. A level 2 (L2) cache 1275 shared by all or a subset of multi-core groups 1265A-1265N stores graphics data and / or instructions for multiple concurrent graphics threads. As shown in the figure, the L2 cache 1275 can be shared across multiple multi-core groups 1265A-1265N. One or more memory controllers 1267 couple the GPU 1280 to a memory 1266, which can be a system memory (e.g., DRAM) and / or a dedicated graphics memory (e.g., GDDR6 memory).

[0171] Input / output (I / O) circuitry 1263 couples GPU 1280 to one or more I / O devices 1262, such as a digital signal processor (DSP), a network controller, or a user input device. On-chip interconnects may be used to couple I / O devices 1262 to GPU 1280 and memory 1266. One or more I / O memory management units (IOMMUs) 1264 of I / O circuitry 1263 couple I / O devices 1262 directly to system memory 1266. Optionally, IOMMU 1264 manages multiple sets of page tables to map virtual addresses to physical addresses in system memory 1266. I / O devices 1262, CPU(s) 1261, and GPU(s) 1280 may then share the same virtual address space.

[0172] In one implementation of the IOMMU 1264, the IOMMU 1264 supports virtualization. In this case, it may manage a first set of page tables to map guest / graphics virtual addresses to guest / graphics physical addresses, and may manage a second set of page tables to map guest / graphics physical addresses to system / host physical addresses (e.g., within system memory 1266). The base address of each of the first set of page tables and the second set of page tables may be stored in a control register and swapped out at context switch (e.g., so that the new context is provided with access to the relevant set of page tables). Although in Fig. 12C Not illustrated, but each of cores 1270, 1271, 1272 and / or multi-core groups 1265A-1265N may include a translation lookaside buffer (TLB) to cache guest virtual to guest physical translations, guest physical to host physical translations, and guest virtual to host physical translations.

[0173] (One or more) CPU 1261, GPU 1280 and I / O device 1262 can be integrated on a single semiconductor chip and / or chip package. The illustrated memory 1266 can be integrated on the same chip or can be coupled to a memory controller 1267 via an off-chip interface. In one implementation, memory 1266 includes GDDR6 memory that shares the same virtual address space with other physical system-level memories, but the basic principles described herein are not limited to this particular implementation.

[0174] Tensor core 1271 may include multiple execution units specifically designed to perform matrix operations, which are basic computational operations used to perform deep learning operations. For example, simultaneous matrix multiplication operations can be used for neural network training and reasoning. Tensor core 1271 can perform matrix processing using a variety of operand precisions, including single-precision floating point (e.g., 32 bits), half-precision floating point (e.g., 16 bits), integer words (16 bits), bytes (8 bits), and half bytes (4 bits). For example, a neural network implementation extracts features of various rendered scenes that potentially combine details from multiple frames to build a high-quality final image.

[0175] In a deep learning implementation, parallel matrix multiplication work can be scheduled for execution on the tensor core 1271. In particular, the training of neural networks requires a large number of matrix dot product operations. To process the inner product formula of N×N×N matrix multiplication, the tensor core 1271 may include at least N dot product processing elements. Before the matrix multiplication begins, a complete matrix is ​​loaded into the tile register, and at least one column of the second matrix is ​​loaded in each cycle of N cycles. In each cycle, N dot products are processed.

[0176] Matrix elements may be stored with different precisions, including 16-bit words, 8-bit bytes (e.g., INT8), and 4-bit nibbles (e.g., INT4), depending on the specific implementation. Different precision modes may be specified for the tensor core 1271 to ensure the most efficient precision for different workloads (e.g., such as inference workloads that can tolerate quantization to bytes and nibbles). Supported formats additionally include 64-bit floating point (FP64) and non-IEEE floating point formats, such as the bfloat16 format (e.g., brain floating point): a 16-bit floating point format with one sign bit, eight exponent bits, and eight valid bits, of which seven valid bits are explicitly stored. An example includes support for reduced precision tensor floating (TF32) mode, which uses the range of FP32 (8 bits) and the precision of FP16 (10 bits) to perform calculations. Reduced precision TF32 operations may be performed on FP32 inputs and produce FP32 outputs with higher performance relative to FP32 and increased precision relative to FP16. In some examples, one or more 8-bit floating point formats (FP8) are supported.

[0177] In some examples, the tensor core 1271 supports a sparse operation mode for matrices in which the vast majority of values ​​are zero. The tensor core 1271 includes support for sparse input matrices encoded in sparse matrix representations (e.g., coordinate list encoding (COO), compressed sparse rows (CSR), compressed sparse columns (CSC), etc.). The tensor core 1271 also includes support for compressed sparse matrix representations in the case where the sparse matrix representation can be further compressed. Compressed, encoded, and / or compressed and encoded matrix data together with associated compression and / or encoding metadata can be read by the tensor core 1271, and non-zero values ​​can be extracted. For example, for a given input matrix A, non-zero values ​​can be loaded from a compressed and / or encoded representation of at least a portion of matrix A. Based on the position of the non-zero value in matrix A (which can be determined from the index or coordinate metadata associated with the non-zero value), the corresponding value in the input matrix B can be loaded. Depending on the operation to be performed (e.g., multiplication), if the corresponding value is a zero value, the loading of the value from the input matrix B can be bypassed. In some examples, the pairing of values ​​for certain operations, such as multiplication operations, can be pre-scanned by the scheduler logic and only operations between non-zero inputs are scheduled. Depending on the dimensions of matrix A and matrix B and the operation to be performed, the output matrix C can be dense or sparse. In the case where the output matrix C is sparse, depending on the configuration of the tensor core 1271, the output matrix C can be output in a compressed format, sparse coding, or compressed sparse coding.

[0178] The ray tracing core 1272 can accelerate ray tracing operations for real-time ray tracing and non-real-time ray tracing implementations. In particular, the ray tracing core 1272 may include a ray traversal / intersection circuit for performing ray traversal using a bounding box hierarchy (BVH) and identifying intersections between rays and primitives enclosed within the BVH box. The ray tracing core 1272 may also include circuits for performing depth testing and culling (e.g., using a Z buffer or similar arrangement). In one implementation, the ray tracing core 1272 performs traversal and intersection operations in coordination with the image denoising techniques described herein, and at least a portion of the traversal and intersection operations may be performed on the tensor core 1271. For example, the tensor core 1271 may implement a deep learning neural network to perform denoising on frames generated by the ray tracing core 1272. However, (one or more) CPUs 1261, graphics core 1270, and / or ray tracing core 1272 may also implement all or part of a denoising and / or deep learning algorithm.

[0179] Additionally, as described above, a distributed denoising approach may be employed, wherein GPU 1280 is in a computing device coupled to other computing devices via a network or high-speed interconnect. In this distributed approach, the interconnected computing devices may share neural network learning / training data to increase the speed at which the entire system learns to perform denoising for different types of image frames and / or different graphics applications.

[0180] The ray tracing core 1272 can handle all BVH traversals and / or ray-primitive intersections, thereby avoiding overloading the graphics core 1270 with thousands of instructions per ray. For example, each ray tracing core 1272 includes a first set of dedicated circuits for performing bounding box tests (e.g., for traversal operations) and / or a second set of dedicated circuits for performing ray-triangle intersection tests (e.g., crossing rays that have been traversed). Thus, for example, the multi-core group 1265A can simply start the ray probe, and the ray tracing core 1272 independently performs ray traversals and intersections and returns hit data (e.g., hits, misses, multiple hits, etc.) to the thread context. While the ray tracing core 1272 performs traversals and intersection operations, the other cores 1270, 1271 are freed up to perform other graphics or computational work.

[0181] Optionally, each ray tracing core 1272 may include a traversal unit that performs BVH test operations and / or a cross unit that performs ray-primitive intersection tests. The cross unit generates a "hit", "miss", or "multiple hits" response, providing the response to the appropriate thread. During traversal and cross operations, execution resources of other cores (e.g., graphics core 1270 and tensor core 1271) are freed to perform other forms of graphics work.

[0182] In some of the examples described below, a hybrid rasterization / ray tracing approach is used in which work is distributed between graphics core 1270 and ray tracing core 1272.

[0183] The ray tracing core 1272 (and / or other cores 1270, 1271) may include hardware support for a ray tracing instruction set such as Microsoft's DirectX Ray Tracing (DXR), which includes the DispatchRays command and ray generation, closest hit, any hit, and miss shaders, which enable unique sets of shaders and textures to be assigned to each object. Another ray tracing platform that can be supported by the ray tracing core 1272, graphics core 1270, and tensor core 1271 is the Vulkan API (e.g., Vulkan version 1.1.85 and later). However, it should be noted that the basic principles described herein are not limited to any particular ray tracing ISA.

[0184] In general, each core 1272, 1271, 1270 may support a ray tracing instruction set that includes instructions / functions for one or more of ray generation, closest hit, any hit, ray-primitive intersection, per-primitive and hierarchy bounding box construction, misses, accesses, and exceptions. More specifically, some examples include ray tracing instructions that perform one or more of the following functions:

[0185] -Ray Generation -Ray generation instructions can be executed for each pixel, sample, or other user-defined work distribution.

[0186] - Closest Hit - A closest hit instruction may be executed to locate the closest intersection of a ray with a primitive within a scene.

[0187] -Any Hit - The Any Hit instruction identifies multiple intersections between rays and primitives within the scene to potentially identify a new closest intersection point.

[0188] -Cross-The -Cross instruction performs a ray-primitive intersection test and outputs the result.

[0189] - Per-primitive bounding box construction - This instruction builds a bounding box around a given primitive or group of primitives (eg, when building a new BVH or other acceleration data structure).

[0190] -Miss - Indicates that the ray missed all geometry within the scene or a specified area of ​​the scene.

[0191] -Visit - Indicates the subbox that the ray will pass through.

[0192] -Exceptions - includes various types of exception handlers (e.g., called for various error conditions).

[0193] In some examples, the ray tracing core 1272 can be adapted to accelerate general computing operations that can be accelerated using computing techniques similar to ray crossing tests. A computing framework can be provided that enables shader programs to be compiled into low-level instructions and / or primitives that perform general computing operations via the ray tracing core. Exemplary computing problems that can benefit from computing operations performed on the ray tracing core 1272 include calculations involving beam, wave, light, or particle propagation in a coordinate space. Interactions associated with the propagation can be calculated relative to a geometry or grid in the coordinate space. For example, calculations associated with the propagation of electromagnetic signals through an environment can be accelerated using instructions or primitives executed via a ray tracing core. The diffraction and reflection of the signal by objects in the environment can be calculated as a direct ray tracing analogy.

[0194] The ray tracing core 1272 can also be used to perform calculations that are not directly similar to ray tracing. For example, the ray tracing core 1272 can be used to accelerate mesh projection, mesh refinement, and volume sampling calculations. General coordinate space calculations, such as nearest neighbor calculations, can also be performed. For example, a set of points near a given point can be found by defining a bounding box in the coordinate space around the point. Then, the BVH and ray detection logic within the ray tracing core 1272 can be used to determine the set of intersections of points within the bounding box. The intersection constitutes the origin and the nearest neighbor of the origin. The calculations performed using the ray tracing core 1272 can be performed in parallel with the calculations performed on the graphics core 1272 and the tensor core 1271. The shader compiler can be configured to compile a compute shader or other general graphics processing program into a low-level primitive that can be parallelized across the graphics core 1270, the tensor core 1271, and the ray tracing core 1272.

[0195] Building larger and larger silicon dies is challenging for a variety of reasons. As silicon dies get larger, manufacturing yields get smaller, and the process technology requirements for different components may be different. On the other hand, in order to have a high-performance system, key components should be interconnected through high-speed, high-bandwidth, low-latency interfaces. These conflicting requirements pose challenges to the development of high-performance chips.

[0196] The embodiments described herein provide techniques that decompose the architecture of a system-on-chip integrated circuit into multiple different chiplets that can be packaged onto a common chassis. In some examples, a graphics processing unit or a parallel processor consists of different silicon chiplets that are manufactured separately. A chiplet is an integrated circuit that is at least partially packaged and includes different logic units that can be assembled with other chiplets into a larger package. Different groups of chiplets with different IP core logic can be assembled into a single device. In addition, chiplets can be integrated into a base die or base chiplet using active inserter technology. The concepts described herein enable interconnection and communication between different forms of IP within a GPU. Developing IP on different processes can be mixed. This avoids the complexity of converging multiple IPs, especially multiple IPs on a large SoC with several flavors of IP, to the same process.

[0197] Enabling the use of multiple process technologies improves time to market and provides a cost-effective way to create multiple product SKUs. For customers, this means getting products that are more tailored to their needs in a cost-effective and timely manner. In addition, decomposed IP is more easily power-gated independently, and components that are not used on a given workload can be powered off, reducing overall power consumption.

[0198] Fig.13A parallel computing system 1300 according to some examples is illustrated. In some examples, the parallel computing system 1300 includes a parallel processor 1320, which may be a graphics processor or a computing accelerator as described herein. The parallel processor 1320 includes a global logic unit 1301, an interface 1302, a thread dispatcher 1303, a media unit 1304, a group of computing units 1305A-1305H, and a cache / memory unit 1306. In some examples, the global logic unit 1301 includes global functions for the parallel processor 1320, including device configuration registers, a global scheduler, power management logic, etc. The interface 1302 may include a front-end interface for the parallel processor 1320. The thread dispatcher 1303 may receive a workload from the interface 1302 and dispatch threads for the workload to the computing units 1305A-1305H. If the workload includes any media operations, at least a portion of these operations may be performed by the media unit 1304. The media unit may also offload some operations to the compute units 1305A-1305H. The cache / memory unit 1306 may include cache memory (eg, L3 cache) and local memory (eg, HBM, GDDR) for the parallel processors 1320.

[0199] FIG. 14A to FIG. 14B A hybrid logical / physical view of a factorized parallel processor according to examples described herein is illustrated. Fig.14A A decomposed parallel computing system 1400 is illustrated. Fig. 14B A chiplet 1430 of the decomposed parallel computing system 1400 is illustrated.

[0200] like Fig.14A As shown, the decomposed computing system 1400 may include a parallel processor 1420, wherein the various components of the parallel processor SoC are distributed across multiple chiplets. Each chiplet may be a different IP core that is independently designed and configured to communicate with other chiplets via one or more common interfaces. The chiplets include, but are not limited to, a computing chiplet 1405, a media chiplet 1404, and a memory chiplet 1406. Each chiplet may be manufactured separately using different process technologies. For example, the computing chiplet 1405 may be manufactured using the smallest or most advanced process technology available at the time of manufacturing, while the memory chiplet 1406 or other chiplets (e.g., I / O, networking, etc.) may be manufactured using a larger or less advanced process technology.

[0201] Various chiplets may be bonded to the base die 1410 and configured to communicate with each other and logic within the base die 1410 via an interconnect layer 1412. In some examples, the base die 1410 may include global logic 1401, which may include a scheduler 1411 and power management 1421 logic unit, an interface 1402, a dispatch unit 1403, and an interconnect fabric module 1408 coupled or integrated with one or more L3 cache banks 1409A-1409N. The interconnect fabric 1408 may be an inter-chiplet fabric integrated into the base die 1410. The fabric 1408 may be used by the logic chiplets to relay messages between the various chiplets. Additionally, the L3 cache banks 1409A-1409N in the base die and / or the L3 cache banks within the memory chiplet 1406 may cache data read from and sent to the DRAM chiplets within the memory chiplet 1406 as well as data read from and sent to the system memory of the host.

[0202] In some examples, global logic 1401 is a microcontroller that can execute firmware to perform scheduler 1411 and power management 1421 functions for parallel processor 1420. The microcontroller executing the global logic can be customized for the target use case of parallel processor 1420. Scheduler 1411 can perform global scheduling operations for parallel processor 1420. Power management 1421 functions can be used to enable or disable individual chiplets within a parallel processor when they are not being used.

[0203] The individual chiplets of the parallel processor 1420 may be designed to perform a specific function that would be integrated into a single die in existing designs. A set of compute chiplets 1405 may include a cluster of compute units (e.g., execution units, streaming multiprocessors, etc.) that include programmable logic to execute compute or graphics shader instructions. Media chiplets 1404 may include hardware logic to accelerate media encoding and decoding operations. Memory chiplets 1406 may include volatile memory (e.g., DRAM) and one or more SRAM cache banks (e.g., L3 banks).

[0204] like Fig. 14BAs shown, each chiplet 1430 may include common components and dedicated components. The chiplet logic 1436 within the chiplet 1430 may include specific components of the chiplet, such as an array of streaming multiprocessors, computational units, or execution units described herein. The chiplet logic 1436 may be coupled to an optional cache or shared local memory 1438, or a cache or shared local memory may be included in the chiplet logic 1436. The chiplet 1430 may include a fabric interconnect node 1442 that receives commands via an inter-chiplet fabric. Commands and data received via the fabric interconnect node 1442 may be temporarily stored in an interconnect buffer 1439. Data sent to and received from the fabric interconnect node 1442 may be stored in the interconnect cache 1440. Power control 1432 and clock control 1434 logic may also be included within the chiplet. The power control 1432 and clock control 1434 logic may receive configuration commands via a fabric that may configure dynamic voltage and frequency scaling of the chiplet 1430. In some examples, individual chiplets can have independent clock and power domains and can be clock-gated and power-gated independently of other chiplets.

[0205] At least a portion of the components within the illustrated chiplet 1430 may also be included in an embedded Fig.14A The base die 1410 may be within logic within the base die 1410. For example, logic within the base die that communicates with the fabric may include a version of the fabric interconnect node 1442. Base die logic that may be independently clocked or power gated may include a version of the power control 1432 and / or clock control 1434 logic.

[0206] Thus, while various examples described herein use the term SoC to describe a device or system having a processor and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, memory circuitry, etc.) integrally integrated into a single integrated circuit (“IC”) die or chip, the present disclosure is not limited in this regard. For example, in various examples of the present disclosure, a device or system may have one or more processors (e.g., one or more processor cores) and associated circuitry (e.g., input / output (“I / O”) circuitry, power delivery circuitry, etc.) arranged in a disaggregated collection of discrete dies, tiles, and / or chiplets (e.g., one or more discrete processor core dies arranged adjacent to one or more other dies such as memory dies, I / O dies, etc.). In such disaggregated devices and systems, the various dies, tiles, and / or chiplets may be physically and electrically coupled together by a packaging structure including, for example, various packaging substrates, interposers, active interposers, photonic interposers, interconnect bridges, etc. The disaggregated collection of discrete dies, tiles, and / or chiplets may also be part of a system on a package (“SoP”).

[0207] Graphics Execution Unit

[0208] FIG. 15A to FIG. 15B Thread execution logic 1500 is illustrated including an array of processing elements employed in a graphics processor core according to examples described herein. FIG. 15A to FIG. 15B Elements in the drawings having the same reference numbers (or names) as elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto. Fig.15A represents an execution unit within a general-purpose graphics processor, and Fig. 15B Represents an execution unit that can be used within a compute accelerator.

[0209] like Fig.15A 1508N), a scalable execution unit array including a plurality of execution units 1508A-1508N, a sampler 1510, a shared local memory 1511, a data cache 1512, and a data port 1514. In some examples, the scalable execution unit array can be dynamically scaled by enabling or disabling one or more execution units (e.g., any one of execution units 1508A, 1508B, 1508C, 1508D to 1508N-1 and 1508N) based on the computational requirements of the workload. In some examples, the included components are interconnected via an interconnect structure connected to the various components. In some examples, thread execution logic 1500 includes one or more connections to a memory such as a system memory or a cache memory through instruction cache 1506, data port 1514, sampler 1510, and one or more of execution units 1508A-1508N. In some examples, each execution unit (e.g., 1508A) is an independent programmable general-purpose computing unit that is capable of executing multiple simultaneous hardware threads while processing multiple data elements in parallel for each thread. In various examples, the array of execution units 1508A-1508N is scalable to include any number of separate execution units.

[0210] In some examples, execution units 1508A to 1508N are primarily used to execute shader programs. Shader processor 1502 can process various shader programs and dispatch execution threads associated with the shader programs via thread dispatcher 1504. In some examples, the thread dispatcher includes logic that arbitrates thread initiation requests from graphics and media pipelines and instantiates the requested threads on one or more execution units in execution units 1508A-1508N. For example, a geometry pipeline can dispatch vertex, tessellation, or geometry shaders to thread execution logic for processing. In some examples, thread dispatcher 1504 can also handle runtime thread derivative requests from executing shader programs.

[0211] In some examples, execution units 1508A-1508N support an instruction set including native support for many standard 3D graphics shader instructions, so that shader programs from graphics libraries (e.g., Direct 3D and OpenGL) are executed with minimal translation. The execution units support vertex and geometry processing (e.g., vertex programs, geometry programs, vertex shaders), pixel processing (e.g., pixel shaders, fragment shaders), and general processing (e.g., compute and media shaders). Each of the execution units 1508A-1508N is capable of multi-issue single instruction multiple data (SIMD) execution, and multi-threaded operations achieve an efficient execution environment in the face of high latency memory access. Each hardware thread within each execution unit has a dedicated high-bandwidth register file and an associated independent thread state. Execution is multi-issue per clock to a pipeline capable of integer, single-precision and double-precision floating-point operations, SIMD branching capabilities, logical operations, transcendental operations, and other miscellaneous operations. While waiting for data from memory or one of the shared functions, dependency logic within execution units 1508A-1508N puts the waiting thread to sleep until the requested data has been returned. While the waiting thread is sleeping, hardware resources can be devoted to processing other threads. For example, during a delay associated with a vertex shader operation, an execution unit can perform operations of a pixel shader, a fragment shader, or another type of shader program (including different vertex shaders). Various examples can be applied to using execution by using single instruction multiple threads (SIMT), as an alternative to or in addition to using SIMD. References to SIMD cores or operations can also be applied to SIMT or to a combination of SIMD and SIMT.

[0212] Each execution unit in execution units 1508A-1508N operates on an array of data elements. The number of data elements is the "execution size" or number of lanes of the instruction. An execution lane is an execution logic unit used for data element access, masking, and flow control within an instruction. The number of lanes may be independent of the number of physical arithmetic logic units (ALUs) or floating point units (FPUs) of a particular graphics processor. In some examples, execution units 1508A-1508N support integer and floating point data types.

[0213] The execution unit instruction set includes SIMD instructions. Various data elements can be stored in registers as compact data types, and the execution unit will process various elements based on the data size of the element. For example, when operating a 256-bit wide vector, 256 bits of the vector are stored in registers, and the execution unit operates the vector as four separate 64-bit compact data elements (quadword (QW) size data elements), eight separate 32-bit compact data elements (doubleword (DW) size data elements), sixteen separate 16-bit compact data elements (word (W) size data elements) or thirty-two separate 8-bit data elements (byte (B) size data elements). However, different vector widths and register sizes are possible.

[0214] In some examples, one or more execution units may be combined into fused execution units 1509A-1509N having thread control logic (1507A-1507N) common to the fused EUs. Multiple EUs may be fused into EU groups. Each EU in the fused EU group may be configured to execute a separate SIMD hardware thread. The number of EUs in the fused EU group may vary according to the example. In addition, each EU may execute various SIMD widths, including but not limited to SIMD8, SIMD16, and SIMD32. Each fused graphics execution unit 1509A-1509N includes at least two execution units. For example, the fused execution unit 1509A includes a first EU 1508A, a second EU 1508B, and a thread control logic 1507A common to the first EU 1508A and the second EU 1508B. The thread control logic 1507A controls the threads executed on the fused graphics execution unit 1509A, thereby allowing each EU within the fused execution unit 1509A-1509N to execute using a common instruction pointer register.

[0215] One or more internal instruction caches (e.g., 1506) are included in thread execution logic 1500 to cache thread instructions for execution units. In some examples, one or more data caches (e.g., 1512) are included to cache thread data during thread execution. Threads executing on execution logic 1500 may also store explicitly managed data in shared local memory 1511. In some examples, sampler 1510 is included to provide texture sampling for 3D operations and media sampling for media operations. In some examples, sampler 1510 includes dedicated texture or media sampling functions to process texture or media data during the sampling process before providing the sampled data to the execution unit.

[0216] During execution, the graphics and media pipeline sends thread initiation requests to the thread execution logic 1500 via thread derivation and dispatch logic. Once a set of geometric objects has been processed and rasterized into pixel data, the pixel processor logic (e.g., pixel shader logic, fragment shader logic, etc.) within the shader processor 1502 is called to further calculate output information and cause the results to be written to the output surface (e.g., color buffer, depth buffer, stencil buffer, etc.). In some examples, the pixel shader or fragment shader calculates the values ​​of various vertex attributes that will be interpolated across the rasterized object. In some examples, the pixel processor logic within the shader processor 1502 then executes the pixel or fragment shader program supplied by the application programming interface (API). To execute the shader program, the shader processor 1502 dispatches the thread to the execution unit (e.g., 1508A) via the thread dispatcher 1504. In some examples, the shader processor 1502 uses the texture sampling logic in the sampler 1510 to access the texture data in the texture map stored in the memory. Arithmetic operations on texture data and input geometry data compute pixel color data for each geometry fragment, or discard one or more pixels from further processing.

[0217] In some examples, data port 1514 provides a memory access mechanism for thread execution logic 1500 to output processed data to memory for further processing on a graphics processor output pipeline. In some examples, data port 1514 includes or is coupled to one or more cache memories (e.g., data cache 1512) to cache data for memory access via the data port.

[0218] In some examples, execution logic 1500 may also include a ray tracer 1505 that may provide ray tracing acceleration functionality. Ray tracer 1505 may support a ray tracing instruction set that includes instructions / functions for ray generation.

[0219] Fig. 15BExemplary internal details of an execution unit 1508 according to an example are illustrated. Graphics execution unit 1508 may include an instruction fetch unit 1537, a general register file array (GRF) 1524, an architectural register file array (ARF) 1526, a thread arbiter 1522, an issue unit 1530, a branch unit 1532, a set of SIMD floating point units (FPUs) 1534, and in some examples, a set of dedicated integer SIMD ALUs 1535. GRF 1524 and ARF 1526 include a set of general register files and architectural register files associated with various simultaneous hardware threads that may be active in graphics execution unit 1508. In some examples, a per-thread architectural state is maintained in ARF 1526, while data used during thread execution is stored in GRF 1524. The execution state of each thread, including the instruction pointer of each thread, may be maintained in thread-specific registers in ARF 1526.

[0220] In some examples, graphics execution unit 1508 has an architecture that is a combination of simultaneous multithreading (SMT) and fine-grained interleaved multithreading (IMT). The architecture has a modular configuration that can be fine-tuned at design time based on the target number of simultaneous threads and the number of registers per execution unit, where execution unit resources are divided across logic for executing multiple simultaneous threads. The number of logical threads that graphics execution unit 1508 can execute is not limited to the number of hardware threads, and multiple logical threads can be assigned to individual hardware threads.

[0221] In some examples, the graphics execution unit 1508 may issue multiple instructions together, each of which may be a different instruction. The thread arbiter 1522 of the graphics execution unit thread 1508 may dispatch the instruction to one of the send unit 1530, the branch unit 1532, or (one or more) SIMD FPUs 1534 for execution. Each execution thread may access 128 general registers within the GRF 1524, wherein each register may store 32 bytes and may be accessed as a SIMD 8 element vector of 32-bit data elements. In some examples, each execution unit thread may access 4K bytes within the GRF 1524, but the example is not limited thereto, and more or less register resources may be provided in other examples. In some examples, the graphics execution unit 1508 is divided into seven hardware threads that may independently perform computing operations, but the number of threads of each execution unit may also vary according to the example. For example, in some examples, up to 16 hardware threads are supported. In an example where seven threads may access 4K bytes, the GRF 1524 may store a total of 28K bytes. With 16 threads able to access 4K bytes, a total of 64K bytes can be stored by GRF 1524. Flexible addressing modes can allow registers to be addressed together to efficiently build wider registers or rectangular block data structures representing strides.

[0222] In some examples, memory operations, sampler operations, and other longer latency system communications are dispatched via "send" instructions executed by the message passing send unit 1530. In some examples, branch instructions are dispatched to a dedicated branch unit 1532 to facilitate SIMD divergence and eventual convergence.

[0223] In some examples, the graphics execution unit 1508 includes one or more SIMD floating point units (FPUs) 1534 that perform floating point operations. In some examples, (one or more) FPUs 1534 also support integer calculations. In some examples, (one or more) FPUs 1534 can SIMD perform up to M 32-bit floating point (or integer) operations, or SIMD perform up to 2M 16-bit integers or 16-bit floating point operations. In some examples, at least one of (one or more) FPUs provides extended math capabilities to support high throughput transcendental math functions and double-precision 64-bit floating points. In some examples, there is also a set of 8-bit integer SIMD ALUs 1535, which can be specifically optimized to perform operations associated with machine learning calculations.

[0224] In some examples, an array of multiple instances of graphics execution unit 1508 may be instantiated in graphics sub-core groupings (e.g., sub-slices). For scalability, product designers may choose the exact number of execution units per sub-core grouping. In some examples, execution unit 1508 may execute instructions across multiple execution lanes. In other examples, individual threads executing on graphics execution unit 1508 execute on different lanes.

[0225] Fig.16 An additional execution unit 1600 according to an example is illustrated. In some examples, the execution unit 1600 includes a thread control unit 1601, a thread state unit 1602, an instruction fetch / prefetch unit 1603, and an instruction decode unit 1604. The execution unit 1600 further includes a register file 1606, which stores registers that can be assigned to hardware threads within the execution unit. The execution unit 1600 further includes a dispatch unit 1607 and a branch unit 1608. In some examples, the dispatch unit 1607 and the branch unit 1608 can be connected to Fig. 15B Send unit 1530 and branch unit 1532 of graphics execution unit 1508 operate similarly.

[0226] The execution unit 1600 also includes a computing unit 1610, which includes a plurality of functional units of different types. In some examples, the computing unit 1610 includes an ALU unit 1611, which includes an array of arithmetic logic units. The ALU unit 1611 may be configured to perform 64-bit, 32-bit, and 16-bit integer and floating-point operations. Integer and floating-point operations may be performed simultaneously. The computing unit 1610 may also include a systolic array 1612 and a mathematical unit 1613. The systolic array 1612 includes a W-wide and D-deep network of data processing units, which may be used to perform vector or other data parallel operations in a systolic manner. In some examples, the systolic array 1612 may be configured to perform matrix operations, such as matrix dot product operations. In some examples, the systolic array 1612 supports 16-bit floating-point operations and 8-bit and 4-bit integer operations. In some examples, the systolic array 1612 may be configured to accelerate machine learning operations. In such an example, the systolic array 1612 may be configured to support the bfloat 16-bit floating-point format. In some examples, math unit 1613 may be included to perform a specific subset of math operations in an efficient and low power manner compared to ALU unit 1611. Math unit 1613 may include a variation of math logic that may be found in shared functional logic of a graphics processing engine provided by other examples. In some examples, math unit 1613 may be configured to perform 32-bit and 64-bit floating point operations.

[0227] Thread control unit 1601 includes logic to control the execution of threads within execution units. Thread control unit 1601 may include thread arbitration logic that starts, stops, and preempts the execution of threads within execution unit 1600. Thread state unit 1602 may be used to store thread states of threads assigned to execute on execution unit 1600. Storing thread states within execution unit 1600 enables quick preemption of threads when they become blocked or idle. Instruction fetch / prefetch unit 1603 may fetch instructions from an instruction cache of higher-level execution logic (e.g., Fig.15A The instruction fetch / prefetch unit 1603 may also issue prefetch requests for instructions to be loaded into the instruction cache based on an analysis of the currently executing thread. The instruction decode unit 1604 may be used to decode instructions to be executed by the computing unit. In some examples, the instruction decode unit 1604 may be used as an auxiliary decoder to decode complex instructions into constituent micro-operations.

[0228] Execution unit 1600 additionally includes a register file 1606 that can be used by hardware threads executing on execution unit 1600. Registers in register file 1606 can be divided across logic for executing multiple simultaneous threads within compute unit 1610 of execution unit 1600. The number of logical threads that graphics execution unit 1600 can execute is not limited to the number of hardware threads, and multiple logical threads can be assigned to individual hardware threads. The size of register file 1606 can vary across examples based on the number of hardware threads supported. In some examples, register renaming can be used to dynamically assign registers to hardware threads.

[0229] Graphics Pipeline

[0230] Fig.17 is a block diagram of another example of a graphics processor 1700 . Fig.17 Elements in the drawings having the same reference numbers (or names) as elements of any other figures herein may operate or function in any manner similar to that described elsewhere herein, but are not limited thereto.

[0231] In some examples, graphics processor 1700 includes a geometry pipeline 1720, a media pipeline 1730, a display engine 1740, a thread execution logic 1750, and a rendering output pipeline 1770. In some examples, graphics processor 1700 is a graphics processor within a multi-core processing system that includes one or more general-purpose processing cores. The graphics processor is controlled by register writes to one or more control registers (not shown) or via commands issued to graphics processor 1700 through ring interconnect 1702. In some examples, ring interconnect 1702 couples graphics processor 1700 to other processing components, such as other graphics processors or general-purpose processors. Commands from ring interconnect 1702 are interpreted by command stream converter 1703, which provides instructions to various components of geometry pipeline 1720 or media pipeline 1730.

[0232] In some examples, command stream converter 1703 directs the operation of vertex fetcher 1705, which reads vertex data from memory and executes vertex processing commands provided by command stream converter 1703. In some examples, vertex fetcher 1705 provides vertex data to vertex shader 1707, which performs coordinate space transformation and lighting operations on individual vertices. In some examples, vertex fetcher 1705 and vertex shader 1707 execute vertex processing instructions by dispatching execution threads to execution units 1752A-1752B via thread dispatcher 1731.

[0233] In some examples, execution units 1752A-1752B are vector processor arrays with instruction sets for performing graphics and media operations. In some examples, execution units 1752A-1752B have an additional L1 cache 1751 that is specific to each array or shared between arrays. The cache can be configured as a data cache, an instruction cache, or a single cache that is partitioned to contain data and instructions in different partitions.

[0234] In some examples, the geometry pipeline 1720 includes a tessellation component that performs hardware accelerated tessellation of 3D objects. In some examples, the programmable shell shader 1711 configures the tessellation operation. The programmable domain shader 1717 provides back-end evaluation of the tessellation output. The tessellation 1713 operates under the direction of the shell shader 1711 and contains dedicated logic that generates a set of detailed geometric objects based on the rough geometric model provided as input to the geometry pipeline 1720. In some examples, if tessellation is not used, the tessellation components (e.g., the shell shader 1711, the tessellation 1713, and the domain shader 1717) can be bypassed.

[0235] In some examples, geometry shader 1719 may process complete geometric objects via one or more threads dispatched to execution units 1752A-1752B, or may proceed directly to clipper 1729. In some examples, the geometry shader operates on entire geometric objects, rather than on vertices or vertex patches as in previous stages of the graphics pipeline. If tessellation is disabled, geometry shader 1719 receives input from vertex shader 1707. In some examples, geometry shader 1719 may be programmed by a geometry shader program to perform geometry tessellation when the tessellation unit is disabled.

[0236] Before rasterization, the clipper 1729 processes the vertex data. The clipper 1729 can be a fixed function clipper or a programmable clipper with clipping and geometry shader functions. In some examples, the rasterizer and depth test component 1773 in the rendering output pipeline 1770 dispatches a pixel shader to convert the geometric object into a per-pixel representation. In some examples, the pixel shader logic is included in the thread execution logic 1750. In some examples, the application can bypass the rasterizer and depth test component 1773 and access the unrasterized vertex data via the outflow unit 1723.

[0237] Graphics processor 1700 has an interconnect bus, interconnect structure, or some other interconnect mechanism that allows data and messages to be passed between the main components of the processor. In some examples, execution units 1752A-1752B and associated logic units (e.g., L1 cache 1751, sampler 1754, texture cache 1758, etc.) are interconnected via data port 1756 to perform memory access and communicate with the rendering output pipeline components of the processor. In some examples, sampler 1754, cache 1751, 1758, and execution units 1752A-1752B each have a separate memory access path. In some examples, texture cache 1758 can also be configured as a sampler cache.

[0238] In some examples, the rendering output pipeline 1770 includes a rasterizer and depth test component 1773 that converts vertex-based objects into associated pixel-based representations. In some examples, the rasterizer logic includes a windower / masker unit that performs fixed-function triangle and line rasterization. In some examples, an associated rendering cache 1778 and depth cache 1779 are also available. A pixel operation component 1777 performs pixel-based operations on data, but in some cases, pixel operations associated with 2D operations (e.g., bit-block image transfers with blending) are performed by the 2D engine 1741, or replaced by the display controller 1743 using an overlay display plane when displaying. In some examples, a shared L3 cache 1775 is available to all graphics components, allowing data to be shared without using main system memory.

[0239] In some examples, graphics processor media pipeline 1730 includes media engine 1737 and video front end 1734. In some examples, video front end 1734 receives pipeline commands from command stream converter 1703. In some examples, media pipeline 1730 includes a separate command stream converter. In some examples, video front end 1734 processes media commands before sending them to media engine 1737. In some examples, media engine 1737 includes thread spawning functionality that spawns threads for dispatching to thread execution logic 1750 via thread dispatcher 1731.

[0240] In some examples, graphics processor 1700 includes display engine 1740. In some examples, display engine 1740 is external to processor 1700 and is coupled to the graphics processor via ring interconnect 1702 or some other interconnect bus or structure. In some examples, display engine 1740 includes 2D engine 1741 and display controller 1743. In some examples, display engine 1740 contains dedicated logic that can operate independently of the 3D pipeline. In some examples, display controller 1743 is coupled to a display device (not shown), which can be a system integrated display device such as in a laptop computer, or an external display device attached via a display device connector.

[0241] In some examples, the geometry pipeline 1720 and the media pipeline 1730 may be configured to perform operations based on multiple graphics and media programming interfaces, and are not specific to any one application programming interface (API). In some examples, the driver software for the graphics processor translates API calls specific to a specific graphics or media library into commands that can be processed by the graphics processor. In some examples, support is provided for the Open Graphics Library (OpenGL), the Open Computing Language (OpenCL) and / or the Vulkan graphics and computing API, all of which are from the Khronos organization. In some examples, support for the Direct3D library from Microsoft Corporation may also be provided. In some examples, a combination of these libraries may be supported. Support for the open source computer vision library (OpenCV) may also be provided. If a mapping from the pipeline of a future API to the pipeline of a graphics processor can be performed, future APIs with compatible 3D pipelines will also be supported.

[0242] Simulation (including binary translation, code deformation, etc.).

[0243] In some cases, the instruction converter can be used to convert instructions from a source instruction set architecture to a target instruction set architecture. For example, the instruction converter can translate (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate or otherwise convert instructions to one or more other instructions to be processed by the core. The instruction converter can be implemented in software, hardware, firmware or a combination thereof. The instruction converter can be on the processor, outside the processor, or partly on the processor and partly outside the processor.

[0244] Fig.18 1 is a block diagram illustrating the use of a software instruction converter according to an example to convert binary instructions in a source ISA into binary instructions in a target ISA. In the illustrated example, the instruction converter is a software instruction converter, but alternatively, the instruction converter can be implemented in software, firmware, hardware, or various combinations thereof. Fig.18 It is shown that a program in a high-level language 1802 can be compiled using a first ISA compiler 1804 to generate first ISA binary code 1806 that can be natively executed by a processor having at least one first ISA core 1816. The processor having at least one first ISA core 1816 represents any processor that can execute programs with the processor having at least one first ISA core in the following manner: Processor substantially the same functionality: compatibly execute or otherwise process (1) a substantial portion of the first ISA or (2) an object code version of an application or other software targeted to run on an Intel processor having at least one first ISA core so as to achieve substantially the same results as a processor having at least one first ISA core. First ISA compiler 1804 represents a compiler operable to generate first ISA binary code 1806 (e.g., object code) that can be executed on a processor having at least one first ISA core 1816 with or without additional link processing. Similarly, Fig.18 It is shown that a program in a high-level language 1802 can be compiled using an alternative ISA compiler 1808 to generate an alternative ISA binary code 1810 that can be natively executed by a processor that does not have a first ISA core 1814. An instruction converter 1812 is used to convert the first ISA binary code 1806 into code that can be natively executed by a processor that does not have a first ISA core 1814. The converted code does not have to be the same as the replaced ISA binary code 1810; however, the converted code will complete the general operation and consist of instructions from the replacement ISA. Thus, the instruction converter 1812 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have a first ISA processor or core to execute the first ISA binary code 1806 through simulation, emulation, or any other process.

[0245] IP core implementation

[0246] One or more aspects of at least some examples may be implemented by representative code stored on a machine-readable medium that represents and / or defines logic within an integrated circuit such as a processor. For example, a machine-readable medium may include instructions representing various logic within a processor. When read by a machine, the instructions may cause the machine to manufacture logic to perform the techniques described herein. This representation (referred to as an "IP core") is a reusable logic unit for an integrated circuit that may be stored on a tangible machine-readable medium as a hardware model describing the structure of the integrated circuit. The hardware model may be provided to various customers or manufacturing facilities that load the hardware model onto a manufacturing machine that manufactures the integrated circuit. The integrated circuit may be manufactured so that the circuit performs the operations described in association with any of the examples described herein.

[0247] Fig.191 is a block diagram illustrating an IP core development system 1900 that can be used to manufacture an integrated circuit to perform operations according to some examples. The IP core development system 1900 can be used to generate modular, reusable designs that can be incorporated into a larger design or used to construct an entire integrated circuit (e.g., a SoC integrated circuit). The design facility 1930 can generate a software simulation 1910 of the IP core design in a high-level programming language (e.g., C / C++). The software simulation 1910 can be used to design, test, and verify the behavior of the IP core using a simulation model 1912. The simulation model 1912 may include functional, behavioral, and / or timing simulations. Then, a register transfer level (RTL) design 1915 can be created or synthesized based on the simulation model 1912. The RTL design 1915 is an abstraction of the behavior of an integrated circuit (including associated logic executed using the modeled digital signals) that models the flow of digital signals between hardware registers. In addition to the RTL design 1915, a low-level design at the logic level or transistor level can also be created, designed, or synthesized. Thus, the specific details of the initial design and simulation can vary.

[0248] The RTL design 1915 or equivalent may be further synthesized by the design facility into a hardware model 1920, which may be a hardware description language (HDL) or some other representation of physical design data. The HDL may be further simulated or tested to verify the IP core design. The IP core design may be stored using a non-volatile memory 1940 (e.g., a hard disk, flash memory, or any non-volatile storage medium) for delivery to a third party manufacturing facility 1965. Alternatively, the IP core design may be transmitted (e.g., via the Internet) via a wired connection 1950 or a wireless connection 1960. The manufacturing facility 1965 may then manufacture an integrated circuit based at least in part on the IP core design. The manufactured integrated circuit may be configured to perform operations according to at least some of the examples described herein.

[0249] Although some embodiments have been described with reference to specific implementations, other implementations are possible according to some embodiments. In addition, the arrangement and / or order of the elements or other features illustrated in the drawings and / or described herein need not be arranged in the particular manner illustrated and described. According to some embodiments, many other arrangements are possible.

[0250] In each system shown in the figure, in some cases, elements can each have the same reference numeral or different reference numerals to suggest that the represented elements can be different and / or similar. However, elements can be flexible enough to have different implementations and work together with some or all of the systems shown or described herein. The various elements shown in the figure can be the same or different. Which one is called the first element and which one is called the second element is arbitrary.

[0251] An embodiment is an implementation or example of the present disclosure. References in the specification to "an embodiment," "an embodiment," "some embodiments," or "other embodiments" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments of the present disclosure, but not necessarily in all embodiments of the present disclosure. Various appearances of "an embodiment," "an embodiment," or "some embodiments" do not necessarily refer to the same embodiment.

[0252] Not all components, features, structures, characteristics, etc. described and illustrated herein need to be included in one or more specific embodiments. If the specification states that a component, feature, structure, or characteristic "may", "might", "can", or "could" be included, for example, that particular component, feature, structure, or characteristic is not necessarily included. If the specification or claims refer to "one" or "an" element, this does not mean that there is only one element. If the specification or claims refer to "additional" elements, this does not exclude the presence of more than one additional element.

[0253] The above description of the illustrated embodiments of the present disclosure (including what is described in the Abstract) is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Although specific embodiments and examples of the present disclosure are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the present disclosure as will be appreciated by those skilled in the relevant art.

[0254] These modifications may be made to the present disclosure in light of the above detailed description. The terms used in the following claims should not be interpreted as limiting the present disclosure to the specific embodiments disclosed in the specification and the drawings. Instead, the scope of the present disclosure is determined entirely by the appended claims, which are to be interpreted in accordance with established principles of claim interpretation.

[0255] Additional Examples

[0256] Example 1 provides a computer processing system, comprising: multiple computing units for executing tasks of one or more applications; multiple sensors, which are coupled to the multiple computing units and are used to collect measurement data of the multiple computing units; and a storage device for storing a hardware health state corresponding to each of the multiple computing units, wherein the health state is based on the measurement data; the multiple computing units are scheduled to perform tasks on the computer processing system for the one or more applications based on the hardware health state of the multiple computing units, wherein, in response to a corresponding first hardware health state of a first computing unit among the multiple computing units indicating that a hardware failure is about to occur, the first computing unit is excluded from the task execution.

[0257] Example 2 includes the substance of Example 1, wherein the error screening content is scheduled to be executed with tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

[0258] Example 3 includes the substance of Examples 1-2, wherein the error screening content includes a test packet for silent data errors (SDE).

[0259] Example 4 includes the substance of Examples 1 to 3, wherein the error screening content is executed as an application.

[0260] Example 5 includes the substance of Examples 1 to 4, wherein the error screening content is executed on the computing unit in response to the computing unit operating below capacity when the tasks of the one or more applications are executed.

[0261] Example 6 includes the substance of Examples 1 to 5, wherein the states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

[0262] Example 7 includes the substance of Examples 1 to 6, wherein the plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component-level built-in self-test (BIST) logic.

[0263] Example 8 includes the substance of Examples 1 to 7, wherein the multiple computing units include one or more of the following entities in the computer processing system: a central processing unit (CPU) core set, a graphics processing unit (GPU) core set, a tensor processing unit (TPU) core set, an intelligent processing unit (IPU) core set, and a memory unit set.

[0264] Example 9 includes the substance of Examples 1 to 8, wherein the plurality of computing units include one or more of: the CPU core set, the GPU core set, the TPU core set, the IPU core set, and the memory unit set.

[0265] Example 10 includes the substance of Examples 1 to 9, wherein machine learning or heuristic methods are used to determine the states of the plurality of computing units.

[0266] Example 11 includes the substance of Examples 1 to 10, wherein in response to a hardware health status of the first group of computing units being no hardware health issues and a hardware health status of the second group of computing units being no failures but impending hardware failures, the second group of computing units are excluded from the task execution.

[0267] Example 12 includes the essence of Examples 1 to 11, wherein, in response to a corresponding second hardware health state of a second computing unit among the multiple computing units being an impending hardware failure and a corresponding third hardware health state of a third computing unit among the multiple computing units being no hardware health issues, a task of the second computing unit is moved to the third computing unit.

[0268] Example 13 provides an exemplary method, comprising: executing tasks of one or more applications by multiple computing units of a computer processing system; collecting measurement data of the multiple computing units by multiple sensors coupled to the multiple computing units; and storing a hardware health state corresponding to each of the multiple computing units in a storage device, wherein the hardware health state is based on the measurement data. The method also includes: scheduling the multiple computing units to perform tasks on the computer processing system for the one or more applications based on the hardware health state of the multiple computing units, wherein in response to a corresponding hardware health state of a first computing unit of the multiple computing units indicating an impending hardware failure, the first computing unit is excluded from the task execution.

[0269] Example 14 includes the substance of Example 13, wherein the error screening content is scheduled to be executed with tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

[0270] Example 15 includes the substance of Examples 13-14, wherein the states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

[0271] Example 16 includes the substance of Examples 13 to 15, wherein the plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component-level built-in self-test (BIST) logic.

[0272] Example 17 provides an exemplary computer-readable storage medium, the non-transitory computer-readable storage medium storing instructions, which, when executed by a processor of a computing system, can cause the computing system to perform the following operations: executing tasks of one or more applications by multiple computing units of the computer processing system; collecting measurement data of the multiple computing units by multiple sensors coupled to the multiple computing units; and storing a hardware health state corresponding to each of the multiple computing units in a storage device, wherein the hardware health state is based on the measurement data. The method also includes: scheduling the multiple computing units to perform tasks on the computer processing system for the one or more applications based on the hardware health state of the multiple computing units, wherein, in response to a corresponding hardware health state of a first computing unit of the multiple computing units indicating an impending hardware failure, the first computing unit is excluded from the task execution.

[0273] Example 18 includes the substance of Example 17, wherein the error screening content is scheduled to be executed with the tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

[0274] Example 19 includes the substance of Examples 17-18, wherein the states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

[0275] Example 20 includes the substance of Examples 17 to 19, wherein the plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component-level built-in self-test (BIST) logic.

[0276] Additional explanation

[0277] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments of the present disclosure. However, it will be apparent that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the appended claims. Therefore, the description and drawings should be considered illustrative rather than restrictive.

[0278] Embodiments of the present disclosure may include the various steps described above. These steps may be specifically implemented in machine executable instructions, which may be used to cause a general or special processor to perform these steps. Alternatively, these steps may be performed by specific hardware components containing hard-wired logic for performing these steps, or by any combination of programmed computer components and custom hardware components.

[0279] As described herein, instructions may refer to a specific configuration of hardware, such as an application-specific integrated circuit (ASIC) configured to perform certain operations or have predetermined functions or software instructions specifically implemented in a non-transient computer-readable medium stored in a memory. Thus, the techniques shown in the figure can be implemented using code and data stored and executed on one or more electronic devices (e.g., terminal stations, network elements, etc.). Such electronic devices use computer machine-readable media (such as non-transient computer machine-readable storage media (e.g., magnetic disks; optical disks; random access memory; read-only memory; flash memory devices; phase change memory) and transient computer machine-readable communication media (e.g., electrical, optical, acoustic or other forms of propagated signals - such as carrier waves, infrared signals, digital signals, etc.)) to store and communicate (internal and / or communicate with other electronic devices through a network) code and data. In addition, such electronic devices typically include a group of one or more processors coupled to one or more other components, such as one or more storage devices (non-transient machine-readable storage media), user input / output devices (e.g., keyboards, touch screens and / or displays), and network connections. The coupling of the processor group and other components is typically performed through one or more buses and bridges (also called bus controllers). The storage device and the signal carrying the network traffic represent one or more machine-readable storage media and machine-readable communication media, respectively. Thus, the storage device of a given electronic device generally stores code and / or data for execution on a group of one or more processors of the electronic device. Of course, one or more parts of the embodiments of the present disclosure can be implemented using different combinations of software, firmware and / or hardware. Throughout the detailed description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure can be practiced without some of these specific details. In some cases, in order to avoid obscuring the subject matter of the present disclosure, known structures and functions are not described in detail. Therefore, the scope and spirit of the present disclosure should be judged according to the appended claims.

Claims

1. A computer processing system comprising: A plurality of computing units for executing tasks of one or more applications; a plurality of sensors, the plurality of sensors being coupled to the plurality of computing units, the plurality of sensors being used to collect measurement data of the plurality of computing units; as well as a storage device for storing a hardware health status corresponding to each of the plurality of computing units, wherein the hardware health status is based on the measurement data; The plurality of computing units are scheduled to perform task execution on the computer processing system for the one or more applications based on the hardware health states of the plurality of computing units, wherein, in response to a corresponding hardware health state of a first computing unit of the plurality of computing units indicating an impending hardware failure, the first computing unit is excluded from the task execution.

2. The computer processing system of claim 1, wherein: Error screening content is scheduled to be executed along with tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

3. The computer processing system of claim 2, wherein: The error screening content includes a test package for silent data errors (SDE).

4. The computer processing system of claim 2, wherein: The error screening content is executed as an application.

5. The computer processing system of claim 2, wherein: The error screening content is executed on the computing unit in response to the computing unit operating below capacity during execution of the task of the one or more applications.

6. The computer processing system according to any one of claims 1 to 5, wherein: The states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

7. The computer processing system according to any one of claims 1 to 5, wherein: The plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component level built-in self-test (BIST) logic.

8. The computer processing system according to any one of claims 1 to 5, wherein: The multiple computing units include one or more of the following entities in the computer processing system: a set of central processing unit (CPU) cores, a set of graphics processing unit (GPU) cores, a set of tensor processing units (TPUs), a set of intelligence processing units (IPUs), and a set of memory units.

9. The computer processing system of claim 8, wherein: The plurality of computing units include one or more of the following components: the CPU core set, the GPU core set, the TPU core set, the IPU core set, and the memory unit set.

10. The computer processing system according to any one of claims 1 to 5, wherein: Machine learning or heuristic methods are used to determine the states of the plurality of computing units.

11. The computer processing system according to any one of claims 1 to 5, wherein: In response to the hardware health status of the first group of computing units being no hardware health issues and the hardware health status of the second group of computing units being no failures but impending hardware failures, the second group of computing units are excluded from the task execution.

12. The computer processing system according to any one of claims 1 to 5, wherein: In response to a second hardware health status of a second computing unit among the plurality of computing units being an impending hardware failure and a third hardware health status of a third computing unit among the plurality of computing units being no hardware health issues, tasks of the second computing unit are moved to the third computing unit.

13. A method comprising: Executing tasks of one or more applications by multiple computing units of a computer processing system; collecting measurement data of the plurality of computing units by a plurality of sensors coupled to the plurality of computing units; storing a hardware health status corresponding to each of the plurality of computing units in a storage device, wherein the hardware health status is based on the measurement data; and The plurality of computing units are scheduled to perform task execution on the computer processing system for the one or more applications based on the hardware health states of the plurality of computing units, wherein, in response to a first computing unit of the plurality of computing units having a corresponding hardware health state indicating an impending hardware failure, the first computing unit is excluded from the task execution.

14. The method according to claim 13, wherein: Error screening content is scheduled to be executed along with tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

15. The method according to claim 13 or 14, wherein: The states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

16. The method according to claim 13 or 14, wherein: The plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component level built-in self-test (BIST) logic.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a computing system, enable the computing system to: Executing tasks of one or more applications by multiple computing units of a computer processing system; collecting measurement data of the plurality of computing units by a plurality of sensors coupled to the plurality of computing units; storing a hardware health status corresponding to each of the plurality of computing units in a storage device, wherein the hardware health status is based on the measurement data; and The plurality of computing units are scheduled to perform task execution on the computer processing system for the one or more applications based on the hardware health states of the plurality of computing units, wherein, in response to a first computing unit of the plurality of computing units having a corresponding hardware health state indicating an impending hardware failure, the first computing unit is excluded from the task execution.

18. The non-transitory computer-readable storage medium of claim 17, wherein: Error screening content is scheduled to be executed with the tasks of the one or more applications to further determine the hardware health status of the plurality of computing units.

19. The non-transitory computer-readable storage medium according to claim 17 or 18, wherein: The states of the plurality of computing units are updated based on the measurement data periodically obtained from the plurality of sensors.

20. The non-transitory computer-readable storage medium according to claim 17 or 18, wherein: The plurality of sensors include one or more of: an in-die variation (IDV) oscillator, a voltage threshold drift sensor, a reference critical path delay drift sensor, a replica path timing / voltage drift sensor, or component level built-in self-test (BIST) logic.

Images