Application programming interface for causing measurement of processor activity

Abstract

The present invention relates to application programming interfaces for causing measurements of processor activity, and specifically discloses apparatuses, systems, and techniques for identifying clock frequencies at which one or more processors are to be run. In at least one embodiment, a processor executes an application programming interface (API) to cause one or more activity levels of one or more processors to be measured at one or more indicated intervals.

Description

[0001] Cross-reference to related applications

[0002] This application is a continuation to International Patent Application No. PCT / CN2024 / 137417, filed on December 6, 2024, entitled “APPLICATION PROGRAMMING INTERFACE TO CAUSE MEASUREMENT OF PROCESSOR ACTIVITY”, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] At least one embodiment relates to processing resources for operating one or more processors. At least one embodiment relates to a processor or computing system for operating processors according to an activity level. Background Technology

[0004] Executing software programs in parallel across multiple processors can lead to inefficient computation. Techniques for executing software programs in parallel across multiple processors can be improved. Attached Figure Description

[0005] Figure 1 The illustration depicts a system for synchronizing processors in a processor group by measuring workload changes, according to at least one embodiment.

[0006] Figure 2 The illustration depicts a process for synchronizing processors in a processor group by measuring workload changes, according to at least one embodiment.

[0007] Figure 3 The illustration depicts a process for synchronizing processors in a processor group by measuring workload changes, according to at least one embodiment.

[0008] Figure 4 The figure illustrates a system flowchart according to at least one embodiment for synchronizing processors in a processor group by measuring workload changes;

[0009] Figure 5A The illustration depicts API calls and responses in a processor group according to at least one embodiment for synchronizing processors in a processor group by measuring workload changes;

[0010] Figure 5B The illustration depicts API calls and responses in a processor group according to at least one embodiment for synchronizing processors in a processor group by measuring workload changes;

[0011] Figure 6A The illustration depicts API calls and responses in a processor group according to at least one embodiment for synchronizing processors in a processor group by measuring workload changes;

[0012] Figure 6B The illustration depicts API calls and responses in a processor group according to at least one embodiment for synchronizing processors in a processor group by measuring workload changes;

[0013] Figure 7 The illustration depicts a system comprising software and hardware for synchronizing processors within a group by measuring workload changes, according to at least one embodiment.

[0014] Figure 8 The illustration depicts a system including a driver and / or a runtime system for synchronizing processors within a group by measuring workload changes, according to at least one embodiment.

[0015] Figure 9 An example data center system according to at least one embodiment is shown;

[0016] Figure 10 A system-on-a-chip (SOC) according to at least one embodiment is shown;

[0017] Figure 11A A parallel processor according to at least one embodiment is shown;

[0018] Figure 11B A processing cluster according to at least one embodiment is shown;

[0019] Figure 11C A graphics multiprocessor according to at least one embodiment is shown;

[0020] Figure 12 An accelerator processor according to at least one embodiment is shown;

[0021] Figure 13A A central processing unit according to at least one embodiment is shown;

[0022] Figure 13B The illustration shows an embodiment according to at least one of the embodiments. Figure 13A The core of the central processing unit;

[0023] Figure 14 Another accelerator processor according to at least one embodiment is shown;

[0024] Figure 15 A neuromorphic processor according to at least one embodiment is shown;

[0025] Figure 16 A supercomputer according to at least one embodiment is shown;

[0026] Figure 17 Another accelerator processor according to at least one embodiment is shown;

[0027] Figure 18 Another processor according to at least one embodiment is shown;

[0028] Figure 19 Another accelerator processor according to at least one embodiment is shown;

[0029] Figure 20 A tensor processing unit according to at least one embodiment is shown;

[0030] Figure 21 A RISC-V compatible processor according to at least one embodiment is shown;

[0031] Figure 22A and Figure 22B A language processing unit according to at least one embodiment is shown;

[0032] Figure 23 A software stack of a programming platform according to at least one embodiment is shown;

[0033] Figure 24 Software supported by a programming platform according to at least one embodiment is shown;

[0034] Figure 25 A method for using at least one embodiment is shown. Figure 24 Compiled code executed on the programming platform;

[0035] Figure 26 An example of an autonomous vehicle and its system architecture according to at least one embodiment is shown;

[0036] Figure 27A The inference and / or training logic according to at least one embodiment is illustrated;

[0037] Figure 27B The inference and / or training logic according to at least one embodiment is shown; and

[0038] Figure 27C Training and deployment of a neural network according to at least one embodiment are illustrated. Detailed Implementation

[0039] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of at least one embodiment. However, those skilled in the art will understand that the inventive concept can be practiced without one or more of these specific details, and that any two or more aspects of any one or more embodiments as described herein can be combined.

[0040] In at least one embodiment, the application programming interface (API) function is referred to as an API. In at least one embodiment, the processor executes different APIs such that performance metrics generated by the processor group are used to calculate the clock frequency at which the processor group will operate when executing a particular job, or as described elsewhere herein. In at least one embodiment, the user invokes the API to cause the processor to receive an identifier for a particular job, an identifier for a particular processor group, and an indication indicating that a workload factor for each processor in the identified processor group should be generated and stored when the job is executed in the identified processor group, or as described elsewhere herein. In at least one embodiment, the application repeatedly invokes the API at regular intervals to cause the processor to measure performance metrics used to generate and store the workload factor for each processor in the identified processor group when executing a job, or as described elsewhere herein. In at least one embodiment, the processor performs calculations to identify the overall average workload factor of the processor group. In at least one embodiment, the user invokes an API function to cause the processor to output the workload factor exhibited by the processors in the processor group when the processor group executes a job to a display of the user interface, or as described elsewhere herein. In at least one embodiment, the user invokes an API to cause the processor to stop the processor from generating and storing the workload factor of each processor in the processor group while the processor group is executing a job, and to calculate the clock frequency at which each processor in the processor group will continue to execute the job, or as described elsewhere herein.

[0041] In at least one embodiment, the processor includes one or more circuits. In at least one embodiment, the processor executes an API to cause one or more activity levels of other processors to be measured at one or more indicated intervals, or as described elsewhere herein. In at least one embodiment, the processor executes an API to cause one or more measurements of one or more activity levels of other processors to be stopped, or as described elsewhere herein. In at least one embodiment, the processor executes an API to cause one or more activity levels of other processors to be indicated to one or more users, or as described elsewhere herein. In at least one embodiment, the processor executes an API to cause one or more users to be indicated to one or more users one or more statistics corresponding to one or more activity levels of one or more processors, or as described elsewhere herein. In at least one embodiment, the technique described herein includes using measured workload changes of processors executing software programs in parallel to calculate the clock frequency to be applied to each processor in the group, thereby improving the synchronization of the software program. One technical effect of the technique described herein includes improving the synchronization of software programs being executed in parallel by processors in the processor group when each processor in the processor group executes its assigned software program instance asynchronously with other processors in the group.

[0042] Figure 1 A block diagram of a system 100 according to at least one embodiment is illustrated. The system includes one or more processors, each processor including one or more circuits for identifying, by using the activity level of a group of processors, that the group of processors will perform any of the operations described herein at its operating clock frequency while executing a software program, or otherwise. In at least one embodiment, this document incorporates… Figure 1 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 2 to 8 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, system 100 includes execution Figure 2 The process 200 system, Figure 3 System 300 Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0043] In at least one embodiment, one or more processors perform one or more operations of system 100. In at least one embodiment, processor 108 performs one or more operations of system 100, and may be any type of processor, part of a processor, processor of a system, or combination of processors as described herein, including logical processors. Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with the processor, or some combination thereof.

[0044] In at least one embodiment, a logical processor refers to a virtualized processor core on which an operating system can schedule tasks. In at least one embodiment, a logical processor is part of a processor architecture that supports parallel processing. In at least one embodiment, a physical processor (such as a core) is the actual hardware component that performs computations within the processor. In at least one embodiment, a logical processor is a virtual representation of a physical core. In at least one embodiment, such as Hyper-Threading TM or Simultaneous Multithreading TMTechniques such as Semi-Mechanical Processing (SMT) divide each physical core of a processor into multiple logical processors. In at least one embodiment, this allows the operating system to treat each physical core as if it were two or more separate cores, thereby doubling the number of tasks that can be processed concurrently. In at least one embodiment, the logical processors may be created or otherwise implemented on any type of processor, part of a processor, processor of a system, or combination of processors as described herein, including Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with the processor, or some combination thereof.

[0045] In at least one embodiment, processor 108 performs operations used by system 100, such as operations of the processor group synchronizing API module 110 with varying workloads. In at least one embodiment, processor 108 performs combined... Figure 2 The one or more operations described herein include, for example, operation 204 for collecting workload factors (WF) from the GPU running the job. In at least one embodiment, a job refers to a software program as further described herein. In at least one embodiment, processor 108 performs a combination of... Figure 3 The one or more operations mentioned above, such as operation 306 for acquiring telemetry data from all GPUs. In at least one embodiment, processor 108 performs a combination of... Figure 4The one or more operations mentioned above, such as operation 414 for executing the JobStartStats API. In at least one embodiment, processor 108 performs a combination of Figure 5A The one or more operations mentioned above, such as operation 502 for invoking the JobStartStats API. In at least one embodiment, processor 108 performs a combination Figure 5B The one or more operations mentioned above, such as operation 508 for invoking the JobGetStats API. In at least one embodiment, processor 108 performs a combination Figure 6A The one or more operations mentioned above, such as operation 602 for calling the GetDeviceFieldValues ​​API. In at least one embodiment, processor 108 performs a combination Figure 6B The one or more operations mentioned above, such as operation 608 for invoking the JobStopStats API. In at least one embodiment, processor 108 performs a combination Figure 7 The one or more operations described herein, such as operations of the API of software library 706. In at least one embodiment, processor 108 performs a combination of Figure 8 The one or more operations mentioned above, such as functions that synchronize the processor group by measuring workload changes in API 810.

[0046] In at least one embodiment, system 100 is any computing system or combination of computing systems, such as a computing system constituting one or more data centers or other facilities housing computing and network equipment. In at least one embodiment, system 100 is Figure 16 At least a portion of, or including, of the system 1600 Figure 16 At least a portion of system 1600. In at least one embodiment, system 100 is used to perform database or distributed database functions. In at least one embodiment, system 100 or the present document at least incorporates... Figures 1 to 8Any other system mentioned herein is referred to as a database system. In at least one embodiment, a distributed database is a type of database distributed across multiple physical locations, which may be on different servers, different geographical regions, or some combination thereof. In at least one embodiment, data stored as part of a distributed database is managed and accessed as if it were a single database, but the data is actually stored in multiple locations. In at least one embodiment, system 100 is used to execute one or more software programs on processor groups in one or more data centers. In at least one embodiment, system 100 is implemented as a non-transitory computer-readable storage medium (further described herein) that stores instructions that, when executed by one or more processors of a computer system, cause the computer system to use the processors or otherwise cause the processors to execute APIs to identify, by using the activity level of the processor group, the clock frequency at which the processor group will operate while executing the software program, or otherwise perform any of the operations described herein. In at least one embodiment, system 100 is implemented as one or more processors including one or more circuits, or a computer system including one or more processors for using or otherwise causing the one or more processors and / or one or more other processors to execute an API to identify, by using the activity level of the processor group, the clock frequency at which the processor group will run when executing a software program, or otherwise perform any of the operations described herein.

[0047] In at least one embodiment, a software program is at least a portion of one or more sets of instructions followed by a computing system to perform operations, solve problems, or automate tasks. In at least one embodiment, a software program exists as a collection of data and code that enables a computer to perform specific functions or activities. In at least one embodiment, a software program serves as an application, providing users with tools and interfaces for performing various tasks on a computing device. In at least one embodiment, a software program is a kernel. In at least one embodiment, a kernel manages system resources and facilitates communication between hardware and software components. In at least one embodiment, a software program runs as a thread, executing a sequence of instructions within a process to perform specific tasks concurrently with other threads.

[0048] In at least one embodiment, system 100 is used to perform high-performance computing tasks, quantization of neural network values, neural network training, neural network inference, or some combination thereof. In at least one embodiment, references to machine learning, artificial intelligence, or deep learning refer to any aspect of a neural network as described herein. In at least one embodiment, system 100 includes an edge computing system, an accelerated computing system, a cloud computing system, a hybrid cloud computing system, or some combination thereof. In at least one embodiment, system 100 is a computing system that includes multiple distributed components connected via a network, such as the Internet. In at least one embodiment, system 100 is used in fields such as generative artificial intelligence (AI), physical modeling, healthcare, genomics, engineering, aerospace, urban planning, graphics processing, finance, data storage and management, data science, e-commerce, meteorology, or some combination thereof. In at least one embodiment, system 100 is used to train a neural network to perform neural network tasks such as language generation, image generation, image classification, image segmentation, object recognition, autonomous driving, manufacturing defect identification, or some combination thereof. In at least one embodiment, the neural network is a component or type of artificial intelligence (AI). In at least one embodiment, system 100 is used as part of a distributed database system.

[0049] In at least one embodiment, system 100 includes one or more data centers 102. In at least one embodiment, the data centers include... Figure 9 At least a portion of, or the entirety of, the data center 900 Figure 9 The data center 900 is at least a portion of the data center. In at least one embodiment, the data center is any facility that houses computers and network equipment. In at least one embodiment, the data center includes processors, such as processor 108, which execute different programs in parallel using massive datasets with multiple dimensions. In at least one embodiment, the data center executes one or more neural network tasks. In at least one embodiment, at least a portion of the computing resources of system 100 is remotely accessed by users via a network. In at least one embodiment, the data center includes two or more processors assigned to execute software programs in parallel, wherein these processors are collectively referred to as a processor group, processor cluster, processing cluster, GPU group, GPU cluster, computing cluster, node, or similar name. In at least one embodiment, data center 102 includes GPU group 116.

[0050] In at least one embodiment, two or more processors of processor 108 are mounted on separate computing machines, such as servers. In at least one embodiment, the separate computing machines are two or more computing machines separated from each other within a server rack, between server racks in a single data center, between separate data centers, or some combination thereof. In at least one embodiment, two or more processors 108 are communicatively connected via a network (such as an Internet network, a managed network (e.g., an enterprise network), a cloud network, the Internet, a local private network, or some combination thereof). In at least one embodiment, two or more processors 108 are communicatively connected via any one or a combination of physical and logical connections (also referred to as interconnects), such as a Super Accelerator Link (UALink). Or some combination thereof.

[0051] In at least one embodiment, system 100 includes user equipment 104. In at least one embodiment, user equipment 104 includes processor 108a. In at least one embodiment, user equipment 104 is a computing system including a user interface. In at least one embodiment, user equipment 104 is referred to as a client device. In at least one embodiment, a user invokes one or more APIs described herein via user equipment 104. In at least one embodiment, a user inputs one or more API parameters described herein via user equipment 104. In at least one embodiment, the interface of user equipment 104 includes a graphical user interface, a command-line interface, or some combination thereof. In at least one embodiment, processor 108a performs operations of user equipment 104 to receive or otherwise acquire APIs, API input parameters, or some combination thereof, which are used to identify the clock frequency at which the processor will operate when executing a software program, or as described elsewhere herein.

[0052] In at least one embodiment, system 100 includes network 106. In at least one embodiment, user equipment 104 is communicatively connected to network 106. In at least one embodiment, network 106 can be one or more of any type of communication network, such as a managed network (e.g., an enterprise network), a cloud network, the Internet, a local private network, or some combination thereof. In one embodiment, network 106 is a local network. In at least one embodiment, network 106 is communicatively connected to any one or more components of data center 106. In at least one embodiment, a neural network training framework uses network 106 at least in part as a cloud-native neural network training framework (such as Red...). Open Data Hub or A portion of a NeMo (NeNative Neural Network Training Framework) performs at least one neural network training operation. In at least one embodiment, a cloud-native neural network training framework refers to a framework that allows a user or application to remotely perform neural network operations via a computing device connected by a network (such as network 106).

[0053] In at least one embodiment, system 100 includes a processor group synchronization API module 110 that uses workload variations, also referred to as processor group synchronization API module 110. In at least one embodiment, processor 108 performs one or more operations of processor group synchronization API module 110. In at least one embodiment, processor group synchronization API module 110 captures per-GPU workload telemetry data in a GPU group, wherein the workload telemetry data is referred to as a workload factor. In at least one embodiment, the workload telemetry data is referred to as an activity level. In at least one embodiment, the workload factor is a type of activity level. In at least one embodiment, the GPU driver calculates the workload factor as a characteristic of the application's dynamic capacitance (Cdyn), where Cdyn represents the application's dynamic activity, or as described elsewhere herein. In at least one embodiment, the driver provides per-GPU telemetry data to a higher-level agent, such as a data center processor management system, or as described elsewhere herein. In at least one embodiment, an example of a data center processor management system is... Data Center GPU Management (DCGM) System. In at least one example, the data center processor management system uses a user-inputted power level, along with information about the clock frequencies of a software program and one or more GPUs, to run computations, or as described elsewhere herein. In at least one embodiment, the software program is referred to as a workload or application. In at least one embodiment, the data center uses user-selected target thermal graphics power (TGP) and workload factor telemetry data, along with the graphics processing core clock (GPCCLK), to execute an algorithm, or as described elsewhere herein. In at least one embodiment, TGP refers to the maximum amount of power a processor is designed to consume under typical operating conditions, as set by the user. In at least one embodiment, TGP refers to the maximum amount of power a processor is designed to consume under typical operating conditions. In at least one embodiment, the algorithm determines the optimal clock frequency for a workload corresponding to the TGP, where the clock frequency may be referred to as a synchronous clock, or as described elsewhere herein.

[0054] In at least one embodiment, the processor group synchronization API module 110 performs one or more operations to initiate and collect telemetry data from one or more processors executing a software program. In at least one embodiment, the data center processor management system operates in a background mode to process the telemetry data and execute algorithms to determine the synchronization clock or the clock frequency at which the processor group will operate while executing a specific software program. In at least one embodiment, the data center program management system identifies and sets the clock frequency and TGP for each GPU to execute the software program. In at least one embodiment, once the data center program management system has identified and set the clock frequency and TGP for the processor group while it executes the software program, a user can invoke the API to make the processor group execute the software program. In at least one embodiment, the identified clock frequency, TGP, or some combination thereof used to operate the processor group while it executes the software program is referred to as a strategy or stats strategy. In at least one embodiment, the process for identifying and setting the clock frequency and / or TGP to be used to operate the processor group includes two steps: a profiling step and a step of setting the strategy for executing the software program.

[0055] In at least one embodiment, as used in any implementation described herein, unless the context explicitly states otherwise or explicitly to the contrary, terms such as “system,” “device,” “component,” “agent,” “manager,” and “module,” as well as nominalized verbs (e.g., coordinator, compiler, scheduler, manager, and / or other terms), each refer to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. In at least one embodiment, any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein is referred to as a component. In at least one embodiment, any component described herein is combined with and / or communicatively connected to at least one other component, regardless of how such components are described in other embodiments as combined and / or communicatively connected. In at least one embodiment, software may be embodied as a software package, code, and / or instruction set or instructions. In at least one embodiment, hardware, individually or in combination, includes hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, and / or firmware storing instructions executed by the programmable circuitry. In at least one embodiment, modules may collectively or individually embody circuitry that forms part of a larger system, such as an integrated circuit (IC), a system-on-a-chip (SoC), and so on. In at least one embodiment, one or more architectures of any circuitry of one or more modules are represented as register-transfer-level (RTL) representations and / or another fabless representation that may be licensed and / or used for tape-out (the final stage in IC design before IC manufacturing).

[0056] In at least one embodiment, system 100 includes a higher-level data center processor manager 112. In at least one embodiment, the higher-level data center processor manager 112 includes an API library that includes one or more APIs as described herein. In at least one embodiment, processor 108 performs one or more operations of the higher-level data center processor manager 112. In at least one embodiment, the higher-level data center processor manager 112 includes a data center processor management system, such as... DCGM Radeon Pro Software for Enterprise ROCm (Radeon Open Compute) Data Center Manager (DCM) VTune Profiler or some combination thereof. In at least one embodiment, the higher-level data center processor manager 112 includes one or more APIs and / or uses a programming language written in a higher-level programming language than another data center processor management system (such as the lower-level data center processor manager 114). In at least one embodiment, a higher-level computing language refers to a language designed to be relatively more user-friendly and more abstract. In at least one embodiment, the higher-level data center processor manager 112 performs one or more operations to cause the activity level of the processor to be measured, stored, computed, or some combination thereof, or as described elsewhere herein. In at least one embodiment, the higher-level data center processor manager 112 performs one or more operations to cause the measurement of the activity level of the processor to be stopped, or as described elsewhere herein. In at least one embodiment, the higher-level data center processor manager 112 performs one or more operations to cause the identification of one or more clock frequencies to be applied to these processors when one or more processors in a processor group are executing a particular software program in parallel, or as described elsewhere herein.

[0057] In at least one embodiment, the higher-level data center processor manager 112 uses user-level code. In at least one embodiment, user-level code refers to a higher-level programming language used by software developers to write applications. In at least one embodiment, the lower-level computational language is closer to machine language, such as x86. In at least one embodiment, instructions written in a computational language are referred to as code. In at least one embodiment, user-level code includes code referred to as source code. In at least one embodiment, examples of user-level code include SQL, Python, Java, and C++. In at least one embodiment, user-level code abstracts hardware details, allowing developers to focus on application logic. In at least one embodiment, user-level code includes lower-level code that includes an intermediate representation (IR) at least partially used to translate the user-level code into executable code. In at least one embodiment, examples include code for representing logical or physical plans, which will be further described herein. In at least one embodiment, lower-level user-level code includes PTX code. In at least one embodiment, PTX code refers to... An intermediate representation of the GPU. In at least one embodiment, PTX code allows users to write parallel programs that can be executed on GPU hardware.

[0058] In at least one embodiment, system 100 includes a lower-level data center processor manager 114. In at least one embodiment, the lower-level data center processor manager 114 includes an API library that includes one or more APIs as described herein. In at least one embodiment, the lower-level data center processor manager 114 uses a lower-level programming language. In at least one embodiment, the lower-level data center processor manager 114 includes... System management interface (nvidiasmi or nvsmi), Radeon Pro Enterprise Edition software ROCm (Radeon Open Compute) Data Center Manager (DCM) A VTune analyzer or some combination thereof. In at least one embodiment, a higher-level data center processor manager 112 communicates with a lower-level data center processor manager 114 such that one or more operations of one or more APIs invoked by a user via user interface 104 and / or higher-level data center processor manager 112 are performed by lower-level data center processor manager 114. In at least one embodiment, lower-level data center processor manager 114 includes one or more drivers for one or more processors. In at least one embodiment, lower-level data center processor manager 114 is installed on a server and includes each driver for running each processor in the processor group. In at least one embodiment, the processor drivers are installed separately from lower-level data center processor manager 114. In at least one embodiment, one or more APIs of higher-level data center processor manager 112 invoke one or more APIs of lower-level data center processor manager 114, or as described elsewhere herein. In at least one embodiment, lower-level data center processor manager 114 performs one or more operations such that the activity level of the processor is measured, stored, computed, or some combination thereof, or as described elsewhere herein. In at least one embodiment, the lower-level data center processor manager 114 performs one or more operations to cause the measurement of the processor's activity level to be stopped, or as described elsewhere herein. In at least one embodiment, the lower-level data center processor manager 114 performs one or more operations to identify one or more clock frequencies to be applied to the processors when one or more processors in the processor group are executing a particular software program in parallel, or as described elsewhere herein.

[0059] In at least one embodiment, system 100 includes a GPU group 116. In at least one embodiment, GPU group 116 is a group of processors of any type as described herein. GPU group 116 includes one or more processor groups. In at least one embodiment, GPU group 116 includes one or more processor groups assigned by a job scheduling system to execute one or more software programs. In at least one embodiment, GPU group 116 includes one or more of the following as described herein: any type of processor, a portion of a processor, a processor of a system, or a combination of processors, including logical processors, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13AProcessor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0060] In at least one embodiment, GPU group 116 is one or more processor clusters within one or more data centers. In at least one embodiment, one or more processors in GPU group 116 are at least partially used to perform artificial intelligence (AI) training and / or inference tasks. In at least one embodiment, two or more processors within GPU group 116 execute the same software program, such as a thread, synchronously (in parallel). In at least one embodiment, a thread is a sequence of computer instructions. In at least one embodiment, two or more processors within GPU group 116 execute the same application asynchronously. In at least one embodiment, two or more processors within the processor group execute different applications asynchronously.

[0061] Figure 2 A block diagram of a process 200 executed by a system according to at least one embodiment is illustrated. The system includes one or more processors, each including one or more circuits for identifying, by using the activity level of the processor group, the clock frequency at which the processor group will operate while executing a software program, or otherwise performing any of the operations described herein. In at least one embodiment, this document incorporates… Figure 2 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figure 1 and Figures 3 to 8 The embodiments described herein are combined in one or more ways. In at least one embodiment, the system for performing one or more operations of process 200 includes... Figure 1System 100 Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0062] In at least one embodiment, the processor of the system executing one or more operations of process 200 is any type of processor, part of a processor, system processor, or combination of processors as described herein, including logical processors, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0063] In at least one embodiment, the processor executing one or more operations of process 200 performs operations used by system 100, such as operations of the processor group synchronizing API module 110 with varying workloads. In at least one embodiment, the processor executing one or more operations of process 200 performs combined operations. Figure 3The one or more operations mentioned above, such as operation 306, which acquires telemetry data from all GPUs. In at least one embodiment, the processor executing one or more operations of process 200 performs a combination of operations. Figure 4 The one or more operations mentioned above, such as executing operation 414 of the JobStartStats API. In at least one embodiment, the processor executing one or more operations of the execution process 200 is combined. Figure 5A The one or more operations mentioned above, such as operation 502 which calls the JobStartStats API. In at least one embodiment, the processor execution of one or more operations of the execution process 200 is combined. Figure 5B The one or more operations mentioned above, such as operation 508 which calls the JobGetStats API. In at least one embodiment, the processor execution of one or more operations of the execution process 200 is combined. Figure 6A The one or more operations mentioned above, such as operation 602, which calls the GetDeviceFieldValues ​​API. In at least one embodiment, the processor executing one or more operations of process 200 performs combined operations. Figure 6B The one or more operations mentioned above, such as operation 608 calling the JobStopStats API. In at least one embodiment, the processor execution of one or more operations of the execution process 200 is combined. Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor executing one or more operations of execution process 200 is combined with Figure 8 The one or more operations mentioned above, such as functions that synchronize the processor group by measuring workload changes in API 810.

[0064] In at least one embodiment, the processor initiates process 200 by performing one or more operations at operation 202 that receive input via the System Management Interface (SMI), the input instructing a GPU group to execute a balanced power profile in which the average TGP is 500 watts. In at least one embodiment, the input is via... Figure 1 User equipment 104 and / or via Figure 1 The higher-level data center management system 112 receives it. In at least one embodiment, SMI refers to a system management interface, such as... Figure 1A lower-level data center management system 114. In at least one embodiment, a balanced power profile refers to constraints applied to one or more processors in a processor group to achieve the power consumption desired by the user. In at least one embodiment, the balanced power profile includes processor constraints such as minimum and maximum power consumption levels, minimum and maximum temperature levels, minimum and maximum processor core clock frequencies, minimum and maximum memory clock frequencies, or some combination thereof.

[0065] In at least one embodiment, the processor continues execution of process 200 by performing one or more operations at operation 204, which enable the data center processor manager to collect workload factor (WF) measurement data from all GPUs running the job. In at least one embodiment, the data center processor manager system of operation 204 is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the processor's workload factor is calculated, at least in part, by the microcontroller using data measured by sensors within the processor. In at least one embodiment, the workload factor is referred to as the activity level. In at least one embodiment, operation 204 collects other metrics besides the workload factor, such as activity factor, power, leakage power, dynamic power, average power, voltage, capacitance, dynamic capacitance, temperature, processor core clock frequency, memory clock frequency, or some combination thereof.

[0066] In at least one embodiment, the workload factor is an activity value that is the product (multiplication) of the activity factor and Cdyn. In at least one embodiment, the workload factor is based at least in part on the total power of the processor detected by a sensor connected to the processor. In at least one embodiment, the workload factor is based at least in part on the analog-to-digital converter (ADC) voltage at a stable frequency. In at least one embodiment, the workload factor is dynamically calculated by subtracting leakage power from the observed total power and dividing the result by the observed voltage and frequency. In at least one embodiment, the leakage power is an estimate based at least in part on a simulated model of a particular processor.

[0067] In at least one embodiment, the processor's workload factor is calculated at least in part by the lower-level data center management module 112, the higher-level data center management module 114, or some combination thereof. In at least one embodiment, the processor's workload factor is calculated at least in part using one or more functions that use the processor's measured dynamic capacitance while executing a particular software program. In at least one embodiment, the processor performs operation 204 to calculate the average workload factor of all processors within the group over a given time period, or as described elsewhere herein.

[0068] In at least one embodiment, the processor continues execution of process 200 by performing one or more operations of the data center processor manager, which calculate one or more TGPs and / or one or more clock frequencies to be applied to each processor when the processor runs a job, or as described elsewhere herein. In at least one embodiment, the data center processor manager is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the data center processor manager uses a workload factor collected via operation 204 to calculate the TGP and / or clock frequency that processors within the group will run when executing a job. In at least one embodiment, the data center processor manager uses an average workload factor collected via operation 204 to calculate the TGP and / or clock frequency that processors within the group will run when executing a job. In at least one embodiment, the data center processor manager calculates the clock frequency of the processor core, the clock frequency of the storage device, or some combination thereof. In at least one embodiment, performing such an operation when a processor of the data center processor manager performs an operation to calculate or otherwise determine the TGP and / or clock frequency is referred to as identifying the TGP and / or clock frequency.

[0069] In at least one embodiment, the processor continues execution of process 200 by executing one or more operations at operation 208 whereby the data center processor manager sets one or more TGPs for each processor while the processor is executing a job. In at least one embodiment, the data center processor manager is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the data center processor manager inputs an instruction for a TGP via an API, such that when a processor starts or is configured to start executing a software program, the processor will run or attempt to run with the indicated TGP. In at least one embodiment, the data center processor manager sets a single TGP value for each processor in the group assigned to execute the software program.

[0070] In at least one embodiment, the processor continues execution of process 200 by performing one or more operations at operation 210 to set the clock frequency for each processor running the job by the data center processor manager. In at least one embodiment, the data center processor manager is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the data center processor manager inputs a clock frequency indication via an API, such that when a processor starts or is configured to start executing a software program, the processor will run or attempt to run at the indicated clock frequency. In at least one embodiment, the data center processor manager sets a single clock frequency value for each processor in the group assigned to execute the software program. In at least one embodiment, the clock frequency set via operation 210 is the clock frequency of the processor or processor core. In at least one embodiment, the clock frequency set via operation 210 is the clock frequency of the processor's memory device.

[0071] Figure 3 A block diagram of a process 200 executed by a system according to at least one embodiment is illustrated. The system includes one or more processors, each including one or more circuits for identifying, by using the activity level of the processor group, the clock frequency at which the processor group will operate while executing a software program, or otherwise performing any of the operations described herein. In at least one embodiment, this document incorporates… Figure 3 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figure 1 and Figure 2 as well as Figures 4 to 8 The embodiments described herein are combined in one or more aspects. In at least one embodiment, the system performing one or more operations of process 300 includes... Figure 1 System 100 Figure 2 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0072] In at least one embodiment, the processor of the system performing one or more operations of process 200 is any type of processor, part of a processor, system processor, or combination of processors as described herein, including logical processors, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0073] In at least one embodiment, the processor executing one or more operations of process 300 performs operations used by system 100, such as operations of the processor group synchronizing API module 110 with varying workloads. In at least one embodiment, the processor executing one or more operations of process 300 performs combined operations. Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency, are described. In at least one embodiment, the processor executing one or more operations of process 300 performs a combination of operations. Figure 4 The one or more operations mentioned above, such as executing operation 414 of the JobStartStats API. In at least one embodiment, the processor execution of one or more operations of the execution process 300 is combined. Figure 5A The one or more operations mentioned above, such as operation 502 which calls the JobStartStats API. In at least one embodiment, the processor execution of one or more operations of the execution process 300 is combined. Figure 5B The one or more operations mentioned above, such as operation 508 which calls the JobGetStats API. In at least one embodiment, the processor execution of one or more operations of execution process 300 is combined. Figure 6AThe one or more operations mentioned above, such as operation 602, which calls the GetDeviceFieldValues ​​API. In at least one embodiment, the processor executing one or more operations of process 300 performs combined operations. Figure 6B The one or more operations mentioned above, such as operation 608 which calls the JobStopStats API. In at least one embodiment, the processor execution of one or more operations of execution process 300 is combined. Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor executing one or more operations of execution process 300 performs in combination. Figure 8 The one or more operations mentioned above, such as functions that synchronize the processor group by measuring workload changes in API 810.

[0074] In at least one embodiment, the processor initiates process 300 by executing one or more operations at operation 302 that receive input from the data center processor manager via the System Management Interface (SMI), the input instructing a group of GPUs to execute a sync_mode policy where the TGP is 400 watts. In at least one embodiment, the input is via... Figure 1 User equipment 104 and / or via Figure 1 The higher-level data center management system 112 receives it. In at least one embodiment, SMI refers to a system management interface, such as... Figure 1 The lower-level data center management system 114. In at least one embodiment, the sync_mode policy refers to the policy by which the data center processor manager calculates the clock frequency and TGP at which each processor in the processor group will execute a particular software program.

[0075] In at least one embodiment, the processor continues process 300 by executing an operation at operation 304 where the data center processor manager sets a TGP of 400 watts for all processors running the job. In at least one embodiment, the data center processor manager is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the data center processor manager inputs an instruction for a TGP via an API, such that when a processor starts or is configured to start executing a software program, the processor will run or attempt to run with the indicated TGP. In at least one embodiment, the TGP value is input by a user via a user interface communicatively connected to the data center processor manager. In at least one embodiment, the data center processor manager sets a single TGP value for each processor in the group assigned to execute the software program.

[0076] In at least one embodiment, one or more processors continue process 300 by performing one or more operations that cause the data center processor manager to collect and calculate the average workload (WL) and average clock frequency (Clk_avg) of all GPUs within the group. In at least one embodiment, the data center processor manager collects workload metrics and clock frequencies over a given time period so that these metrics and clock frequencies can be used to calculate the average workload and average clock frequency. In at least one embodiment, the average workload is the average workload factor when all GPUs in the group execute software programs in parallel over a given time period. In at least one embodiment, the average clock frequency is the average clock frequency when each GPU in the GPU group executes software programs in parallel over a given time period. In at least one embodiment, the data center processor manager calculates the average workload and average clock frequency of the GPU group by operation 308 instead of operation 306.

[0077] In at least one embodiment, one or more processors continue process 300 by executing one or more operations at operation 308, where the data center processor manager computes the GPU group to run at its operating clock frequency. In at least one embodiment, operation 308 includes one or more algorithms executed in the mathematical layer of a higher-level data center processor manager. In at least one embodiment, one or more algorithms of operation 308 are provided by a specific set of management interfaces within the GPU, which allows for advanced system-level monitoring and control, typically used to manage power consumption, thermal throttling, and other critical aspects of GPU operation within the larger system. In at least one embodiment, the specific set of management interfaces within the GPU includes... NVML System Management Group (SSG). In at least one embodiment, a specific set of management interfaces within the GPU is implemented on a higher-level data center processor manager, a lower-level data center processor manager, a GPU driver, or some combination thereof.

[0078] In at least one embodiment, the data center processor manager operating 308 is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, operation 308 includes a data center processor manager that calculates the average workload and average clock frequency as described in operation 306. In at least one embodiment, the data center processor manager calculates the minimum clock frequency among all clock frequencies exhibited by the GPUs in the GPU group. In at least one embodiment, the data center processor manager uses the average clock frequency and the minimum clock frequency to at least partially calculate the clock frequency at which each GPU within the group will run while executing a software program, where this clock frequency is referred to as Sync_clk. In at least one embodiment, Figure 3The exemplary formula shown is used to solve for Sync_clk. In at least one embodiment, Figure 3 K0-K7 represent coefficients. In at least one embodiment, A, B, and C represent additional coefficients generated using coefficients K0-K7 at least in part. In at least one embodiment, an exemplary equation for solving Sync_clk is WL = A*Sync_clk^2 + B*Sync_clk + C, solving for Sync_clk. In at least one embodiment, calculating Sync_clk includes guardrails and checks for exceeding constraints imposed on the GPU, such as:

[0079] If (sync_clk >= Max GPCCLK) then set TGP to the target client setpoint, and check if Sync_clk is between min_clk and avg_clk.

[0080] In at least one embodiment, if the Sync_clk value is greater than Max GPCCLK, the guardrail limits the power consumption of the GPU group by setting the TGP to the user-input TGP, where Max GPCCLK refers to the maximum graphics processing cluster clock. In at least one embodiment, maximum GPCCLK is the maximum total clock frequency of the graphics processing cluster (GPC) within the GPU.

[0081] In at least one embodiment, it is checked whether the calculated Sync_clk value is between the minimum clock frequency and the average clock frequency. In at least one embodiment, a Sync_clk value higher than the average clock frequency may exceed a power consumption threshold set by the user or application. In at least one embodiment, a Sync_clk value lower than the minimum clock frequency will not improve the performance of the software program.

[0082] In at least one embodiment, one or more processors continue process 300 by performing one or more operations at operation 310 to set Sync_clk for all GPUs within the group running the job by the data center processor manager. In at least one embodiment, the data center processor manager is Figure 1 A higher-level data center processor manager 112. In at least one embodiment, the data center processor manager uses an API to input an indication (such as a value) of Sync_clk, such that when each GPU in a group begins executing a software program or is configured to execute a software program, each GPU receives or otherwise obtains Sync_clk as the clock frequency at which that GPU will run while executing the software program.

[0083] Figure 4The illustration depicts a system 400 according to at least one embodiment, the system including one or more processors, each processor including one or more circuitry for identifying, by using the activity level of the processor group, the clock frequency at which the processor group will operate while executing a software program, or otherwise performing any of the operations described herein. In at least one embodiment, this document incorporates... Figure 4 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1-3 and Figures 5A-8 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 400 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0084] In at least one embodiment, the processor of system 400 is any type of processor, part of a processor, system processor, or combination of processors as described herein, including a logic processor, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0085] In at least one embodiment, the processor of system 400 performs operations used by system 100, such as operations of the API module 110 using a processor group that synchronizes with varying workloads. In at least one embodiment, the processor of system 400 performs combined... Figure 2 The one or more operations, such as operation 204, which calculates TGP and clock frequency. In at least one embodiment, the processor of system 400 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 400 performs the combination... Figure 5A The one or more operations mentioned above, such as operation 502 which calls the JobStartStats API. In at least one embodiment, the processor of system 400 performs the combined... Figure 5B The one or more operations mentioned above, such as operation 508 calling the JobGetStats API. In at least one embodiment, the processor of system 400 performs the combination Figure 6A The one or more operations mentioned above, such as operation 602, which calls the GetDeviceFieldValues ​​API. In at least one embodiment, the processor of system 400 performs a combination of... Figure 6B One or more operations, such as operation 608, which calls the JobStopStats API. In at least one embodiment, the processor of system 400 performs a combination of Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor of system 400 performs the combination Figure 8 The one or more operations mentioned above.

[0086] In at least one embodiment, system 400 includes a user interface (UI) 404. In at least one embodiment, UI 404 includes... Figure 1 At least a portion of, or itself is, user equipment 104. Figure 1At least a portion of the user equipment 104. In at least one embodiment, the user invokes one or more APIs described herein by typing the names of one or more APIs via UI 404. In at least one embodiment, the user invokes one or more APIs by entering the names of one or more APIs into the command line of UI 404. In at least one embodiment, the user invokes the API via operation 412 to cause a higher-level data center manager 406 to initiate measurement and storage of the activity level of each processor in the processor group at given intervals, or as described elsewhere herein. In at least one embodiment, the higher-level processor manager 406 includes Figure 1 It is part of, or at least part of, the higher-level processor manager 112.

[0087] In at least one embodiment, the API invoked via operation 412 is named JobStartStats or jobstartstats for illustrative purposes. In at least one embodiment, the details of the API invoked via operation 412 are described below with code and comments:

[0088]

[0089]

[0090] In at least one embodiment, an API (such as JobStartStats) invoked via operation 412 facilitates the user's notification to the DCGM of the job to be started. In at least one embodiment, the API invocation is performed as part of the job preamble. In at least one embodiment, the parameter pDcgmHandle is used as input representing a DCGM handle. In at least one embodiment, the DCGM handle indicates an instance of the Data Center Processor Manager. In at least one embodiment, another parameter groupId is used as input identifying a collection or group of one or more GPUs, with more details available via an API called dcgmGroupCreate. In at least one embodiment, passing a group ID as DCGM_GROUP_ALL_GPUS allows operations to be performed on all GPUs. In at least one embodiment, the parameter jobId is used as input where the user provides a string to identify the job to be executed on the GPU group. In at least one embodiment, the parameter jobStatPolicy optionally provides job statistics (stat) settings, such as which statistics to measure and store. In at least one embodiment, jobStatPolicy allows the user to input a type of metric (such as activity level) for measurement and storage, and for calculating the Sync_clk value. In at least one embodiment, the return value of DCGM_ST_OK indicates a successful call. In at least one embodiment, the return value of DCGM_ST_BADPARAM indicates an invalid parameter. In at least one embodiment, the return value of DCGM_ST_DUPLICATE_KEY indicates that the specified jobId is already in use.

[0091] In at least one embodiment, in response to a call to an API of a higher-level data center processor manager 406, the processor of the higher-level data center processor manager 406 performs an operation of that API via operation 414. In at least one embodiment, one or more operations of operation 414 include the higher-level data center processor manager 406 calling an API of a lower-level data center processor manager 408. In at least one embodiment, the higher-level data center processor manager 406 repeatedly calls the API of the lower-level data center processor manager 408 to obtain an activity level measurement (such as a workload factor) for each GPU in the GPU group, as indicated by calling an API (such as JobStartStats) at operation 414. In at least one embodiment, the higher-level data center processor manager 406 repeatedly calls the API of the lower-level data center processor manager 408 to obtain, at regular intervals, activity level measurements (such as a workload factor) as indicated by calling an API (such as JobStartStats) at operation 414. In at least one embodiment, the higher-level data center processor manager 406 repeatedly calls the API (such as DeviceGetFieldValues) of the lower-level data center processor manager 408 to obtain, at regular intervals, activity level measurements (such as workload factors) indicated by structures (such as dcgmJobStatPolicy_v1) further described herein.

[0092] In at least one embodiment, a data structure such as jobStatPolicy is defined before the API is invoked via operation 412. In at least one embodiment, the definition of such a data structure is detailed using the following code and comments:

[0093]

[0094] In at least one embodiment, the typedef enumeration `dcgmJobStatPolicy_enum` defines job stat policies. In at least one embodiment, `DCGM_JOB_STAT_NONE` represents a policy with no specific job statistics. In at least one embodiment, `DCGM_JOB_STAT_MULTI_GPU_CLOCK_SYNC` indicates a policy for synchronizing the clocks of multiple GPUs within a group. In at least one embodiment, the enumeration is named `dcgmJobStatPolicy_t`. In at least one embodiment, the structure `dcgmJobStatPolicy_v1` includes multiple fields. In at least one embodiment, the unsigned integer `version` specifies the API version number. In at least one embodiment, the `statPolicy` field of type `DcgmJobStatPolicy_t` specifies a specific job stat policy. In at least one embodiment, the unsigned integer `jobGPUCount` indicates the total number of GPUs assigned to a job across all nodes. In at least one embodiment, the unsigned integer `syncFrequency` specifies the time interval in seconds between two applications of a specified job policy.

[0095] In at least one embodiment, the higher-level data center processor manager 406 repeatedly invokes the API (such as DeviceGetFieldValues) of the lower-level data center processor manager 408. In at least one embodiment, when invoked, the lower-level data center manager 408 performs the operation of the API (such as DeviceGetFieldValues) to measure the activity level of each GPU in the GPU group via operation 416. In at least one embodiment, the lower-level data center manager 408 performs the operation of the API (such as DeviceGetFieldValues) to communicate with each driver of each GPU in the GPU group assigned to execute software programs.

[0096] In at least one embodiment, execution of an API (such as DeviceGetFieldValues) causes each GPU driver to return a processor performance metric as indicated by the API via operation 420. In at least one embodiment, the processor performance metric returned via operation 420 is sent to a lower-level data center manager 408 for storage and use in calculating a workload factor. In at least one embodiment, the performance metric returned via operation 420 is collected from processors in processor group 411. In at least one embodiment, the performance metric returned via operation 420 includes capacitance values ​​or dynamic capacitance values ​​measured over a given interval as indicated by an API or data structure as further described herein. In at least one embodiment, the lower-level data center processor manager 408 uses these capacitance values ​​to calculate a workload factor when the performance metric is received via operation 420 or otherwise obtained. In at least one embodiment, each GPU driver in processor driver 410, not the lower-level data center manager 408, uses the performance metric returned via operation 420 to calculate the workload factor for each GPU. In at least one embodiment, the performance metric returned via operation 420 is sent to a higher-level data center processor manager 406 for storage and use in calculating a workload factor.

[0097] In at least one embodiment, the API (such as DeviceGetFieldValues) is described using code and comments as follows:

[0098]

[0099]

[0100] In at least one embodiment, the API requests values ​​from a list of fields of the device, enabling simultaneous querying of multiple fields. In at least one embodiment, if any underlying fieldId is populated by the same driver call, the results for those fieldIDs are derived from a single call, rather than a separate driver call for each fieldId. In at least one embodiment, the fieldID is an indication of the type of metric (such as workload factor) to be measured on each processor in the processor group.

[0101] In at least one embodiment, the parameter `device` represents a device handle to a specific GPU within the group for which the field values ​​are requested. In at least one embodiment, the parameter `valuesCount` specifies the number of entries among the values ​​to be retrieved. In at least one embodiment, the parameter `values` is an array of structures, where each structure stores a field value, and the `fieldId` of each value must be populated prior to this call.

[0102] In at least one embodiment, the return value of NVML_SUCCESS indicates that all values ​​in the array have been filled, although the individual state requires checking the nvmlReturn field for each value. In at least one embodiment, the return value of NVML_ERROR_INVALID_ARGUMENT indicates that the device is invalid or the values ​​parameter is NULL.

[0103] In at least one embodiment, a user invokes an API (e.g., JobGetStats) via operation 420, causing a higher-level data center manager 406 to perform operations to return statistics related to performance metrics, which are collected in part by invoking an API (e.g., JobStartStats) via operation 412. In at least one embodiment, in response to the API call at operation 420, the higher-level data center manager 406 performs one or more operations of that API at operation 422 to access data storage of activity levels, processor metrics, or some combination thereof. In at least one embodiment, the activity level of the storage includes workload factor, average workload factor, or some combination thereof.

[0104] In at least one embodiment, the processor generates statistical data by using dynamic capacitance measurements of the processor. In at least one embodiment, utilizing dynamic capacitance measurements enables the processor to generate various statistical data reflecting the processor's performance, efficiency, and reliability. In at least one embodiment, the statistical data includes values ​​related to power consumption, energy efficiency, switching activity, thermal distribution, high capacitance variation, voltage scaling efficiency, frequency response, or some combination thereof.

[0105] In at least one embodiment, analyzing the processor's dynamic capacitance measurements can estimate power consumption under different workloads. In at least one embodiment, measuring energy efficiency can reveal the processor's energy usage efficiency, typically expressed as performance per watt. In at least one embodiment, observing switching activity can indicate the frequency and intensity of processor state changes, which affect power consumption and heat generation. In at least one embodiment, understanding heat distribution through dynamic capacitance can highlight areas requiring enhanced cooling solutions. In at least one embodiment, identifying high capacitance variations can pinpoint potential performance bottlenecks. In at least one embodiment, high dynamic capacitance represents stress on components, which affects long-term reliability. In at least one embodiment, measuring voltage scaling efficiency can assess the processor's ability to maintain performance at different voltage levels. In at least one embodiment, analyzing frequency response through dynamic capacitance helps optimize processor clock speeds for various tasks.

[0106] In at least one embodiment, the higher-level data center manager 406 performs one or more operations of the API invoked via operation 420 to return, send, transmit, display, or otherwise output the activity level to the user via UI 404 through operation 424. In at least one embodiment, the higher-level data center manager 406 performs one or more operations of the API invoked via operation 420 to return, send, transmit, display, or otherwise output information related to the activity level to the user via UI 404, wherein the information includes statistics regarding maximum, minimum, or average activity levels, power consumption, current TGP, the number of GPUs being measured, or some combination thereof.

[0107] In at least one embodiment, the API (such as JobGetStats) is described using code and comments as follows:

[0108]

[0109] In at least one embodiment, an API such as JobGetStats retrieves statistics for a job identified by a job ID generated by the data center processor manager, and this statistics are accessible at any point during job processing. In at least one embodiment, the data center processor manager is... DCGM. In at least one embodiment, to reuse the jobId, dcgmJobRemove is called after this call.

[0110] In at least one embodiment, the parameter pDcgmHandle is used as input representing a DCGM handle. In at least one embodiment, the DCGM handle is an identifier for an instance of the Data Center Processor Manager. In at least one embodiment, the parameter jobId is used as input, where the user provides a string to identify the job. In at least one embodiment, the parameter pJobInfo is used as both input and output, returning information about the identified job, whose version is set to dcgmJobInfo_version prior to this call.

[0111] In at least one embodiment, the return value DCGM_ST_OK indicates a successful call. In at least one embodiment, the return value DCGM_ST_BADPARAM indicates an invalid parameter. In at least one embodiment, the return value DCGM_ST_NO_DATA indicates that the jobId is not a valid job identifier. In at least one embodiment, the return value DCGM_ST_VER_MISMATCH indicates that the version is not set or is invalid.

[0112] In at least one embodiment, a user invokes an API (such as JobStopStats) via operation 426, causing a higher-level data center manager 406 to perform an operation to stop the measurement and / or storage of the activity levels of individual GPUs in the GPU group. In at least one embodiment, in response to the API call via operation 426, the higher-level data center manager 406 performs an operation of the API to calculate or otherwise identify a Sync_clk value that all GPUs in the group will run at when the software program is executed. In at least one embodiment, the higher-level data center manager 406 performs an API operation such as JobStopStats to access data storage of the overall average workload factor of the GPU group over one or more given time periods while the GPU group is executing the software program.

[0113] In at least one embodiment, the API (such as JobStopStats) is described using code and comments as follows:

[0114]

[0115]

[0116] In at least one embodiment, an API such as JobStopStats allows the client to notify DCGM to stop collecting statistics for the job represented by the job ID; the call occurs as part of the job's end. In at least one embodiment, the job ID can still be used to view statistics at any time, but cannot be reused to start a new job. In at least one embodiment, an API (e.g., dcgmWatchJobFields()) must be called before this API to enable measurement of the activity level of the GPU executing the job. In at least one embodiment, the parameter pDcgmHandle is used as input representing a DCGM handle. In at least one embodiment, the parameter jobId is used as input, where the user provides a string to identify the job.

[0117] In at least one embodiment, the return value DCGM_ST_OK indicates a successful call. In at least one embodiment, the return value DCGM_ST_BADPARAM indicates an invalid parameter. In at least one embodiment, the return value DCGM_ST_NO_DATA indicates that the jobId is not a valid job identifier.

[0118] In at least one embodiment, a processor of the higher-level data center processor manager 406 performs operations of an API such as JobStopStats to calculate or otherwise identify the clock frequency (such as Sync_clk) at which one or more processors in the processor group will run when executing the identified job. In at least one embodiment, processor driver 410, one or more drivers of the lower-level data center manager 408, or some combination thereof perform operations to calculate or otherwise identify the clock frequency (such as Sync_clk) at which one or more processors in the processor group will run when executing the identified job. In at least one embodiment, processor driver 410, one or more drivers of the lower-level data center manager 408, the higher-level data center manager 406, or some combination thereof perform operations to calculate or otherwise identify the clock frequency (such as Sync_clk) at which one or more processors in the processor group will run when executing the identified job.

[0119] In at least one embodiment, the higher-level data center processor manager 406 performs an API operation such as JobStopStats to return, send, transmit, display, or otherwise output via UI 404 the clock frequency (such as Sync_clk) at which one or more processors in the processor group will operate when performing the identified job via operation 430. In at least one embodiment, the user inputs an indication of the clock frequency returned via operation 430, a job identifier, a GPU group identifier, or some combination thereof into an API called to set the clock frequency at which one or more processors in the processor group will operate when performing the identified job via operation 432. In at least one embodiment, in response to the API call via operation 432, the higher-level data center manager 408 performs an operation to set the clock frequency at which one or more processors in the processor group will operate when performing the identified job via operation 434.

[0120] Figure 5A The illustration depicts a system 500 according to at least one embodiment, the system including one or more API calls that, when executed by a processor, cause one or more circuits of the processor to identify, at least in part, the clock frequency at which the processor group will operate while executing a software program, or otherwise perform any of the operations described herein, by using the activity level of the processor group. In at least one embodiment, this document incorporates... Figure 5A One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1-4 and Figures 5B-8The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 500 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0121] In at least one embodiment, the processor of system 500 is any type of processor, part of a processor, system processor, or combination of processors described herein, including logic processors, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0122] In at least one embodiment, the processor of system 500 performs operations used by system 100, such as operations of the API module 110 using a processor group with varying workloads. In at least one embodiment, the processor of system 500 performs combined... Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency. In at least one embodiment, the processor of system 500 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 500 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 calling the JobStartStats API. In at least one embodiment, the processor of system 500 performs the combined... Figure 5B The one or more operations mentioned above, such as operation 508 calling the JobGetStats API. In at least one embodiment, the processor of system 500 performs the combined... Figure 6A The one or more operations mentioned above, such as operation 602, which calls the GetDeviceFieldValues ​​API. In at least one embodiment, the processor of system 500 performs the combination Figure 6B The one or more operations mentioned above, such as operation 608 calling the JobStopStats API. In at least one embodiment, the processor of system 500 performs the combined... Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor of system 500 performs the combination Figure 8 The one or more operations mentioned above.

[0123] In at least one embodiment, JobStartStats API call 502 is a... Figure 1 The call to one or more APIs of the processor group synchronization API module 110 is used to control workload changes. In at least one embodiment, the JobStartStats API call 502 is using... Figure 4Operation 412 is a call to the API. In at least one embodiment, JobStartStats API call 502 is used (e.g., by a user, application, or library) to receive one or more of the following parameters: DCGM handle, processor group identifier (groupID), job identifier (jobID), job statistics policy, or any combination thereof, or as described elsewhere herein. In at least one embodiment, JobStartStats API call 502 is a call to an API function in an API library used as part of a data center processor management system. In at least one embodiment, the API function is referred to as an API command. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as inputs. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as indications. In at least one embodiment, parameters received according to JobStartStats API call 502 are referred to as prompts.

[0124] In at least one embodiment, the JobStartStats API response 504 includes one or more calls to another API to obtain activity level measurements, or as at least in combination with Figure 4 As described separately. In at least one embodiment, the JobStartStats API response 504 returns an indication of whether the JobStartStats API call 502 was successful, an indication of whether the parameters input to the JobStartStats API were invalid, an indication of whether the identified job is in use, or some combination thereof, or as at least in combination herein. Figure 4 As described separately.

[0125] In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or as described elsewhere herein. In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more activity levels of one or more processors to be measured, thereby identifying one or more processors at one or more clock frequencies to operate at, or as otherwise described herein. In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more activity levels of one or more processors to be measured, the measurement being based at least in part on one or more indications of one or more processor groups comprising one or more processors, or as described elsewhere herein. In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more activity levels of one or more processors to be measured, the measurement being based at least in part on one or more indications of one or more instances of processor management software, or as described elsewhere herein. In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more activity levels of one or more processors to be measured, the measurement being based at least in part on one or more indications of one or more types of activity levels to be measured, or as described elsewhere herein. In at least one embodiment, the processor of system 500 executes JobStartStats API call 502 and / or JobStartStats API response 504 to cause one or more processors to concurrently execute one or more software programs as part of one or more data centers.

[0126] Figure 5B The illustration depicts a system 506 according to at least one embodiment, which includes one or more API calls that, when executed by a processor, at least partially cause one or more circuits of the processor to identify, by using the activity level of the processor group, that the processor group will perform any of the operations described herein at its operating clock frequency while executing a software program, or otherwise. In at least one embodiment, this document incorporates... Figure 5B One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1 to 5A and Figures 6A to 8 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 506 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 6A System 600 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of the system, or is itself at least a part of the system.

[0127] In at least one embodiment, the processor of system 506 is any type of processor, part of a processor, processor of a system, or combination of processors as described herein, including a logic processor, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0128] In at least one embodiment, the processor of system 506 performs operations used by system 100, such as operations of the API module 110 using a processor group with varying workloads. In at least one embodiment, the processor of system 506 performs combined... Figure 2 The one or more operations, such as operation 204, which calculates TGP and clock frequency. In at least one embodiment, the processor of system 506 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 506 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 for invoking the JobStartStats API. In at least one embodiment, the processor of system 506 performs a combination of Figure 5B The one or more operations mentioned above, such as operation 508 for calling the JobGetStats API. In at least one embodiment, the processor of system 506 performs the combination Figure 6A The one or more operations mentioned above, such as operation 602 for calling the GetDeviceFieldValues ​​API. In at least one embodiment, the processor of system 506 performs the combination Figure 6B The one or more operations mentioned above, such as operation 608 for invoking the JobStopStats API. In at least one embodiment, the processor of system 506 performs a combination of Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor of system 506 performs the combined... Figure 8 The one or more operations mentioned above.

[0129] In at least one embodiment, JobGetStats API call 508 is a... Figure 1 The call to one or more APIs of the processor group synchronization API module 110 is used to handle workload changes. In at least one embodiment, the JobGetStats API call 508 is using... Figure 4Operation 420 is a call to the API. In at least one embodiment, JobGetStats API call 508 is used (e.g., by a user, application, or library) to receive one or more of the following parameters: a DCGM handle, a job identifier (jobID), a data structure for returning job information, or some combination thereof, or as described elsewhere herein. In at least one embodiment, JobGetStats API call 508 is a call to an API function in an API library that is part of a data center processor management system. In at least one embodiment, the API function is referred to as an API command. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as inputs. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as indications. In at least one embodiment, parameters received according to the data center processor management system or used as system-related parameters are referred to as prompts.

[0130] In at least one embodiment, the JobGetStats API response 510 includes an indication of whether the JobGetStats API call 508 was successful, an indication of whether the parameters input to the JobGetStats API are invalid, an indication of whether the identified job is invalid or in use, an indication of whether the version of the data structure is invalid, and an indication of the data structure used to return job information, or as at least in combination herein. Figure 4 As described separately.

[0131] In at least one embodiment, the processor of system 506 executes JobGetStats API call 508 and / or JobGetStats API call 510 to indicate to one or more users one or more statistical data corresponding to one or more activity levels of one or more processors, or as described elsewhere herein. In at least one embodiment, the processor of system 506 executes JobGetStats API call 508 and / or JobGetStats API call 510 to indicate that one or more activity levels of one or more processors are used to identify one or more clock frequencies at which one or more processors should operate, or as described elsewhere herein. In at least one embodiment, the processor of system 506 executes JobGetStats API call 508 and / or JobGetStats API call 510 to indicate to one or more users, at least in part, based on one or more indications from those one or more processors, one or more statistical data corresponding to one or more activity levels of one or more processors, or as described elsewhere herein. In at least one embodiment, the processor of system 506 executes JobGetStats API call 508 and / or JobGetStats API call 510 to indicate, at least in part, to one or more users, one or more statistical data corresponding to one or more activity levels of those one or more processors, or as described elsewhere herein, based on one or more indications of one or more types of activity levels to be measured. In at least one embodiment, the processor of system 506 executes JobGetStatsAPI call 508 and / or JobGetStatsAPI call 510 to indicate to one or more users, at least in part, one or more statistics corresponding to one or more activity levels of one or more processors, or as described elsewhere herein, based on one or more indications of one or more types of activity levels to be measured.In at least one embodiment, one or more processors of system 506 execute JobGetStatsAPI call 508 and / or JobGetStatsAPI call 510 to cause one or more processors to concurrently execute one or more software programs as part of one or more data centers.

[0132] Figure 6A The illustration depicts a system 600 according to at least one embodiment, the system including one or more API calls that, when executed by a processor, cause one or more circuits of the processor to identify, at least in part, the clock frequency at which the processor group will operate while executing a software program, or otherwise perform any of the operations described herein, by using the activity level of the processor group. In at least one embodiment, this document incorporates... Figure 5A One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1 to 5B and Figures 6B to 8 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 600 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6B System 606 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0133] In at least one embodiment, the processor of system 600 is any type of processor, part of a processor, processor of a system, or combination of processors as described herein, including a logic processor, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0134] In at least one embodiment, the processor of system 600 performs operations used by system 100, such as operations of the API module 110 using a processor group that synchronizes with varying workloads. In at least one embodiment, the processor of system 600 performs combined... Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency. In at least one embodiment, the processor of system 600 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 600 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 for invoking the JobStartStats API. In at least one embodiment, one or more processors of system 600 perform the combined... Figure 5A One or more operations, such as JobStartStats API call 512. In at least one embodiment, the processor of system 600 performs a combination of Figure 5B The one or more operations mentioned above, such as operation 508 for calling the JobGetStats API. In at least one embodiment, the processor of system 600 performs the combination Figure 6B The one or more operations mentioned above, such as operation 608 for invoking the JobStopStats API. In at least one embodiment, the processor of system 600 performs the combination Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor of system 600 performs the combination Figure 8 The one or more operations mentioned above.

[0135] In at least one embodiment, DeviceGetFieldValues ​​API call 602 is a... Figure 1The processor group uses the synchronous API module 110 to call one or more APIs that vary with workload. In at least one embodiment, the DeviceGetFieldValues ​​API call 602 is using... Figure 4 Operation 414 is a call to the API. In at least one embodiment, DeviceGetFieldValues ​​API call 602 is used (e.g., by a user, application, or library) to receive one or more of the following parameters: a DCGM handle, the number of entries for each field value to be retrieved, an indication of the data structure used to store the field values, or any combination thereof, or as described elsewhere herein. In at least one embodiment, DeviceGetFieldValues ​​API call 602 is a call to an API function of an API library used as part of a data center processor management system. In at least one embodiment, the API function is referred to as an API command. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as inputs. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as indications. In at least one embodiment, parameters received by or used as system-related parameters according to the data center processor management system are referred to as prompts.

[0136] In at least one embodiment, the processor executes a DeviceGetFieldValues ​​API response 604 to return an indication of whether any field value was successfully populated, an indication of whether a particular GPU is invalid, and an indication of a data structure populated with field values, or any combination thereof, or as at least elsewhere herein. Figure 4 Described.

[0137] In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to indicate to one or more users one or more activity levels of one or more processors, or as described elsewhere herein. In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to indicate that one or more activity levels of one or more processors are used to identify the clock frequencies at which one or more processors will operate, or as described elsewhere herein. In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to indicate to one or more users, at least in part, the activity levels of those one or more processors, or as described elsewhere herein. In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to indicate, at least in part, to one or more users, one or more activity levels of one or more processors based on one or more indications of one or more instances of processor management software, or as described elsewhere herein.

[0138] In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to indicate, at least in part, one or more activity levels of one or more processors to one or more users based on one or more indications of the activity levels of one or more types to be measured, or as described elsewhere herein. In at least one embodiment, the processor of system 600 executes DeviceGetFieldValues ​​API call 602 and / or DeviceGetFieldValues ​​API response 604 to cause one or more activity levels of one or more processors to be used to cause those one or more processors to concurrently execute one or more software programs as part of one or more data centers, or as described elsewhere herein.

[0139] Figure 6B The illustration depicts a system 606 according to at least one embodiment, the system including one or more API calls that, when executed by a processor, cause one or more circuits of the processor to identify, at least in part, the clock frequency at which the processor group will operate while executing a software program, or otherwise perform any of the operations described herein, by using the activity level of the processor group. In at least one embodiment, this document incorporates... Figure 6B One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1 to 6A as well as Figure 7 and Figure 8 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 600 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 7 System 700, Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0140] In at least one embodiment, the processor of system 606 is any type of processor, part of a processor, processor of a system, or combination of processors described herein, including a logic processor, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11AParallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0141] In at least one embodiment, the processor of system 606 performs operations used by system 100, such as operations of the API module 110 using a processor group with varying workloads. In at least one embodiment, the processor of system 606 performs combined... Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency, are performed by the processor of system 606 in at least one embodiment. Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 606 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 for invoking the JobStartStats API. In at least one embodiment, one or more processors of system 606 perform the combined... Figure 5A One or more operations, such as JobStartStats API call 512. In at least one embodiment, the processor of system 606 performs a combination of Figure 5B The one or more operations mentioned above, such as operation 508 for calling the JobGetStats API. In at least one embodiment, the processor of system 606 performs the combination Figure 6AThe one or more operations mentioned above, such as operation 602 for invoking the DeviceGetStats API. In at least one embodiment, the processor of system 606 performs a combination of... Figure 7 The one or more operations mentioned above, such as operations of the API of software library 706. In at least one embodiment, the processor of system 606 performs the combined... Figure 8 The one or more operations mentioned above.

[0142] In at least one embodiment, JobStopStats API call 608 is a... Figure 1 The call to one or more APIs of the processor group synchronization API module 110 is used to control workload changes. In at least one embodiment, the JobStopStats API call 608 is using... Figure 4 Operation 426 is a call to the API. In at least one embodiment, JobStopStats API call 608 is used (e.g., by a user, application, or library) to receive one or more of the following parameters: a DCGM handle, a job identifier (jobID), or some combination thereof, or as described elsewhere herein. In at least one embodiment, JobStopStats API call 608 is a call to an API function of an API library used as part of a data center processor management system. In at least one embodiment, the API function is referred to as an API command. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as inputs. In at least one embodiment, parameters received by the API or otherwise obtained are referred to as indications. In at least one embodiment, parameters received by or used as system-related parameters according to the data center processor management system are referred to as prompts.

[0143] In at least one embodiment, the processor executes the JobStopStats API response 610 to return an indication of whether the JobStopStats API call 608 was successful, an indication of whether the parameters input to the JobStopStats API call 608 were invalid, an indication of whether the identified job was invalid, or some combination thereof, or as at least in combination herein. Figure 4 As described separately. In at least one embodiment, the processor executes the JobStopStats API response 610 to perform operations such as stopping the collection of measurements of the processor group's activity level, calculating the clock frequency, outputting the clock frequency, or some combination thereof, or as described herein in at least the following combination. Figure 4 As described separately.

[0144] In at least one embodiment, the processor of system 606 executes a JobStopStats API call 608 and / or a JobStopStats API response 610 to cause one or more measurements of one or more activity levels of one or more processors to be stopped, or as described elsewhere herein.

[0145] In at least one embodiment, the processor of system 606 executes JobStopStats API call 608 and / or JobStopStats API response 610 such that one or more activity levels of one or more processors are used to identify which one or more processors are to operate at one or more clock frequencies, or as described elsewhere herein.

[0146] In at least one embodiment, the processor of system 606 executes a JobStopStats API call 608 and / or a JobStopStats API response 610 such that one or more measurements of one or more activity levels of one or more processors are stopped at least in part based on one or more indications from that one or more processors, or as described elsewhere herein.

[0147] In at least one embodiment, the processor of system 606 executes a JobStopStats API call 608 and / or a JobStopStats API response 610 such that measurements of one or more activity levels of one or more processors are based at least in part on one or more indications from one or more instances of the processor management software being stopped, or as described elsewhere herein.

[0148] In at least one embodiment, the processor of system 606 executes a JobStopStats API call 608 and / or a JobStopStats API response 610 such that one or more measurements of one or more activity levels of one or more processors are stopped at least in part based on one or more indications of the activity level of one or more types to be measured.

[0149] In at least one embodiment, the processor of system 606 executes JobStopStats API call 608 and / or JobStopStats API response 610 to cause one or more activity levels of one or more processors to be used to concurrently execute one or more software programs as part of one or more data centers, or as described elsewhere herein.

[0150] In at least one embodiment, the processor of system 606 executes a JobStopStats API call 608 and / or a JobStopStats API response 610 such that one or more measurements of one or more activity levels of one or more processors are based at least in part on one or more indications of one or more software programs to be executed by one or more processors being stopped, or as described elsewhere herein.

[0151] Figure 7 The illustration depicts a system 700 according to at least one embodiment, the system including one or more API calls that, when executed by a processor, cause one or more circuits of the processor to identify, at least in part, the clock frequency at which the processor group will operate while executing a software program, or otherwise perform any of the operations described herein, by using the activity level of the processor group. In at least one embodiment, this document incorporates... Figure 7 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1 to 6B and Figure 8 The embodiments described herein are combined in one or more aspects. In at least one embodiment, the processor of the execution system 700 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 8 The system 800 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0152] In at least one embodiment, the processor of system 700 is any type of processor, part of a processor, system processor, or combination of processors as described herein, including a logic processor, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU2590, Figure 26 The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0153] In at least one embodiment, the processor of system 700 performs operations used by system 100, such as operations of the API module 110 using a processor group that synchronizes with varying workloads. In at least one embodiment, the processor of system 700 performs combined... Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency. In at least one embodiment, the processor of system 700 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 700 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 for invoking the JobStartStats API. In at least one embodiment, the processor of system 700 performs the combination Figure 5A One or more operations, such as JobStartStats API call 512. In at least one embodiment, the processor of system 700 performs a combination of Figure 5B The one or more operations mentioned above, such as operation 508 for calling the JobGetStats API. In at least one embodiment, the processor of system 700 performs the combination Figure 6A The one or more operations mentioned above, such as operation 602 for invoking the DeviceGetStats API. In at least one embodiment, the processor of system 700 performs the combination Figure 6B The one or more operations mentioned above, such as operation 608 for invoking the JobStopStats API. In at least one embodiment, the processor of system 700 performs the combination Figure 8 The one or more operations mentioned above.

[0154] System 700 includes software and hardware according to at least one embodiment for determining, by utilizing the activity level of a processor group, the clock frequency at which the processor group will operate while executing a software program, or otherwise performing any of the operations described herein. In at least one embodiment, system 700 includes software and hardware for executing an API to cause the activity level of other processors to be measured at one or more indicated intervals, or otherwise performing any of the operations described herein. In at least one embodiment, system 700 includes software and hardware for executing an API to cause one or more measurements of one or more activity levels of other processors to be stopped, or otherwise performing any of the operations described herein. In at least one embodiment, system 700 includes software and hardware for executing an API to cause one or more activity levels of other processors to be indicated to one or more users, or otherwise performing any of the operations described herein. In at least one embodiment, system 700 includes software and hardware according to at least one embodiment for executing an API to cause one or more users to be indicated to one or more users one or more statistics corresponding to one or more activity levels of one or more processors, or otherwise performing any of the operations described herein.

[0155] System 700 may include storage device 702 and processor 708. Storage device 702 may include, for example, memory, cache, or other storage devices as further described herein. Storage device 702 may be separate from processor 708, or storage device 702 may be included in processor 708 (e.g., in storage device 712). In at least one embodiment, software program 704 and / or software library (or instructions) 706 may be stored in memory, cache, or other storage devices and provided to processor 708 to cause one or more circuits of processor 708 to perform the operations as described herein. In at least one embodiment, software program 704 and / or software library (or instructions) 706 may be integrated into one or more circuits of processor 708. Software program 704, which can be used to perform any of the operations described herein, may be stored on storage device 702.

[0156] In at least one embodiment, the software program 704 may include one or more software modules. In at least one embodiment, the software program 704 includes... Figure 1 At least a portion of the processor group synchronization API module 110 is used to handle workload variations. In at least one embodiment, the software program 704 includes... Figure 1 At least a portion of the higher-level data center processor manager 112, at least a portion of the lower-level data center processor manager 114, or some combination thereof.

[0157] In at least one embodiment, as used in any implementation described herein, unless the context explicitly states otherwise, a module refers to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. In at least one embodiment, software is embodied as a software package, code, and / or instruction set or instructions, and "hardware" as used in any implementation described herein includes, for example, hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, and / or firmware storing instructions executed by programmable circuitry, either individually or in any combination. In at least one embodiment, modules are collectively or individually embodied as circuitry forming part of a larger system, such as an integrated circuit (IC), a system-on-a-chip (SoC), etc. In at least one embodiment, a module performs one or more processes in conjunction with any suitable processing unit and / or combination of processing units (such as one or more CPUs, GPUs, GPGPUs, PPUs, and / or variants thereof, including those further described herein).

[0158] In at least one embodiment, software program 704 may include a collection of software code, commands, instructions, or other text sequences for instructing a computing device to perform one or more computational operations and / or invoke one or more other instruction sets, such as APIs or API functions or instruction set architecture (ISA) level instructions, to be executed or implemented. In at least one embodiment, software program 704 includes an API as described herein, which is used to identify the clock frequency at which the processor group will operate by using measurements of the activity level, or as described elsewhere herein. Instructions (e.g., hardware instructions) or microcode may relate to ISA-level instructions, which may include native ISA instructions or non-native ISA instructions. Software program 704 and / or software library (or instruction) 706 (e.g., one or more modules) may be distributed among multiple processors communicating via buses, networks, writes to shared memory, and / or any suitable communication procedures (such as those described herein).

[0159] In at least one embodiment, system 700 may include one or more software libraries 706, which may, for example, provide one or more API and / or ISA instructions. In at least one embodiment, one or more API and / or ISA instructions may be used to identify the clock frequency at which the processor group will operate, or as described elsewhere herein, by using measurements of the activity level. In at least one embodiment, one or more software libraries 706 may be included in a driver and / or runtime. In at least one embodiment, software library 706 (e.g., which includes one or more API and / or ISA instructions) may include a set of software instructions that, if executed or otherwise implemented, cause processor 708 to perform one or more computational operations, such as any operations described herein. In at least one embodiment, one or more API and / or ISA instructions may be distributed or otherwise provided as part of one or more software libraries 706, runtime, driver, and / or any other grouping of software and / or executable code as further described herein. In at least one embodiment, one or more API and / or ISA instructions may perform one or more computational operations in response to a call by software program 704.

[0160] Processor 708 may include any number of processors and any suitable processing units and / or combinations of processing units, such as, but not limited to, a central processing unit (“CPU”), a graphics processing unit (“GPU”), or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, parallel processors, GPGPUs, DPUs, and / or variations thereof, including those further described herein), including any processors as described herein, such as, but not limited to, those described herein. Figure 10-22BThe processor 708 is a processor in a memory device 702. In at least one embodiment, the processor 708 may retrieve or fetch instructions (e.g., one or more API and / or ISA instructions) from the memory device 702, for example, using instruction fetch 716 (e.g., for an instruction fetch phase). The instructions may include instructions for identifying the clock frequency at which the processor group will operate by using measurements of the activity level, or as described elsewhere herein. In at least one embodiment, the processor 708 may include a memory device 712 and an instruction queue 710 for storing and queuing instructions fetched from the memory device 702. In at least one embodiment, the fetched instructions may be decoded by a decoder 718 to determine what operation the processor 708 should perform (e.g., during an instruction decoding phase). In at least one embodiment, the processor 708 may fetch additional operands (data) that can be used for the instructions, and the operands may be stored in, for example, a register or memory device 712. In at least one embodiment, micro-operations 720 may perform operations on the data stored in one or more registers or memory devices 712. For example, each step of the instructions fetched by the processor 708 may be broken down during execution, such that the processor 708 executes the instructions step-by-step through a series of micro-operations 720. In at least one embodiment, the program counter (PC) 714 may store the address of the next instruction and may be updated to point to the next instruction to be executed by the processor 708.

[0161] In at least one embodiment, processor 708 may execute instructions (e.g., during execution phase). For example, processor 708 may perform operations specified by the instructions, such as arithmetic operations, logical operations, or data transfers. In at least one embodiment, computing unit 722 may execute instructions to perform any of the operations described herein. In at least one embodiment, computing unit may include ALU 724 (Arithmetic Logic Unit), which may be used to perform arithmetic and logical operations. In at least one embodiment, computing unit may include FPU (Floating Point Unit) 726, which may be used to perform floating-point calculations. In at least one embodiment, other circuitry 728 may be used to perform other operations, such as vector and / or scalar operations. In at least one embodiment, accelerator 730 may include one or more matrix multiplication accelerators, one or more parallel processing units (PPUs) (such as GPUs), or any other accelerators or processors further described herein. In at least one embodiment, software program 704 may utilize one or more API and / or ISA instructions to perform various computational operations, such as matrix multiplication, arithmetic operations, or any other computational operations further described herein, via accelerator 730. In at least one embodiment, one or more computational operations using accelerator 730 may include at least one or more groups of computational operations that are accelerated by being executed at least in part by accelerator 730, including determining the clock frequency at which the processor group will operate by using the average activity level of the processor group, or as described elsewhere herein.

[0162] In at least one embodiment, system 700 can be used to execute one or more instructions including functions or operations, such as combining... Figures 1 to 6B as well as Figure 8The instructions described herein. In at least one embodiment, according to at least one embodiment, a system 700 including one or more processors causes one or more circuits to identify, by using the activity level of the processor group, the clock frequency at which these processors will operate while executing a software program, and / or otherwise perform any of the operations described herein. In at least one embodiment, a system 700 including one or more processors causes one or more circuits to execute an API to cause the activity level of other processors to be measured at one or more indicated intervals, and / or otherwise perform any of the operations described herein. In at least one embodiment, a system 700 including one or more processors causes one or more circuits to execute an API to cause one or more measurements of one or more activity levels of other processors to be stopped, and / or otherwise perform any of the operations described herein. In at least one embodiment, a system 700 including one or more processors causes one or more circuits to execute an API to cause one or more users to be instructed on the activity level of other processors, and / or otherwise perform any of the operations described herein. In at least one embodiment, a system 700 including one or more processors causes one or more circuits to execute an API to indicate to one or more users one or more statistics corresponding to one or more activity levels of one or more processors, and / or otherwise perform any of the operations described herein.

[0163] In at least one embodiment, system 700 is included and / or otherwise includes a combination of... Figures 1 to 6B and Figure 8 The systems shown or discussed are configured such that one or more circuits identify, by using the activity level of the processor group, that these processors will perform any of the operations described herein at their operating clock frequency while executing a software program, and / or otherwise. In at least one embodiment, system 700 is included and / or otherwise includes in conjunction with... Figures 1 to 6B and Figure 8 The system shown or discussed is configured to cause one or more circuits to execute an API, to cause the activity level of other processors to be measured at one or more indicated intervals, and / or to otherwise perform any of the operations described herein. In at least one embodiment, system 700 is included and / or otherwise includes a combination of Figures 1 to 6B The system illustrated or discussed in Figure 8 is configured to cause one or more circuits to execute an API, such that one or more measurements of one or more activity levels of other processors are stopped, and / or otherwise perform any of the operations described herein. In at least one embodiment, system 700 is included and / or otherwise includes a combination of Figures 1 to 6B and Figure 8 The system shown or discussed is configured to cause one or more circuits to execute an API, to instruct one or more users on one or more activity levels of other processors, and / or otherwise perform any of the operations described herein. In at least one embodiment, system 700 is included and / or otherwise includes a combination of... Figures 1 to 6B and Figure 8 The illustrated or discussed systems are designed to cause one or more circuits to execute an API, to instruct one or more users to provide one or more statistics corresponding to one or more activity levels of one or more processors, and / or to otherwise perform any of the operations described herein.

[0164] In at least one embodiment, system 700 includes Figures 17 to 27C The one or more hardware components shown herein, such as those used to identify the clock frequency at which these processors will operate while executing a software program, and / or otherwise perform any of the operations described herein, according to at least one embodiment, by using the activity level of the processor group. In at least one embodiment, system 700 includes Figures 17 to 27C The one or more hardware components shown herein, such as those used to execute an API to cause the activity level of other processors to be measured at one or more indicated intervals, and / or otherwise perform any of the operations described herein. In at least one embodiment, system 700 includes Figures 17 to 27C The one or more hardware components shown herein, such as those used to execute APIs to stop one or more measurements of one or more activity levels of other processors, and / or otherwise perform any of the operations described herein. In at least one embodiment, system 700 includes... Figures 17 to 27C The illustrated hardware includes one or more components, such as those used to execute an API to instruct one or more users on one or more activity levels of other processors, and / or otherwise perform any of the operations described herein. In at least one embodiment, system 700 includes... Figures 17 to 27C The one or more hardware shown herein, such as those used to execute APIs to instruct one or more users on one or more statistics corresponding to one or more activity levels of one or more processors, and / or otherwise perform any of the operations described herein.

[0165] Figure 8A system 800 according to at least one embodiment is illustrated, the system including a driver and / or runtime, the driver and / or runtime including one or more libraries for providing one or more application programming interfaces (APIs) to be executed by one or more processors including one or more circuits, these circuits being used at least in part to identify, by using the activity level of the processor group, the clock frequency at which the processor group will operate when executing a software program, or otherwise to perform any of the operations described herein. In at least one embodiment, this document incorporates... Figure 8 One or more aspects of the one or more embodiments described herein are at least in conjunction with this document. Figures 1 to 7 The embodiments described herein may be combined in one or more aspects. In at least one embodiment, the processor of the execution system 800 includes... Figure 1 System 100 Figure 2 The system Figure 3 The system Figure 4 System 400 Figure 5A System 500 Figure 5B System 506 Figure 6A System 600 Figure 6B System 606 Figure 7 The system 700 or some combination thereof is at least a part of, or is itself at least a part of, these systems.

[0166] In at least one embodiment, the processor of system 800 is any type of processor, part of a processor, system processor, or combination of processors described herein, including logic processors, Figure 9 processor 908, Figure 10 Processor Complex 1010 Figure 11A Parallel processor 1100, Figure 11B Graphics multiprocessor 1134, Figure 12 Processor 1200, Figure 13A Processor 1300, Figure 13B Core 1312 Figure 14 Accelerator 1400, Figure 15 Processor 1555, Figure 16 Processor 1632, Figure 17 Acceleration processing unit 1700, Figure 18 Processor 1800, Figure 19 Core 1900 Figure 20 TPU 2000 Figure 21 Vector processor 2100 Figure 22A Multi-core sharding processor 2200, Figure 23 Hardware 2308 Figure 25 CPU 2590, Figure 26The GPU 2608's streaming multiprocessor (SM), Figure 26 The processor 2610, and Figure 27A and Figure 27B The processor used in conjunction with the logic 2715 shown is... Figure 27C The training framework 2724 is used in conjunction with a processor or some combination thereof.

[0167] In at least one embodiment, the processor of system 800 performs operations used by system 100, such as operations of the API module 110 using a processor group that synchronizes with varying workloads. In at least one embodiment, the processor of system 800 performs combined... Figure 2 The one or more operations, such as operation 204 for calculating TGP and clock frequency. In at least one embodiment, the processor of system 800 performs the combined... Figure 3 The one or more operations, such as operation 308 calculating the Sync_clk value, are described. In at least one embodiment, the processor of system 800 performs the combination... Figure 4 The one or more operations mentioned above, such as operation 412 for invoking the JobStartStats API. In at least one embodiment, one or more processors of system 800 perform the combined... Figure 5A One or more operations, such as JobStartStats API call 512. In at least one embodiment, the processor of system 800 performs a combination of Figure 5B The one or more operations mentioned above, such as operation 508 for calling the JobGetStats API. In at least one embodiment, the processor of system 800 performs the combination Figure 6A The one or more operations mentioned above, such as operation 602 for invoking the DeviceGetStats API. In at least one embodiment, the processor of system 800 performs the combination Figure 6B The one or more operations mentioned above, such as operation 608 for calling the JobStopStats API. In at least one embodiment, the processor of system 800 performs the combination Figure 7 The one or more operations mentioned above.

[0168] In at least one embodiment, system 800 is any computing system or combination of computing systems, such as computing systems that constitute one or more data centers or other facilities housing computing and networking equipment.

[0169] In at least one embodiment, software program 802 is a software module. In at least one embodiment, software program 802 includes one or more software modules. In at least one embodiment, one or more APIs 810 are software instruction sets that, when executed, cause one or more processors to perform one or more computational operations. In at least one embodiment, one or more APIs 810 are distributed or otherwise provided as part of one or more runtimes 804, drivers 804, libraries 806, and / or any other grouping of software and / or executable code further described herein. In at least one embodiment, one or more APIs 810 perform one or more computational operations in response to a call to software program 802. In at least one embodiment, software program 802 is a collection of software code, commands, instructions, or other text sequences for instructing a computing device to perform one or more computational operations and / or to call one or more other instruction sets (such as APIs 810 or functions 812) to be executed. In at least one embodiment, the functionality provided by one or more APIs 810 includes software functions, such as those that can be used to accelerate one or more portions of software program 802 using one or more parallel processing units (PPUs) (such as graphics processing units (GPUs)). In at least one embodiment, the software program is a compiler.

[0170] In at least one embodiment, API 810 is a hardware interface for one or more circuits to perform one or more computational operations. In at least one embodiment, one or more software APIs 810 described herein are implemented as one or more circuits to perform one or more techniques described herein. In at least one embodiment, one or more software programs 802 include instructions that, when executed, cause one or more hardware devices and / or circuits to perform one or more techniques further described herein.

[0171] In at least one embodiment, software program 802 (such as a user-implemented software program) utilizes one or more application programming interfaces (APIs) 810 to perform various computational operations or any computational operations performed by a parallel processing unit (PPU) (such as a graphics processing unit (GPU)), as further described herein. In at least one embodiment, one or more APIs 810 provide a set of callable functions 812 (referred to herein as APIs, API functions, and / or functions), which each perform one or more computational operations, such as computational operations related to parallel computing. For example, in one embodiment, one or more APIs 810 provide functions 812 such that a processor executes the function to identify the clock frequency at which one or more processors in a processor group will operate, or as described elsewhere herein. In at least one embodiment, API 810 provides one or more functions 812, which are one or more neural networks, such as pre-trained LLMs.

[0172] In at least one embodiment, one or more software programs 802 interact with or otherwise communicate with one or more APIs 810 to perform one or more computational operations using one or more PPUs (such as GPUs). In at least one embodiment, the one or more computational operations using one or more PPUs comprise at least one or more sets of computational operations that are accelerated by being executed at least partially by said one or more PPUs. In at least one embodiment, one or more software programs 802 interact with one or more APIs 810 to facilitate parallel computation using remote or local interfaces.

[0173] In at least one embodiment, the interface is software instructions that, when executed, provide access to one or more functions 812 provided by one or more APIs 810. In at least one embodiment, when a software developer compiles one or more software programs 802 in conjunction with one or more libraries 806 (which include one or more APIs 810 or otherwise provide access to one or more APIs 810), the software program 802 uses the native interface. In at least one embodiment, one or more software programs 802 are statically compiled in conjunction with precompiled libraries 806 or uncompiled source code including instructions for executing one or more APIs 810. In at least one embodiment, one or more software programs 802 are dynamically compiled, and the one or more software programs are linked to one or more precompiled libraries 806 that include one or more APIs 810 using a linker.

[0174] In at least one embodiment, when a software developer executes a software program that utilizes a library 806 including one or more APIs 810 or communicates with the library via a network or other remote communication medium, the software program 802 uses a remote interface. In at least one embodiment, the one or more libraries 806 including one or more APIs 810 will be executed by a remote computing service, such as a computing resource service provider. In another embodiment, the one or more libraries 806 including one or more APIs 810 will be executed by any other computing host that provides the one or more APIs 810 to the one or more software programs 802.

[0175] In at least one embodiment, a processor executing or using one or more software programs 802 calls, uses, executes, or otherwise implements one or more APIs 810 to allocate and otherwise manage memory to be used by the software program 802. In at least one embodiment, one or more software programs 802 utilize one or more APIs 810 to allocate and manage memory to be used by one or more portions of the software program 802 (which will be accelerated using one or more PPUs (such as GPUs or any other accelerators or processors further described herein)). Those software programs 802 may be executed by one or more processors, at least in part, based on the latency of the interconnect coupled to one or more processors, using functions 812 provided by one or more APIs 810 in one embodiment.

[0176] In at least one embodiment, API 810 is an API for facilitating parallel computing. In at least one embodiment, API 810 is any other API as further described herein. In at least one embodiment, API 810 is provided by a driver and / or runtime 804. In at least one embodiment, API 810 is provided by a CUDA user-mode driver. In at least one embodiment, API 810 is provided by a CUDA runtime. In at least one embodiment, driver 804 is a data value and software instructions that, if executed, perform or otherwise facilitate the operation of one or more functions 812 of API 810 during the loading and execution of one or more portions of software program 802. In at least one embodiment, runtime 804 is a data value and software instructions that, when executed, perform or otherwise facilitate the operation of one or more functions 812 of API 810 during the execution of software program 802. In at least one embodiment, one or more software programs 802 utilize one or more APIs 810 implemented by or otherwise provided by a driver and / or runtime 804 to perform combined arithmetic operations by the one or more software programs 802 during execution by one or more PPUs (such as GPUs).

[0177] In at least one embodiment, one or more software programs 802 utilize one or more APIs 810 provided by a driver and / or runtime 804 to perform combinatorial arithmetic operations on one or more PPUs (such as GPUs). In at least one embodiment, one or more APIs 810 provide combinatorial arithmetic operations via a driver and / or runtime 804, as described above. In at least one embodiment, one or more software programs 802 utilize one or more APIs 810 provided by a driver and / or runtime 804 to allocate or otherwise reserve one or more blocks of memory 814 for one or more PPUs (such as GPUs). In at least one embodiment, one or more software programs 802 utilize one or more APIs 810 provided by a driver and / or runtime 804 to allocate or otherwise reserve blocks of memory. In at least one embodiment, one or more APIs 810 are used to perform combinatorial mathematical functions as described herein.

[0178] In at least one embodiment, to improve the availability of software program 802 and / or optimize one or more portions of said software program 802 (which will be accelerated by one or more PPUs (such as GPUs)), one or more APIs 810 provide one or more API functions 812 to perform a scheduling system that can be used by or by one or more computing devices as described herein. In at least one embodiment, the processor executes one or more software programs to combine two or more application programming interfaces (APIs) into a single API. In at least one embodiment, the processor uses the APIs to cause the scheduler to select a thread selection mechanism and / or otherwise perform the operations described herein. In at least one embodiment, the APIs invoke the scheduler to induce resource allocation. In at least one embodiment, the processor utilizes exemplary APIs to schedule one or more instructions for execution by one or more processors based at least in part on latency coupled to one or more interconnects of said one or more processors.

[0179] In at least one embodiment, memory 814 is system memory 1090 of SOC 1000. In at least one embodiment, memory 814 is processor memory. In at least one embodiment, memory 814 is any form of hardware for storing data and is referred to as a storage device or data storage device. In at least one embodiment, memory 814 stores data for the various operations described herein, including at least in conjunction with... Figures 4 to 6B The API parameters mentioned above.

[0180] In at least one embodiment, memory 814 is a computer-readable storage medium and / or code stored on the computer-readable storage medium in the form of a computer program comprising a plurality of computer-readable instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable medium. In at least one embodiment, at least some of the computer-readable instructions that can be used to perform the operations described herein are not stored using only transient signals (e.g., propagating transient electrical or electromagnetic transmissions). In at least one embodiment, the non-transitory computer-readable medium does not necessarily include non-transitory data storage circuitry (e.g., buffers, caches, and queues) within a transceiver having transient signals. In at least one embodiment, memory 814 is implemented as a non-transitory computer-readable storage medium storing executable instructions that, if executed by one or more processors of a computer system, cause one or more neural networks to generate software to be executed by one or more GPUs, at least in part, based on software to be executed by one or more CPUs.

[0181] Data Center

[0182] Figure 9 An example data center 900 according to at least one embodiment is illustrated. The data center 900 may include one or more rooms having racks 902 and auxiliary equipment for housing one or more racks 902 and one or more substrates 904. A rack 902 may include one or more substrates 904. A rack 902 may include a housing for housing and supporting individual substrates 904. Operational aspects of the rack 902 may be adjustable at the rack level (corresponding to a group of substrates 904) or at the substrate level (corresponding to an individual substrate 904), among other options. The rack 902 or substrate 904 may have specific selected maximum operating parameters, such as, but not limited to, power consumption, operating frequency, etc. The data center 900 may be supported by various cooling systems, such as, but not limited to, cooling towers, cooling loops, pumps, and other support systems. The cooling system may include sensors and controllers for monitoring and managing the cooling characteristics of the rack 902. The substrates 904 within the rack 902 may draw operating power from one or more power distribution units (PDUs; not shown). PDUs can be arranged within racks 902, for example, between racks 902 that include substrates 904, or within racks 902 that also house substrates 904.

[0183] The rack 902 and substrate 904 may include subsystems, modules, add-in cards, and other semiconductor components. Substrate 904 may include one or more computing units 906, each computing unit 906 including one or more processors 908, one or more memories 910, and an interface controller 912. The computing unit 906 may include any number of processors, such as, but not limited to, a central processing unit (“CPU”), a graphics processing unit (“GPU”), or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors, etc.), including any processor described herein, such as, but not limited to, those described herein. Figures 10-22B The processor in the computing unit 906 may include one or more memory storage devices 910 (e.g., dynamic read-only memory, solid-state storage devices, or disk drives), as well as network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power supply modules, and cooling modules, etc. One or more computing units 906 may be a server having the aforementioned one or more computing resources.

[0184] Computing unit 906 may include individual computing unit groups housed in one or more racks (not shown), or in numerous racks within data centers in different geographical locations (also not shown). Individual computing unit groups may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. Several computing units (e.g., including CPUs and / or other processors) may be grouped within one or more racks to provide computing resources to support one or more workloads. Resource coordinator 914 may configure or otherwise control one or more computing units 906 or groups of computing units. Resource coordinator 914 may include a Software Design Infrastructure (“SDI”) management entity for data center 900. Resource coordinator 914 may include hardware, software, or some combination thereof.

[0185] Data center 900 may include any one or any combination of the framework layer 920, software layer 930, and application layer 940. For example... Figure 9 As shown, framework layer 920 includes a job scheduler 922, a configuration manager 924, a resource manager 926, and a distributed file system 928. Framework layer 920 may include a framework for supporting software 932 of software layer 930 and / or one or more applications 942 of application layer 940. Software 932 or application 942 may respectively include web-based service software or applications, such as, but not limited to, software or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 920 may be a type of free and open-source software web application framework, such as, but not limited to, Apache Spark. TM (Hereinafter referred to as "Spark"), which can utilize the distributed file system 928 for large-scale data processing (e.g., "big data"). The job scheduler 922 may include Spark drivers, which facilitate the scheduling of workloads supported by various layers of the data center 900. The configuration manager 924 may be able to configure different layers, such as, but not limited to, the software layer 930 and the framework layer 920 (which includes Spark and the distributed file system 928 for supporting large-scale data processing). The resource manager 926 may be able to manage clustered or grouped compute units 906 mapped to or allocated to support the distributed file system 928 and the job scheduler 922. The resource manager 926 may coordinate with the resource coordinator 914 to manage these mapped or allocated compute resources.

[0186] Software 932 may be included in software layer 930, and may include software used by at least a portion of computing units 906, one or more computing units 906, groups of computing units 906, and / or the distributed file system 928 of framework layer 920. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0187] Application 942 may be included in application layer 940 and may include one or more types of applications used by at least the portions of computing unit 906, one or more computing units 906, groups of computing units 906, and / or the distributed file system 928 of framework layer 920. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing applications, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0188] Any of the Configuration Manager 924, Resource Manager 926, and Resource Coordinator 914 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can alleviate the burden on data center operators of Data Center 900 to make potentially erroneous configuration decisions and may prevent underutilized and / or poorly performing portions of the data center.

[0189] Data center 900 may include tools, services, software, or other resources for training one or more machine learning models according to one or more embodiments described herein, or for using one or more machine learning models to predict or infer information. For example, a machine learning model can be trained by calculating weight parameters based on a neural network architecture using the software and computing resources described above regarding data center 900. The trained machine learning model corresponding to one or more neural networks can be used with the resources described above regarding data center 900 to infer or predict information using weight parameters calculated through one or more training techniques described herein.

[0190] Data Center 900 can use CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, or other hardware (e.g., Figures 10-22BThe embodiments described herein can be used to perform some or all of the processes and techniques described elsewhere, such as, but not limited to, training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above can be configured as a service to allow a user to train or perform information inference, such as, but not limited to, image recognition, speech recognition, or other artificial intelligence services.

[0191] In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0192] In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0193] In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause the identification of one or more processors in the processor group to operate at one or more clock frequencies by obtaining one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0194] In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may include one or more processors and / or circuitry that can be used to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0195] In at least one embodiment, processor 908 may be configured by software 932 to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may be configured by software 932 to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0196] In at least one embodiment, processor 908 may be configured by software 932 to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may be configured by software 932 to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0197] In at least one embodiment, processor 908 may be configured by software 932 to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may be configured by software 932 to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0198] In at least one embodiment, processor 908 may be configured by software 932 to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 908 may be configured by software 932 to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0199] processor

[0200] In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any operations described above or elsewhere herein. In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobStartStats API to at least partially cause, by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, one or more processors in a processor group to be identified to operate at one or more clock frequencies, or otherwise perform any operations described above or elsewhere herein.

[0201] In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobStopStats API to at least partially cause identification of one or more processors in a processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0202] In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any operations described above or elsewhere herein. In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify one or more processors in a processor group that will operate at one or more clock frequencies, or otherwise perform any operations described above or elsewhere herein.

[0203] In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any operations described above or elsewhere herein. In at least one embodiment, the following figures illustrate, but are not limited to, example processors and processing systems that can be used to execute the JobGetStats API to at least partially cause the identification of one or more processors in a processor group that will operate at one or more clock frequencies, or otherwise perform any operations described above or elsewhere herein.

[0204] In at least one embodiment, the example processor and processing system may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the example processor and processing system may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in a processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors that will be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0205] In at least one embodiment, the example processor and processing system may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the example processor and processing system may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in a processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0206] In at least one embodiment, the example processor and processing system may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the example processor and processing system may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in a processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0207] In at least one embodiment, the example processor and processing system may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the example processor and processing system may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in a processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0208] In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in a processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0209] In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in a processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0210] In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause the identification of one or more processors in a processor group to operate at one or more clock frequencies by obtaining one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0211] In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry that can be used to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the processor and / or processing system described herein may include one or more circuitry that can be used to execute the JobGetStats API to at least partially cause the identification of one or more processors in a processor group to operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0212] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0213] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0214] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0215] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0216] Figure 27A and Figure 27B Logic 2715, as described elsewhere herein, is illustrated and can be used in one or more devices to perform, for example (but not limited to), the operations discussed herein according to at least one embodiment. Logic can refer to, for example, any combination of software logic, hardware logic, and / or firmware logic for providing the functions and / or operations described herein, wherein the logic can be collectively or individually embodied as circuitry forming part of a larger system, such as an integrated circuit (IC), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a system-on-a-chip (SoC), or one or more processors (e.g., a CPU, a GPU).

[0217] Figure 10 A processor according to at least one embodiment is illustrated, which is a system-on-a-chip (SOC) 1000 (which may be referred to as a system-on-a-chip, superchip, or other names). The SOC 1000 may include processor complex 1010 and processor complex 1040. The SOC 1000 may include any number of processor complexes 1010 and / or processor complexes 1040, which may include any number of processors described herein in any combination, such as, but not limited to, processors described herein in any combination. Figures 10-22B The processor in the system. For example, processor 1010 may include a central processing unit (CPU), and processor 1040 may include a graphics processor. Alternatively, processor 1010 may include a graphics processor, and processor 1040 may include a graphics processor. The SOC 1000 may include any number of display controllers 1092, any number of multimedia engines 1094, any number of I / O interfaces 1070, any number of memory controllers 1080, and any number of fabrics 1060 in any combination. For ease of explanation, this document uses reference numbers identifying objects and bracket numbers identifying instances (if needed) to denote multiple instances of similar objects. The SOC 1000 may include a processor from Broadcom Corporation, Palo Alto, California.

[0218] Processor complex 1010 may include a CPU, processor complex 1040 may include a GPU, and SOC 1000 may include a processing unit integrating processor complex 1010 and processor complex 1040 onto a single chip. Certain tasks may be assigned to processor complex 1010, while other tasks may be assigned to processor complex 1040. Processor complex 1010 may be configured to execute main control software associated with SOC 1000, such as, but not limited to, an operating system. Processor complex 1010 may be the main processor of SOC 1000, controlling and coordinating the operation of other processors. Processor complex 1010 may issue commands that control the operation of processor complex 1040 to perform some or all of the operations described herein. Processor complex 1010 may be configured to execute host-executable code derived from CUDA or other source code (e.g., HIP source code), while processor complex 1040 may be configured to execute device-executable code derived from CUDA or other source code to perform any of the operations described herein.

[0219] The processor complex 1010 may include cores 1020(1)-1020(4) and a cache (e.g., L3 cache) 1030 for storing information for performing the operations described herein. The processor complex 1010 may include any number of cores 1020 in any combination and any number and type of cache. The cores 1020 may be configured to execute instructions of a specific instruction set architecture (“ISA”) to perform some or all of the operations described herein. Each core 1020 may include a CPU core. Cores 1020(1)-1020(4) may be referred to as compute units or arithmetic units. The SOC 1000 may include any number of processor complexes 1010, architecture 1060, I / O interface 1070, and memory controller 1080.

[0220] Each core 1020 may include a fetch / decode unit 1022, an integer execution engine 1024, a floating-point execution engine 1026, and an L2 cache 1028. The fetch / decode unit 1022 may fetch instructions to perform some or all of the operations described herein (e.g., but not limited to APIs compiled into instructions) and decode those instructions, generate micro-operations, and dispatch individual micro-instructions to the integer execution engine 1024 and / or the floating-point execution engine 1026. The fetch / decode unit 1022 may concurrently dispatch one micro-instruction to the integer execution engine 1024 and another micro-instruction to the floating-point execution engine 1026. The integer execution engine 1024 may perform integer and memory operations. The floating-point engine 1026 may perform floating-point and vector operations. The fetch / decode unit 1022 may dispatch micro-instructions to one or more execution engines, which may replace both the integer execution engine 1024 and the floating-point execution engine 1026.

[0221] Each core 1020(i) (where i is an integer representing a specific instance of core 1020) can access the L2 cache 1028(i) included in core 1020(i). Each core 1020 included in core complex 1010(j) (where j is an integer representing a specific instance of core complex 1010) can be connected to other cores 1020 included in core complex 1010(j) via the L3 cache 1030(j) included in core complex 1010(j). The cores 1020 included in core complex 1010(j) (where j is an integer representing a specific instance of core complex 1010) can access all L3 caches 1030(j) included in core complex 1010(j). The L3 cache 1030 can include any number of slices.

[0222] Processor complex 1040 may be a graphics complex that can be configured to perform computational operations (e.g., the computational operations described herein) in a highly parallel manner. Processor complex 1040 may be configured to perform graphics pipeline operations, such as, but not limited to, drawing commands, pixel operations, geometric calculations, and other operations associated with rendering an image to a display. Processor complex 1040 may be configured to perform graphics-independent operations, such as, but not limited to, neural network training and / or simulation. Processor complex 1040 may be configured to perform both graphics-related and graphics-independent operations.

[0223] The processor complex 1040 may include any number of compute units 1050(1)-1050(N) (where N is any integer greater than 1) and an L2 cache 1042. The compute units 1050 may share the L2 cache 1042, which may store information that will be used to perform some or all of the operations described herein. The L2 cache 1042 may be partitioned. The processor complex 1040 may include any number of compute units 1050 and any number (including zero) and type of cache. The processor complex 1040 may include any number of dedicated graphics hardware.

[0224] Each compute unit 1050 may include any number of SIMD units 1052(1)-1052(N) (where N is any integer greater than 1) and shared memory 1054. Each SIMD unit 1052 may implement a SIMD architecture and may be configured to perform some or all of the operations described herein in parallel. Each compute unit 1050 may execute any number of thread blocks, but each thread block may execute on a single compute unit 1050, although in some embodiments, the thread block may execute on multiple compute units. A thread block may include any number of execution threads. A workgroup may be a thread block. Each SIMD unit 1052 may execute a set of threads. A set of threads (e.g., 16 threads), also referred to as a warp, subgroup, or wavefront (e.g., used by AMD and Intel), may belong to a single thread block and be configured to process different datasets based on a single instruction set. Prediction may be used to disable one or more threads in a warp, subgroup, or wavefront. A lane may be a thread. A work item can be a thread, such as (but not limited to) an OpenCL thread. Different thread bundles, subgroups, or wavefronts within a thread block can be synchronized together and communicate via shared memory 1054. Each compute unit 1050 can include one or more thread block clusters, where thread block clusters can implement programmable control over locality at a larger granularity than a single thread block in a single streaming multiprocessor (SM). Thread block clusters (also referred to as “clusters”) can support multiple thread blocks running concurrently across streaming multiprocessors, thereby synchronously and cooperatively acquiring, exchanging, or otherwise using data. In at least one embodiment, a streaming multiprocessor (“SM”) can refer to a streaming microprocessor, a streaming processor (“SP”), a streaming processing unit (“SPU”), a compute unit (“CU”), an execution unit (“EU”), and / or a slice, where a slice in this context can refer to a portion of the processing resources within a processing unit (e.g., 16 cores, a ray tracing unit, a thread bootstrap, or a scheduler).

[0225] Structure 1060 may be a system interconnect that facilitates data and control transfers across processor complex 1010, processor complex 1040, I / O interface 1070, memory controller 1080, display controller 1092, and multimedia engine 1094, for example, to perform some or all of the operations described herein. SOC 1000 may include any number and type of system interconnects other than or replacing structure 1060, facilitating data and control transfers across any number and type of directly or indirectly linked components within or outside SOC 1000. I / O interface 1070 may represent any number and type of I / O interfaces (e.g., PCI, PCI extensions (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). Various types of peripheral devices may be coupled to I / O interface 1070. Peripheral devices that may be coupled to I / O interface 1070 may include keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0226] Display controller 1092 can display images on one or more display devices, such as, but not limited to, liquid crystal displays (“LCD”) devices. Multimedia engine 1094 can include any number and type of circuitry related to multimedia, such as, but not limited to, video decoders, video encoders, image signal processors, etc. Memory controller 1080 can facilitate data transfer between SOC 1000 and unified system memory 1090. Processor complex 1010 and processor complex 1040 can share unified system memory 1090. Unified system memory 1090 can include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. Unified system memory 1090 can include 3D stacked memory, including but not limited to high bandwidth memory (HBM), HBM2e, or HDM3.

[0227] The SOC 1000 can implement a memory subsystem including any number and type of memory controllers 1080 and memory devices (e.g., shared memory 1054), which may be dedicated to a single component or shared among multiple components to perform any of the operations described herein. The SOC 1000 can implement a cache subsystem including one or more cache memories (e.g., L2 cache 1028, L3 cache 1030, and L2 cache 1042), each cache memory may be dedicated to any number of components (e.g., core 1020, core complex 1010, SIMD unit 1052, compute unit 1050, and processor complex 1040), or may be shared among any number of components (e.g., core 1020, core complex 1010, SIMD unit 1052, compute unit 1050, and processor complex 1040).

[0228] In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0229] In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group that will operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0230] In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0231] In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the SOC 1000 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0232] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0233] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0234] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0235] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0236] Figure 11A A parallel processor 1100 according to at least one embodiment is illustrated. The parallel processor 1100 may be implemented using one or more circuits and may be referred to as a programmable processor (e.g., CPU and / or GPU), logic, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other hardware (e.g., Figures 10-22B (The embodiments in the document) are used to perform any of the operations described above or elsewhere herein.

[0237] Parallel processor 1100 may include parallel processing unit 1102 for performing any of the operations described above or elsewhere herein. Parallel processing unit 1102 may include I / O unit 1104 that enables communication with other devices, including other instances of parallel processing unit 1102. I / O unit 1104 may be directly connected to other devices. I / O unit 1104 may be connected to other devices via the use of a hub or switch interface, such as, but not limited to, memory hub 1105. The connection between memory hub 1105 and I / O unit 1104 may form a communication link 1113. I / O unit 1104 may be connected to host interface 1106 and memory crossbar switch 1116, wherein host interface 1106 receives commands directed to perform processing operations, and memory crossbar switch 1116 receives commands directed to perform memory operations.

[0238] When host interface 1106 receives a command buffer via I / O unit 1104, host interface 1106 can route the work operations that execute these commands to front-end 1108. Front-end 1108 can be coupled to scheduler 1110 (which may be referred to as sequencer), which is configured to distribute commands or other work items to processing cluster array 1112. Scheduler 1110 can ensure that processing cluster array 1112 is correctly configured and in an active state before tasks are distributed to the cluster of processing cluster array 1112. Scheduler 1110 can be implemented via firmware logic executed on a microcontroller. The microcontroller-implemented scheduler 1110 can be configured to perform complex scheduling and work distribution operations at both coarse and fine granular levels, thereby enabling fast preemption and context switching of threads executing on processing array 1112. Host software can validate workloads scheduled on processing cluster array 1112 via one of multiple graphics processing paths. The workload can then be automatically distributed to the processing array cluster 1112 by the scheduler 1110 logic within the microcontroller, which includes the scheduler 1110.

[0239] Processing cluster array 1112 can perform any of the operations described above or elsewhere herein, and may include up to “N” processing clusters (e.g., clusters 1114A, 1114B through 1114N), where “N” represents a positive integer (which may be a different integer “N” than used in other diagrams). Each cluster 1114A-1114N in processing cluster array 1112 can execute a large number of concurrent threads. Scheduler 1110 may use various scheduling and / or work distribution algorithms to distribute work to clusters 1114A-1114N in processing cluster array 1112, which may vary depending on the workload generated by each type of program or computation. Scheduling may be dynamically handled by scheduler 1110 or may be assisted by compiler logic during the compilation of program logic configured to be executed by processing cluster array 1112. Different clusters 1114A-1114N of processing cluster array 1112 may be assigned to process different types of programs or perform different types of computations.

[0240] The processing cluster array 1112 can be configured to perform various types of parallel processing operations, such as, but not limited to, any of the operations described above or elsewhere herein. The processing cluster array 1112 can be configured to perform general-purpose parallel computing operations. For example, the processing cluster array 1112 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations (including physical operations), and performing data transformations.

[0241] Processing cluster array 1112 can be configured to perform parallel graphics processing operations. Processing cluster array 1112 may include additional logic for supporting the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. Processing cluster array 1112 can be configured to execute shader programs related to graphics processing, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. Parallel processing unit 1102 can transfer data from system memory via I / O unit 1104 for processing. During processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 1122) during processing and then written back to system memory.

[0242] When the parallel processing unit 1102 is used to perform graphics processing, the scheduler 1110 can be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations to multiple clusters 1114A-1114N of the processing cluster array 1112. Each part of the processing cluster array 1112 can be configured to perform different types of processing. For example, a first part can be configured to perform vertex shading and topology generation, a second part can be configured to perform tessellation and geometry shading, and a third part can be configured to perform pixel shading or other screen-space operations to produce a rendered image for display. Intermediate data generated by one or more clusters 1114A-1114N can be stored in a buffer to allow intermediate data to be transferred between clusters 1114A-1114N for further processing.

[0243] Processing cluster array 1112 can receive processing tasks to be executed via scheduler 1110, which receives commands defining the processing tasks from front end 1108. Processing tasks may include indexes of data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as state parameters and commands defining how to process the data (e.g., which program to execute). Scheduler 1110 can be configured to retrieve the index corresponding to the task, or can receive the index from front end 1108. Front end 1108 can be configured to ensure that processing cluster array 1112 is configured to be active before the workload specified by the incoming command buffer (e.g., batch buffer, push buffer, etc.) is initiated.

[0244] Each instance of one or more instances of parallel processing unit 1102 may be coupled to parallel processor memory 1122 to perform any of the operations described above or elsewhere herein. Parallel processor memory 1122 may be accessed via memory crossbar switch 1116, which may receive memory requests from processing cluster array 1112 and I / O unit 1104. Memory crossbar switch 1116 may access parallel processor memory 1122 via memory interface 1118. Memory interface 1118 may include multiple partition units (e.g., partition units 1120A, 1120B through 1120N), each partition unit may be coupled to a portion (e.g., a memory cell) of parallel processor memory 1122. The number of partition units 1120A-1120N can be configured to be equal to the number of memory units, such that the first partition unit 1120A has a corresponding first memory unit 1124A, the second partition unit 1120B has a corresponding memory unit 1124B, and the Nth partition unit 1120N has a corresponding Nth memory unit 1124N. The number of partition units 1120A-1120N may not be equal to the number of memory units.

[0245] Memory cells 1124A-1124N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as, but not limited to, synchronous graphics random access memory (SGRAM), which includes graphics double data rate (GDDR) memory. Memory cells 1124A-1124N may also include 3D stacked memory, including but not limited to high-bandwidth memory (HBM), HBM2e, or HDM3. Render targets (e.g., but not limited to framebuffers or texture maps) may be stored in memory cells 1124A-1124N, allowing partitioning cells 1120A-1120N to write portions of each render target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 1122. A local instance of the parallel processor memory 1122 may not be included to support a unified memory design that combines system memory with local cache memory.

[0246] Any cluster 1114A-1114N in the processing cluster array 1112 can process data to be written to any memory cell 1124A-1124N within the parallel processor memory 1122. The memory crossbar switch 1116 can be configured to transfer the output of each cluster 1114A-1114N to any partition cell 1120A-1120N, or to another cluster 1114A-1114N where additional processing operations can be performed on the output. Each cluster 1114A-1114N can communicate with the memory interface 1118 via the memory crossbar switch 1116 to read from or write to various external memory devices. The memory crossbar switch 1116 can be connected to the memory interface 1118 to communicate with the I / O unit 1104, or to a local instance of the parallel processor memory 1122, enabling processing units within different processing clusters 1114A-1114N to communicate with system memory or other memory local to the non-parallel processing unit 1102. The memory crossbar switch 1116 can use virtual channels to separate traffic flows between clusters 1114A-1114N and partition units 1120A-1120N.

[0247] Multiple instances of the parallel processing unit 1102 can be mounted on a single add-in card, or multiple add-in cards can be interconnected. Even if different instances of the parallel processing unit 1102 have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences, these different instances can be configured to interoperate. For example, some instances of the parallel processing unit 1102 may include higher-precision floating-point units relative to other instances. Systems including one or more instances of the parallel processing unit 1102 or the parallel processor 1100 can be implemented in a variety of configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0248] Figure 11A It also includes a block diagram of a partitioning unit 1120 according to at least one embodiment. The partitioning unit 1120 is... Figure 11AAn example of one of the partition units 1120A-1120N in the parallel processor memory. Partition unit 1120 may include an L2 cache 1121, a frame buffer interface 1125, and a ROP 1126 (raster operation unit). The L2 cache 1121 may be a read / write cache configured to perform load and store operations received from the memory crossbar switch 1116 and the ROP 1126. Read misses and urgent write-back requests may be output from the L2 cache 1121 to the frame buffer interface 1125 for processing. Updates may also be sent to the frame buffer via the frame buffer interface 1125 for processing. The frame buffer interface 1125 may interface with one of the memory cells in the parallel processor memory, such as, but not limited to, Figure 11A The memory cells 1124A-1124N (shown as 1124) are located in the parallel processor memory 1122 (for example, within the parallel processor memory 1122).

[0249] ROP 1126 can be a processing unit that performs raster operations, such as, but not limited to, stenciling, z-testing, blending, etc. ROP 1126 can then output processed graphics data stored in graphics memory. ROP 1126 may include compression logic for compressing depth or color data written to memory and decompressing depth or color data read from memory. The compression logic can be lossless compression logic that utilizes one or more compression algorithms. The type of compression performed by ROP 1126 can vary based on the statistical characteristics of the data to be compressed. For example, incremental color compression is performed on depth and color data on a per-tile basis.

[0250] ROP 1126 can be included in each processing cluster (e.g., Figure 11A The data is stored within clusters 1114A-1114N, not within partition unit 1120. Read and write requests for pixel data (not pixel fragment data) can be transferred via memory crossbar switch 1116. Processed graphics data can be displayed on the monitor and routed for further processing by the processor, or routed to... Figure 11A One of the processing entities within the parallel processor 1100 in the system is further processed.

[0251] In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0252] In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements at one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements at one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0253] In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0254] In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the parallel processor 1100 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0255] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0256] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0257] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0258] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0259] Figure 11B A block diagram including a processing cluster 1114 within a parallel processing unit according to at least one embodiment. The processing cluster may be... Figure 11A An instance of one of the processing clusters 1114A-1114N is provided, which can be used to perform any of the operations described above or elsewhere herein. Processing cluster 1114 can be configured to execute many threads in parallel, where a “thread” refers to an instance of a specific program executed on a specific input dataset. Single Instruction Multiple Data (SIMD) instruction issuing techniques can be used to support the parallel execution of a large number of threads without providing multiple independent instruction units. Single Instruction Multiple Thread (SIMT) techniques can be used to support the parallel execution of a large number of typically synchronous threads using a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster.

[0260] The operation of cluster 1114 can be controlled via pipeline manager 1132, which distributes processing tasks to SIMT parallel processors. Pipeline manager 1132 can... Figure 11AThe scheduler 1110 receives instructions and manages the execution of these instructions via the graphics multiprocessor 1134 and / or texture unit 1136. The graphics multiprocessor 1134 may be an example instance of a SIMT parallel processor. However, the processing cluster 1114 may include various types of SIMT parallel processors with different architectures. The processing cluster 1114 may include one or more instances of the graphics multiprocessor 1134. The graphics multiprocessor 1134 can process data and can use the data cross switch 1140 to distribute the processed data to one of several possible destinations, including other shader units. The pipeline manager 1132 can facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data cross switch 1140.

[0261] Each graphics multiprocessor 1134 within the processing cluster 1114 may include a set of identical functional execution logic (e.g., arithmetic logic units, load-memory units, etc.) for performing computations for any of the operations described above or elsewhere herein. The functional execution logic may be configured in a pipelined manner, where new instructions can be issued before previous instructions complete. The functional execution logic may support a wide range of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and computation of various algebraic functions. Different operations can be performed using the same functional unit hardware, and arbitrary combinations of functional units are possible.

[0262] Instructions transmitted to the processing cluster 1114 can form threads, which may also be called thread bundles, subgroups, waves, or wavefronts. A group of threads executing across a set of parallel processing engines can be called a thread group. Thread groups can execute a common program on different input data. Each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 1134. The number of threads in a thread group can be less than the number of processing engines within the graphics multiprocessor 1134. When the number of threads in a thread group is less than the number of processing engines, one or more processing engines may be idle during the processing cycle of that thread group. The number of threads in a thread group can also be more than the number of processing engines within the graphics multiprocessor 1134. When the number of threads in a thread group is more than the number of processing engines within the graphics multiprocessor 1134, processing can be performed in consecutive clock cycles. Multiple thread groups can execute concurrently on the graphics multiprocessor 1134.

[0263] The graphics multiprocessor 1134 includes an internal cache memory for performing load and store operations, such as, but not limited to, any of the operations described above or elsewhere herein. The graphics multiprocessor 1134 may forgo the internal cache and instead use a cache memory within the processing cluster 1114 (e.g., L1 cache 1148). Each graphics multiprocessor 1134 may also access partition units that can be shared across all processing clusters 1114 (e.g., ...). Figure 11A The L2 cache within partition units 1120A-1120N is used for transferring data between threads. The graphics multiprocessor 1134 can also access off-chip global memory, which may include one or more of the local parallel processor memory and / or system memory. Any memory outside of the parallel processing unit 1102 can be used as global memory. The processing cluster 1114 may include multiple instances of the graphics multiprocessor 1134 and can share common instructions and data, which can be stored in the L1 cache 1148.

[0264] Each processing cluster 1114 may include an MMU 1145 (Memory Management Unit), which can be configured to map virtual addresses to physical addresses. One or more instances of the MMU 1145 may reside in... Figure 11A The MMU 1145 is located within the memory interface 1118. It may include: a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles; and optional cache line indexes. The MMU 1145 may include address translation lookup buffers (TLBs) or caches that may reside within the graphics multiprocessor 1134 or L1 1148 cache or processing cluster 1114. Physical addresses can be processed to distribute surface data access locally, allowing for efficient request interleaving between partition units. The cache line indexes can be used to determine whether a request for a cache line is a hit or a miss.

[0265] Processing cluster 1114 can be configured such that each graphics multiprocessor 1134 is coupled to a texture unit 1136 for performing texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. Texture data can be read from an internal texture L1 cache (not shown) or an L1 cache within the graphics multiprocessor 1134, and can be retrieved as needed from an L2 cache, local parallel processor memory, or system memory. Each graphics multiprocessor 1134 can output a processed task to a data crossbar switch 1140 to provide the processed task to another processing cluster 1114 for further processing, or store the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 1116. Pre-ROP 1142 (pre-raster operation unit) can be configured to receive data from the graphics multiprocessor 1134 and direct the data to ROP units, which can be associated with partitioning units (e.g., ...) described herein. Figure 11A The PreROP 1142 unit is located together with partition units 1120A-1120N. The PreROP 1142 unit can perform color blending optimization, organize pixel color data, and perform address translation.

[0266] In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using the activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0267] In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0268] In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause the identification of one or more processors in the processor group to operate at one or more clock frequencies by obtaining one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0269] In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processing cluster 1114 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0270] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0271] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0272] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0273] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0274] Figure 11C A graphics multiprocessor 1134 according to at least one embodiment is illustrated, for example, to perform any of the operations described above or elsewhere herein. The graphics multiprocessor 1134 may be coupled to a pipeline manager 1132 of a processing cluster 1114. The graphics multiprocessor 1134 may include an execution pipeline including, but not limited to, an instruction cache 1152 (e.g., which may store instructions, such as, but not limited to, compiled API instructions), instruction units 1154, address mapping units 1156, register files 1158, one or more general-purpose graphics processing unit (GPGPU) cores 1162, and one or more load / store units 1166, one or more of which may perform load / store operations to load / store instructions corresponding to the execution operations. The GPGPU cores 1162 and load / store units 1166 may be coupled to cache memory 1172 and shared memory 1170 via a memory and cache interconnect 1168. The GPGPU cores 1162 may be part of a SoC, such as, but not limited to, [other components]. Figure 10 It is part of the integrated circuit 1000.

[0275] Instruction cache 1152 can receive a stream of instructions to be executed from pipeline manager 1132 (e.g., to perform any of the operations described above or elsewhere herein). Instructions can be cached in instruction cache 1152 and dispatched for execution by instruction unit 1154. Instruction unit 1154 can dispatch instructions into thread groups (e.g., thread bundles, subgroups, wavefronts, or waves), with each thread in the thread group assigned to a different execution unit within GPGPU core 1162. Instructions can access any of the local, shared, or global address spaces by specifying an address within a unified address space. Address mapping unit 1156 can be used to translate addresses in the unified address space into different memory addresses accessible by load / store unit 1166.

[0276] Register file 1158 provides a set of registers for the functional units of graphics multiprocessor 1134. Register file 1158 provides temporary storage for operands on data paths connected to functional units of graphics multiprocessor 1134 (e.g., GPGPU core 1162, load / store unit 1166). Register file 1158 can be partitioned among the functional units such that each functional unit is allocated a dedicated portion of register file 1158. Register file 1158 can be partitioned among different thread bundles (which may be referred to as wavefronts, subgroups, and / or waves or threads) executed by graphics multiprocessor 1134.

[0277] Each GPGPU core 1162 may include a floating-point unit (FPU) and / or an integer arithmetic logic unit (ALU) for executing instructions of the graphics multiprocessor 1134. The architectures of the GPGPU cores 1162 may be similar or different. A first part of the GPGPU core 1162 may include a single-precision FPU and an integer ALU, while a second part of the GPGPU core may include a double-precision FPU. The FPU may implement IEEE 754-2008 standard floating-point arithmetic or enable variable-precision floating-point arithmetic. The graphics multiprocessor 1134 may also include one or more fixed-function or special-function units for performing specific functions, such as, but not limited to, copying rectangles or pixel blending operations. One or more of the GPGPU cores 1162 may also include fixed-function or special-function logic.

[0278] The GPGPU core 1162 may include SIMD logic capable of executing a single instruction on multiple sets of data. The GPGPU core 1162 can physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. The SIMD instructions for the GPGPU core may be generated by the shader compiler at compile time, or may be automatically generated when executing programs written and compiled for Single Program Multiple Data (SPMD) or SIMT architectures. Multiple threads of the program can be configured for a SIMT execution model that can be executed via a single SIMD instruction. For example, eight SIMT threads performing the same or similar operations can be executed in parallel via a single SIMD8 logic unit.

[0279] The memory and cache interconnect 1168 may include an interconnect network that connects each functional unit of the graphics multiprocessor 1134 to the register file 1158 and shared memory 1170. The memory and cache interconnect 1168 may be a cross-switch interconnect that allows the load / store unit 1166 to perform load and store operations between the shared memory 1170 and the register file 1158. The register file 1158 may operate at the same frequency as the GPGPU core 1162, thus data transfer between the GPGPU core 1162 and the register file 1158 can have very low latency. The shared memory 1170 can be used to implement communication between threads executing on functional units within the graphics multiprocessor 1134. The cache memory 1172 can be used as a data cache, for example, for caching texture data transferred between functional units and texture units 1136. The shared memory 1170 can also be used as a program-managed cache. In addition to automatically caching the data stored in the cache memory 1172, threads executing on the GPGPU core 1162 can also programmatically store data in shared memory.

[0280] The parallel processor or GPGPU described herein can be communicatively coupled to a host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. The GPU can be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., high-speed interconnects, such as, but not limited to, PCIe or NVLink). A System-on-a-Chip (SoC) may include the parallel processor or GPGPU described herein, which executes on the SoC. The GPU may be integrated as a core on a package or chip and communicatively coupled to the core via an internal processor bus / interconnect within the package or chip. Regardless of the GPU's connection method, the processor core can assign work to the GPU in the form of a sequence of commands / instructions contained in a job descriptor. The GPU can then use dedicated circuitry / logic to efficiently process these commands / instructions to perform any of the operations described above or elsewhere herein.

[0281] In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more processors in the processor group to be identified at one or more clock frequencies of operation by using the activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0282] In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0283] In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause the identification of one or more processors in the processor group to operate at one or more clock frequencies by obtaining one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0284] In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the graphics multiprocessor 1134 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group to operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0285] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0286] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0287] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0288] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0289] Figure 12A processor 1200 according to at least one embodiment is illustrated. The processor 1200 may include a hybrid architecture processor (e.g., Lunar Lake or Meteor Lake) from Intel Corporation, Santa Clara, California, or other processors sharing at least some of the components described herein. The processor 1200 may include one or more central processing units (CPU 1202), one or more graphics processing units (GPU 1206), and / or one or more neural processing units (NPU 1208), which may be, for example, dedicated AI accelerators for offloading artificial intelligence (AI) workloads from the CPU 1202 and GPU 1206. The processor 1200 may use instructions that, if executed, cause the processor 1200 and / or any of its components to perform some or all of the processes and techniques described elsewhere herein. The processor 1200 may include any number of memory and cache units 1210 for facilitating processing between different components of the processor 1200. The memory and cache 1210 on processor 1200 may include one or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination. Regarding processor 1200 and any components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device (e.g., cache and / or memory) internal or external to processor 1200. The results of APIs may be stored in storage devices internal or external to processor 1200, including registers, DRAM, flash memory, SRAM, cache, or other memory. One or more APIs described herein may include calls.

[0290] Processor 1200 may include a computing engine as CPU 1202, and may include any number of cores, such as, but not limited to, up to 16 cores / 22 threads. The cores in CPU 1202 may include P-cores (performance), E-cores (high efficiency), and LP-E cores (low-power, high-efficiency). Performance cores can be used for low-latency, single-threaded, computationally intensive workloads, while high-efficiency cores can be used for multi-threaded, less computationally intensive workloads. Low-power, high-efficiency cores can be used for scalable multi-threaded execution and offloading background tasks. P-cores can be used for single-threaded and limited-threaded execution, while E-cores and LP-E cores are used for multi-threaded throughput and power efficiency.

[0291] The GPU 1206 can include any number of graphics engines, such as, but not limited to, those with 8 Xe cores (up to 128 execution units or EUs). Arc TM Graphics engine (Xe LPG). For example... Figure 12 As shown, GPU 1206 may include vector engine 1210 and matrix engine 1212, which, for example, can run FP, INT, and matrix operation tasks simultaneously, individually, or in batches. GPU 1206 may include load / store unit 1214, as well as other memories, such as, but not limited to, instruction cache (I$) 1216 and L1 cache / subsystem local memory (SLM) 1218, which may, for example, store instructions for performing any of the operations described above or elsewhere herein.

[0292] The NPU 1204 may include one or more AI Boost integrates a Neural Processing Unit (NPU). The NPU 1204 can be enumerated as an integrated PCIe device to the host processor. The NPU 1204 may include one or more (e.g., two) Neural Computation Engine (NCE) tiles 1230. Each tile may be configured with any combination of, but not limited to, the following: (e.g., 2000) Multiply-Accumulate (MAC) engines 1234, a post-processing engine (not shown), an AEP processor (not shown), and memory per tile (2MB dedicated SRAM), such as... Figure 12 As shown. For general computing needs, the neural computing engine 1230 may include an inference pipeline 1232, an activation function (AF) 1236, a data transformation 1238, a load / store 1240, and a streaming hybrid architecture vector engine (SHAVE) 1228 for high-performance parallel computing, which may include a DMA (Direct Memory Access) engine 1224 for transporting data between system memory DRAM (Dynamic Random Access Memory) 1226 and a software-managed cache. The built-in device MMU (Memory Management Unit) 1222, plus the IOMMU (Input-Output Memory Management Unit) (not shown), can support multiple concurrent hardware contexts and provide secure isolation between execution contexts according to the MCDM (Microsoft Computing Driver Model) architecture. The processor 1200 may also include a media unit (not shown), which may be included on or separate from the XCD or other components of the processor 1200 to enable video playback and video processing of compressed or uncompressed data, such as using HEVC, AV1, VP9, ​​and AVC hardware-accelerated decoding support and HEVC, VP9, ​​and AVC hardware-accelerated encoding support.

[0293] Thread bootstrap ( The Thread Director (which includes firmware built into the processor 1200) can prioritize and manage the distribution of workloads, thereby sending tasks to optimized cores. For example, the Thread Director can tie P cores, E cores, and / or LP-E cores (as described above) together with task scheduling capabilities and the ability to send less demanding tasks to E cores or LP-E cores. Deep learning acceleration ( DLBoost (not shown) can provide built-in AI acceleration for training and inference workloads and may include support for VNNI (for CPU) and DP4a (for GPU) instruction sets. This instruction set can be used with OpenVINO. TM The toolkit and oneAPI are optimized to accelerate INT8 inference. For example, the software stack described elsewhere in this document can be used to leverage OpenVINO. TM The toolkit enables AI inference. The processor 1200 can be configured to execute applications, such as, but not limited to, CUDA programs.

[0294] In at least one embodiment, processor 1200 may include one or more circuits for using a neural network to generate software to be executed by a GPU by modifying software to be executed by a CPU, or otherwise perform any of the operations described above or elsewhere herein.

[0295] One or more circuits may be configured by software to use neural networks to generate software to be executed by the GPU by modifying software to be executed by the CPU, or otherwise perform any of the operations described above or elsewhere in this document.

[0296] Processor 1200 may alternatively include a processor based on Qualcomm's AIEngine Direct architecture from Santa Clara, California, or other processors sharing at least some of the components described herein. It may include any number of NPUs, GPUs, CPUs, and other associated components, such as, but not limited to, an NPU 1204 as a Hexagon NPU, a GPU 1206 as an Adreno GPU, a CPU 1202 as a Kryo or Qualcomm Oryon CPU, and a Qualcomm Sensing Hub (not shown) and a memory subsystem 1210, in any combination. Hexagon NPU 1204 may include power rails, micro-tile inference units, hardware acceleration units, tensor units, scalar units, and vector units (all not shown), which may have dedicated or shared memory (e.g., cache or memory, such as HBM3) for storing, for example, instructions for performing any of the operations described above or elsewhere herein. The Adreno GPU 1206 can provide graphics and parallel processing for AI, in formats including but not limited to 32-bit floating-point (FP32), 16-bit floating-point (FP16), and 8-bit integer (INT8). The Kryo or Qualcomm Oryon CPU 1202 can execute AI workloads and handle the contextualization of ubiquitous generative AI applications. The CPU 1202 may also include an instruction fetch unit, a renaming and deprecation unit, a memory management unit, a vector execution unit, an integer execution unit, and a load and store unit for processing and instruction management. Regarding the processor 1200 and any of its components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by the instruction fetch unit, decoded by the processor decoder or equivalent, scheduled (e.g., sequentially or out of order) for execution by the scheduler or equivalent, executed by execution logic or equivalent, reordered, and then deprecated by the renaming and deprecation unit. The API (and / or compiled instructions including the API) can be stored in any storage device (e.g., cache and / or memory) inside or outside the processor 1200. An arbitrary number of CPU cores 1202 can be included in an arbitrary number of CPU clusters, which can be coupled to memory and / or cache, such as, but not limited to, a shared L2 cache. Memory can be separate or shared; for example, the CPU clusters of CPU cores 1202 can be coupled to a memory subsystem 1210, which can include structures capable of reading and writing to memory (e.g., DRAM), system-level caches, and an arbitrary number of memory management units.The Qualcomm sensing hub (not shown) includes a miniature NPU, power rails, and conventional sensors (such as gyroscopes, accelerometers, or even barometers) that support voice and data streaming. The memory subsystem 1210 may include memory and cache on the processor 1200, which may include L1 or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination, for example, for storing information and / or instructions for performing any of the operations described above or elsewhere herein. All or part of the memory and / or cache in the memory subsystem 1210 may be shared or used individually by any component or combination of components on the processor 1200 (e.g., GPU 1206, NPU 1204, and CPU 1202).

[0297] The Qualcomm AI Engine 1200 can be programmed and controlled using a software stack to perform some or all of the operations described herein, including, for example... A neural processing SDK is provided for inference on Android, Linux, and Windows. Developer libraries and services support programming languages, virtual platforms, and compilers. At lower levels of the software stack, system software includes a basic real-time operating system (RTOS), system interfaces, and drivers. The software stack supports various operating systems, including Android, Windows, Linux, and QNX, as well as deployment and monitoring infrastructures such as Prometheus, Kubernetes, and Docker. OpenCL and DirectML are supported for direct cross-platform access to the GPU 1206. For the CPU 1202, LLVM compiler infrastructure optimizations enable accelerated and efficient AI inference. Regarding the Qualcomm AI Engine 1200 and any of its components described above or elsewhere herein, one or more APIs described herein can, for example, be compiled into instructions that can be fetched by instruction fetching logic or equivalents, decoded by processor decoders or equivalents, scheduled (e.g., sequentially or out of order) for execution by a scheduler or equivalent, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. The API (and / or compiled instructions including the API) can be stored in any storage device (e.g., cache and / or memory) inside or outside the Qualcomm AI Engine 1200. The results of the API can be stored in storage devices inside or outside the Qualcomm AI Engine 1200, including registers, DRAM, flash memory, SRAM, cache, or other memory.

[0298] In at least one embodiment, processor 1200 or Qualcomm AI engine 1200 may include one or more circuitry that can be used to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1200 or Qualcomm AI engine 1200 may include one or more circuitry that can be used to execute the JobStartStats API to at least partially cause one or more processors in the processor group to be identified at one or more clock frequencies of operation by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0299] In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry that can be used to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry that can be used to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0300] In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry that can be used to execute the DeviceGetFieldValues ​​API to at least partially cause the identification of one or more processors in the processor group to operate at one or more clock frequencies by obtaining one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0301] In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1200 or QUALCOMM AI engine 1200 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0302] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0303] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0304] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0305] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0306] Figure 13AA processor 1300 according to at least one embodiment is illustrated. The processor 1300 may include a Scalable family processor from Intel Corporation, Santa Clara, California, or other processors that share at least some of the components described herein. The processor 1300 may include one or more cores 1312(1)-1312(N) capable of performing the operations described elsewhere herein, where N is any integer greater than 1. Cores 1312(1)-1312(N) may be interconnected using ring and / or mesh interconnects. Utilizing a mesh interconnect architecture, arrays of vertical and horizontal communication paths may allow traversal from one core to another 1312(1)-1312(N) via the shortest path (jumping to the correct row along the vertical path and to the correct column along the horizontal path). For the mesh interconnect, a die may accommodate cores 1312(1)-1312(N) and may include a Converged Mesh Stop Point (CMS) grid that may be associated with cores 1312(1)-1312(N) (e.g., 1:1). Each core can be associated with a low-level cache (LLC) slice 1314(1)-1314(N), or cores 1312(1)-1312(N) can share a cache, such as a low-level cache. LLC 1314(1)-1314(N) can be inclusive or non-inclusive (having blocks not present in the high-level cache) by merging blocks from a higher-level cache (e.g., L2 cache). Each core and LLC slice can include a caching and home agent (CHA) (not shown), which can maintain cache consistency by providing resource scalability via the mesh interconnect. Super Path Interconnect ( UPI 1316 provides cache coherency capabilities. UPI 1316 enables coherent interconnects for scalable systems and allows multiple processors to share a single shared address space via links, such as, but not limited to, two or three UPI links per processor.

[0307] The processor 1300 may also include a system agent 1310, which may house and / or perform various functions, such as, but not limited to, memory management, display functions, and / or input / output (I / O) functions. For example, the processor 1300 may include one or more integrated memory controllers (IMCs) 1308. IMCs 1308 may control and manage memory, such as, but not limited to, different memory types, such as DDR RAM, such as DDR4, or other memory described elsewhere herein. The system agent 1310 may include a display controller (not shown) for supporting one or more displays. The system agent 1310 may also integrate a PCIe 1304 (e.g., up to 20 PCIe lanes), which may, for example, be connected to an external dedicated graphics connector via a DMI bus (e.g., Intel's DMI 3.0 bus) 1306. The system agent 1310 may include an image processing unit (IPU) (not shown) that integrates an on-die image signal processor (ISP). Structure 1302 provides scalability for connecting to other nodes (e.g., processors, such as processor 1300) and can, for example, be connected to Cornelis Networks (…). It can be used together with elements of a scalable system framework that provides performance for high-performance computing (HPC) workloads and the ability to scale to tens of thousands of nodes.

[0308] Figure 13B Components within a core 1312 according to at least one embodiment are illustrated. The core 1312 may include a front-end 1318, a back-end or execution engine 1332, and a memory subsystem 1342. The front-end 1318 may provide operations (e.g., operations described elsewhere herein) to the execution engine 1332 by decoding instructions stored in memory. For example, the front-end 1318 may include micro-operation (μOps) cache paths and / or conventional paths, and a branch prediction unit 1321 capable of determining path instructions. A conventional path of instructions might involve fetching variable-length (e.g., x86) instructions from an L1 instruction cache 1320 using instruction fetching and pre-decoding 1322, queuing these instructions into an instruction queue 1324, and decoding the instructions into μOps that can be provided to an allocation queue 1328 using a decoder 1326. Alternatively, the μOps cache path may include a cache that includes decoded μOps (μOps 1330) that can be sent to the allocation queue 1328. The allocation queue 1328 can act as an interface between the front end 1318 and the execution engine 1332, and can provide instructions to the execution engine 1332. For example, one or more APIs described herein can be compiled into instructions that can be stored, processed, and executed by the front end 1318 and the execution engine 1332, and stored in the memory subsystem 1342.

[0309] Execution engine 1332 can receive micro-operations into reordering buffer 1334, which can register, rename, and deregister μOPs. μOPs can be sent from the reordering buffer to scheduler 1336, which can be connected to one or more different execution units 1338, which can be connected to address generation units (AGUs) 1340. Execution units 1338 can perform operations such as basic arithmetic logic unit (ALU) operations, multiplication, division, and / or more complex operations, such as, but not limited to, various vector operations. Scheduler 1336 can manage the queuing of μOPs for one or more execution units 1338 based on, for example, the operations that need to be performed.

[0310] The memory subsystem 1342 can handle load and store requests as well as sorting operations. For example, μOPs may be associated with memory accesses (e.g., load and store), and these μOPs can be sent through dedicated scheduler ports that can perform these memory operations. For example, store and load operations can be sent to load and store buffers 1344. The memory subsystem 1342 may also include shared or separate L1 data and instruction caches 1346, and an L2 cache 1348 that can be used and shared by the L1 data and instruction caches 1346. (As described above regarding...) Figure 13A Each core 1312 can be connected to a slice of a level 3 cache (e.g., LLC1314), which can be shared by all cores 1312.

[0311] In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0312] In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0313] In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0314] In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, processor 1300 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0315] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0316] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0317] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0318] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0319] Figure 14 An AI accelerator 1400 according to at least one embodiment is illustrated. Processor 1400 may include a processor with an AI accelerator architecture manufactured by Intel Corporation, Santa Clara, California, or other processors sharing at least some of the components described herein. AI accelerator 1400 may use instructions that, if executed by AI accelerator 1400, cause AI accelerator 1400 to perform some or all of the processes and techniques described elsewhere herein. For example, with respect to AI accelerator 1400 and any components described above or elsewhere herein, one or more APIs described herein may, for example, be compiled into instructions that may be fetched by instruction fetching logic or equivalents, decoded by processor decoders or equivalents, scheduled (e.g., sequentially or out of order) by a scheduler or equivalent for execution, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. APIs (and / or compiled instructions including APIs) may be stored in any storage device internal or external to AI accelerator 1400 (e.g., in cache and / or memory). The results of the API can be stored in internal or external storage devices of the AI ​​accelerator 1400, including registers, DRAM, flash memory, SRAM, cache, or other memory. The AI ​​accelerator 1400 may include one or more compute dies, which may include homogeneous or heterogeneous processors. The compute dies may include one or more central processing units (CPUs), one or more graphics processing units (GPUs), or a combination of both.

[0320] In at least one embodiment, the computational die may include a computational engine for performing AI computations. In at least one embodiment, the computational die of the AI ​​accelerator 1400 may be split into any number (e.g., four) clusters, which may be referred to as DCORE (Deep Learning Core) 1406, and include any number of matrix multiplication engines (MME) 1408, tensor processor cores (TPC) 1410, memory management units 1412, and L2 caches 1414 in any combination. The MME 1408 may perform operations using matrix multiplication, such as fully connected layers, convolutions, and batch general matrix multiplication (GEMM). The MME 1408 may be equipped with a multiplication-accumulation unit (MAC) (not shown), which may perform general matrix multiplication (GEMM) operations, such as, but not limited to, AxB multiplication, which involves generating a tensor C [NxM] from two input tensors A [NxK] and B [KxN]. The MME 1408 may be programmed with array dimensions, positions, data types, and various operands. The MME 1408 can retrieve tensors A and B from memory and pull them into its streaming buffer for parallel matrix multiplication by the MAC. After completion, the MME 1408 can push tensor C back to memory. The TPC 1410 may include any number of scalar units for performing scalar operations, any number of vector units for performing vector operations, any number of register files or local memory units (e.g., vector local memory), and load and store components for instructions, which may be coupled to memory or caches (e.g., HBM, L3 cache, and / or L2 cache) (all not shown). The TPC can support different types of parallel processing, such as Very Long Instruction Word (VLIW) Single Instruction Multiple Data (SIMD) data types such as, but not limited to, FP32, BF16, FP16, and FP8 (both E4M3 and E5M2), UINT32, INT32, UINT16, INT16, UINT8, and INT8 data types. Any number of computational dies can be interconnected. The interconnection of connectable computing dies can be via an intermediary bridge, which is transparent to the software, for example.

[0321] The memory on the AI ​​accelerator 1400 may include one or more levels of cache (e.g., L1, L2, L3, and / or last-level cache) and high-bandwidth memory (e.g., HBM2e or HBM3) in any combination. The memory and / or cache system may be unified or separate. The compute die of the AI ​​accelerator 1400 may include on-chip memory comprising one or more levels (e.g., two levels) of cache. On-chip SRAM or other memory described elsewhere herein may be used as a unified last-level cache (L3) or split into multiple slices of L2 cache accessible to the MME 1408 and TPC 1410 groups. Using on-chip memory as an L2 or L3 cache is entirely software-configurable, and the software can dynamically determine its optimal cache allocation based on I / O tensors. AI accelerator 1400 may include one or more memory management units (MMUs) 1422 for managing memory, such as allowing the AI ​​accelerator 1400 memory subsystem to run in virtual space when accessing VRAM.

[0322] AI accelerator 1400 may include a communication port (e.g., a PCIe Gen5 x16 port) 1402 for communicating with a host and scheduling and synchronization unit 1404. AI accelerator 1400 may include a media unit 1416, which may include any number or combination of media decoder engines (DECs) 1420 and rotation engines (ROTs) 1418. AI accelerator 1400 may include a network unit 1424, which may include any number or combination of network ports 1426 and an accompanying RDMA engine 1428, L2 cache, and memory (e.g., HBM2e or HBM3) stack. AI accelerator 1400 may include a programmable control path entity (not shown) for managing the parallel and efficient execution of the various engines. The control path may include a submission queue (SQ) that can be issued by the runtime system, a completion queue (CQ) that can be used for job completion reporting, a programmable scheduling mechanism that can be used for task scheduling, a programmable hardware synchronization mechanism or "synchronization manager (SM)" that can be used for hardware synchronization, and a programmable interrupt service mechanism or "interrupt manager (INTR)" that can pass asynchronous events to drivers.

[0323] AI Accelerator 1400 may include media decoding units supporting video formats such as, but not limited to, HEVC, Progressive H.264, SVC base layer, MVC, VP9, ​​JPEG, and Progressive JPEG. AI Accelerator 1400 may support post-processing of the decoded media stream, such as, but not limited to, image downsizing (image resizing), vertical and horizontal scaling at different scaling ratios, image enlargement, image cropping, bilinear scaling, and Lancos scaling. AI Accelerator 1400 may implement two post-processing channels per decoder unit, one for scalar (up and down) and the other solely for outputting the original image. AI Accelerator 1400 may include a hardware rotation engine that performs the following transformations on the input image: 2D rotation, 3D rotation, projection, image warping and de-warping, resampling of the input data at user-defined coordinates, and rescaling.

[0324] The RDMA 1428 based on converged Ethernet on the AI ​​accelerator 1400 enables scaling from a single node (i.e., from a single AI accelerator 1400 to hundreds or thousands of nodes or AI accelerators 1400). The network subsystem 1424 may include... The accelerator 1400 includes an In-Gigabit Ethernet Communication Library (IGCL), a master controller coordinating data movement, and a programmable scheduling mechanism that enables smooth engine activation while maintaining task dependencies. The accelerator network subsystem may include a Gigabit Ethernet NIC port 1426, a Layer 2 MAC (not shown), and an RDMA engine 1428. The AI ​​accelerator 1400 may include an aggregation engine for performing summation activities. All engines in the processor 1400 can run in parallel; for example, the MME 1408, TPC 1410, and NIC 1426 can all operate simultaneously. Dependencies may exist between operations running on different engines; for example, the output of one engine may be used as the input of another engine, and / or the MME, TPC, and NIC may be scheduled to run in parallel. When one engine completes its execution, another engine can be scheduled to begin working on the next operation (executed immediately after its input is ready).

[0325] The AI ​​accelerator 1400 can be operated and controlled using a software layer 1428, which may include low-level components such as, but not limited to, a graph compiler, an automatic kernel fusionist and pre-compiled kernel libraries, and integrations with the AI ​​ecosystem such as, but not limited to, PyTorch, DeepSpeed, Hugging Face, vLLM, Ray, etc., or as described elsewhere in this document regarding software and programming platforms. The software layer 1428 may include implementations of algorithms such as, but not limited to, paged attention, flash attention, etc. The software layer 1428 can generate optimized binary code that implements a given model topology, such as, but not limited to, performing operator fusion, data layout management, parallelization, pipeline and memory management, and graph-level optimization.

[0326] In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in a processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0327] In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group that will operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0328] In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify one or more processors in a processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0329] In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, the AI ​​accelerator 1400 may include one or more circuitry configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0330] In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStartStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies by using one or more activity levels of one or more processors to be measured at one or more indicated intervals, or otherwise perform any of the operations described above or elsewhere herein.

[0331] In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause one or more measurements of one or more activity levels of one or more processors to be stopped, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobStopStats API to at least partially cause identification of one or more processors in the processor group to operate at one or more clock frequencies by stopping measurements of one or more activity levels of one or more processors, or otherwise perform any of the operations described above or elsewhere herein.

[0332] In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the DeviceGetFieldValues ​​API to at least partially cause, by obtaining one or more activity levels of one or more processors, to identify that one or more processors in the processor group will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0333] In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause one or more statistics corresponding to one or more activity levels of one or more processors to be indicated to one or more users, or otherwise perform any of the operations described above or elsewhere herein. In at least one embodiment, one or more circuits may be software-configured to execute the JobGetStats API to at least partially cause the identification of one or more processors in the processor group that will operate at one or more clock frequencies, or otherwise perform any of the operations described above or elsewhere herein.

[0334] This paper describes a neuromorphic computing system employing a multi-core architecture, where each core houses computing elements including neurons, synapses with on-chip learning capabilities, and local memory for storing synaptic weights and routing tables. Figure 15This is a simplified block diagram 1500 illustrating at least a portion of an example of such a neuromorphic computing device 1505 according to at least one embodiment. The neuromorphic computing device 1505 may include a neuromorphic processor from Intel Corporation, Santa Clara, California, or other processors that include at least a portion of the components described herein. As shown in this example, the device 1505 may be equipped with a network 1510 consisting of multiple neural network cores interconnected by a network on the device, thereby potentially defining multiple distinct connections between the cores. For example, the device 1505 may provide a network 1510 of spiking neural network cores, each core communicating via short packet spiking messages sent from one core to another through network channels. Each core (e.g., 1515) may have processing and memory resources, as well as logic, for implementing a number of primitive nonlinear time computation elements, such as, but not limited to, multiple (e.g., more than 1000) distinct artificial neurons (referred to herein as “neurons”). For example, each core may be able to implement multiple neurons concurrently, allowing the neuromorphic core to implement many, many neurons using the device 1505. With respect to the neuromorphic computing device 1505 and any components described above or elsewhere herein, one or more APIs or equivalents described herein may, for example, be compiled into instructions or equivalents that may be fetched by instruction fetching logic or equivalents, decoded by processor decoder or equivalents, scheduled (e.g., sequentially or out of order) for execution by scheduler or equivalents, executed by execution logic or equivalents, reordered, and then retired by retirement logic or equivalents. The APIs (and / or compiled instructions including the APIs) may be stored in any storage device (e.g., cache and / or memory) internal or external to the neuromorphic computing device 1505. The results of the APIs may be stored in storage devices internal or external to the neuromorphic computing device 1505, including registers, DRAM, flash memory, SRAM, cache, or other memory equivalents.

[0335] continue Figure 15For example, neuromorphic computing device 1505 may also include processor 1520 and system memory 1525 for implementing one or more components to manage and provide the functionality of neuromorphic computing device 1505. For instance, a system manager 1530 may be provided to manage the global attributes and operations of neuromorphic computing device 1505 (e.g., attributes affecting core network 1510, multiple cores in network 1510, interconnection of neuromorphic computing device 1505 with other devices, managing access to global system memory 1525, and other potential examples). In one example, system manager 1530 may manage the definition and configuration of specific routing tables for individual routers in network 1510, orchestration of network definitions and attributes to be applied to network 1510 (e.g., weights, attenuation rates, etc.), core synchronization and time multiplexing management, routing input to appropriate cores, and other potential functions.

[0336] As another example, the neuromorphic computing device 1505 may also include a programming interface 1535 through which a user or system can specify the neural network definition to be applied (e.g., via routing tables and individual neuron attributes), implemented by the neuromorphic core grid 1510. A software-based programming tool may be provided with or separately from the neuromorphic computing device 1505, through which a user can provide a definition for a specific neural network to be implemented using the neuromorphic core network 1510. The programming interface 1535 can receive input from a programmer, then generate the corresponding routing table and populate the specified parameters into the local memory of each neuromorphic core (e.g., 1515) to implement the corresponding custom artificial neural network implemented by the neuromorphic core 1515.

[0337] In certain circumstances, the neuromorphic computing device 1505 can advantageously engage and interoperate with other devices, including general-purpose computing devices, to enable specific applications and use cases. Therefore, in some cases, external interface logic 1540 may be provided to communicate with one or more other devices (e.g., via one or more defined communication protocols). External interface 1540 may be used to accept input data from another device or an external memory controller used as an input data source. External interface 1540 may additionally or alternatively be used to allow the results or outputs of computations performed using the neural network implemented using the neuromorphic computing device 1505 to be provided to another device (e.g., another general-purpose processor implementing machine learning algorithms) to enable additional applications and enhancements, among other examples.

[0338] like Figure 15The diagram illustrates a network 1510 of multiple neural network cores interconnected by a network on the device, showing a portion of a network structure interconnecting multiple neuromorphic cores (e.g., 1515a-d). For example, several neuromorphic cores (e.g., 1515a-d) can be provided in a mesh, each core interconnected via a network including multiple routers (e.g., 1550). In one implementation, each neuromorphic core (e.g., 1515a-d) can be connected to a single router (e.g., 1550) in the routers, and the router can be connected to at least one other router (e.g., [other router]). Figure 15 (As shown at 1510 in [reference]). As an example, in one particular implementation, four neuromorphic cores (e.g., 1515a-d) can be connected to a single router (e.g., 1550), and each router 1550 can be connected to two or more other routers to form a multi-core mesh, thereby allowing each neuromorphic core to interconnect with every other neuromorphic core in the neuromorphic computing device 1505. Furthermore, since each neuromorphic core can be configured to implement multiple different neurons, the router network of the neuromorphic computing device 1505 can similarly implement connections or artificial synapses (or simply "synapses") defined between any two of the potentially many (e.g., 30,000+) neurons defined using the neuromorphic core network 1510 provided in the neuromorphic computing device 1505.

[0339] Figure 15A block diagram of the internal components of an example implementation of the neuromorphic core 1515 is shown. In one example, a single neuromorphic core may implement a number of neurons (e.g., 1024) that share the architectural resources of the neuromorphic core 1515 in a time-division multiplexing manner. In one example, each neuromorphic core 1515 may include a processor block 1555 capable of executing arithmetic functions and routing related to the implementation of the digitally implemented artificial neurons, such as, but not limited to, those explained herein. Each neuromorphic core 1515 may also provide local memory in which routing tables of the neural network can be stored and accessed, accumulated potentials of each cell body of each neuron implemented using core 1515 can be tracked, parameters of each neuron implemented by core 1515 can be recorded, and other data and usage can be recorded. Components or architectural resources of the neuromorphic core 1515 may also include: an input interface 1565 for receiving input spike messages generated by other neurons on other neuromorphic cores; and an output interface 1570 for sending spike messages to other neuromorphic cores via a mesh network 1510. In some instances, the routing logic of the neuromorphic core 1515 can be implemented at least partially using the output interface 1570. Furthermore, in some cases, the core (e.g., 1515) can implement multiple neurons within an example SNN, and some of these neurons can be interconnected. In this case, spiking messages sent between neurons hosted on the core 1515 can forgo communication via the routing structure of the neuromorphic computing device 1505 and can be managed locally within the specific neuromorphic core 1515.

[0340] Each neuromorphic core may also include logic for implementing artificial dendrites 1580 and artificial cell bodies 1585 (hereinafter referred to as “dendrites” and “cell bodies”, respectively) for each neuron 1575. Dendrite 1580 may be a hardware-implemented process for receiving impulses from network 1510. Cell body 1585 may be a hardware-implemented process for receiving the current time-accumulated neurotransmitter mass of each dendrite and evolving the potential states of each dendrite and cell body to generate outgoing impulse messages at appropriate times. Dendrite 1580 may be defined for each connection receiving input from another source (e.g., another neuron). In one implementation, the dendritic process 1580 may receive and process the impulse message as it arrives serially from network 1510 in a time-division multiplexed manner. With the reception of impulses, neuronal activation (tracked using cell body 1585 (and local memory 1560)) may increase. When the activation of a neuron 1575 exceeds a threshold set for neuron 1575, neuron 1575 generates a spike message, which is propagated via output interface 1570 to a fixed set of fan-out neurons. The network distributes the spike messages to all destination neurons, which in turn can update their activation in a transient, time-dependent manner in response. This can lead to some of the destination neurons also exceeding their corresponding thresholds and triggering further spike messages, just as in real biological neural networks.

[0341] As described above, the neuromorphic computing device 1505 can reliably implement spiking-based neural computing models. Such models are also referred to as spiking neural networks (SNNs). In addition to neuronal and synaptic states, SNNs incorporate temporal concepts. For example, in SNNs, communication occurs via event-driven action potentials or spiking, which convey no explicit information other than the spiking time and the implicit source and destination neuron pairs corresponding to the spiking transmission. The computation of the result of a dynamic nonlinear integral as a weighted spiking input occurs in each neuron. In some implementations, recurrent and dynamic feedback can be incorporated into the SNN computation model. Furthermore, various network connectivity models can be employed to model a wide range of real-world networks or relationships, including fully connected (all-to-all) networks, feedforward trees, completely random projections, "small-world" networks, and other examples. Isomorphic two-dimensional networks at the neuromorphic core (e.g., but not limited to...) Figure 15The network shown in the example can advantageously support all these network models. Since some or all of the cores of the neuromorphic computing device 1505 can be connected, some or all of the neurons defined in a core can also be fully connected via a certain number of router hops. The neuromorphic computing device 1505 may also include fully configurable routing tables for defining various neural networks by allowing neurons in each core to distribute their spurs to any number of cores in the grid 1510 to achieve a completely arbitrary connection graph.

[0342] In improved implementations of systems capable of supporting SNNs, for example, but not limited to... Figure 15 The example illustrates a very large-scale integrated circuit (VLSI) hardware device that can provide high-speed, reliable circuitry to implement SNNs (Spiritual Neural Networks) to model the information processing algorithms used by the brain, but in a more programmable way. For instance, while a biological brain can only perform a specific set of defined behaviors (a consequence of years of development), a neuromorphic processor device can provide the ability to rapidly reprogram all neural parameters. Therefore, a single neuromorphic processor can be used to implement a wider range of behaviors than a single slice of biological brain tissue. This distinction can be achieved by employing neuromorphic processors with neuromorphic designs that are radically different from those found in natural neural circuitry.

[0343] As an example, a neuromorphic processor can implement a spontaneous neural network (SNN) using time-multiplexed computation in both a spiking communication network and the neuronal mechanism of the neuromorphic computing device 1505. Therefore, the physical circuitry of the neuromorphic computing device 1505 can be shared by many neurons to achieve a higher neuron density. Through time multiplexing, the network can connect N cores with a total wiring length of O(N), while the length of discrete point-to-point wiring will be extended to O(N). 2 This significantly reduces wiring resources to accommodate planar and non-plastic VLSI routing techniques, among other examples. In the neuromorphic core, time multiplexing can be implemented through dense memory allocation, for example, using static random access memory (SRAM) with a shared bus, address decoding logic, and other multiplexed logic elements. The state of each neuron can be stored in the processor's memory, where data describing the state of each neuron includes the state of the collective synapse of each neuron, all currents and voltages on its membrane, and other example information (e.g., but not limited to configuration and other information).

[0344] Neuromorphic processors can be implemented in a “digital” manner, unlike other processors that employ more “analog” or “isomorphic” neuromorphic approaches. For example, a digital implementation can use digital adder and multiplier circuitry to integrate synaptic currents, in contrast to an analog isomorphic neuromorphic approach that accumulates charge on capacitors in a manner similar to how neurons accumulate synaptic charge on their lipid membranes. For instance, the accumulated synaptic charge for each neuron can be stored in the local memory of the corresponding core. Furthermore, at the architectural level of an example digital neuromorphic processor, reliable and deterministic operation can be achieved through time synchronization across the core network, ensuring that any two executions of the design, given the same initial conditions and configuration, will produce the same results. Asynchronicity can be reserved at the circuit level to allow individual cores to operate as quickly and freely as possible while maintaining determinism at the system level. Therefore, in neural computing, the concept of time as a time variable can be abstracted away from the “wall clock” time used by the hardware to perform computations. Thus, in some implementations, a time synchronization mechanism can be provided that globally synchronizes the neuromorphic cores at discrete time intervals. Synchronization mechanisms allow neural computation to be completed at the fastest speed allowed by the circuitry, and there is a difference between the runtime and the biological time for modeling neuromorphic systems.

[0345] In operation, the neuromorphic computing device 1505 can start in an idle state when all neuromorphic cores are inactive. As each core asynchronously loops through its neurons, it generates impulse messages, which are routed by the mesh interconnect to the appropriate destination core containing all destination neurons. The implementation of multiple neurons on a single neuromorphic core can be time-multiplexed, and time steps can be defined, where all impulses involving multiple neurons can be processed and considered using the shared resources of the respective cores. When each core completes its service to its neurons within the corresponding time step, in some implementations, the core can communicate with neighboring cores using synchronization messages (e.g., using a handshake) to refresh the mesh of all transmitted impulse messages, allowing the core to safely determine that all impulses have been serviced within a certain time step. At this point, all cores can be considered synchronized, allowing them to advance their time steps and return to the initial state to begin the next time step.

[0346] Given this context, as described above, a device (e.g., 1505) can be provided to realize an interconnected neuromorphic core network 1510, wherein the core 1515 can realize multiple artificial neurons capable of interconnecting to realize an SNN. Each neuromorphic core (e.g., 1515) can provide two loosely coupled asynchronous processes: an input dendrite process (e.g., 1580) that receives impulses from the network 1510 and applies them to the appropriate destination dendritic chamber at an appropriate future time; and an output c...

Claims

1. A processor, comprising: One or more circuits, said one or more circuits for executing an application programming interface (API) to cause one or more activity levels of one or more processors to be measured at one or more indicated intervals.

2. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more activity levels of the one or more processors are measured to identify one or more clock frequencies at which the one or more processors are to operate.

3. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more activity levels of the one or more processors are measured at least in part based on one or more indications of one or more processor groups including the one or more processors.

4. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more activity levels of the one or more processors are measured at least in part based on one or more indications of one or more instances of processor management software.

5. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more activity levels of the one or more processors are measured at least in part based on one or more indications of one or more software programs to be executed by the one or more processors.

6. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more activity levels of the one or more processors are measured at least in part based on one or more indications of one or more types of the activity level to be measured.

7. The processor of claim 1, wherein the one or more circuits are configured to execute the API such that the one or more processors concurrently execute one or more software programs as part of one or more data centers.

8. A system comprising: One or more processors, the one or more processors being configured to execute an application programming interface (API) such that one or more activity levels of the one or more processors are measured at one or more indicated intervals.

9. The system of claim 8, wherein the one or more processors are configured to execute the API such that the one or more activity levels of the one or more processors are measured to identify the one or more processors at one or more clock frequencies in which they operate while executing one or more software programs.

10. The system of claim 8, wherein the one or more processors are configured to execute the API such that the activity levels of the one or more processors are measured at least in part based on one or more indications of a data center processor group comprising the one or more processors.

11. The system of claim 8, wherein the one or more processors are configured to execute the API such that the activity levels of the one or more processors are measured at least in part based on one or more indications of one or more instances of data center processor management software.

12. The system of claim 8, wherein the one or more processors are configured to execute the API such that the activity level of the one or more processors is measured at least in part based on one or more indications of one or more software programs to be executed concurrently by the one or more processors.

13. The system of claim 8, wherein the one or more processors are configured to execute the API such that the activity levels of the one or more processors are measured at least in part based on one or more indications of the workload factor to be measured.

14. The system of claim 8, wherein the one or more processors are configured to execute the API such that the one or more processors, as part of one or more data centers, improve the synchronization of one or more software programs.

15. A method comprising: Execute the application programming interface (API) to make one or more activity levels of one or more processors measured at one or more indicated intervals.

16. The method of claim 15, further comprising: The API is executed such that the activity levels of the one or more processors are measured, thereby identifying the one or more processor groups at one or more clock frequencies when executing one or more software programs.

17. The method of claim 15, further comprising: The API is executed such that the activity levels of the one or more processors are measured at least in part based on one or more indications from one or more processor groups in one or more data centers that include the one or more processors.

18. The method of claim 15, further comprising: The API is executed such that the activity levels of the one or more processors are measured, at least in part, based on one or more indications from one or more instances of data center processor management software used to communicate with one or more drivers of the one or more processors.

19. The method of claim 15, further comprising: The API is executed such that the activity levels of the one or more processors are measured at least in part based on one or more indications of one or more software programs that will be executed concurrently by one or more processor groups including the one or more processors.

20. The method of claim 15, further comprising: The API is executed such that the activity levels of the one or more processors are measured and used to calculate the average activity level of the one or more processors.

Images