PROCESSOR SCHEDULING

Abstract

Devices, systems, and methods for allocating processors used by software are disclosed. In at least one embodiment, it is prevented that one or more processors of a group of processors used to execute a software program are assigned to execute a second software program until at least a certain period of time has elapsed.

Description

Area

[0001] At least one embodiment relates to the processing of resources for scheduling or scheduling software jobs. At least one embodiment relates to processors or computer systems for rescheduling software that is to be executed by one or more processors when a processor executing this software fails. background

[0002] Software running across multiple processors restarts if one of those processors fails. However, sometimes other software may occupy some of the remaining working processors while the first software is restarting. Brief description of the drawings Fig. Figure 1 shows a block diagram of a processor scheduling system according to at least one embodiment; Fig. 2 shows a block diagram of a system with a control plane according to at least one embodiment; Fig. Figure 3 shows a block diagram of a system with processor process monitoring according to at least one embodiment; Fig. Figure 4 shows a block diagram of a method for assigning one or more processors to a group according to at least one embodiment; Fig. Figure 5 shows an example of a processor according to at least one embodiment; Fig. 6. A block diagram of a driver and / or runtime environment comprising one or more libraries to provide one or more application programming interfaces (APIs), according to at least one embodiment; Fig. Figure 7 illustrates a distributed system according to at least one embodiment; Fig. Figure 8 illustrates an exemplary data center according to at least one embodiment; Fig. Figure 9 illustrates a client-server network according to at least one embodiment; Fig. 10 illustrates an example of a computer network according to at least one embodiment; Fig. 11A illustrates a networked computer system according to at least one embodiment; Fig. 11B illustrates a networked computer system according to at least one embodiment; Fig. 11C illustrates a networked computer system according to at least one embodiment; Fig. 12 illustrates, according to at least one embodiment, one or more components of a system environment in which services can be offered as third-party network services; Fig. Figure 13 illustrates a cloud computing environment according to at least one embodiment; Fig. Figure 14 illustrates, according to at least one embodiment, a set of functional abstraction layers provided by a cloud computing environment; Fig. 15 illustrates a chip-level supercomputer according to at least one embodiment; Fig. Figure 16 illustrates a rack-module-level supercomputer according to at least one embodiment; Fig. Figure 17 illustrates a rack-level supercomputer according to at least one embodiment; Fig. 18 illustrates a supercomputer at the overall system level according to at least one embodiment; Fig. 19A illustrates an inference and / or training logic according to at least one embodiment; Fig. 19B illustrates an inference and / or training logic according to at least one embodiment; Fig. Figure 20 illustrates the training and deployment of a neural network according to at least one embodiment; Fig. 21 illustrates an architecture of a system of a network according to at least one embodiment; Fig. 22 illustrates an architecture of a system of a network according to at least one embodiment; Fig. 23 illustrates a control-level protocol stack according to at least one embodiment; Fig. 24 illustrates a user-level protocol stack according to at least one embodiment; Fig. 25 illustrates components of a core network according to at least one embodiment; Fig. Figure 26 illustrates components of a system for supporting network function virtualization (NFV) according to at least one embodiment; Fig. 27 illustrates a processing system according to at least one embodiment; Fig. 28 illustrates a computer system according to at least one embodiment; Fig. 29 illustrates a system according to at least one embodiment; Fig. Figure 30 illustrates an exemplary integrated circuit according to at least one embodiment; Fig. 31 illustrates a computing system according to at least one embodiment; Fig. 32 illustrates an APU according to at least one embodiment; Fig. 33 illustrates a CPU according to at least one embodiment; Fig. Figure 34 illustrates an exemplary accelerator integration slice according to at least one embodiment; Fig. 35A-35B illustrate exemplary graphics processors according to at least one embodiment; Fig. 36A illustrates a graphics kernel according to at least one embodiment; Fig. 36B illustrates a GPCPU according to at least one embodiment; Fig. 37A illustrates a parallel processor according to at least one embodiment; Fig. Figure 37B illustrates a processing cluster according to at least one embodiment; Fig. 37C illustrates a graphics multiprocessor according to at least one embodiment; Fig. 38 illustrates a software stack of a programming platform according to at least one embodiment; Fig. Figure 39 illustrates a CUDA implementation of a software stack consisting of Fig. 38 according to at least one embodiment; Fig. Figure 40 illustrates a ROCm implementation of a software stack consisting of Fig. 38 according to at least one embodiment; Fig. Figure 41 illustrates an OpenCL implementation of a software stack consisting of Fig. 38 according to at least one embodiment; Fig. 42 illustrates, according to at least one embodiment, software supported by a programming platform; and Fig. Figure 43 illustrates compiling code for execution on programming platforms from the Fig. 3-41 according to at least one embodiment. Detailed description

[0003] The following description provides numerous specific details to facilitate a more thorough understanding of at least one embodiment. However, it is obvious to a person skilled in the art that the inventive concepts can also be implemented without one or more of these specific details.

[0004] In at least one embodiment, a processor comprises one or more circuits to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain time interval has elapsed. In at least one embodiment, this first software program is restarted to be executed by one or more processors of this group of processors. In at least one embodiment, this first software program is reset to a checkpoint to be executed by one or more processors of this group of processors. In at least one embodiment, this one or more processors are assigned to execute only this first software program before this checkpoint expires.In at least one embodiment, this sequence includes a time period that must elapse after a failure of this group of processors. In at least one embodiment, this sequence includes a predetermined time after a failure of this group of processors. In at least one embodiment, this second software differs from this first software program.

[0005] Fig. Figure 1 shows a block diagram of a processor scheduling system 100 (“System 100”) according to at least one embodiment. In at least one embodiment, the System 100 comprises one or more processors comprising one or more circuits to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least one time interval has elapsed.

[0006] In at least one embodiment, one or more aspects of one or more embodiments that are associated with Fig. 1 are described, combined with one or more aspects of one or more embodiments, including those which, at least in conjunction with the Fig. 2 to 43 are described. In at least one embodiment, one or more processors perform one or more operations of the system 100. In at least one embodiment, one or more processors performing one or more operations of the system 100 are any processor or a combination of processors, including processor(s) 108 and those that are at least in conjunction with the Fig. 2-43 are described.

[0007] In at least one embodiment, as used in each implementation described herein, terms such as "system," "facility," "components," and "module," as well as nominalized verbs (e.g., compiler, scheduler, manager, and / or other terms), each refer to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. In at least one embodiment, each combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein is referred to as a component.In at least one embodiment, all components described herein are combined and / or communicatively connected to at least one other component, regardless of how such components are described as combined and / or communicatively connected in other embodiments. In at least one embodiment, software may be embodied as a software package, code, and / or instruction set or instructions. In at least one embodiment, the hardware comprises, individually or in any combination, hardwired circuits, programmable circuits, state machine circuits, fixed-function circuits, execution unit circuits, and / or firmware that stores instructions executed by programmable circuits. In at least one embodiment, modules may be implemented collectively or individually as a circuit that is part of a larger system, for example, an integrated circuit (IC), a system-on-a-chip (SoC), etc.In at least one embodiment, one or more architectures of arbitrary circuits of one or more modules are represented as a register-transfer-level (RTL) representation and / or another fabless representation that can be licensed and / or used in tape-out, a final stage in IC design prior to use in the manufacture of an IC.

[0008] In at least one embodiment, System 100 is any computing system comprising one or more data centers or other facilities that house computing and networking equipment. In at least one embodiment, System 100 is used to perform high-performance computing tasks, neural network training, neural network inference, or a combination thereof. In at least one embodiment, System 100 comprises an edge computing system, an accelerated computing system, a cloud computing system, a hybrid cloud computing system, or a combination thereof. In at least one embodiment, System 100 is a computer system comprising multiple distributed components connected by a network, such as an internet network.In at least one embodiment, System 100 is used in fields such as healthcare, genomics, engineering, aerospace, urban planning, graphics processing, finance, data storage and management, e-commerce, meteorology, physics modeling, or a combination thereof. In at least one embodiment, System 100 is used to perform artificial intelligence (AI) tasks such as image classification, image segmentation, autonomous driving, manufacturing defect identification, or a combination thereof. In at least one embodiment, neural networks are a type of AI.

[0009] In at least one embodiment, the system 100 has a user interface 102 through which a user provides inputs that supply information about one or more software workloads. In at least one embodiment, a software workload is referred to as a job. In at least one embodiment, the user interface 102 is a user interface of the job scheduler 110. In at least one embodiment, at least one section of the job scheduler 110 is implemented on a computing device that operates the user interface 102. In at least one embodiment, the user interface 102 is a user interface of a processor management application. In at least one embodiment, a processor management application is any combination of hardware, firmware, or software, such as the data center processor management module 120, which is further described herein.In at least one embodiment, at least one section of the data center processor management module 120 is implemented on a computing device that operates the user interface 102.

[0010] In at least one embodiment, the user interface 102 is communicatively connected to the network 104. In at least one embodiment, the network 104 can be one or more networks of any type, for example, a managed network (e.g., a corporate network), a cloud network, the internet, a local private network, or a combination thereof. In one embodiment, the network 104 is a local network. In at least one embodiment, the network 104 is communicatively connected to one or more components of the data center 106.

[0011] In at least one embodiment, the system 100 includes a data center 106. In at least one embodiment, the data center 106 is one or more data centers. In at least one embodiment, a data center is any facility that houses computing equipment and network equipment. In at least one embodiment, a data center includes processors that perform operations in parallel to process massive, multidimensional datasets. In at least one embodiment, a data center performs one or more AI tasks. In at least one embodiment, a user remotely accesses at least a portion of the computing resources of the data center 106 via the network 104 to schedule and execute jobs.

[0012] In at least one embodiment, the system 100 comprises one or more processors 108, which are any processor or combination of processors, including those associated with the Fig. 2 to 44 processors described herein. In at least one embodiment, each processor described herein, including processors 108, comprises one or more circuits. In at least one embodiment, processor(s) 108 is one or more processors implemented in a computing system designed to perform AI tasks such as image classification, autonomous driving, or a combination thereof. In at least one embodiment, processor(s) 108 is one or more processors implemented in an edge computing device, a workstation, a server, or a combination thereof, such as an NVIDIA® DGX™ workstation. In at least one embodiment, processor(s) 108 is one or more AMD® Epics™ embedded processors and / or one or more NVIDIA® A100™ GPUs.In at least one embodiment, the processor(s) 108 is / are one or more different types of processors implemented as part of a heterogeneous computing system.

[0013] In at least one embodiment, the processors 108 are a group of processors. In at least one embodiment, two or more processors 108 are installed at different locations, for example, in two different data centers connected to each other via a network. In at least one embodiment, the processors 108 are one or more graphics processing units (GPUs) of a group of GPUs. In at least one embodiment, a group of GPUs is referred to as a GPU cluster. In at least one embodiment, the processor(s) 108 are one or more sections of one or more GPUs, each section comprising a section of GPU memory and a section of GPU computing hardware configured to operate as an independent, separate, and complete GPU. In at least one embodiment, a section of GPU computing hardware is a section of streaming multiprocessors (SMs) of a GPU.In at least one embodiment, the processor(s) are 108 sections of one or more GPUs and are referred to as partitions. In at least one embodiment, the processor(s) are 108 sections of one or more GPUs configured by a GPU partitioning system such as NVIDIA® Multi-Instance GPUs (MIG).

[0014] In at least one embodiment, the processor(s) 108 execute one or more jobs 109, for example, jobs 0, 109a to n, 109n. In at least one embodiment, one or more of the processors 108 execute one of the jobs 109. In at least one embodiment, one or more of the processors 108 execute more than one of the jobs 109. In at least one embodiment, the jobs 109 are executed in parallel by the processors 108. In at least one embodiment, one or more of the jobs 109 are executed by more than one of the processors 108. In at least one embodiment, one or more of the jobs 109 are executed by one of the processors 108. In at least one embodiment, two or more of the jobs 109a-109n are each executed by a different number of processors 108.In at least one embodiment, two or more jobs 109a-109n are each executed by an equal number of processors 108. In at least one embodiment, one or more processors 108 are not used to execute a job 109 and / or are provided as replacements in case of failure of another processor 108.

[0015] In at least one embodiment, one or more of the jobs 109a-109n are restarted if one or more processors 108 executing that job fail. In at least one embodiment, a failure of one or more processors 108 includes a hardware malfunction. In at least one embodiment, a failure of one or more processors 108 includes a software malfunction. In at least one embodiment, a failure of one or more processors 108 includes an event that causes reduced performance of one or more of the jobs 109a-109n executed by that processor. In at least one embodiment, restarting a job 109 includes resetting that job 109 to a checkpoint. In at least one embodiment, a checkpoint includes a saved state that preserves the progress of a job 109 and to which that job can be reset.

[0016] In at least one embodiment, the system 100 includes a job scheduler 110. In at least one embodiment, the job scheduler 110 is implemented on one or more of the processors 108 or other processors contained in the data center 106. In at least one embodiment, any description of a scheduler or module that performs an operation refers to a processor that executes that scheduler or module to perform that operation. In at least one embodiment, a job scheduler is referred to as a software workload scheduler, scheduling software application, or scheduler. In at least one embodiment, the job scheduler 110 is any combination of hardware, firmware, or software that manages when and how jobs are to be executed by one or more processors. In at least one embodiment, a job is any software workload or...A software workload, software instruction, or any set of software instructions identifiable as a unit of work to be executed by one or more processors. In at least one embodiment, a software instruction is referred to as an instruction. In at least one embodiment, the Job Scheduler 110 is at least part of a computer management system such as SchedMD® SLURM®, Oracle® Grid Engine, Oracle® Scheduler, IBM® Spectrum LSF, or a combination thereof. In at least one embodiment, the Job Scheduler 110 is at least part of a distributed resource management (DRM) system. In at least one embodiment, a job is any computational workload defined by a user or application. In at least one embodiment, a job is referred to as a set of one or more tasks, processes, or operations.In at least one embodiment, a job is a kernel, which is a set of instructions to be executed in parallel by one or more processors. In at least one embodiment, a job is a container, which is a set of instructions that can be executed by one or more processors in different computing environments using different hardware, firmware, software, or a combination thereof. In at least one embodiment, different types of jobs include jobs related to physics modeling, image classification, cloud-based document management, web hosting, or a combination thereof.

[0017] In at least one embodiment, the system 100 has a control level 112. In at least one embodiment, the control level 112 is any combination of hardware or software for creating, terminating, modifying, managing, or otherwise controlling jobs 109. In at least one embodiment, the control level 112 is implemented by one or more processors 108. In at least one embodiment, the control level 112 is implemented by other hardware or software within the data center 106. In at least one embodiment, the control level 112 is implemented by other hardware or software outside the data center 106.

[0018] In at least one embodiment, the system 100 includes a processor flow database 116. In at least one embodiment, the processor flow database 116 is contained in one or more data storage devices that store information about a time period for reserving processors in the event of a failure of a processor executing a job. In at least one embodiment, a job is restarted if a processor executing that job fails. In at least one embodiment, a failed processor is marked in the processor flow database 116 or elsewhere as unavailable for future use. In at least one embodiment, processors that have not failed are reserved for use by a job after its restart. In at least one embodiment, these processors are reserved for a period of time specified in the processor flow database 116 or for another duration.In at least one embodiment, the processor flow database 116 stores all other suitable information. In at least one embodiment, the processor flow database 116 is updated by processors and / or workers. In at least one embodiment, the processor flow database is updated by the control plane 112 or by another suitable component. In at least one embodiment, the processor flow database is part of the job scheduler 110. In at least one embodiment, the information stored by the processor flow database is stored by the jobs 109 themselves.

[0019] In at least one embodiment, the system 100 includes a data center processor management module 120. In at least one embodiment, the data center processor management module 120 is hardware, firmware, software, or a combination thereof, used to set processor setting values ​​of data center processors, such as the processor(s) 108. In at least one embodiment, processor settings are values ​​used by the data center processor management module 120 to configure one or more processors to operate with one or more of the processor setting values ​​or within a range of these processor setting values. In at least one embodiment, a range of processor setting values ​​is calculated based on a percentage of the processor setting value.In at least one embodiment, the data center processor management module 120 configures one or more processors to operate at a given processor setting value or within a range of processor setting values ​​by managing or modifying how instructions are input to or executed by a processor; by causing devices (e.g., microcontrollers, voltage regulator modules, switches) to control power consumption and fan speed; by causing certain circuits or sections of circuits of a processor to be used; by physically modifying an aspect of a processor (e.g., modifying a logic component); by using techniques known to those skilled in the art; or by a combination thereof. In at least one embodiment, the data center processor management module 120 specifies job processor requirements 114.

[0020] In at least one embodiment, processor setting values ​​are referred to as processor settings. In at least one embodiment, the data center processor management module 120 is referred to as a compute resource manager, resource manager (RM), or processor management application. In at least one embodiment, the data center processor management module 120 comprises any combination of hardware, firmware, or software that manages communication between components of a data center, for example, between a job scheduler and a processor, using a communication protocol. In at least one embodiment, one or more sections of the data center processor management module 120 that manage communication between components of a data center are implemented as a separate module.In at least one embodiment, one or more sections of the data center processor management module 120 are implemented in a computer network, in a computer system, on a node, or a combination thereof, which is separate from another computer network, computer system, node, or combination thereof on which another section of the data center processor management module 120 is implemented. In at least one embodiment, a section of a module that is implemented separately from another section of that module or from another module is referred to as "out-of-band," "remote," or "distributed."

[0021] In at least one embodiment, the data center processor management module 120 includes at least one section of the NVIDIA® Data Center GPU Manager (DCGM), including one or more API functions of that system. In at least one embodiment, at least one section of the data center processor management module 120 manages processor settings and / or processor configuration at a low level. In at least one embodiment, low-level processor management refers to management that includes commands and / or instructions that are sent to and usable by a processor driver. In at least one embodiment, at least one section of a data center processor management module 120 includes an interface (e.g., a user interface) and an API library that a user or application (e.g., a software application) can access.a job scheduler) can use a section of the Data Center Processor Management Module 120 that performs low-level processor management. In at least one embodiment, each section of the Data Center Processor Management Module 120 that performs low-level processor management, has an interface for performing low-level processor management, has API functions for performing low-level processor management, or a combination thereof, is referred to as a Resource Manager System Management Interface (RMSMI). In at least one embodiment, a section of an RMSMI is one or more sections of an NVIDIA® System Management Interface (SMI) system, including one or more API functions of that system.In at least one embodiment, a section of an RMSMI is located on one or more sections of a processor management library, such as the AMD® ROCm SMI Library or the NVIDIA® Management Library (NVML).

[0022] In at least one embodiment, at least one section of the data center processor management module 120 is a baseboard management controller (BMC) that is used, at least in part, to monitor and control processors of a computer system. In at least one embodiment, at least one section of the data center processor management module 120 is an interface (e.g., a user interface) and an API library that a user or an application (e.g., a job scheduler) can use with a section of the data center processor management module 120 that performs baseboard management.One or more sections of the Data Center Processor Management Module 120 that perform baseboard management, include an interface for performing baseboard management, include API functions for performing baseboard management, or a combination thereof, are referred to as a Resource Manager Baseboard Management Interface (RMBMCI). In at least one embodiment, a section of an RMBMCI is one or more sections of an NVIDIA® Baseboard Management Controller (BMC), including one or more API functions of that system or similar.

[0023] In at least one embodiment, at least one section of the Data Center Processor Management Module 120 is one or more processor drivers, for example, GPU drivers. In at least one embodiment, a processor executes a processor driver to configure that processor and / or another processor according to processor settings that are selected and / or identified as otherwise described herein. In at least one embodiment, a processor driver of the Data Center Processor Management Module 120 is referred to as a Resource Manager Driver (RM Driver).

[0024] In at least one embodiment, the data center processor management module 120 is implemented on a processor of one or more of the processors 108, which differs from a processor of one or more of the processors 108 on which the job scheduler 110 is implemented. In at least one embodiment, the data center processor management module 120 is implemented on a computing device (e.g., a server) that differs from a computing device on which the job scheduler 110 is implemented. In at least one embodiment, the data center processor management module 120 is implemented in a data center that differs from a data center in which the job scheduler 110 is implemented.

[0025] Fig. Figure 2 shows a block diagram of a system 200 with a control level 212 according to at least one embodiment. In at least one embodiment, the control level 212 causes one or more processors of a group of processors that have been executing a software program to be assigned such that they continue to execute this software program until at least a certain period of time has elapsed after the failure of at least one or more of these processors. In at least one embodiment, one or more aspects of one or more embodiments, which are discussed here in conjunction with Fig. 2 are described, combined with one or more aspects of one or more embodiments described herein, including those that are at least in conjunction with the Fig. 1-43 are described. In at least one embodiment, one or more processors perform one or more operations of System 200. In at least one embodiment, one or more processors performing one or more operations of System 200 are any processor or combination of processors described herein, including in conjunction with the Fig. 1-43.

[0026] In at least one embodiment, the control level 212 comprises a processor flow monitor 214, a processor flow database 216, and a job controller 218. In at least one embodiment, the processor flow monitor 214, the processor flow database 216, and the job controller 218 may be contained in other components of the system 200. In at least one embodiment, the processor flow database 216 is the processor flow database 116, which, with respect to Fig. 1 was explained.

[0027] In at least one embodiment, the processor flow monitor 214 communicates with the processor(s) 208 to determine whether one of these processors 208 is reserved for use by one or more jobs for a time equal to or longer than an execution time. In at least one embodiment, the processor(s) 208 execute one or more jobs, for example, jobs 0 to n, 209a-209n. In at least one embodiment, jobs 209a-209n are jobs 0 to n, 109a-109n, which are defined by reference to Fig. 1 explained. In at least one embodiment, the jobs 209a-209n comprise one or more containers. In at least one embodiment, after one or more processors executing a job have failed, other processors executing that job are reserved for future use by that job. In at least one embodiment, this reservation is maintained until an expiration time. In at least one embodiment, a failure of one or more processors 208 comprises a hardware failure, a software failure, or any condition that adversely affects the performance of the jobs 209a-209n. In at least one embodiment, this expiration time is specified in the processor flow database 216. In at least one embodiment, the processor flow monitor 214 monitors a period of time for which the processor(s) 208 are reserved for use by one or more jobs 209a-209n.are, and causes this processor(s) 208 to maintain or maintain this reservation, or to release or release this reservation, or to terminate or terminate (or otherwise terminate) depending on whether a corresponding expiration time has been reached. In at least one embodiment, an expiration time is a specific point in time. In at least one embodiment, an expiration time is specified as a duration. In at least one embodiment, an expiration time is predetermined. In at least one embodiment, an expiration time is determined dynamically. In at least one embodiment, the processor execution monitor is part of a scheduler, for example, the one in . Fig. 1 Job Scheduler shown 110.

[0028] In at least one embodiment, for example as with reference to Fig. As explained in more detail in Section 3, jobs 209a-209n record the (health) status of one or more processors 208, each of which executes jobs 209a-209n. In at least one embodiment, the status of one or more processors 208 includes an indication of whether that processor has failed. In at least one embodiment, the status of one or more processors 208 includes all other information indicating factors that affect the performance of that processor.

[0029] In at least one embodiment, the processor flow database 216 is the one associated with Fig. 1. Processor flow database 216 described. In at least one embodiment, the processor flow database 216 is software and / or hardware that is contained in the control plane 212 or is at least partially provided outside the control plane 212. In at least one embodiment, the processor flow database 216 communicates with the processor flow monitor 214 and / or the processor(s) 208 to store (health) status information, including an expiration time of the processor(s) 208.

[0030] In at least one embodiment, the job controller 218 is hardware and / or software for creating and terminating jobs, e.g., jobs 209a-209n. In at least one embodiment, the job controller 218 is hardware and / or software that is contained in the control level 212 or is at least partially provided outside of the control level 212. In at least one embodiment, the job controller 218 communicates with the processor flow database 216 and / or the processor flow monitor 214 to determine the status of the processor(s) 208, e.g., whether a processor is executing a job, is reserved, or whether a reservation has expired.In at least one embodiment, in response to the finding that one or more of the processors 208 have failed, one or more other processors 208 performing the same job are reserved for future use by that job for a period of time determined on the basis of a value stored in the processor flow database 216.

[0031] Fig. Figure 3 shows a block diagram of a system 300 that includes a processor flow monitor 314 according to at least one embodiment. In at least one embodiment, the processor flow monitor 314 is configured to inform the processor(s) 308 whether this processor is to be reserved and / or whether a reservation has expired. In at least one embodiment, one or more aspects of one or more embodiments, which are discussed here in conjunction with Fig. 3 are described, combined with one or more aspects of one or more embodiments described herein, including those that are at least in conjunction with the Fig. 1-43 are described. In at least one embodiment, one or more processors perform one or more operations of System 300. In at least one embodiment, one or more processors performing one or more operations of System 300 are any processor or combination of processors described herein, including in conjunction with the Fig. 1-43.

[0032] In at least one embodiment, the system 300 includes a processor flow monitor 314. In at least one embodiment, the processor flow monitor 314 is at least a section of the processor flow monitor 214. Fig. 2. In at least one embodiment, the processor flow monitor 314 receives a command issued by a user via a user interface, for example the user interface 102. Fig. 1, is entered. In at least one embodiment, the processor flow monitor 314 receives an instruction to monitor a time period for which processor(s) 308 are reserved, and / or to determine, based on information from the processor flow database 316, whether this reservation has expired.

[0033] In at least one embodiment, the processor flow database 316 receives and stores information from the processor flow monitor 314. In at least one embodiment, the processor flow database stores, for a number of processors 312a (corresponding, for example, to processors 308), a job 312b executed by that processor, a status 312c of that processor, and an expiration (or expiration time) 312d of that processor. In at least one embodiment, this status 312c indicates whether the processor is in use, reserved, available, or failed. In at least one embodiment, this status 312c indicates any other information about the processor. In at least one embodiment, the expiration or expiration time 312d indicates a period of time for which a corresponding processor is to be reserved. In at least one embodiment, the expiration 312d indicates a specific point in time.In at least one embodiment, expiry 312d is a remaining period of time until the corresponding processor becomes available.

[0034] In at least one embodiment, a processor 308 executing a failed job notifies the processor flow monitor 314 (or another suitable component) of this failure, which stores this failure as status 312, for example as in Fig. 3, wherein a processor 308, corresponding to processor 312a number 2 and executing job 312b number 0, has failed, as indicated by status 312d. In at least one embodiment, the processors 308 and / or workers write their status directly to the processor flow database 316 or via a component other than the processor flow monitor 314. In at least one embodiment, upon detecting a failure status 312d, the processor flow monitor 314 notifies other processors 312a of the same job of this failure, and these processors are reserved for future execution by this job, for example, as in Fig. As shown in Figure 3, processors 312a, numbered 0 and 1, which correspond to job 312b, numbered 0, are reserved in response to the failure of processor 312a, numbered 2. In at least one embodiment, this reservation lasts for a period specified by the expiration time or expiration 312d.

[0035] Fig. Figure 4 shows a block diagram of a method 400 for assigning one or more processors to a group of processors according to at least one embodiment. In at least one embodiment, the method 400 prevents one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least one time interval has elapsed. In at least one embodiment, one or more aspects of one or more embodiments, which are in conjunction with Fig. 3 are described, combined with one or more aspects of one or more embodiments, which are at least in conjunction with the Fig. 1-43 are described.

[0036] In at least one embodiment, a job is started (or restarted) at 402. In at least one embodiment, a job is one of jobs 109a-109n, which are defined by reference to Fig. 1 were discussed. In at least one embodiment, a job is executed by a worker 209a-209n, which, with reference to Fig. 2 was discussed.

[0037] In at least one embodiment, a processor failure (e.g., by the processor flow monitor 314 or another suitable component) is detected or recorded at 404. In at least one embodiment, a processor failure is a hardware or software failure of a processor (e.g., one of the processors 108) that is executing a job. In at least one embodiment, a processor failure includes a failure predicted by a job. In at least one embodiment, the detection of a processor failure at 404 involves updating a processor flow database (e.g., the processor flow database 216) to indicate that this processor has failed. In at least one embodiment, an entry in a processor flow database indicating that a processor has failed prevents this processor from being used to execute future jobs.In at least one embodiment, information indicating that a processor has failed is stored in another suitable location. In at least one embodiment, the detection of a processor failure involves performing tests to determine which processor has failed.

[0038] In at least one embodiment, after detecting this failure, an expiration time (an expiration) is set at 406 for other processors executing the same job. In at least one embodiment, different expiration times are set for different processors. In at least one embodiment, setting an expiration at 406 involves setting a reservation for a processor.

[0039] In at least one embodiment, an attempt is made to reschedule (or queue for rescheduling) this job at 408. In at least one embodiment, this job is rescheduled to use processors that were previously used to execute this job. In at least one embodiment, rescheduling a job involves resetting a job to a previously verified state.

[0040] In at least one embodiment, the method 400 checks at 410 whether an expiry time specified at 406 has elapsed. In at least one embodiment, the elapse of the expiry time involves the passage of a period of time. In at least one embodiment, the elapse of an expiry time involves reaching a specific point in time. In at least one embodiment, when the expiry time has elapsed, a processor with this elapsed expiry time is released for use by another job 413.

[0041] In at least one embodiment, method 400 checks at 412 whether this job has been successfully rescheduled. In at least one embodiment, method 400 returns to 410 if this new job has not been successfully rescheduled. In at least one embodiment, method 400 returns to 402 to start this newly scheduled job if it has been successfully rescheduled.

[0042] Fig. Figure 5 shows an example of a processor 500 according to at least one embodiment. In at least one embodiment, the processor 500 executes one or more processes or methods as described with reference to the Fig. The measures described in sections 1 to 4 are designed to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain time interval has elapsed. In at least one embodiment, this first software program is restarted to be executed by one or more processors of this group of processors. In at least one embodiment, this first software program is reset to a checkpoint to be executed by one or more processors of this group of processors. In at least one embodiment, this one or more processors are configured to be assigned only to execute this first software program before this checkpoint expires.In at least one embodiment, this sequence includes a time period that must elapse after a failure of this group of processors. In at least one embodiment, this sequence includes a predetermined time after the failure of this group of processors. In at least one embodiment, this second software differs from this first software program.

[0043] In at least one embodiment, the processor 500 comprises one or more processors such as those used in conjunction with the Fig. 7-43. In at least one embodiment, the processor 500 is any suitable processing unit or combination of processing units, such as one or more CPUs, GPUs, GPGPUs, or PPUs. In at least one embodiment, the processor 500 comprises a job control module 510, a process module 520, a job scheduler module 530, and a process database module 540. In at least one embodiment, the job control module 510, the process module 520, the job scheduler module 530, and the process database module 540 are part of the processor 500, as described, for example, in Fig. 5 shown, or can be part of one or more other processors. In at least one embodiment, the job control module 510, the execution module 520, the job scheduler module 530, and the execution database module 540 are distributed across multiple processors, which communicate via a bus, a network, by writing to a common memory, or any suitable communication process, such as the one described in the Fig. 7-43 described, communicate.

[0044] In at least one embodiment, the job control module 510 comprises circuits that cause a job to be executed, terminated, or otherwise controlled by one or more processors and / or workers. In at least one embodiment, for example, the job control module 510 can perform operations to control the operations specified in Fig. to implement steps 402 and 404 as shown in the diagram.

[0045] In at least one embodiment, the execution module comprises 520 circuits that cause processors and / or workers to set an execution time and / or to determine whether an execution time for a processor has expired. In at least one embodiment, for example, the execution module can perform 620 operations to determine the Fig. to implement steps 406, 410 and 413 as shown.

[0046] In at least one embodiment, the job scheduler module 530 comprises circuits that cause jobs to be scheduled for execution using one or more processors. In at least one embodiment, for example, the job scheduler module 530 can perform operations to schedule the jobs in Fig. to implement steps 408, 412 and 413 as shown.

[0047] In at least one embodiment, the sequence database module 540 comprises circuits that cause sequence information or other information about processors to be stored, which, for example, the sequence database module 520 can access. In at least one embodiment, the sequence database module 540 can, for example, perform operations to store the information contained in Fig. 3. To implement the processor flow database 316 shown.

[0048] Fig. Figure 6 shows a block diagram of a driver and / or a runtime environment comprising one or more libraries to provide one or more application programming interfaces (APIs) according to at least one embodiment. In at least one embodiment, any processor or any combination of processors executes API(s) 610, including the processor(s) 108. Fig. 1, the processor(s) 208 from Fig. 2 and / or any processor(s) that refer to the Fig. 7-43. In at least one embodiment, the API(s) 610 are further described herein, comprising the API(s) used to specify a number of processors to be used to execute a job. In at least one embodiment, a call to the API(s) 610 causes one or more operations of one or more components or modules described herein, such as the Job Scheduler 110 or the Data Center Processor Management Module 120, to be executed. Fig. 1, are executed by one or more processors. In at least one embodiment, a call to API(s) 610 causes one or more processors to execute one or more of the functions associated with the Fig. Perform the operations described in 1-43.

[0049] In at least one embodiment, a software program 602 is a software module. In at least one embodiment, a software program 602 comprises one or more software modules. In at least one embodiment, the one or more APIs 610 are sets of software instructions that, when executed, cause one or more processors to perform one or more arithmetic operations. In at least one embodiment, the one or more APIs 610 are distributed or otherwise provided as part of one or more libraries 606, runtime environments 604, drivers 604, and / or another grouping of software and / or executable code, which are further described herein. In at least one embodiment, the one or more APIs 610 perform one or more arithmetic operations in response to a call by the software program 602.In at least one embodiment, the software program 602 is a collection of software code, commands, instructions, or other text sequences for instructing a computing device to perform one or more arithmetic operations and / or to call one or more other sets of instructions, such as APIs 610 or functions 612, to be executed. In at least one embodiment, the functions provided by the one or more APIs 610 include software functions, such as those that can be used to accelerate one or more sections of the software program 602 using one or more parallel processing units (PPUs), such as graphics processing units (GPUs). In at least one embodiment, the software program is a compiler.

[0050] In at least one embodiment, the APIs 610 are hardware interfaces to one or more circuits for performing one or more arithmetic operations. In at least one embodiment, one or more of the software APIs 610 described herein are implemented as one or more circuits for performing one or more of the techniques described herein. In at least one embodiment, one or more software programs 602 comprise instructions which, when executed, cause one or more hardware devices and / or circuits to perform one or more techniques further described herein.

[0051] In at least one embodiment, the software program 602, such as a user-implemented software program, uses one or more application programming interfaces (APIs) 610 to perform various arithmetic operations, or any arithmetic operation, performed by parallel processing units (PPUs), such as graphics processing units (GPUs), as further described herein. In at least one embodiment, one or more APIs 610 provide a set of callable functions 612, referred to herein as APIs, API functions, and / or functions, which individually perform one or more arithmetic operations, such as arithmetic operations related to parallel data processing.For example, in one embodiment, one or more of the APIs 610 provide functions 612 to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain period of time has elapsed.

[0052] In at least one embodiment, one or more software programs 602 interact with or otherwise communicate with one or more APIs 610 to perform one or more computational operations using one or more PPUs, such as GPUs. In at least one embodiment, one or more computational operations using one or more PPUs comprise at least one or more groups of computational operations that are to be accelerated by execution at least partially by the one or more PPUs. In at least one embodiment, the one or more software programs 602 interact with the one or more APIs 610 to enable parallel data processing using a remote or local interface.

[0053] In at least one embodiment, an interface corresponds to software instructions which, when executed, provide access to one or more functions 612 provided by the one or more APIs 610. In at least one embodiment, the interface is the user interface 102. Fig. 1. In at least one embodiment, a software program 602 uses a local interface when a software developer compiles the one or more software programs 602 in conjunction with one or more libraries 606 that include or otherwise provide access to the one or more APIs 610. In at least one embodiment, the one or more software programs 602 are compiled statically in conjunction with precompiled libraries 606 or uncompiled source code that includes instructions for executing the one or more APIs 610. In at least one embodiment, the one or more software programs 602 are compiled dynamically, and the one or more software programs use a linker to establish a connection to one or more precompiled libraries 606 that include the one or more APIs 610.

[0054] In at least one embodiment, the software program 602 uses a remote interface when a software developer executes a software program that uses or otherwise communicates with a library 606 comprising one or more APIs 610 over a network or other remote communication medium. In at least one embodiment, one or more libraries 606 comprising one or more APIs 610 are executed by a remote computing service, such as a computing resource service provider. In another embodiment, the one or more libraries 606 comprising one or more APIs 610 are executed by any other computing host that provides one or more APIs 610 to the one or more software programs 602.

[0055] In at least one embodiment, a processor executing or using the one or more software programs 602 calls, uses, executes, or otherwise implements one or more APIs 610 to allocate and otherwise manage memory to be used by the software programs 602. In at least one embodiment, the one or more software programs 602 use the one or more APIs 610 to allocate and otherwise manage memory to be used by one or more sections of the software programs 602, which are to be accelerated using one or more PPUs, such as GPUs, or any other accelerator or processor further described herein.These software programs 602 can be executed, at least partially, based on the latency of connections coupled to one or more processors, by one or more processors, using functions 612 which in one embodiment are provided by one or more APIs 610.

[0056] In at least one embodiment, API 610 is an API for facilitating parallel data processing. In at least one embodiment, API 610 is any other API further described herein. In at least one embodiment, API 610 is provided by a driver and / or a runtime environment 604. In at least one embodiment, API 610 is provided by a CUDA user-mode driver. In at least one embodiment, API 610 is provided by a CUDA runtime environment. In at least one embodiment, a driver 604 corresponds to data values ​​and software instructions that, when executed, perform or otherwise facilitate the operation of one or more functions 612 of API 610 during the loading and execution of one or more sections of the software program 602.In at least one embodiment, the runtime environment 604 comprises data values ​​and software instructions which, when executed, perform or otherwise facilitate the operation of one or more functions 612 of the API 610 during the execution of a software program 602. In at least one embodiment, the one or more software programs 602 use the one or more APIs 610, which are implemented or otherwise provided by a driver and / or the runtime environment 604, to perform combined arithmetic operations by the one or more software programs 602 during execution by one or more PPUs, such as GPUs.

[0057] In at least one embodiment, the one or more software programs 602 use the one or more APIs 610 provided by a driver and / or the runtime environment 604 to perform combined arithmetic operations on one or more PPUs, such as GPUs. In at least one embodiment, the one or more APIs 610 provide combined arithmetic operations via a driver and / or the runtime environment 604, as described above. In at least one embodiment, the one or more software programs 602 use the one or more APIs 610 provided by a driver and / or the runtime environment 604 to allocate or otherwise reserve one or more memory blocks 614 on one or more PPUs, such as GPUs.In at least one embodiment, the one or more software programs 602 use the one or more APIs 610 provided by a driver and / or the runtime environment 604 to allocate or otherwise reserve memory blocks. In at least one embodiment, the one or more APIs 610 are configured to perform combined mathematical functions as described herein.

[0058] In at least one embodiment, the one or more APIs 610 provide one or more API functions 612 to execute a scheduling system that can be used, or is used, by one or more computing units as described herein to improve the usability of the software programs 602 and / or to optimize one or more sections of the software programs 602 that are to be accelerated by one or more PPUs, such as GPUs. In at least one embodiment, a processor executes the one or more software programs to combine two or more application programming interfaces (APIs) into a single API.In at least one embodiment, a processor uses an API to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain period of time has elapsed.

[0059] In at least one embodiment, the memory 614 is a system memory. In at least one embodiment, the memory 614 is any form of hardware that stores data and is referred to as memory or data storage. In at least one embodiment, the memory 614 stores data that is used in various operations described herein, including information about the number of processors to be used by a job, as is the case here at least in conjunction with the Fig. 1 to 5 are described.

[0060] In at least one embodiment, the memory 614 is a computer-readable storage medium and / or code stored on this computer-readable storage medium in the form of a computer program containing a plurality of computer-readable instructions executable by one or more processors. In at least one embodiment, a computer-readable storage medium is a non-volatile computer-readable medium. In at least one embodiment, at least some computer-readable instructions usable to perform the operations described herein are not stored exclusively using volatile signals (e.g., a propagating volatile electrical or electromagnetic transmission). In at least one embodiment, a non-volatile computer-readable medium does not necessarily comprise non-volatile data storage circuits (e.g.,Buffer, caches, and queues) within transceivers for volatile signals. In at least one embodiment, the memory 614 is implemented as a non-volatile, computer-readable storage medium that stores executable instructions which, when executed by one or more processors of a computer system, cause the computer system to perform one or more operations of an API to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain period of time has elapsed. SERVERS AND DATA CENTERS

[0061] The following figures represent, without limitation, exemplary network servers and data center-based systems that can be used to implement at least one embodiment.

[0062] Fig. Figure 7 illustrates a distributed system 700 according to at least one embodiment. In at least one embodiment, the distributed system 700 includes one or more client computing devices 702, 704, 706, and 708 configured to run and operate a client application, such as a web browser, a proprietary client, and / or variations thereof, over one or more networks 710. In at least one embodiment, the server 712 can be communicatively coupled to remote client computing devices 702, 704, 706, and 708 over the network 710.

[0063] In at least one embodiment, the Server 712 can be configured to run one or more services or one or more software applications, such as services and applications capable of managing single sign-on (SSO) session access across multiple data centers. In at least one embodiment, the Server 712 can also provide other services or software applications that include non-virtual and virtual environments. In at least one embodiment, these services can be offered to users of the Client Computing Devices 702, 704, 706, and / or 708 as web-based or cloud services, or under a Software as a Service (SaaS) model.In at least one embodiment, users operating client computing devices 702, 704, 706 and / or 708 can in turn use one or more client applications to interact with the server 712 in order to use services provided by these components.

[0064] In at least one embodiment, the software components 718, 720, and 722 of System 700 are implemented on Server 712. In at least one embodiment, one or more components of System 700 and / or services provided by these components may also be implemented by one or more of the client computing devices 702, 704, 706, and / or 708. In at least one embodiment, users operating client computing devices may then use one or more client applications to access services provided by these components. In at least one embodiment, these components may be implemented in hardware, software, firmware, or combinations thereof. It is understood that various different system configurations are possible, which may differ from the distributed System 700. Fig. The embodiment shown in Figure 7 is therefore an example of a distributed system for implementing an embodiment of a system and is not intended to be restrictive.

[0065] In at least one embodiment, the client computing devices 702, 704, 706, and / or 708 can include various types of computing systems. In at least one embodiment, a client computing device can include portable devices (e.g., an iPhone®, mobile phone, iPad®, computer tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® with a head-mounted display) running software such as Microsoft Windows Mobile® and / or a variety of mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and / or variations thereof. In at least one embodiment, devices can support various applications, such as various internet-related applications, email, short message service (SMS) applications, and can use various other communication protocols.In at least one embodiment, client computing devices may also include general-purpose personal computers, including, by way of example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. In at least one embodiment, client computing devices may be workstation computers running any of a wide variety of commercially available UNIX® or UNIX-like operating systems, including, without limitation, a wide variety of GNU / Linux operating systems, such as Google Chrome OS. In at least one embodiment, client computing devices may also include electronic devices, such as a thin client computer, an internet-enabled gaming system (e.g.,a Microsoft Xbox game console (with or without Kinect® gesture input device) and / or a personal messaging device capable of communicating over network(s) 710. Even if the distributed system 700 is in . Fig. Figure 7 shows four client computing devices; however, any number of client computing devices can be supported. Other devices, such as devices with sensors, etc., can interact with the Server 712.

[0066] In at least one embodiment, the network(s) 710 in the distributed system 700 can be any type of network capable of supporting data communications using any of a wide variety of available protocols, including, without limitation, TCP / IP (transmission control protocol / Internet protocol), SNA (System Network Architecture), IPX (Internet Packet Exchange), AppleTalk, and / or variations thereof. In at least one embodiment, the network(s) 710 can be a local area network (LAN), networks based on Ethernet, Token Ring, a wide area network, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g.,a network operating according to any of the protocol suites from the Institute of Electrical and Electronics (IEEE) 802.11, Bluetooth® and / or any other wireless protocol) and / or any combination of these and / or other networks.

[0067] In at least one embodiment, the Server 712 can consist of one or more general-purpose computers, specialized server computers (including, by way of example, PC (personal computer) servers, UNIX® servers, midrange servers, mainframes, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. In at least one embodiment, the Server 712 can include one or more virtual machines running virtual operating systems or other computing architectures with virtualization. In at least one embodiment, one or more flexible pools of logical storage devices can be virtualized to manage virtual storage devices for a server. In at least one embodiment, virtual networks can be controlled by the Server 712 using software-defined networking.In at least one embodiment, the Server 712 can be designed to run one or more services or one or more software applications.

[0068] In at least one embodiment, the Server 712 can run any operating system as well as any commercially available server operating system. In at least one embodiment, the Server 712 can also run any of a variety of additional server applications and / or mid-tier applications, including HTTP (Hypertext Transport Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Java® servers, database servers, and / or variations thereof. In at least one embodiment, exemplary database servers include, without limitation, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and / or variations thereof.

[0069] In at least one embodiment, the Server 712 may include one or more applications for analyzing and consolidating data feeds and / or event updates received by users of the Client Computing Devices 702, 704, 706, and 708. In at least one embodiment, data feeds and / or event updates may include Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party information sources, and continuous data streams that may include, but are not limited to, real-time events relating to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, and / or variations thereof.In at least one embodiment, the server 712 may also include one or more applications to display data feeds and / or event updates via one or more display devices of the client computing devices 702, 704, 706 and 708.

[0070] In at least one embodiment, the distributed system 700 can also include one or more databases 714 and 716. In at least one embodiment, the databases can provide a mechanism for storing information, such as user interaction information, usage pattern information, adaptation rule information, and other information. In at least one embodiment, the databases 714 and 716 can be located in a plurality of locations. In at least one embodiment, one or more of the databases 714 and 716 can reside locally on (and / or in) the server 712 on a non-transient storage medium. In at least one embodiment, the databases 714 and 716 can be located remotely from the server 712 and communicate with the server 712 via a network-based or dedicated connection.In at least one embodiment, databases 714 and 716 can reside in a storage-area network (SAN). In at least one embodiment, all files required to perform functions attributed to server 712 can be stored locally on server 712 and / or remotely, as needed. In at least one embodiment, databases 714 and 716 can include relational databases, such as databases designed to store, update, and retrieve data in response to SQL-formatted commands.

[0071] Fig. Figure 8 illustrates an exemplary data center 800 according to at least one embodiment. In at least one embodiment, the data center 800 includes, without limitation, a data center infrastructure layer 810, a framework layer 820, a software layer 830, and an application layer 840.

[0072] In at least one embodiment, as in Fig. Figure 8 shows that the data center infrastructure layer 810 includes a resource orchestrator 812, clustered computing resources 814, and node computing resources (“node CRs”) 816(1)-816(N), where “N” is any positive integer. In at least one embodiment, the node CRs 816(1)-816(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (“FPGAs”), graphics processing units, etc.), memory devices (e.g., dynamic read-only memory), data storage devices (e.g., solid-state or hard disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc.In at least one embodiment, one or more of the node CRs among node CRs 816(1)-816(N) may be a server that includes one or more of the aforementioned computing resources.

[0073] In at least one embodiment, the grouped compute resources 814 can include separate groupings of node CRs located in one or more racks (not shown) or many racks located in data centers at different geographic locations (also not shown). Separate groupings of node CRs within the grouped compute resources 814 can include grouped compute, network, memory, or data storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node CRs containing CPUs or processors can be grouped in one or more racks to provide compute resources to support one or more workloads.In at least one embodiment, one or more racks can also include any number of power modules, cooling modules and network switches in any combination.

[0074] In at least one embodiment, the resource orchestrator 812 can configure or otherwise control one or more node CRs 816(1)-816(N) and / or grouped computing resources 814. In at least one embodiment, the resource orchestrator 812 can include a software design infrastructure (“SDI”) management unit for the data center 800. In at least one embodiment, the resource orchestrator 812 can include hardware, software, or a combination thereof.

[0075] In at least one embodiment, the framework layer includes layer 820, as in Fig. Figure 8 shows, without limitation, a task scheduler 832, a configuration manager 834, a resource manager 836, and a distributed file system 838. In at least one embodiment, the framework layer 820 can include a framework to support software 852 of software layer 830 and / or one or more application(s) 842 of application layer 840. In at least one embodiment, the software 852 or the application(s) 842 can each include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 820 can be, but is not limited to, a type of free and open-source software web application framework, such as Apache Spark™ (hereinafter “Spark”), which can utilize the distributed file system 838 for large-scale data processing (e.g., “Big Data”).In at least one embodiment, the task scheduler 832 can include a Spark driver to facilitate the scheduling of workloads supported by different layers of the data center 800. In at least one embodiment, the configuration manager 834 can be able to configure different layers, such as the software layer 830 and the framework layer 820, including Spark and the distributed file system 838, to support large-scale data processing. In at least one embodiment, the resource manager 836 can be able to manage clustered or grouped compute resources mapped or allocated to support the distributed file system 838 and the task scheduler 832. In at least one embodiment, clustered or grouped compute resources can include the grouped compute resource 814 on the data center infrastructure layer 810.In at least one embodiment, the resource manager 836 can coordinate with the resource orchestrator 812 to manage these mapped or allocated computing resources.

[0076] In at least one embodiment, the software included in software layer 830 may include software 852 that is used by at least sections of the node CRs 816(1)-816(N), grouped computing resources 814, and / or the distributed file system 838 of framework layer 820. One or more types of software may include, but are not limited to, web browsing software, email scanning software, database software, and streaming video content software.

[0077] In at least one embodiment, the application(s) 842 included in application layer 840 may include one or more types of applications that are used by at least sections of node CRs 816(1)-816(N), grouped compute resources 814, and / or the distributed file system 838 of network layer 820. In at least one embodiment, one or more types of applications may include, without limitation, CUDA applications, 5G network applications, artificial intelligence applications, data center applications, and / or variations thereof.

[0078] In at least one embodiment, any configuration manager 834, resource manager 836, and resource orchestrator 812 can implement any number and type of self-modifying actions based on any set and type of data acquired in any technically feasible way. In at least one embodiment, self-modifying actions can relieve a data center operator of data center 800 from potentially making poor configuration decisions and potentially prevent sections of a data center from being underutilized and / or underperforming.

[0079] Fig. Figure 9 illustrates a client-server network 904 formed by a plurality of interconnected network server computers 902, according to at least one embodiment. In at least one embodiment, each network server computer 902 in a system 900 stores data that can be accessed by other network server computers 902 and client computers 906 and networks 908 connected in a wide area network 904. In at least one embodiment, the configuration of a client-server network 904 can change over time as client computers 906 and one or more networks 908 connect to and disconnect from a network 904, and as one or more mainline server computers 902 are added to or removed from a network 904.In at least one embodiment, when a client computer 906 and a network 908 are connected to network server computers 902, the client-server network includes such a client computer 906 and a network 908. In at least one embodiment, the term computer includes any device or machine capable of accepting data, applying prescribed processes to data, and providing results of processes.

[0080] In at least one embodiment, the client-server network 904 stores information that network server computers 902, remote networks 908, and client computers 906 can access. In at least one embodiment, the network server computers 902 are formed by mainframe computers, minicomputers, and / or microcomputers, each with one or more processors. In at least one embodiment, the server computers 902 are interconnected by wired and / or wireless transmission media, such as conductive wires, fiber optic cables, and / or microwave transmission media, satellite transmission media, or other conductive, optical, or electromagnetic wave transmission media. In at least one embodiment, client computers 906 access a network server computer 902 by means of a similar wired or wireless transmission medium.In at least one embodiment, a client computer 906 can connect to a client-server network 904 using a modem and a standard telephone communication network. In at least one embodiment, alternative carrier systems such as cable and satellite communication systems can also be used to connect to the client-server network 904. In at least one embodiment, other non-public or time-shared carrier systems can be used. In at least one embodiment, the network 904 is a global information network, such as the Internet. In at least one embodiment, the network is a non-public intranet that uses protocols similar to the Internet, but with additional security measures and restricted access controls.In at least one embodiment, the 904 network is a non-public or semi-private network that uses proprietary communication protocols.

[0081] In at least one embodiment, the client computer 906 is any end-user computer and may also be a mainframe computer, a minicomputer, or a microcomputer with one or more microprocessors. In at least one embodiment, the server computer 902 may sometimes act as a client computer accessing another server computer 902. In at least one embodiment, the remote network 908 may be a local area network, a network added to a wide area network by an independent service provider (ISP) for the Internet, or any other group of computers connected by wired or wireless transmission media with a configuration that is either fixed or changes over time.In at least one embodiment, client computers 906 can connect to and access a network 904 independently or via a remote network 908.

[0082] Fig. Figure 10 illustrates an example of a computer network 1008 connecting one or more computing machines according to at least one embodiment. In at least one embodiment, the network 1008 can be any type of electronically connected group of computers, including, for example, the following networks: Internet, intranet, local area networks (LANs), wide area networks (WANs), or an interconnected combination of these network types. In at least one embodiment, the connectivity within a network 1008 can be a remote modem, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber Distributed Datalink Interface (FDDI), Asynchronous Transfer Mode (ATM), or any other communication protocol.In at least one embodiment, computing devices connected to a network can be a desktop, a server, a portable device, a handheld device, a set-top box, a personal digital assistant (PDA), an end device, or any other desired type or configuration. In at least one embodiment, network-connected devices can vary significantly in processing power, internal memory, and other performance aspects, depending on their functionality. In at least one embodiment, communication within a network and to or from computing devices connected to a network can be either wired or wireless.In at least one embodiment, the network 1008 can include at least part of the worldwide public internet, which generally connects a large number of users according to a client-server model as specified in a Transmission Control Protocol / Internet Protocol (TCP / IP) specification. In at least one embodiment, the client-server network is a predominant model for communication between two computers. In at least one embodiment, a client computer ("client") issues one or more commands to a server computer ("server"). In at least one embodiment, the server executes client commands by accessing available network resources and returning information to a client in accordance with the client commands.In at least one embodiment, client computer systems and network resources residing on network servers are assigned a network address for identification during communication between elements of a network. In at least one embodiment, communications from other networked systems to servers include a network address of a relevant server / network resource as part of the communication, so that a suitable destination of data or a request is identified as a recipient. In at least one embodiment, where a network comprises the global internet, a network address is an IP address in TCP / IP format that can route data, at least partially, to an email account, a website, or another internet tool residing on a server.In at least one embodiment, information and services that reside on network servers can be made available to a web browser of a client computer via a domain name (e.g., www.site.com) that is associated with an IP address of a network server.

[0083] In at least one embodiment, a plurality of clients 1002, 1004, and 1006 are connected to a network 1008 via respective communication links. In at least one embodiment, each of these clients can access a network 1008 via any desired communication method, such as a dial-up modem connection, a cable connection, a digital subscriber line (DSL), a wireless or satellite connection, or any other form of communication. In at least one embodiment, each client can communicate using any machine compatible with a network 1008, such as a personal computer (PC), a workstation, a dedicated terminal, a personal data assistant (PDA), or other similar device. In at least one embodiment, the clients 1002, 1004, and 1006 may or may not be located in the same geographic area.

[0084] In at least one embodiment, a plurality of servers 1010, 1012, and 1014 are connected to a network 1008 to serve clients communicating with a network 1018. In at least one embodiment, each server is typically a high-performance computer or device that manages network resources and responds to client commands. In at least one embodiment, the servers include computer-readable data storage media, such as hard disk drives and RAM, which store program instructions and data. In at least one embodiment, the servers 1010, 1012, and 1014 execute application programs that respond to client commands. In at least one embodiment, the server 1010 can execute a web server application to respond to client requests for HTML pages and can also execute a mail server application to receive and forward electronic mail.In at least one embodiment, other application programs, such as an FTP server or a media server for streaming audio / video data to clients, can also run on a Server 1010. In at least one embodiment, different servers can be designated to perform different tasks. In at least one embodiment, the Server 1010 can be a dedicated web server that manages website resources for different users, while a Server 1012 can be designated to provide electronic mail (email) management. In at least one embodiment, other servers can be designated for media (audio, video, etc.), a file transfer protocol (FTP), or a combination of any two or more services that are typically available or provided over a network.In at least one embodiment, each server can be located in a location that is identical to or different from that of other servers. In at least one embodiment, there can be multiple servers that perform mirrored tasks for users, thereby reducing data congestion or minimizing traffic to and from a single server. In at least one embodiment, servers 1010, 1012, and 1014 are under the control of a web hosting provider within a company for maintaining and delivering third-party content over a network 1008.

[0085] In at least one embodiment, web hosting providers deliver services to two different types of clients. In at least one embodiment, one type, which can be called a browser, requests content from servers 1010, 1012, 1014, such as web pages, email messages, video clips, etc. In at least one embodiment, a second type, which can be called a user, commissions a web hosting provider to maintain a network resource, such as a website, and make it available to browsers. In at least one embodiment, users enter into a contract with a web hosting provider to make storage space, processing capacity, and communication bandwidth available for their desired network resource according to a set of server resources that the user wishes to use.

[0086] In order for a web hosting provider to provide services to these two clients, application programs that manage server-hosted network resources must be correctly configured in at least one embodiment. In at least one embodiment, the program configuration process involves defining a set of parameters that at least partially control an application program's response to browser requests and also at least partially define server resources available to a specific user.

[0087] In one embodiment, an intranet server 1016 communicates with a network 1008 via a communication link. In at least one embodiment, the intranet server 1016 communicates with a server manager 1018. In at least one embodiment, the server manager 1018 includes a database containing configuration parameters of an application program, which are used in the servers 1010, 1012, and 1014. In at least one embodiment, users modify a database 1020 via an intranet 1016, and a server manager 1018 interacts with servers 1010, 1012, and 1014 to modify application program parameters so that they match the contents of a database. In at least one embodiment, a user logs on to an intranet server 1016 by establishing a connection to an intranet 1016 via the computer 1002 and entering authentication information, such as a username and password.

[0088] In at least one embodiment, an intranet server 1016 authenticates a user when the user wishes to log on to a new service or modify an existing service, and provides the user with an interactive screen display / control panel that allows the user to access configuration parameters for a specific application program. In at least one embodiment, a user is presented with a series of modifiable text fields that describe aspects of a user website or other network resource configuration. In at least one embodiment, when a user wishes to increase the storage space reserved on a server for their website, a field is provided in which the user specifies the desired storage space.In at least one embodiment, an intranet server 1016 updates a database 1020 in response to receiving this information. In at least one embodiment, the server manager 1018 forwards this information to a suitable server, and a new parameter is used during application program operation. In at least one embodiment, an intranet server 1016 is configured to provide users with access to configuration parameters of hosted network resources (e.g., websites, email, FTP sites, media sites, etc.) for which a user has a contract with a web hosting service provider.

[0089] Fig. Figure 11A illustrates a networked computer system 1100A according to at least one embodiment. In at least one embodiment, the networked computer system 1100A comprises a plurality of nodes or personal computers (“PCs”) 1102, 1118, 1120. In at least one embodiment, the personal computer or node 1102 comprises a processor 1114, a memory 1116, a video camera 1104, a microphone 1106, a mouse 1108, speakers 1110, and a monitor 1112. In at least one embodiment, the PCs 1102, 1118, 1120 can, for example, each run one or more desktop servers of an internal network within a given company or can be servers of a general network that is not limited to a specific environment. In at least one embodiment, there is one server per PC node of a network, such that each PC node of a network represents a specific network server with a specific network URL address.In at least one embodiment, each server uses by default a standard web page for the user of that server, which itself may contain embedded URLs that point to further subpages of that user on that server or to other servers or pages on other servers in a network.

[0090] In at least one embodiment, nodes 1102, 1118, 1120, and other nodes of a network are interconnected via a medium 1122. In at least one embodiment, the medium 1122 can be a communication channel, such as an Integrated Services Digital Network (“ISDN”). In at least one embodiment, different nodes of a networked computer system can be connected through a variety of communication media, including local area networks (“LAN”), plain-old telephone lines (“POTS”), sometimes referred to as the public switched telephone network (“PSTN”), and / or variations thereof. In at least one embodiment, different nodes of a network can also represent computer system users who are interconnected via a network, such as the Internet.In at least one embodiment, each server in a network (which is executed by a specific node of a network in a given instance) has a unique address or identification within a network, which can be specified in the form of a URL.

[0091] In at least one embodiment, a plurality of multi-point conferencing units (“MCUs”) can be used to transmit data to and from different nodes or “endpoints” of a conferencing system. In at least one embodiment, nodes and / or MCUs can be interconnected via an ISDN connection or a local area network (“LAN”) in addition to various other communication media, such as nodes connected via the Internet. In at least one embodiment, nodes of a conferencing system can generally be connected directly to a communication medium, such as a LAN, or via an MCU, and this conferencing system can include other nodes or elements, such as routers, servers, and / or variations thereof.

[0092] In at least one embodiment, the processor 1114 is a programmable general-purpose processor. In at least one embodiment, the processors of the nodes of the networked computer system 1100A can also be specialized video processors. In at least one embodiment, various peripheral devices and components of a node, such as those of node 1102, can differ from those of other nodes. In at least one embodiment, node 1118 and node 1120 can be configured identically to node 1102 or differently. In at least one embodiment, a node can be implemented on any suitable computer system in addition to PC systems.

[0093] Fig. Figure 11B illustrates a networked computer system 1100B according to at least one embodiment. In at least one embodiment, the system 1100B illustrates a network, such as a LAN 1124, which can be used to connect a plurality of nodes that can communicate with each other. In at least one embodiment, a plurality of nodes are connected to the LAN 1124, such as PC nodes 1126, 1128, and 1130. In at least one embodiment, a node can also be connected to the LAN via a network server or other means. In at least one embodiment, the system 1100B comprises other types of nodes or elements, with one example including routers, servers, and nodes.

[0094] Fig. Figure 11C illustrates a networked computer system 1100C according to at least one embodiment. In at least one embodiment, the system 1100C illustrates a WWW system that has communications via a backbone communication network, such as the Internet 1132, which can be used to connect a plurality of nodes of a network. In at least one embodiment, the WWW is a set of protocols that operates on the Internet and enables a graphical interface system to operate on it to access information about the Internet. In at least one embodiment, a plurality of nodes are connected to the Internet 1132 in the WWW, such as PC nodes 1140, 1142, 1144. In at least one embodiment, a node forms an interface with other nodes of the WWW via a WWW HTTP server, such as servers 1134, 1136.In at least one embodiment, the PC 1144 can be a PC that forms a node of the network 1132 and itself runs its server 1136, although PC 1144 and server 1136 are shown in for illustrative purposes. Fig. 11C are shown separately.

[0095] In at least one embodiment, WWW is a distributed application type characterized by WWW HTTP, the WWW protocol running on the transmission control protocol / Internet protocol (“TCP / IP”) of the Internet. In at least one embodiment, the WWW can thus be characterized by a set of protocols (i.e., HTTP) running on the Internet as its “backbone”.

[0096] In at least one embodiment, a web browser is an application that runs on a node of a network and, in WWW-compatible network systems, allows users of a specific server or node to view such information, thus enabling a user to search for graphical and text-based files linked together via hypertext links embedded in documents or files available from servers in a network that understand HTTP. In at least one embodiment, a retrieved document can contain various embedded hypertext links and a local copy of a page that is created locally for a requesting user when a given web page of a first server associated with a first node is retrieved by a user using another server in a network such as the Internet.When a user clicks on a hypertext link, in at least one embodiment, locally stored information relating to a selected hypertext link is typically sufficient to enable a user's machine to open a connection over the Internet to a server specified by a hypertext link.

[0097] In at least one embodiment, more than one user can be connected to each HTTP server, for example, via a LAN, such as LAN 1138, as illustrated with respect to the WWW-HTTP server 1134. In at least one embodiment, the system 1100C can also include other types of nodes or elements. In at least one embodiment, a WWW-HTTP server is an application running on a machine, such as a PC. In at least one embodiment, each user can be considered to have a unique "server," as illustrated with respect to PC 1144. In at least one embodiment, a server can be considered to be a server, such as the WWW-HTTP server 1134, that provides network access for a LAN or a plurality of nodes or a plurality of LANs.In at least one embodiment, there are multiple users, each with a desktop PC or network node, and each desktop PC potentially acts as a server for one of those users. In at least one embodiment, each server is associated with a specific network address or URL, which, when accessed, provides a standard web page for that user. In at least one embodiment, a web page can contain further links (embedded URLs) that point to further subpages for that user on that server, or to other servers in a network, or to pages on other servers in a network. CLOUD COMPUTING AND SERVICES

[0098] The following figures represent, without limitation, exemplary cloud-based systems that can be used to implement at least one embodiment.

[0099] In at least one embodiment, cloud computing is a type of computing system in which dynamically scalable and often virtualized resources are provided as a service over the internet. In at least one embodiment, users do not need to have any knowledge, expertise, or control over the technology infrastructure that can be described as "in the cloud" that supports them. In at least one embodiment, cloud computing includes infrastructure-as-a-service, platform-as-a-service, software-as-a-service, and other variations that share a common guiding principle of relying on the internet to meet users' computing needs. In at least one embodiment, a typical cloud deployment, such as in a private cloud (e.g., a corporate network) or a data center (DC) in a public cloud (e.g., a cloud service), may be...The cloud (Internet) consists of thousands of servers (or alternatively, virtual machines), hundreds of Ethernet, Fibre Channel, or Fibre Channel over Ethernet (FCoE) ports, switching and storage infrastructure, etc. In at least one embodiment, the cloud can also consist of network service infrastructure such as IPsec VPN hubs, firewalls, load balancers, wide area network (WAN) optimizers, etc. In at least one embodiment, remote participants can securely access cloud applications and services by connecting via a VPN tunnel, such as an IPsec VPN tunnel.

[0100] In at least one embodiment, cloud computing is a model for enabling convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

[0101] In at least one embodiment, cloud computing is characterized by on-demand self-service, where a consumer can unilaterally and automatically provision computing resources, such as server time and network storage, as needed, without requiring human interaction with the respective service provider. In at least one embodiment, cloud computing is characterized by broad network access, where resources are available via a network and accessed through standard mechanisms that promote use by heterogeneous thin- or thick-client platforms (e.g., mobile phones, laptops, and PDAs).In at least one embodiment, cloud computing is characterized by resource pooling, where a provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with various physical and virtual resources being dynamically allocated and reallocated according to consumer demand. In at least one embodiment, a sense of location independence arises from the fact that a customer generally has no control over or knowledge of the precise location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center). In at least one embodiment, examples of resources include data storage, processing, memory, network bandwidth, and virtual machines.In at least one embodiment, cloud computing is characterized by rapid elasticity, where capabilities can be rapidly and elastically provisioned, in some cases automatically, for rapid scale-out and rapidly released for rapid scale-in. In at least one embodiment, the capabilities available for provisioning often appear unlimited to a consumer and can be acquired in any quantity at any time. In at least one embodiment, cloud computing is characterized by a metered service, where cloud systems automatically control and optimize resource utilization by using a metering capability at a specific level of abstraction appropriate for a particular type of service (e.g., storage, processing, bandwidth, and active user accounts).In at least one embodiment, resource usage can be monitored, controlled and reported to provide transparency for both a provider and a consumer of a service used.

[0102] In at least one embodiment, cloud computing can be associated with various services. In at least one embodiment, cloud Software-as-a-Service (SaaS) can refer to a service where a capability provided to a consumer is to use a provider's applications running on a cloud infrastructure. In at least one embodiment, applications are accessible from various client devices via a thin-client interface, such as a web browser (e.g., web-based email). In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with a possible exception of limited user-specific application configuration settings.

[0103] In at least one embodiment, Cloud Platform as a Service (PaaS) can refer to a service where a capability provided to a consumer is to deploy applications created or purchased by the consumer, using programming languages ​​and tools supported by a provider, on a cloud infrastructure. In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure, including the networks, servers, operating systems, or storage, but has control over the deployed applications and potentially the configurations of the application hosting environment.

[0104] In at least one embodiment, Cloud Infrastructure as a Service (IaaS) can refer to a service where a capability provided to a consumer consists of access to processing, storage, networking, and other basic computing resources, with the consumer being able to deploy and run freely chosen software, which may include operating systems and applications. In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control over selected network components (e.g., host firewalls).

[0105] In at least one embodiment, cloud computing can be used in various ways. In at least one embodiment, a private cloud can refer to a cloud infrastructure operated exclusively for one organization. In at least one embodiment, a private cloud can be managed by an organization or a third-party provider and can exist on or off-site. In at least one embodiment, a shared cloud can refer to a cloud infrastructure used jointly by multiple organizations and supporting a specific community with common concerns (e.g., objectives, security requirements, policies, and compliance considerations).In at least one embodiment, a shared cloud can be managed by an organization or a third-party provider and can exist on or off-site. In at least one embodiment, a public cloud can refer to cloud infrastructure made available to a broad public or a large industry group and owned by an organization that provides cloud services. In at least one embodiment, a hybrid cloud can refer to cloud infrastructure that is a composition of two or more clouds (private, shared, or public) that remain independent entities but are interconnected by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load balancing between clouds).In at least one embodiment, a cloud computing environment is service-oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability.

[0106] Fig. Figure 12 illustrates one or more components of a system environment 1200 in which services can be offered as third-party network services, according to at least one embodiment. In at least one embodiment, a third-party network can be referred to as a cloud, cloud network, cloud computing network, and / or variations thereof. In at least one embodiment, the system environment 1200 includes one or more client computing devices 1204, 1206, and 1208 that can be used by users to interact with a third-party network infrastructure system 1202 that provides third-party network services, which can be referred to as cloud computing services. In at least one embodiment, the third-party network infrastructure system 1202 can comprise one or more computers and / or servers.

[0107] It goes without saying that this is in Fig. The third-party network infrastructure system 1202 shown may have components other than those shown. Furthermore, it shows Fig. 12 an embodiment of a third-party network infrastructure system. In at least one embodiment, the third-party network infrastructure system 1202 may have more or fewer components than in Fig. As shown in 12, it can combine two or more components or can have a different configuration or arrangement of components.

[0108] In at least one embodiment, the client computing devices 1204, 1206, and 1208 can be configured to run a client application, such as a web browser, a proprietary client application, or another application, which a user of a client computing device can use to interact with the third-party network infrastructure system 1202 to utilize services provided by the third-party network infrastructure system 1202. Although the exemplary system environment 1200 is shown with three client computing devices, any number of client computing devices can be supported. In at least one embodiment, other devices, such as devices with sensors, etc., can interact with the third-party network infrastructure system 1202.In at least one embodiment, the network(s) 1210 can enable communication and data exchange between the client computing devices 1204, 1206 and 1208 and the third-party network infrastructure system 1202.

[0109] In at least one embodiment, services provided by the third-party network infrastructure system 1202 may include a host of services that are made available to users of a third-party network infrastructure system on demand. In at least one embodiment, various services may also be offered, including, without limitation, online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database management and processing, managed technical support services, and / or variations thereof. In at least one embodiment, services provided by a third-party network infrastructure system may dynamically scale to meet the needs of their users.

[0110] In at least one embodiment, a specific instantiation of a service provided by the third-party network infrastructure system 1202 can be referred to as a "service instance." In at least one embodiment, any service made available to a user over a communications network, such as the Internet, by a third-party network service provider's system is generally referred to as a "third-party network service." In at least one embodiment, in a public third-party network environment, the servers and systems that constitute the third-party network service provider's system are distinct from a customer's own local servers and systems. In at least one embodiment, the third-party network service provider's system can host an application, and a user can request and use an application over a communications network such as the Internet.

[0111] In at least one embodiment, a service in a third-party computer network infrastructure may include protected computer network access to storage, a hosted database, a hosted web server, a software application, or another service provided to a user by a third-party network vendor. In at least one embodiment, a service may include password-protected access to remote storage in a third-party network over the internet. In at least one embodiment, a service may include a web service-based hosted relational database and a scripting middleware engine for private use by a networked developer. In at least one embodiment, a service may include access to an email software application hosted on the website of a third-party network vendor.

[0112] In at least one embodiment, the third-party network infrastructure system 1202 can include a suite of applications, middleware, and database service offerings that are delivered to a customer in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. In at least one embodiment, the third-party network infrastructure system 1202 can also provide "big data"-related computing and analytics services. In at least one embodiment, the term "big data" is used generally to refer to extremely large datasets that can be stored and manipulated by analysts and researchers to visualize large volumes of data, identify trends, and / or otherwise interact with data. In at least one embodiment, big data and related applications can be hosted and / or manipulated by an infrastructure system at many levels and at different scales.In at least one embodiment, dozens, hundreds, or thousands of parallel-connected processors can act on such data to represent it or to simulate external forces on the data or what it represents. In at least one embodiment, these datasets can involve structured data, such as data organized in a database or otherwise according to a structured model, and / or unstructured data (e.g., emails, images, data blobs (large binary objects), web pages, complex event processing).In at least one embodiment, by utilizing the ability of an embodiment to relatively quickly concentrate more (or fewer) computing resources on a target, a third-party network infrastructure system can be made more available to perform tasks on large datasets based on demand from a business, government agency, research institution, private individual, group of like-minded individuals or organizations, or other entity.

[0113] In at least one embodiment, the third-party network infrastructure system 1202 can be adapted to automatically provision, manage, and track a customer's subscription to services offered by the third-party network infrastructure system 1202. In at least one embodiment, the third-party network infrastructure system 1202 can provide third-party network services through various deployment models. In at least one embodiment, services can be provided under a public third-party network model in which the third-party network infrastructure system 1202 is owned by an organization that sells third-party network services and makes them available to the general public or various industry enterprises.In at least one embodiment, services can be provided under a non-public third-party network model, in which the third-party network infrastructure system 1202 is operated exclusively for a single organization and can provide services to one or more entities within that organization. In at least one embodiment, third-party network services can also be provided under a shared third-party network model, in which the third-party network infrastructure system 1202 and the services provided by the third-party network infrastructure system 1202 are shared by multiple organizations in a cohesive community. In at least one embodiment, third-party network services can also be provided under a hybrid third-party network model, which is a combination of two or more different models.

[0114] In at least one embodiment, services provided by the third-party network infrastructure system 1202 may include one or more services provided under the Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), Infrastructure-as-a-Service (IaaS) categories, or other categories of services, including hybrid services. In at least one embodiment, a customer may order one or more services provided by the third-party network infrastructure system 1202 via a subscription order. In at least one embodiment, the third-party network infrastructure system 1202 then performs the processing to provide services in the customer's subscription order.

[0115] In at least one embodiment, services provided by the third-party network infrastructure system 1202 can include, without limitation, application services, platform services, and infrastructure services. In at least one embodiment, application services can be provided by a third-party network infrastructure system via a SaaS platform. In at least one embodiment, the SaaS platform can be configured to provide third-party network services that fall into a SaaS category. In at least one embodiment, the SaaS platform can provide capabilities for building and delivering a suite of on-demand applications on an integrated development and deployment platform. In at least one embodiment, the SaaS platform can manage and control the underlying software and infrastructure for delivering SaaS services.In at least one embodiment, customers can use applications running on a third-party network infrastructure system by utilizing services provided by a SaaS platform. In at least one embodiment, customers can acquire application services without having to purchase separate licenses and support. In at least one embodiment, various different SaaS services can be provided. In at least one embodiment, examples without limitation include services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0116] In at least one embodiment, platform services can be provided by the third-party network infrastructure system 1202 via a PaaS platform. In at least one embodiment, the PaaS platform can be configured to provide third-party network services that fall into a PaaS category. In at least one embodiment, examples of platform services can, without limitation, include services that enable organizations to consolidate existing applications on a shared, common architecture, as well as the ability to create new applications that utilize shared services provided by a platform. In at least one embodiment, the PaaS platform can manage and control the underlying software and infrastructure for providing PaaS services.In at least one embodiment, customers can purchase PaaS services provided by the third-party network infrastructure system 1202 without having to purchase separate licenses and support.

[0117] In at least one embodiment, customers can use programming languages ​​and tools supported by a third-party network infrastructure system and control the deployed services by utilizing services provided by a PaaS platform. In at least one embodiment, platform services provided by a third-party network infrastructure system can include third-party database network services, third-party middleware network services, and third-party network services. In at least one embodiment, third-party database network services can support common service deployment models that allow organizations to pool database resources and offer customers a database-as-a-service in the form of a third-party database network.In at least one embodiment, third-party middleware network services can provide a platform for customers to develop and deploy various business applications, and the third-party network services can provide a platform for customers to deploy applications in a third-party network infrastructure system.

[0118] In at least one embodiment, various different infrastructure services can be provided by an IaaS platform within a third-party network infrastructure system. In at least one embodiment, infrastructure services enable the management and control of underlying computing resources, such as storage, networks, and other fundamental computing resources, for customers using services provided by a SaaS platform and a PaaS platform.

[0119] In at least one embodiment, the third-party network infrastructure system 1202 can also include infrastructure resources 1230 for providing resources used to deliver various services to customers of a third-party network infrastructure system. In at least one embodiment, the infrastructure resources 1230 can include pre-integrated and optimized combinations of hardware, such as servers, storage, and network resources, for running services provided by a PaaS platform and a SaaS platform, and other resources.

[0120] In at least one embodiment, resources in the third-party network infrastructure system 1202 can be shared by multiple users and dynamically reassigned as needed. In at least one embodiment, resources can be assigned to users in different time zones. In at least one embodiment, the third-party network infrastructure system 1202 can allow a first group of users in a first time zone to use resources of a third-party network infrastructure system for a specified number of hours, and then allow the same resources to be reassigned to another group of users in a different time zone, thereby maximizing resource utilization.

[0121] In at least one embodiment, a set of internal shared services 1232 can be provided, which are shared by various components or modules of the third-party network infrastructure system 1202 to enable the third-party network infrastructure system 1202 to provide services. In at least one embodiment, these internal shared services can include, without limitation, a security and identity service, an integration service, an enterprise repository service, an enterprise manager service, a virus scanning and whitelisting service, a high availability, backup and recovery service, a service to enable third-party network support, an email service, a notification service, a file transfer service and / or variations thereof.

[0122] In at least one embodiment, the third-party network infrastructure system 1202 can provide comprehensive management of third-party network services (e.g., SaaS, PaaS, and IaaS services) within a third-party network infrastructure system. In at least one embodiment, the third-party network management functionality can include capabilities for provisioning, managing, and tracking a customer subscription received through the third-party network infrastructure system 1202 and / or variations thereof.

[0123] In at least one embodiment, as in Fig. Figure 12 shows that third-party network management functionality is provided by one or more modules, such as a job management module 1220, a job orchestration module 1222, a job delivery module 1224, a job management and monitoring module 1226, and an identity management module 1228. In at least one embodiment, these modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, specialized server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0124] In at least one embodiment, a customer using a client device, such as the client computing devices 1204, 1206, or 1208, can interact with the third-party network infrastructure system 1202 in step 1234 by requesting one or more services provided by the third-party network infrastructure system 1202 and placing an order for a subscription to one or more services offered by the third-party network infrastructure system 1202. In at least one embodiment, a customer can access a user interface (UI) of a third-party network, such as a third-party network UI 1212, a third-party network UI 1214, and / or a third-party network UI 1216, and place a subscription order through that UI.In at least one embodiment, order information received by the third-party network infrastructure system 1202 in response to a customer placing an order may include information that identifies a customer and one or more services offered by a third-party network infrastructure system 1202 that a customer wishes to subscribe to.

[0125] In at least one embodiment, order information received from a customer in step 1236 can be stored in an order database 1218. In at least one embodiment, if it is a new order, a new record can be created for the order. In at least one embodiment, the order database 1218 can be one of several databases operated by the third-party network infrastructure system 1218 and operated in conjunction with other system elements.

[0126] In at least one embodiment, at step 1238 order information can be forwarded to an order management module 1220, which can be configured to perform billing and accounting functions in relation to an order, such as verifying an order and, after verification, posting an order.

[0127] In at least one embodiment, at step 1240, information regarding an order can be transmitted to an order orchestration module 1222, which is configured to coordinate the provisioning of services and resources for an order placed by a customer. In at least one embodiment, the order orchestration module 1222 can use services of the order provisioning module 1224 for provisioning. In at least one embodiment, the order orchestration module 1222 enables the management of business processes associated with each order and applies business logic to determine whether an order should proceed to provisioning.

[0128] In at least one embodiment, at step 1242, upon receiving a new subscription order, the order orchestration module 1222 sends a request to the order provisioning module 1224 to allocate and configure resources needed to fulfill a subscription order. In at least one embodiment, the order provisioning module 1224 enables the allocation of resources for services requested by a customer. In at least one embodiment, the order provisioning module 1224 provides an abstraction layer between third-party network services provided by the third-party network infrastructure system 1200 and a physical implementation layer used to provision resources for delivering requested services.In at least one embodiment, this makes it possible to isolate the job orchestration module 1222 from implementation details, such as whether services and resources are actually provided in real time or pre-provided and only allocated / assigned on request.

[0129] In at least one embodiment, at step 1244, once services and resources are provisioned, a notification can be sent to subscription customers indicating that a requested service is now ready for use. In at least one embodiment, information (e.g., a link) can be sent to a customer that enables them to begin using the requested services.

[0130] In at least one embodiment, in step 1246, a customer's subscription order can be managed and tracked by an order management and monitoring module 1226. In at least one embodiment, the order management and monitoring module 1226 can be configured to collect usage statistics regarding customer use of subscribed services. In at least one embodiment, statistics can be collected for the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime, and / or variations thereof.

[0131] In at least one embodiment, the third-party network infrastructure system 1200 can include an identity management module 1228 configured to provide identity services, such as access management and authorization services within the third-party network infrastructure system 1200. In at least one embodiment, the identity management module 1228 can manage information about customers who wish to use services provided by the third-party network infrastructure system 1202. In at least one embodiment, such information can include information that authenticates the identities of such customers and information that describes the actions these customers are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.).In at least one embodiment, the identity management module 1228 can also include the management of descriptive information about each customer and how and by whom this descriptive information can be accessed and modified.

[0132] Fig. Figure 13 illustrates a cloud computing environment 1302 according to at least one embodiment. In at least one embodiment, the cloud computing environment 1302 comprises one or more computer systems / servers 1304 with which computing devices such as a personal digital assistant (PDA) or mobile phone 1306A, a desktop computer 1306B, a laptop computer 1306C, and / or an automotive computer system 1306N communicate. In at least one embodiment, this allows infrastructure, platforms, and / or software to be offered as services by the cloud computing environment 1302, so that not each client has to maintain these resources separately. It is understood that the Fig. The 13 types of computing devices shown in 1306A-N are for illustrative purposes only, and the cloud computing environment 1302 can communicate with any type of computer device over any type of network and / or network / addressable connection (e.g., via a web browser).

[0133] In at least one embodiment, a Computer System / Server 1304, which may be referred to as a cloud computing node, is compatible with numerous other general-purpose or special-purpose computing environments or configurations. In at least one embodiment, computing systems, environments, and / or configurations suitable for use with the Computer System / Server 1304 include personal computer systems, server computer systems, thin clients, thick clients, hang devices or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments, including any of the aforementioned systems or devices, and / or variations thereof, without limitation.

[0134] In at least one embodiment, the computer system / server 1304 can be described in a general context of instructions executable by a computer system, such as program modules executed by a computer system. In at least one embodiment, the program modules include routines, programs, objects, components, logic, data structures, etc., that perform concrete tasks or implement concrete abstract types of data. In at least one embodiment, the exemplary computer system / server 1304 can be implemented in distributed cloud computing environments in which tasks are performed by remote processing devices interconnected via a communication network.In at least one embodiment, program modules in a distributed cloud computing environment can reside on both local and remote computer system storage media, including storage devices.

[0135] Fig. Figure 14 illustrates a set of functional abstraction layers provided by the cloud computing environment 1302 ( Fig. 13) shall be provided, according to at least one embodiment. It should be understood in advance that the in Fig. The 14 components, layers and functions shown are for illustrative purposes only and components, layers and functions may vary.

[0136] In at least one embodiment, the hardware and software layer 1402 comprises hardware and software components. Examples of hardware components in at least one embodiment include mainframes, various servers based on a RISC (Reduced Instruction Set Computer) architecture, various computing systems, supercomputers, storage devices, networks, network components, and / or variations thereof. Examples of software components in at least one embodiment include network application server software, various application server software, various database software, and / or variations thereof.

[0137] In at least one embodiment, the virtualization layer 1404 provides an abstraction layer from which the following exemplary virtual units can be provided: virtual servers, virtual storage, virtual networks, including virtual private networks, virtual applications, virtual clients and / or variations thereof.

[0138] In at least one embodiment, the management layer 1406 provides various functions. In at least one embodiment, resource provisioning provides dynamic procurement of compute resources and other resources used to perform tasks within a cloud computing environment. In at least one embodiment, consumption tracking provides usage tracking when resources are used within a cloud computing environment and billing or invoicing for the consumption of these resources. In at least one embodiment, resources may include application software licenses. In at least one embodiment, security provides identity verification for users and tasks, as well as protection for data and other resources. In at least one embodiment, the user interface provides access to a cloud computing environment for both users and system administrators.In at least one embodiment, service level management provides the allocation and management of cloud computing resources to ensure that the required service levels are met. In at least one embodiment, service level agreement (SLA) management provides a pre-arranging and procurement mechanism for cloud computing resources that are expected to be needed in the future according to an SLA.

[0139] In at least one embodiment, workload layer 1408 provides functionality that utilizes a cloud computing environment. Examples of workloads and functions that can be provided by this layer in at least one embodiment include: mapping and navigation, software development and management, educational services, data analysis and processing, transaction processing, and service delivery. SUPERCOMPUTER

[0140] The following figures represent, without limitation, exemplary supercomputer-based systems that can be used to implement at least one embodiment.

[0141] In at least one embodiment, a supercomputer can refer to a hardware system that is essentially parallel and comprises at least one chip, wherein chips in a system are interconnected by a network and placed in hierarchically organized enclosures. In at least one embodiment, a large hardware system that fills a machine room with multiple racks, each containing multiple boards / rack modules, each containing multiple chips, all interconnected by a scalable network, is a particular example of a supercomputer. In at least one embodiment, a single rack of such a large hardware system is another example of a supercomputer.In at least one embodiment, a single chip that exhibits considerable parallelism and contains multiple hardware components can equally be considered a supercomputer, since as feature sizes decrease, the amount of hardware that can be integrated into a single chip can also increase.

[0142] Fig. Figure 15 illustrates a chip-level supercomputer according to at least one embodiment. In at least one embodiment, the main computational work is performed within an FPGA or ASIC chip in finite state machines (1504), which are referred to as thread units. In at least one embodiment, task and synchronization networks (1502) connect finite state machines and are used to dispatch threads and execute operations in the correct sequence. In at least one embodiment, a partitioned multilevel cache hierarchy (1508, 1512) on a chip is accessed using memory networks (1506, 1510). In at least one embodiment, off-chip memory is accessed using memory controllers (1516) and an off-chip memory network (1514).In at least one embodiment, an I / O controller (1518) is used for cross-chip communication when an embodiment does not fit into a single logic chip.

[0143] Fig. Figure 16 illustrates a rack-module-level supercomputer according to at least one embodiment. In at least one embodiment, there are multiple FPGA or ASIC chips (1602) within a rack module, connected to one or more DRAM units (1604) that form a main accelerator memory. In at least one embodiment, each FPGA / ASIC chip is connected to its adjacent FPGA / ASIC chip using wide buses on a board with differential high-speed signaling (1606). In at least one embodiment, each FPGA / ASIC chip is also connected to at least one serial high-speed communication cable.

[0144] Fig. Figure 17 illustrates a rack-level supercomputer according to at least one embodiment. Fig. Figure 18 illustrates a supercomputer at the overall system level according to at least one embodiment. In at least one embodiment, with reference to Fig. 17 and Fig. 18. Serial high-speed optical or copper cables (1702, 1802) are used between rack modules in a rack and across racks throughout an entire system to implement a scalable, potentially incomplete hypercube network. In at least one embodiment, one of the FPGA / ASIC chips of an accelerator is connected to a host system via a PCI Express connection (1804). In at least one embodiment, the host system comprises a host microprocessor (1808) on which a software portion of an application runs, and memory consisting of one or more host memory DRAM units (1806) that are kept coherent with the memory on an accelerator. In at least one embodiment, the host system can be a separate module on one of the racks or can be integrated into one of the modules of a supercomputer.In at least one embodiment, a cube-connected cycle topology provides communication links to create a hypercube network for a large supercomputer. In at least one embodiment, a small group of FPGA / ASIC chips on a rack module can function as a single hypercube node, thus increasing the total number of external connections for each group compared to a single chip. In at least one embodiment, a group comprises chips A, B, C, and D on a rack module with internal wide differential buses that connect A, B, C, and D in a torus organization. In at least one embodiment, there are 12 serial communication cables connecting a rack module to the outside world. In at least one embodiment, chip A on a rack module is connected to serial communication cables 0, 1, and 2. In at least one embodiment, chip B is connected to cables 3, 4, and 5.In at least one embodiment, chip C is connected to 6, 7, 8. In at least one embodiment, chip D is connected to 9, 10, 11. In at least one embodiment, an entire group {A, B, C, D}, forming a rack module, can form a hypercube node within a supercomputer system with up to 212 = 4096 rack modules (16384 FPGA / ASIC chips). In at least one embodiment, in order for chip A to send a message externally on connection 4 of the group {A, B, C, D}, a message must first be routed to chip B using an integrated differential-width bus connection. In at least one embodiment, a message arriving at connection 4 (i.e., arriving at B) in a group {A, B, C, D} and destined for chip A must also first be internally routed to a correct destination chip (A) within the group {A, B, C, D}. In at least one embodiment, parallel supercomputer systems of other sizes can also be implemented. ARTIFICIAL INTELLIGENCE

[0145] The following figures represent, without limitation, exemplary artificial intelligence-based systems that can be used to implement at least one embodiment.

[0146] Fig. Figure 19A illustrates inference and / or training logic 1915, which is used to perform inference and / or training operations in association with one or more embodiments. Details regarding the inference and / or training logic 1915 are given below in conjunction with the Fig. 19A and / or 19B provided.

[0147] In at least one embodiment, the inference and / or training logic 1915 may, without limitation, include a code and / or data store 1901 to store forward and / or output weighting and / or input / output data and / or other parameters for configuring neurons or layers of a neural network that is trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the training logic 1915 may include or be coupled to code and / or data stores 1901 to store graph code or other software for controlling the timing and / or order in which weighting and / or other parameter information is to be loaded to configure logic, including integer and / or floating-point units (collectively referred to as arithmetic logic units (ALUs)).In at least one embodiment, code, such as graph code, loads weighting or other parameter information into processor ALUs based on a neural network architecture to which such code corresponds. In at least one embodiment, the code and / or data memory 1901 stores weighting parameters and / or input / output data of each layer of a neural network that is trained or used in conjunction with one or more embodiments during the forward propagation of input / output data and / or weighting parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of the code and / or data memory 1901 may be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or system memory of a processor.

[0148] In at least one embodiment, any section of the code and / or data memory 1901 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or code and / or data memory 1901 can be cache memory, dynamic randomly addressable memory (“DRAM”), static randomly addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other memory.In at least one embodiment, the choice of whether the code and / or code and / or data storage 1901 in an example to a processor is internal or external, or includes DRAM, SRAM, Flash or another type of memory, may depend on the available chip-internal or chip-external memory, the latency requirements of the training and / or inference functions being performed, the batch size of the data being used in inferring and / or training a neural network, or a combination of these factors.

[0149] In at least one embodiment, the inference and / or training logic 1915 may, without limitation, include a code and / or data store 1905 for storing backward and / or output weighting and / or input / output data corresponding to neurons or layers of a neural network that is trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the code and / or data store 1905 stores weighting parameters and / or input / output data of each layer of a neural network that is trained or used in conjunction with one or more embodiments during backward propagation of input / output data and / or weighting parameters during training and / or inference using aspects of one or more embodiments.In at least one embodiment, the training logic 1915 may include or be coupled to code and / or data storage 1905 to store graph code or other software for controlling the timing and / or sequence in which weighting and / or other parameter information is to be loaded to configure logic, including integer and / or floating-point units (collectively referred to as arithmetic logic units (ALUs)).

[0150] In at least one embodiment, code, such as graph code, causes weighting or other parameter information to be loaded into processor ALUs based on a neural network architecture to which such code corresponds. In at least one embodiment, any portion of the code and / or data memory 1905 can be contained in another chip-internal or chip-external data memory, including the L1, L2, or L3 cache or system memory of a processor. In at least one embodiment, any portion of the code and / or data memory 1905 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data memory 1905 can be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory.In at least one embodiment, the choice of whether the code and / or data storage 1905 is, for example, internal or external to a processor, or comprises DRAM, SRAM, Flash or another type of memory, may depend on available chip-internal versus chip-external memory, latency requirements of executed training and / or inference functions, batch size of data used in inferring and / or training a neural network, or a combination of these factors.

[0151] In at least one embodiment, the code and / or data memory 1901 and the code and / or data memory 1905 can be separate memory structures. In at least one embodiment, the code and / or data memory 1901 and the code and / or data memory 1905 can be a combined memory structure. In at least one embodiment, the code and / or data memory 1901 and the code and / or data memory 1905 can be partially combined and partially separate. In at least one embodiment, any section of the code and / or data memory 1901 and the code and / or data memory 1905 can be included in another chip-internal or chip-external data memory, including the L1, L2, or L3 cache or system memory of a processor.

[0152] In at least one embodiment, the inference and / or training logic 1915 may, without limitation, include one or more arithmetic logic units (“ALU(s)”) 1910, including integer and / or floating-point units, to perform logical and / or mathematical operations that are based at least in part on or specified by training and / or inference code (e.g., graph code), wherein a result thereof may produce activations (e.g., output values ​​of layers or neurons within a neural network) that are stored in an activation memory 1920 and are functions of input / output and / or weighting parameter data that are stored in the code and / or data memory 1901 and / or the code and / or data memory 1905.In at least one embodiment, activations stored in the activation memory 1920 are generated according to linear algebraic and / or matrix-based mathematics, which are carried out by the ALU(s) 1910 in response to the execution of instructions or other code, wherein weight values ​​stored in the code and / or data memory 1905 and / or the data memory 1901 are used as operands together with other values, such as distortion values, gradient information, moment values ​​or other parameters or hyperparameters, any or all of which may be stored in the code and / or data memory 1905 or the code and / or data memory 1901 or in another chip-internal or -external memory.

[0153] In at least one embodiment, the ALU(s) 1910 are contained within one or more processors or other hardware logic devices or circuits, while in another embodiment, the ALU(s) 1910 may be external to a processor or other hardware logic device or circuit that uses them (e.g., a coprocessor). In at least one embodiment, the ALU(s) 1910 may be contained within the execution units of a processor or otherwise within a bank of ALUs that the execution units of a processor can access, either within the same processor or distributed across different processors of different types (e.g., central processing units, graphics processing units, fixed function units, etc.).In at least one embodiment, the code and / or data memory 1901, the code and / or data memory 1905, and the activation memory 1920 can share a processor or other hardware logic device or circuit, while in another embodiment they can be located in different processors or other hardware logic devices or circuits, or in a combination of identical and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of the activation memory 1920 can be included in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or system memory of a processor.Furthermore, the inference and / or training code may be stored with other code that a processor or other hardware logic or circuitry can access and that is retrieved and / or processed using the retrieval, decoding, scheduling, execution, elimination, and / or other logic circuitry of a processor.

[0154] In at least one embodiment, the activation memory 1920 can be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory. In at least one embodiment, the activation memory 1920 can be located wholly or partially inside or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the activation memory 1920 is internal or external to a processor, or whether it comprises DRAM, SRAM, flash, or another type of memory, can depend on available on-chip versus off-chip memory, latency requirements of executed training and / or inference functions, batch size of data used in inferring and / or training a neural network, or a combination of these factors.

[0155] In at least one embodiment, the inference and / or training logic 1915, which is in Fig. Figure 19A illustrates that the inference and / or training logic can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® (e.g., “Lake Crest”) processor. In at least one embodiment, the inference and / or training logic 1915, which is illustrated in Figure 19A, ... Fig. Figure 19A illustrates how it can be used in conjunction with hardware of a central processing unit (“CPU”), hardware of a graphics processing unit (“GPU”) or other hardware, such as field-programmable gate arrays (“FPGAs”).

[0156] Fig. Figure 19B illustrates inference and / or training logic 1915 according to at least one embodiment. In at least one embodiment, the inference and / or training logic 1915 can, without limitation, include hardware logic in which computing resources are dedicated or otherwise used exclusively in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, the inference and / or training logic 1915, which is described in Fig. 19B, illustrated, can be used in conjunction with an application-specific integrated circuit (ASIC), such as a Google TensorFlow® processing unit, a Graphcore™ inference processing unit (IPU), or an Intel Corp. Nervana® (e.g., “Lake Crest”) processor. In at least one embodiment, the inference and / or training logic 1915, which is illustrated in Fig. Figure 19B illustrates the inference and / or training logic 1915, which is used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as field-programmable gate arrays (FPGAs). In at least one embodiment, the inference and / or training logic 1915 includes, without limitation, the code and / or data memory 1901 and the code and / or data memory 1905, which can be used to store code (e.g., graph code), weight values, and / or other information, including distortion values, gradient information, instantaneous values, and / or other parameter or hyperparameter information. In at least one embodiment, which is illustrated in Figure 19B, the inference and / or training logic 1915 includes, without limitation, the code and / or data memory 1901 and the code and / or data memory 1905, which can be used to store code (e.g., graph code), weight values, and / or other information, including distortion values, gradient information, instantaneous values, and / or other parameter or hyperparameter information. Fig. As illustrated in Figure 19B, each of the code and / or data memory 1901 and the code and / or data memory 1905 is associated with a dedicated computational resource, such as the computational hardware 1902 and the computational hardware 1906, respectively. In at least one embodiment, each of the computational hardware 1902 and the computational hardware 1906 comprises one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on information stored in the code and / or data memory 1901 and the code and / or data memory 1905, respectively, with the result being stored in the activation memory 1920.

[0157] In at least one embodiment, each of the code and / or data stores 1901 and 1905 and the corresponding computing hardware 1902 and 1906, respectively, corresponds to different layers of a neural network, such that the resulting activation from one memory / computing pair 1901 / 1902 of the code and / or data store 1901 and the computing hardware 1902 is provided as input to the next memory / computing pair 1905 / 1906 of the code and / or data store 1905 and the computing hardware 1906, in order to reflect a conceptual organization of a neural network. In at least one embodiment, each of the memory / computing pairs 1901 / 1902 and 1905 / 1906 can correspond to more than one layer of a neural network. In at least one embodiment, additional memory / computing pairs (not shown) may be included following or in parallel to the memory / computing pairs 1901 / 1902 and 1905 / 1906 in the inference and / or training logic 1915.

[0158] Fig. Figure 20 illustrates the training and deployment of a deep neural network according to at least one embodiment. In at least one embodiment, the untrained neural network 2006 is trained using a training dataset 2002. In at least one embodiment, the training framework 2004 is a PyTorch framework, whereas in other embodiments, the training framework 2004 is a TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training framework. In at least one embodiment, the training framework 2004 trains an untrained neural network 2006 and allows it to be trained using processing resources described herein to generate a trained neural network 2008. In at least one embodiment, the weights can be chosen randomly or by pretraining using a deep belief network.In at least one embodiment, the training can be carried out in a supervised, partially supervised or unsupervised manner.

[0159] In at least one embodiment, the untrained neural network 2006 is trained using supervised learning, wherein the training dataset 2002 contains an input paired with a desired output for that input, or wherein the training dataset 2002 contains an input with a known output, and an output of the neural network 2006 is manually evaluated. In at least one embodiment, the untrained neural network 2006 is trained in a supervised manner and processes inputs from the training dataset 2002 and compares the resulting outputs with a set of expected or desired outputs. In at least one embodiment, errors are then backpropagated by the untrained neural network 2006. In at least one embodiment, the training framework 2004 adjusts weights that control the untrained neural network 2006.In at least one embodiment, the training framework 2004 includes tools to monitor how well the untrained neural network 2006 converges to a model, such as the trained neural network 2008, capable of producing correct answers, such as in the result 2014, based on input data, such as a new dataset 2012. In at least one embodiment, the training framework 2004 repeatedly trains the untrained neural network 2006 while adjusting weights to refine an output of the untrained neural network 2006 using a loss function and a fitting algorithm, such as stochastic gradient descent. In at least one embodiment, the training framework 2004 trains the untrained neural network 2006 until the untrained neural network 2006 achieves a desired accuracy.In at least one embodiment, the trained neural network can then be used in 2008 to implement any number of machine learning operations.

[0160] In at least one embodiment, the untrained neural network 2006 is trained using unsupervised learning, wherein the untrained neural network 2006 attempts to train itself using unlabeled data. In at least one embodiment, the training dataset 2002 for unsupervised learning includes input data without associated output data or ground-truth data. In at least one embodiment, the untrained neural network 2006 can learn groupings within the training dataset 2002 and determine how individual inputs relate to the untrained dataset 2002. In at least one embodiment, unsupervised training can be used to generate a self-organizing map in the trained neural network 2008, which is capable of performing operations useful in reducing the dimensionality of the new dataset 2012.In at least one embodiment, unsupervised training can also be used to perform anomaly detection, enabling the identification of data points in the new 2012 dataset that deviate from normal patterns of the new 2012 dataset.

[0161] In at least one embodiment, semi-supervised learning can be used, a technique in which the training dataset 2002 contains a mixture of labeled and unlabeled data. In at least one embodiment, the training framework 2004 can be used to perform incremental learning, such as through transfer learning techniques. In at least one embodiment, the incremental learning enables the trained neural network 2008 to adapt to the new dataset 2012 without forgetting the knowledge that the trained neural network 2008 acquired during the initial training. 5G NETWORKS

[0162] The following figures represent, without limitation, exemplary 5G network-based systems that can be used to implement at least one embodiment.

[0163] Fig. Figure 21 illustrates an architecture of a system 2100 of a network according to at least one embodiment. In at least one embodiment, it is shown that the system 2100 includes user equipment (UE) 2102 and UE 2104. In at least one embodiment, UEs 2102 and 2104 are illustrated as smartphones (e.g., portable mobile touchscreen computing devices that can be connected to one or more cellular networks), but they can also include any mobile or non-mobile computing device, such as personal data assistants (PDAs), pagers, laptop computers, desktop computers, wireless handheld devices, or any computing device with a wireless communication interface.

[0164] In at least one embodiment, each of the UEs 2102 and 2104 can include an Internet of Things (IoT) UE, which can include a network access layer designed for low-power IoT applications that utilize short-lived UE connections. In at least one embodiment, an IoT UE can utilize technologies such as machine-to-machine (M2M) or machine-type communication (MTC) to exchange data with an MTC server or device over a public land mobile network (PLMN), a proximity-based service (ProSe), or device-to-device (D2D) communication, sensor networks, or IoT networks. In at least one embodiment, an M2M or MTC data exchange can be machine-initiated.In at least one embodiment, an IoT network describes the connection of IoT UEs, which may include uniquely identifiable embedded computing devices (within the internet infrastructure), via short-lived connections. In at least one embodiment, IoT UEs may execute background applications (e.g., keep-alive messages, status updates, etc.) to enable connections of an IoT network.

[0165] In at least one embodiment, the UEs 2102 and 2104 can be configured to connect to a radio access network (RAN) 2116, e.g., to communicate. In at least one embodiment, the RAN 2116 can, in one example, be an Evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN), a NextGen RAN (NG RAN), or another type of RAN. In at least one embodiment, the UEs 2102 and 2104 use connections 2112 and 2114, respectively, each of which comprises a physical communication interface or layer.In at least one embodiment, the connections 2112 and 2114 are illustrated as an air interface to enable communicative coupling and can correspond to cellular communication protocols such as a Global System for Mobile Communications (GSM) protocol, a Code Division Multiple Access (CDMA) network protocol, a Push-to-Talk (PTT) protocol, a PTT-over-Cellular (POC) protocol, a Universal Mobile Telecommunications System (UMTS) protocol, a 3GPP Long Term Evolution (LTE) protocol, a Fifth Generation (5G) protocol, a New Radio (NR) protocol and variations thereof.

[0166] In at least one embodiment, the UEs 2102 and 2104 can further exchange communication data directly via a ProSe interface 2106. In at least one embodiment, the ProSe interface 2106 can alternatively be referred to as a sidelink interface comprising one or more logical channels, including, but not limited to, a Physical Sidelink Control Channel (PSCCH), Physical Sidelink Shared Channel (PSSCH), Physical Sidelink Discovery Channel (PSDCH), and Physical Sidelink Broadcast Channel (PSBCH).

[0167] In at least one embodiment, the UE 2104 is configured to access an access point (AP) 2110 via connection 2108. In at least one embodiment, connection 2108 can comprise a local wireless connection, such as a connection compliant with any IEEE 802.11 protocol, with AP 2110 comprising a Wireless Fidelity (WiFi®) router. In at least one embodiment, AP 2110 is shown to be connected to the internet without connecting to a core network of a wireless system.

[0168] In at least one embodiment, the RAN 2116 can include one or more access nodes enabling connections 2112 and 2114. In at least one embodiment, these access nodes (AN) can be designated as base stations (BS), NodeBs, evolved NodeBs (eNBs), next-generation NodeBs (gNBs), RAN nodes, etc., and can include ground stations (e.g., terrestrial access points) or satellite stations providing coverage within a geographic area (e.g., a cell). In at least one embodiment, the RAN 2116 can include one or more RAN nodes for providing macrocells, e.g., macro-RAN nodes 2118, and one or more RAN nodes for providing femtocells or picocells (e.g., cells with larger coverage areas, smaller user capacity, or greater bandwidth compared to macrocells). B. RAN node 2120 with low power (LP).

[0169] In at least one embodiment, any one of the RAN nodes 2118 and 2120 can terminate an air interface protocol and can be a first contact point for the UEs 2102 and 2104. In at least one embodiment, any one of the RAN nodes 2118 and 2120 can perform various logical functions for the RAN 2116, including, but not limited to, radio network controller (RNC) functions such as radio carrier management, dynamic uplink and downlink radio resource management, data packet scheduling, and mobility management.

[0170] In at least one embodiment, the UEs 2102 and 2104 can be configured to communicate with each other or with any RAN node 2118 and 2120 over a multi-carrier communication channel using orthogonal frequency division multiplexing (OFDM) communication signals according to various communication techniques, such as an orthogonal frequency division multiple access (OFDMA) communication technique (e.g., for downlink communication) or a single carrier frequency division multiple access (SC-FDMA) communication technique (e.g., for uplink and ProSe or sidelink communication) and / or variations thereof, without limitation. In at least one embodiment, OFDM signals can comprise a plurality of orthogonal subcarriers.

[0171] In at least one embodiment, a downlink resource grid can be used for downlink transmissions from any of the RAN nodes 2118 and 2120 to the UEs 2102 and 2104, while uplink transmissions can use similar techniques. In at least one embodiment, a grid can be a time-frequency grid, referred to as a resource grid or time-frequency resource grid, which represents a physical resource in a downlink in each time window. In at least one embodiment, such a time-frequency plane representation is common practice for OFDM systems, making it intuitive for radio resource allocation. In at least one embodiment, each column and each row of a resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. In at least one embodiment, a duration of a resource grid in a time domain corresponds to a time window in a radio frame.In at least one embodiment, the smallest time-frequency unit in a resource grid is referred to as a resource element. In at least one embodiment, each resource grid comprises a set of resource blocks that describe an allocation of specific physical channels to resource elements. In at least one embodiment, each resource block comprises a collection of resource elements. In at least one embodiment, this can represent the smallest set of resources currently allocable in a frequency domain. In at least one embodiment, there are multiple distinct physical downlink channels that are transmitted using such resource blocks.

[0172] In at least one embodiment, a Physical Downlink Shared Channel (PDSCH) can transmit user data and higher-layer signaling to UEs 2102 and 2104. In at least one embodiment, a Physical Downlink Control Channel (PDCCH) can transmit, among other things, information about a transport format and resource allocations with respect to the PDSCH channel. In at least one embodiment, it can also inform UEs 2102 and 2104 about a transport format, resource allocation, and HARQ (Hybrid Automatic Repeat Request) information regarding a shared uplink channel. In at least one embodiment, downlink scheduling (allocating control and resource blocks of a shared channel to UE 2102 within a cell) can typically be performed at any of the RAN nodes 2118 and 2120 based on channel quality information returned by any of the UEs 2102 and 2104.In at least one embodiment, downlink resource allocation information can be sent on a PDCCH that is used (e.g., allocated) for each of the UEs 2102 and 2104.

[0173] In at least one embodiment, a PDCCH can use control channel elements (CCEs) to transmit control information. In at least one embodiment, complex-valued PDCCH symbols can first be organized into quadruples before being assigned to resource elements, which can then be permuted using a subblock interleaver for rate matching. In at least one embodiment, each PDCCH can be transmitted using one or more of these CCEs, each CCE being able to correspond to nine sets of four physical resource elements known as resource element groups (REGs). In at least one embodiment, each REG can be assigned four quadrature phase shift keying (QPSK) symbols.In at least one embodiment, the PDCCH can be transmitted using one or more CCEs, depending on the amount of downlink control information (DCI) and the channel state. In at least one embodiment, four or more different PDCCH formats can be defined in LTE with different numbers of CCEs (e.g., aggregation level, L = 1, 2, 4, or 8).

[0174] In at least one embodiment, an Enhanced Physical Downlink Control Channel (EPDCCH) utilizing PDSCH resources can be used to transmit control information. In at least one embodiment, the EPDCCH can be transmitted using one or more enhanced control channel elements (ECCEs). In at least one embodiment, each ECCE can correspond to nine sets of four physical resource elements known as enhanced resource element groups (EREGs). In at least one embodiment, an ECCE can have a different number of EREGs in some situations.

[0175] In at least one embodiment, the RAN 2116 is communicatively coupled to a core network (CN) 2138 via an S1 interface 2122. In at least one embodiment, the CN 2138 can be an EPC (Evolved Packet Core) network, an NPC (NextGen Packet Core) network, or another type of CN. In at least one embodiment, the S1 interface 2122 is divided into two parts: an S1-U interface 2126, which transmits traffic data between the RAN nodes 2118 and 2120 and the servicing gateway (S-GW) 2130, and an S1 mobility management entity (MME) interface 2124, which is a signaling interface between the RAN nodes 2118 and 2120 and the MMEs 2128.

[0176] In at least one embodiment, the CN 2138 comprises the MMEs 2128, the S-GW 2130, the Packet Data Network (PDN) gateway (P-GW) 2134, and a Home Subscriber Server (HSS) 2132. In at least one embodiment, the MMEs 2128 can function similarly to a control plane of Serving General Packet Radio Service (GPRS) legacy support nodes (SGSN). In at least one embodiment, the MMEs 2128 can manage mobility aspects of access, such as gateway selection and the management of tracking area lists. In at least one embodiment, the HSS 2132 can include a database for network users, including subscription-related information, to support the handling of communication sessions by a network entity.In at least one embodiment, the CN 2138 can include one or more HSSs 2132s, depending on the number of mobile subscribers, the capacity of an equipment, the organization of a network, etc. In at least one embodiment, the HSS 2132 can provide support for routing / roaming, authentication, authorization, name / address resolution, location dependencies, etc.

[0177] In at least one embodiment, the S-GW 2130 can terminate an S1 interface 2122 towards RAN 2116 and route data packets between RAN 2116 and CN 2138. In at least one embodiment, the S-GW 2130 can be a local mobility anchor point for inter-RAN node handoffs and can also provide an anchor for inter-3GPP mobility. In at least one embodiment, other responsibilities can include legal interception, fees, and enforcement of certain policies.

[0178] In at least one embodiment, the P-GW 2134 can terminate an SGi interface towards a PDN. In at least one embodiment, the P-GW 2134 can route data packets between an EPC network 2138 and external networks, such as a network containing the application server 2140 (alternatively referred to as an application function (AF)), via an Internet Protocol (IP) interface 2142. In at least one embodiment, the application server 2140 can be an element that provides applications that use IP carrier resources with a core network (e.g., UMTS packet services (PS) domain, LTE PS data services, etc.). In at least one embodiment, the P-GW 2134 is shown to be communicatively coupled to an application server 2140 via an IP communication interface 2142. In at least one embodiment, the application server 2140 can also be configured to provide one or more communication services (e.g.,to support Voice-over-Internet Protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, etc.) for UEs 2102 and 2104 via CN 2138.

[0179] In at least one embodiment, the P-GW 2134 can further be a policy enforcement and charge data collection node. In at least one embodiment, the policy and charge enforcement function (PCRF) 2136 is a policy and charge control element of the CN 2138. In at least one embodiment, in a non-roaming scenario, there can be a single PCRF in a home public land mobile network (HPLMN) associated with a session of an UE's Internet Protocol Connectivity Access Network (IP-CAN).In at least one embodiment, in a roaming scenario with a local traffic outbreak, there can be two PCRFs associated with an IP-CAN session of a UE: a home PCRF (H-PCRF) within a home mobile network (HPLMN) and a visited PCRF (V-PCRF) within a visited public land mobile network (VPLMN). In at least one embodiment, the PCRF 2136 can be communicatively coupled to an application server 2140 via the P-GW 2134. In at least one embodiment, the application server 2140 can signal the PCRF 2136 to specify a new service flow and to select a suitable quality of service (QoS) and charge parameters.In at least one embodiment, the 2136 can provide this rule in a Policy and Charging Enforcement Function (PCEF) (not shown) with a suitable Traffic Flow Template (TFT) and QoS class of identifier (QCI) that initiates a QoS and charges as determined by Application Server 2140.

[0180] Fig. Figure 22 illustrates an architecture of a System 2200 of a network according to some embodiments. In at least one embodiment, it is shown that the System 2200 includes a UE 2202, a 5G access node or RAN node (shown as (R)AN node 2208), a user-level function (shown as UPF 2204), a data network (DN 2206), which in one example may be operator services, internet access or third-party services, and a 5G core network (5GC) (shown as CN 2210).

[0181] In at least one embodiment, CN 2210 includes an Authentication Server Function (AUSF 2214); a Core Access and Mobility Function (AMF 2212); a Session Management Function (SMF 2218); a Network Exposure Function (NEF 2216); a Policy Control Function (PCF 2222); a Network Function (NF) Repository Function (NRF 2220); a Unified Data Management Function (UDM 2224); and an Application Function (AF 2226). In at least one embodiment, CN 2210 may also include other elements not shown, such as a structured data storage network function (SDSF), an unstructured data storage network function (UDSF), and variations thereof.

[0182] In at least one embodiment, the UPF 2204 can serve as an anchor point for intra-RAT and inter-RAT mobility, an external PDU session point for interconnection with the DN 2206, and a branch point to support a multi-networked PDU session. In at least one embodiment, the UPF 2204 can also perform packet routing and forwarding, packet inspection, enforcement of some user-level policy rules, legal packet interception (UP collection), traffic usage reporting, user-level QoS handling (e.g., packet filtering, gating, UL / DL rate enforcement), uplink traffic verification (e.g., SDF-to-QoS flow mapping), transport-level packet marking in uplink and downlink, downlink packet buffering, and downlink data notification triggering.In at least one embodiment, the UPF 2204 can include an uplink classifier to support the routing of data traffic flows to a data network. In at least one embodiment, the DN 2206 can represent various network operator services, internet access, or third-party services.

[0183] In at least one embodiment, AUSF 2214 can store data for authenticating UE 2202 and handle authentication-related functionality. In at least one embodiment, AUSF 2214 can enable a common authentication framework for different access types.

[0184] In at least one embodiment, the AMF 2212 can be responsible for registration management (e.g., for registering the UE 2202, etc.), connection management, reachability management, mobility management, and the legal interception of AMF-related events, as well as access authentication and authorization. In at least one embodiment, the AMF 2212 can provide the transport of SM messages for the SMF 2218 and act as a transparent proxy for routing SM messages. In at least one embodiment, the AMF 2212 can also handle the transport of Short Message Service (SMS) messages between the UE 2202 and an SMS Function (SMSF) (in Fig. (22 not shown). In at least one embodiment, the AMF 2212 can function as a Security Anchor Function (SEA) that may involve interaction with the AUSF 2214 and the UE 2202 and the receipt of an intermediate key generated as a result of the UE 2202's authentication process. In at least one embodiment using USIM-based authentication, the AMF 2212 can retrieve security material from the AUSF 2214. In at least one embodiment, the AMF 2212 can also include a Security Context Management (SCM) function that receives a key from the SEA, which it uses to derive access network-specific keys.In at least one embodiment, the AMF 2212 can also be a termination point of the RAN-CP interface (N2 reference point), a termination point of the NAS(NI) signaling, and perform NAS encryption and integrity protection.

[0185] In at least one embodiment, the AMF 2212 can also support NAS signaling with a UE 2202 via an N3 Interworking Function (IWF) interface. In at least one embodiment, the N3IWF can be used to provide access to untrusted entities. In at least one embodiment, the N3IWF can be a termination point for N2 and N3 interfaces for the control plane and user plane, respectively, and can thus handle N2 signaling from the SMF and the AMF for PDU sessions and QoS, encapsulate / decapsulate data packets for IPSec and N3 tunneling, tag N3 user-plane packets in the uplink, and enforce QoS according to the N3 packet tagging, taking into account the QoS requirements associated with such tagging received via N2.In at least one embodiment, the N3IWF can also forward uplink and downlink control plane NAS(NI) signaling between the UE 2202 and the AMF 2212, and forward uplink and downlink user plane packets between the UE 2202 and the UPF 2204. In at least one embodiment, the N3IWF also provides mechanisms for establishing IPsec tunnels with the UE 2202.

[0186] In at least one embodiment, the SMF 2218 can be responsible for session management (e.g., session setup, modification, and release, including maintaining the tunnel between the UPF and the AN node); UE IP address assignment and management (including optional authorization); selection and control of the UP function; configuration of traffic control at the UPF to route traffic to the correct destination; termination of interfaces to policy control functions; control of part of policy enforcement and QoS; legal interception (for SM events and interface to the LI system); termination of SM portions of NAS messages; downlink data notification; initiator of AN-specific SM information sent to the AN via AMF over N2; and determining the SSC mode of a session.In at least one embodiment, the SMF 2218 can include the following roaming functionality: handling local enforcement to apply QoS SLAB (VPLMN); charge data collection and charge interface (VPLMN); legal interception (in the VPLMN for SM events and interface to the LI system); support for interaction with the external DN for transport of signaling for PDU session authorization / authentication by the external DN.

[0187] In at least one embodiment, the NEF 2216 can provide means for the secure disclosure of services and capabilities provided to third parties by 3GPP network functions, internal disclosure / redisclosure, application functions (e.g., AF 2226), edge computing or fog computing systems, etc. In at least one embodiment, the NEF 2216 can authenticate, authorize, and / or throttle AFs. In at least one embodiment, the NEF 2216 can also translate information exchanged with the AF 2226 and information exchanged with internal network functions. In at least one embodiment, the NEF 2216 can translate between an AF service identifier and internal 5GC information. In at least one embodiment, the NEF 2216 can also receive information from other network functions (NFs) based on disclosed capabilities of other network functions.In at least one embodiment, this information can be stored as structured data in the NEF 2216 or in a data storage NF using standardized interfaces. In at least one embodiment, stored information can then be disclosed again by the NEF 2216 to other NFs and AFs and / or used for other purposes, such as analyses.

[0188] In at least one embodiment, the NRF 2220 can support service detection functions, receive NF detection requests from NF instances, and provide information about detected NF instances to NF instances. In at least one embodiment, the NRF 2220 also maintains information about available NF instances and their supported services.

[0189] In at least one embodiment, the PCF 2222 can provide policy rules for the function(s) at the control level to enforce them and can also support a unified policy framework to regulate network behavior. In at least one embodiment, the PCF 2222 can also implement a front end (FE) to access subscription information relevant for policy decisions in a UDR of the UDM 2224.

[0190] In at least one embodiment, the UDM 2224 can process subscription-related information to support the handling of communication sessions between network units and can store subscription data from the UE 2202. In at least one embodiment, the UDM 2224 can comprise two parts: an application frontend and a user data repository (UDR). In at least one embodiment, the UDM can include a UDM frontend responsible for processing access credentials, location management, subscription management, and so on. In at least one embodiment, multiple different frontends can serve the same user in different transactions.In at least one embodiment, the UDM-FE accesses subscription information stored in a UDR and performs authentication access data processing; user identification handling; access authorization; registration / mobility management; and subscription management. In at least one embodiment, the UDR can interact with the PCF 2222. In at least one embodiment, the UDM 2224 can also support SMS management, with an SMS-FE implementing similar application logic to that discussed previously.

[0191] In at least one embodiment, the AF 2226 can provide application influence over traffic routing, access to a Network Capability Exposure (NCE), and interact with a policy framework for policy control. In at least one embodiment, the NCE can be a mechanism that allows a 5GC and an AF 2226 to provide each other with information via NEF 2216 that can be used for edge computing implementations. In at least one embodiment, network operator and third-party services can be hosted near the UE 2202 access point to achieve efficient service delivery through reduced end-to-end latency and load on a transport network.In at least one embodiment, the 5GC for edge computing implementations can select a UPF 2204 near the UE 2202 and perform traffic control from the UPF 2204 to the DN 2206 via the N6 interface. In at least one embodiment, this can be based on UE subscription data, the UE location, and information provided by the AF 2226. In at least one embodiment, the AF 2226 can influence the UPF (re)selection and traffic routing. In at least one embodiment, a network operator can allow the AF 2226 to interact directly with relevant network fields, based on the operator's deployment, if the AF 2226 is considered a trusted entity.

[0192] In at least one embodiment, the CN 2210 can include an SMSF that can be responsible for SMS subscription checks and verification and for forwarding SMS messages to / from the UE 2202 to / from other units, such as an SMS-GMSC / IWMSC / SMS router. In at least one embodiment, SMS can also interact with AMF 2212 and UDM 2224 for a notification procedure indicating that the UE 2202 is available for SMS transmission (e.g., setting a UE unreachable flag and notifying the UDM 2224 when the UE 2202 is available for SMS).

[0193] In at least one embodiment, the System 2200 may include the following service-based interfaces: Namf: service-based interface shown by AMF; Nsmf: service-based interface shown by SMF; Nnef: service-based interface shown by NEF; Npcf: service-based interface shown by PCF; Nudm: service-based interface shown by UDM; Naf: service-based interface shown by AF; Nnrf: service-based interface shown by NRF; and Nausf: service-based interface shown by AUSF.

[0194] In at least one embodiment, the System 2200 can include the following reference points: N1: reference point between UE and AMF; N2: reference point between (R)AN and AMF; N3: reference point between (R)AN and UPF; N4: reference point between SMF and UPF; and N6: reference point between UPF and a data network. In at least one embodiment, there can be many more reference points and / or service-based interfaces between NF services in NF; however, these interfaces and reference points have been omitted for clarity. In at least one embodiment, an NS reference point can be located between the PCF and the AF; an N7 reference point can be located between the PCF and the SMF; an N11 reference point can be located between the AMF and the SMF; and so on.In at least one embodiment, the CN 2210 can include an Nx interface, which is an inter-CN interface between MME and AMF 2212, to enable cooperation between the CN 2210 and the CN 7221.

[0195] In at least one embodiment, the system 2200 can include multiple RAN nodes (such as (R)AN nodes 2208), wherein an Xn interface is defined between two or more (R)AN nodes 2208 (e.g., gNBs) connected to the 5GC 410, between an (R)AN node 2208 (e.g., gNB) connected to the CN 2210 and an eNB (e.g., a macro-RAN node), and / or between two eNBs connected to the CN 2210.

[0196] In at least one embodiment, the Xn interface can include an Xn user-level (Xn-U) interface and an Xn control-level (Xn-C) interface. In at least one embodiment, Xn-U can provide non-guaranteed delivery of user-level PDUs and support / provide data routing and flow control functionality. In at least one embodiment, Xn-C can provide management and error handling functionality, functionality for managing an Xn-C interface, and mobility support for the UE 2202 in a connected mode (e.g., CM-CONNECTED), including functionality for managing UE mobility for the connected mode between one or more (R)AN nodes 2208.In at least one embodiment, the mobility support can include a context transfer from an old operating (R)AN node 2208 (source) to a new operating (R)AN node 2208 (destination); and control of user-level tunnels between the old operating (R)AN node 2208 (source) and the new operating (R)AN node 2208 (destination).

[0197] In at least one embodiment, an Xn-U protocol stack can include a transport network layer built on top of an Internet Protocol (IP) transport layer and a GTP-U layer on top of a UDP and / or IP layer(s) to deliver user-level PDUs. In at least one embodiment, the Xn-C protocol stack can include an application-layer signaling protocol (referred to as the Xn application protocol (Xn-AP)) and a transport network layer built on top of an SCTP layer. In at least one embodiment, the SCTP layer can reside on top of an IP layer. In at least one embodiment, the SCTP layer provides guaranteed delivery of application-layer messages. In at least one embodiment, point-to-point transmission is used in a transport IP layer to deliver signaling PDUs.In at least one embodiment, the Xn-U protocol stack and / or an Xn-C protocol stack may be the same as or similar to a user-level and / or control-level protocol stack (or stacks) shown and described herein.

[0198] Fig. Figure 23 illustrates a control-level protocol stack according to at least one embodiment. In at least one embodiment, a control level 2300 is shown as a communication protocol stack between the UE 2102 (or alternatively the UE 2104), the RAN 2116, and the MME(s) 2128.

[0199] In at least one embodiment, the PHY layer 2302 can send or receive information used by the MAC layer 2304 over one or more air interfaces. In at least one embodiment, the PHY layer 2302 can further perform link matching or adaptive modulation and coding (AMC), power control, cell search (e.g., for initial synchronization and handover purposes), and other measurements used by higher layers, such as an RRC layer 2310. In at least one embodiment, the PHY layer 2302 can also perform transport channel error detection, transport channel forward error correction (FEC) encoding / decoding, physical channel modulation / demodulation, interleaving, rate matching, physical channel mapping, and multiple-input multiple-output (MIMO) antenna processing.

[0200] In at least one embodiment, the MAC layer 2304 can perform mapping between logical channels and transport channels, multiplexing of MAC service data units (SDUs) from one or more logical channels onto transport blocks (TBs) to be delivered to PHY via transport channels, demultiplexing of MAC-SDUs to one or more logical channels of transport blocks (TBs) to be delivered by PHY via transport channels, multiplexing of MAC-SDUs onto TBs, scheduling of information reports, error correction through hybrid automatic repeat request (HARD), and logical channel prioritization.

[0201] In at least one embodiment, the RLC layer 2306 can operate in a variety of modes, including: Transparent Mode (TM), Unacknowledged Mode (UM), and Acknowledged Mode (AM). In at least one embodiment, the RLC layer 2306 can perform upper-layer protocol data unit (PDU) transfers, automatic repeat request (ARQ) error correction for AM data transfers, and concatenation, segmentation, and reassembly of RLC SDUs for UM and AM data transfers. In at least one embodiment, the RLC layer 2306 can also perform resegmentation of RLC data PDUs for AM data transfers, rearrange RLC data PDUs for UM and AM data transfers, detect duplicate data for UM and AM data transfers, discard RLC SDUs for UM and AM data transfers, detect protocol errors for AM data transfers, and perform RLC recovery.

[0202] In at least one embodiment, the PDCP layer 2308 can perform header compression and decompression of IP data, maintain PDCP sequence numbers (SNs), perform sequential delivery of upper layer PDUs during lower layer recovery, eliminate duplicate lower layer SDUs during lower layer recovery for RLC-AM-associated radio carriers, encrypt and decrypt control plane data, perform integrity protection and integrity verification of control plane data, control timer-based data discard, and perform security operations (e.g., encryption, decryption, integrity protection, integrity verification, etc.).

[0203] In at least one embodiment, the main services and functions of an RRC layer 2310 may include sending system information (e.g., in master information blocks (MIB) or system information blocks (SIB) with respect to a non-access stratum (NAS)); sending system information with respect to an access stratum (AS); paging; establishing, maintaining, and releasing an RRC link between a UE and E-UTRAN (e.g., RRC link paging, RRC link establishment, RRC link modification, and RRC link release); setting up, configuring, maintaining, and releasing point-to-point radio carriers; security functions, including key management; mobility between radio access technology (RAT); and measurement configuration for UE measurement reports.In at least one embodiment, the MIBs and SIBs can comprise one or more information elements (IEs), each of which can comprise individual data fields or data structures.

[0204] In at least one embodiment, the UE 2102 and the RAN 2116 can use a Uu interface (e.g., an LTE Uu interface) to exchange control plane data via a protocol stack comprising the PHY layer 2302, the MAC layer 2304, the RLC layer 2306, the PDCP layer 2308, and the RRC layer 2310.

[0205] In at least one embodiment, non-access layer (NAS) protocols (NAS protocols 2312) form a highest layer of a control plane between the UE 2102 and the MME(s) 2128. In at least one embodiment, the NAS protocols 2312 support the mobility of the UE 2102 and session management procedures to establish and maintain IP connectivity between the UE 2102 and the P-GW 2134.

[0206] In at least one embodiment, the Si application protocol (S1-AP) layer (Si-AP layer 2322) can support Si interface functions and include elementary procedures (EPs). In at least one embodiment, an EP is an interaction unit between the RAN 2116 and the CN 2128. In at least one embodiment, S1-AP layer services can comprise two groups: UE-associated services and non-UE-associated services. In at least one embodiment, these services perform functions that include, but are not limited to: E-UTRAN radio access bearer (E-RAB) management, UE capability indication, mobility, NAS signaling transport, RAN information management (RIM), and configuration transmission.

[0207] In at least one embodiment, the Stream Control Transmission Protocol (SCTP) layer (alternatively referred to as the Stream Control Transmission Protocol / Internet Protocol (SCTP / IP) layer) (SCTP layer 2320) can ensure reliable delivery of signaling messages between the RAN 2116 and the MME(s) 2128, partly based on an IP protocol supported by an IP layer 2318. In at least one embodiment, the L2 layer 2316 and an L1 layer 2314 can refer to communication links (e.g., wired or wireless) used by a RAN node and an MME to exchange information.

[0208] In at least one embodiment, the RAN 2116 and the MME(s) 2128 can use an S1-MME interface to exchange control plane data via a protocol stack comprising an L1 layer 2314, an L2 layer 2316, an IP layer 2318, an SCTP layer 2320 and a Si-AP layer 2322.

[0209] Fig. Figure 24 illustrates a user-level protocol stack according to at least one embodiment. In at least one embodiment, a user level 2400 is shown as a communication protocol stack between a UE 2102, the RAN 2116, the S-GW 2130, and the P-GW 2134. In at least one embodiment, the user level 2400 can utilize the same protocol layers as the control level 2300. For example, in at least one embodiment, the UE 2102 and the RAN 2116 can use a Uu interface (e.g., an LTE Uu interface) to exchange user-level data via a protocol stack comprising the PHY layer 2302, the MAC layer 2304, the RLC layer 2306, and the PDCP layer 2308.

[0210] In at least one embodiment, the General Packet Radio Service (GPRS) tunneling protocol can be used for a user-level layer (GTP-U) (GTP-U layer 2404) to transmit user data within a GPRS core network and between a radio access network and a core network. In at least one embodiment, the transmitted user data can be packets in any of the IPv4, IPv6, or PPP formats. In at least one embodiment, the UDP and IP security (UDP / IP) layer (UDP / IP layer 2402) can provide checksums for data integrity, port numbers for addressing different functions at a source and a destination, and encryption and authentication for selected data flows.In at least one embodiment, the RAN 2116 and the S-GW2130 can use an S1-U interface to exchange user-level data over a protocol stack comprising L1 layer 2314, L2 layer 2316, UDP / IP layer 2402, and GTP-U layer 2404. In at least one embodiment, the S-GW 2130 and the P-GW 2134 can use an S5 / S8a interface to exchange user-level data over a protocol stack comprising L1 layer 2314, L2 layer 2316, UDP / IP layer 2402, and GTP-U layer 2404. In at least one embodiment, as described above with reference to the... Fig. As discussed in section 22, the NAS protocols support UE 2102 mobility and session management procedures to establish and maintain IP connectivity between the UE 2102 and the P-GW 2134.

[0211] Fig. Figure 25 illustrates components 2500 of a core network according to at least one embodiment. In at least one embodiment, the components of CN 2138 can be implemented in a physical node or separate physical nodes, including components for reading and executing instructions from a machine-readable or computer-readable medium (e.g., a non-transient machine-readable storage medium). In at least one embodiment, network functions virtualization (NFV) is used to virtualize one or all of the network node functions described above via executable instructions stored in one or more computer-readable storage media (described in more detail below). In at least one embodiment, a logical instantiation of CN 2138 can be referred to as a network slice 2502 (e.g.,The network slice 2502 includes, as shown in the HSS 2132, the MME(s) 2128 and the S-GW 2130. In at least one embodiment, a logical instantiation of a section of the CN 2138 can be designated as a network sub-slice 2504 (e.g., the network sub-slice 2504 includes, as shown in the representation, the P-GW 2134 and the PCRF 2136).

[0212] In at least one embodiment, NFV architectures and infrastructures can be used to virtualize one or more network functions, alternatively performed by proprietary hardware, on physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches. In at least one embodiment, NFV systems can be used to run virtual or reconfigurable implementations of one or more EPC components / functions.

[0213] Fig. Figure 26 is a block diagram illustrating components of a System 2600 for supporting Network Function Virtualization (NFV) according to at least one embodiment. In at least one embodiment, the System 2600 is illustrated to include a virtualized infrastructure manager (shown as VIM 2602), a network function virtualization infrastructure (shown as NFVI 2604), a VNF manager (shown as VNFM 2606), virtualized network functions (shown as VNF 2608), an element manager (shown as EM 2610), an NFV orchestrator (shown as NFVO 2612), and a network manager (shown as NM 2614).

[0214] In at least one embodiment, the VIM 2602 manages resources of the NFVI 2604. In at least one embodiment, the NFVI 2604 may include physical or virtual resources and applications (including hypervisors) used to run the System 2600. In at least one embodiment, the VIM 2602 may manage a virtual resource lifecycle with the NFVI 2604 (e.g., creating, maintaining, and dismantling virtual machines (VMs) associated with one or more physical resources), track VM instances, monitor performance, errors, and security of VM instances and associated physical resources, and expose VM instances and associated physical resources to other management systems.

[0215] In at least one embodiment, the VNFM 2606 can manage the VNF 2608. In at least one embodiment, the VNF 2608 can be used to execute EPC components / functions. In at least one embodiment, the VNFM 2606 can manage a lifecycle of the VNF 2608 and track the performance, faults, and safety of virtual aspects of the VNF 2608. In at least one embodiment, the EM 2610 can track the performance, faults, and safety of functional aspects of the VNF 2608. In at least one embodiment, tracking data from the VNFM 2606 and the EM 2610 can, for example, include performance measurement (PM) data used by the VIM 2602 or the NFVI 2604. In at least one embodiment, both the VNFM 2606 and the EM 2610 can scale a set of the VNFs of the System 2600 up / down.

[0216] In at least one embodiment, the NFVO 2612 can coordinate, authorize, release, and deploy resources of the NFVI 2604 to provide a requested service (e.g., to execute an EPC function, component, or slice). In at least one embodiment, the NM 2614 can provide a package of end-user functions responsible for managing a network, which may include network elements with VNF, non-virtualized network functions, or both (the management of the VNF may be handled by the EM 2610). COMPUTER-BASED SYSTEMS

[0217] The following figures represent, without limitation, exemplary computer-based systems that can be used to implement at least one embodiment.

[0218] Fig. Figure 27 illustrates a processing system 2700 according to at least one embodiment. In at least one embodiment, the processing system 2700 includes one or more processors 2702 and one or more graphics processors 2708, and it can be a single-processor desktop system, a multi-processor workstation system, or a server system comprising a large number of processors 2702 or processor cores 2707. In at least one embodiment, the processing system 2700 is a processing platform integrated into an integrated circuit as a system-on-a-chip (“SoC”) for use in mobile, portable, or embedded devices.

[0219] In at least one embodiment, the processing system 2700 can include or be integrated with a server-based gaming platform, a game console, a media console, a mobile gaming console, a handheld gaming console, or an online gaming console. In at least one embodiment, the processing system 2700 is a mobile phone, a smartphone, a tablet computer, or a mobile internet device. In at least one embodiment, the processing system 2700 can also include, be coupled to, or be integrated with a wearable device, such as a wearable smartwatch device, a smart eyewear device, an augmented reality device, or a virtual reality device.In at least one embodiment, the processing system 2700 is a television or set-top box device comprising one or more processors 2702 and a graphical interface generated by one or more graphics processors 2708.

[0220] In at least one embodiment, one or more processors 2702 each include one or more processor cores 2707 for processing instructions that, upon execution, perform operations for system and user software. In at least one embodiment, each of the one or more processor cores 2707 is configured to process a specific instruction set 2709. In at least one embodiment, the instruction set 2709 can enable Complex Instruction Set Computing (“CISC”), Reduced Instruction Set Computing (“RISC”), or Very Long Instruction Word Computing (“VLIW”). In at least one embodiment, the processor cores 2707 can each process a different instruction set 2709, which may contain instructions to facilitate the emulation of other instruction sets.In at least one embodiment, the processor core 2707 can also include other processing devices, such as a digital signal processor (“DSP”).

[0221] In at least one embodiment, the processor 2702 includes a cache memory (“cache”) 2704. In at least one embodiment, the processor 2702 can have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory is shared by different components of the processor 2702. In at least one embodiment, the processor 2702 also uses an external cache (e.g., a Level 3 (“L3”) cache or a Last Level Cache (“LLC”)) (not shown) that can be shared by the processor cores 2707 using known cache coherence techniques. In at least one embodiment, the processor 2702 additionally includes a register bank 2706, which can contain different types of registers for storing different types of data (e.g.,Integer registers, floating-point registers, status registers, and an instruction pointer register). In at least one embodiment, the register bank may include 2706 general-purpose registers or other registers.

[0222] In at least one embodiment, one or more processors 2702 are coupled to one or more interface buses 2710 to transmit communication signals, such as address, data, or control signals, between the processor 2702 and other components in the processing system 2700. In at least one embodiment, the interface bus 2710 can be a processor bus, such as a version of a Direct Media Interface (“DMI”) bus. In at least one embodiment, the interface bus 2710 is not limited to a DMI bus and can include one or more peripheral component interconnect buses (e.g., “PCI”, PCI Express (“PCIe”)), memory buses, or other types of interface buses. In at least one embodiment, the processor(s) 2702 include an integrated memory controller 2716 and a platform control hub 2730.In at least one embodiment, the storage controller 2716 enables communication between a storage device and other components of the processing system 2700, while the platform controller hub (“PCH”) 2730 provides connections to input / output (“I / O”) devices via a local I / O bus.

[0223] In at least one embodiment, the storage device 2720 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or another storage device with suitable performance to serve as processor memory. In at least one embodiment, the storage device 2720 can operate as system memory for the processing system 2700 to store data 2722 and instructions 2721 for use when one or more processors 2702 execute an application or process. In at least one embodiment, the memory controller 2716 is also coupled to an optional external graphics processor 2712, which can communicate with one or more graphics processors 2708 in the processors 2702 to perform graphics and media operations.In at least one embodiment, a display device 2711 can be connected to the processor(s) 2702. In at least one embodiment, the display device 2711 can include one or more internal display devices, such as in a mobile electronic device or a laptop device, or external display devices connected via a display interface (e.g., DisplayPort, etc.). In at least one embodiment, the display device 2711 can include a head-mounted display (HMD), such as a stereoscopic display device for use in virtual reality (VR) or augmented reality (AR) applications.

[0224] In at least one embodiment, the platform control hub 2730 enables peripheral devices to be connected to the storage device 2720 and the processor 2702 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include an audio controller 2746, a network controller 2734, a firmware interface 2728, a wireless transceiver 2726, touch sensors 2725, and a data storage device 2724 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 2724 can connect via a storage interface (e.g., SATA) or via a peripheral bus, such as PCI or PCIe. In at least one embodiment, the touch sensors 2725 can include touchscreen sensors, pressure sensors, or fingerprint sensors.In at least one embodiment, the wireless transceiver 2726 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a cellular network transceiver, such as a 3G, 4G, or Long-Term Evolution (“LTE”) transceiver. In at least one embodiment, the firmware interface 2728 enables communication with the system firmware and, in one example, can be a unified extensible firmware interface (“UEFI”). In at least one embodiment, the network controller 2734 can enable a network connection to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to the interface bus 2710. In at least one embodiment, the audio controller 2746 is a multi-channel high-definition audio controller.In at least one embodiment, the system 2700 includes an optional legacy I / O controller 2740 for coupling legacy devices (e.g., Personal System 2 (“PS / 2”)) to the processing system 2700. In at least one embodiment, the platform control hub 2730 can also be connected to one or more universal serial bus (“USB”) controllers 2742, which are connected to input devices such as keyboard and mouse combinations 2743, a camera 2744, or other USB input devices.

[0225] In at least one embodiment, an instance of the memory controller 2716 and the platform control hub 2730 can be integrated into a discrete external graphics processor, such as the external graphics processor 2712. In at least one embodiment, the platform control hub 2730 and / or the memory controller 2716 can be external to one or more processor(s) 2702. For example, in at least one embodiment, the processing system 2700 can include an external memory controller 2716 and a platform control hub 2730, which can be configured as a memory control hub and peripheral device control hub within a system chipset that communicates with processor(s) 2702.

[0226] Fig. Figure 28 illustrates a computer system 2800 according to at least one embodiment. In at least one embodiment, the computer system 2800 can be a system with connected devices and components, a system-on-a-chip (SOC), or a combination thereof. In at least one embodiment, the computer system 2800 comprises a processor 2802, which may include execution units for executing an instruction. In at least one embodiment, the computer system 2800 can, without limitation, include a component, such as a processor 2802, to employ execution units that include logic for performing algorithms for processing data.In at least one embodiment, the Computer System 2800 may include processors such as the PENTIUM® processor family, Xeon™, Itanium®, XScale™ and / or StrongARM™ microprocessors, Intel® Core™ or Intel® Nervana™ processors, available from Intel® Corporation of Santa Clara, California, although other systems (including PCs with other microprocessors, engineering workstations, set-top boxes, and the like) may also be used. In at least one embodiment, the Computer System 2800 may run a version of the WINDOWS operating system available from Microsoft Corporation in Redmond, Washington, although other operating systems (in one example, UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0227] In at least one embodiment, the Computer System 2800 can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices are mobile phones, Internet Protocol devices, digital cameras, personal digital assistants (“PDAs”), and portable PCs. In at least one embodiment, embedded applications can include a microcontroller, a digital signal processor (DSP), a system-on-a-chip (SoC), network computers (“NetPCs”), set-top boxes, network hubs, wide area network (“WAN”) switches, or any other system capable of executing one or more instructions.

[0228] In at least one embodiment, the computer system 2800 can, without limitation, include a processor 2802, which can, without limitation, include one or more execution units 2808 that can be configured to execute a Compute Unified Device Architecture (“CUDA”) program (CUDA® is developed by NVIDIA Corporation, Santa Clara, California). In at least one embodiment, a CUDA program is at least one section of a software application written in a CUDA programming language. In at least one embodiment, the computer system 2800 is a desktop or server system with a single processor. In at least one embodiment, the computer system 2800 can be a multiprocessor system.In at least one embodiment, the processor 2802 can, without limitation, include a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing a combination of instruction sets, or any other processing device, such as a digital signal processor. In one example, the processor 2802 can be coupled to a processor bus 2810, which can transmit data signals between the processor 2802 and other components in the computer system 2800.

[0229] In at least one embodiment, the processor 2802 can include, without limitation, an internal level 1 cache (“L1”) 2804. In at least one embodiment, the processor 2802 can have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache can be located external to the processor 2802. In at least one embodiment, the processor 2802 can also include a combination of both internal and external caches. In at least one embodiment, a register bank 2806 can store different types of data in different registers, including, without limitation, integer registers, floating-point registers, status registers, and an instruction pointer register.

[0230] In at least one embodiment, the execution unit 2808, including without limitation the logic for performing integer and floating-point operations, is also located in the processor 2802. The processor 2802 may also include read-only memory (“ROM”) for microcode (“µcode”), which stores microcode for specific macro instructions. In at least one embodiment, the execution unit 2808 may include logic for handling a packed instruction set 2809. In at least one embodiment, operations used by many multimedia applications can be performed using packed data in a general-purpose processor 2802 by including the packed instruction set 2809 in an instruction set of a general-purpose processor 2802, together with associated circuitry for executing instructions.In at least one embodiment, many multimedia applications can be accelerated and run more efficiently by using the full width of a processor's data bus to perform operations on packed data, thereby eliminating the need to transfer smaller data units across the processor's data bus to perform one or more operations on one data element at a time.

[0231] In at least one embodiment, the execution unit 2808 can also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, the computer system 2800 can include a memory 2820 without restriction. In at least one embodiment, the memory 2820 can be implemented as a DRAM device, an SRAM device, a flash memory device, or another storage device. The memory 2820 can store instructions 2819 and / or data 2821, represented by data signals that can be executed by the processor 2802.

[0232] In at least one embodiment, a system logic chip can be coupled to the processor bus 2810 and the memory 2820. In at least one embodiment, a system logic chip can, without restriction, include a memory control hub (“MCH”) 2816, and the processor 2802 can communicate with the MCH 2816 via the processor bus 2810. In at least one embodiment, the MCH 2816 can provide the memory 2820 with a high-bandwidth memory path 2818 for instruction and data storage, as well as for storing graphics instructions, data, and textures. In at least one embodiment, the MCH 2816 can route data signals between the processor 2802, the memory 2820, and other components in the computer system 2800, and bridge data signals between the processor bus 2810, the memory 2820, and a system I / O 2822. In at least one embodiment, the system logic chip can provide a graphics port for coupling with a graphics controller.In at least one embodiment, the MCH 2816 can be coupled to the memory 2820 via a high-bandwidth memory path 2818, and the graphics / video card 2812 can be coupled to the MCH 2816 via an interconnection of an accelerated graphics port (“AGP”) 2814.

[0233] In at least one embodiment, the computer system 2800 can use the system I / O 2822, which is a proprietary hub interface bus, to couple the MCH 2816 to the I / O controller hub (“ICH”) 2830. In at least one embodiment, the ICH 2830 can provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus can, without limitation, include a high-speed I / O bus for connecting peripheral devices to the memory 2820, a chipset, and the processor 2802. Examples can include, without limitation, an audio control 2829, a firmware hub (“flash BIOS”) 2828, a wireless transceiver 2826, a data storage device 2824, a legacy I / O control 2823 which includes a user input interface 2825 and a keyboard interface, a serial expansion port 2877, such as a USB, and a network control 2834.The 2824 data storage device can include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or another mass storage device.

[0234] Illustrated in at least one embodiment Fig. 28 a system comprising interconnected hardware devices or “chips”. In at least one embodiment, Fig. 28 illustrates an exemplary SoC. In at least one embodiment, the Fig. The devices illustrated in Figure 28 are interconnected using proprietary interconnects, standardized interconnects (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of the System 2800 are interconnected using Compute Express Link (“CXL”) connections.

[0235] Fig. Figure 29 illustrates a system 2900 according to at least one embodiment. In at least one embodiment, the system 2900 is an electronic device utilizing a processor 2910. In at least one embodiment, the system 2900 can, by way of example and without limitation, be a notebook, a tower server, a rack server, a blade server, a laptop, a desktop computer, a tablet, a mobile device, a telephone, an embedded computer, or any other suitable electronic device.

[0236] In at least one embodiment, the system 2900 can, without limitation, include a processor 2910 that is communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, the processor 2910 is coupled using a bus or interface, such as an I2C bus, a system management bus (SMBus), a low pin count bus (LPC), a serial peripheral interface (SPI), a high-definition audio (HDA) bus, a serial advance technology attachment (SATA) bus, USB (versions 1, 2, or 3), or a universal asynchronous receiver / transmitter (UART) bus. In at least one embodiment, the following is illustrated: Fig. 29 a system comprising interconnected hardware devices or “chips”. In at least one embodiment, Fig. 29 illustrates an exemplary SoC. In at least one embodiment, the Fig. The devices illustrated in Figure 29 are interconnected using proprietary interconnects, standardized interconnects (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components are made of Fig. 29 interconnected using CXL interconnections.

[0237] In at least one embodiment, Fig. 29 The following include: a display 2924, a touchscreen 2925, a touchpad 2930, a near field communications unit (NFC) 2945, a sensor hub 2940, a thermal sensor 2946, an Express chipset (EC) 2935, a trusted platform module (TPM) 2938, BIOS / firmware / flash memory (BIOS, FW Flash) 2922, a DSP 2960, a solid state disk (SSD) or hard disk drive (HDD) 2920, a wireless local area network (WLAN) unit 2950, ​​a Bluetooth unit 2952, a wireless wide area network (WWAN) unit 2956, a global positioning system (“GPS”) 2955, a camera (“USB 3.0 camera”) 2954, such as a USB 3.A 0-camera, or a low-power double data rate storage unit (LPDDR3), implemented in an example in the LPDDR3 standard. These components can each be implemented in any suitable way.

[0238] In at least one embodiment, other components can be communicatively coupled to the processor 2910 via the components discussed above. In at least one embodiment, an accelerometer 2941, an ambient light sensor (ALS) 2942, a compass 2943, and a gyroscope 2944 can be communicatively coupled to the sensor hub 2940. In at least one embodiment, a thermal sensor 2939, a fan 2937, a keyboard 2946, and a touchpad 2930 can be communicatively coupled to the EC 2935. In at least one embodiment, a loudspeaker 2963, headphones 2964, and a microphone (mic) 2965 can be communicatively coupled to an audio unit (audio codec and Class-D amplifier) ​​2964, which in turn can be communicatively coupled to the DSP 2960. In at least one embodiment, the audio unit 2964 can, for example and without limitation, include an audio encoder / decoder (“codec”) and a class-D amplifier.In at least one embodiment, a SIM card (“SIM”) 2957 can be communicatively coupled to the WWAN unit 2956. In at least one embodiment, components such as the WLAN unit 2950 and the Bluetooth unit 2952, as well as the WWAN unit 2956, can be implemented in a Next Generation Form Factor (“NGFF”).

[0239] Fig. Figure 30 illustrates an exemplary integrated circuit 3000 according to at least one embodiment. In at least one embodiment, the exemplary integrated circuit 3000 is a SoC that can be fabricated using one or more IP cores. In at least one embodiment, the integrated circuit 3000 includes one or more application processor(s) 3005 (e.g., CPUs), at least one graphics processor 3010, and it may additionally include an image processor 3015 and / or a video processor 3020, any one of which may be a modular IP core. In at least one embodiment, the integrated circuit 3000 includes peripheral or bus logic, which includes a USB controller 3025, a UART controller 3030, an SPI / SDIO controller 3035, and an I2S / I2C controller 3040.In at least one embodiment, the integrated circuit 3000 may include a display device 3045 coupled to one or more display interfaces 3055 provided by a high-definition multimedia interface (“HDMI”) controller 3050 and a mobile industry processor interface (“MIPI”). In at least one embodiment, storage may be provided by a flash memory subsystem 3060 comprising flash memory and a flash memory controller. In at least one embodiment, a memory interface may be provided via a memory controller 3065 for accessing SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits additionally include an embedded security engine 3070.

[0240] Fig. Figure 31 illustrates a computing system 3100 according to at least one embodiment. In at least one embodiment, the computing system 3100 comprises a processing subsystem 3101 with one or more processors 3102 and a system memory 3104, which communicates via an interconnection path that may include a memory hub 3105. In at least one embodiment, the memory hub 3105 may be a separate component within a chipset component or may be integrated into one or more processors 3102. In at least one embodiment, the memory hub 3105 is coupled to an I / O subsystem 3111 via a communication link 3106. In at least one embodiment, the I / O subsystem 3111 includes an I / O hub 3107, which may enable the computing system 3100 to receive inputs from one or more input devices 3108.In at least one embodiment, the I / O hub 3107 enables a display controller, which may be included in one or more processor(s) 3102, to provide outputs to one or more display devices 3110A. In at least one embodiment, one or more display devices 3110A coupled to the I / O hub 3107 may include a local, internal, or embedded display device.

[0241] In at least one embodiment, the processing subsystem 3101 includes one or more parallel processors 3112, which are coupled to the memory hub 3105 via a bus or other communication link 3113. In at least one embodiment, the communication link 3113 can be one of any number of standards-based communication link technologies or protocols, such as, but not limited to, PCIe, or can be a vendor-specific communication interface or communication structure. In at least one embodiment, one or more parallel processors 3112 form a computationally focused parallel or vector processing system, which can include a large number of processing cores and / or processing clusters, such as a multi-core processor.In at least one embodiment, one or more parallel processors 3112 form a graphics processing subsystem that can output pixels to one or more display devices 3110A, which are coupled via the I / O hub 3107. In at least one embodiment, one or more parallel processors 3112 can also include a display controller and a display interface (not shown) to enable a direct connection to one or more display devices 3110B.

[0242] In at least one embodiment, a system storage unit 3114 can be connected to the I / O hub 3107 to provide a storage mechanism for the computing system 3100. In at least one embodiment, an I / O switch 3116 can be used to provide an interface mechanism to enable connections between the I / O hub 3107 and other components, such as a network adapter 3118 and / or a wireless network adapter 3119, which may be integrated into a platform, and various other devices that may be added via one or more expansion devices 3120. In at least one embodiment, the network adapter 3118 can be an Ethernet adapter or another wired network adapter.In at least one embodiment, the wireless network adapter 3119 may include one or more wireless radio devices from a Wi-Fi, Bluetooth, NFC or other network device.

[0243] In at least one embodiment, the computing system 3100 may include other components not explicitly shown, which include USB or other port connections, optical storage drives, video recording devices and / or variations thereof, and which may also be connected to the I / O hub 3107. In at least one embodiment, communication paths connecting various components in Fig. 31 interconnect, be implemented using any suitable protocols, such as PCI-based protocols (e.g. PCIe) or other bus or point-to-point communication interfaces and / or protocol(s), such as NVLink high-speed interconnect, or interconnection protocols.

[0244] In at least one embodiment, one or more parallel processors 3112 include a circuit optimized for graphics and video processing, which in one example includes a video output circuit, and constitutes a graphics processing unit (“GPU”). In at least one embodiment, one or more parallel processor(s) 3112 include circuits optimized for general-purpose processing. In at least one embodiment, the components of the computing system 3100 can be integrated into one or more other system elements on a single integrated circuit. In at least one embodiment, one or more parallel processor(s) 3112, the memory hub 3105, the processor(s) 3102, and the I / O hub 3107 can be integrated into an integrated SoC circuit.In at least one embodiment, the components of the 3100 computing system can be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least one section of the components of the 3100 computing system can be integrated into a multi-chip module (MCM), which can be interconnected with other multi-chip modules to form a modular computing system. In at least one embodiment, the I / O subsystem 3111 and the display devices 3110B are omitted from the 3100 computing system. PROCESSING SYSTEMS

[0245] The following figures represent, without limitation, exemplary processing systems that can be used to implement at least one embodiment.

[0246] Fig. Figure 32 illustrates an accelerated processing unit (“APU”) 3200 according to at least one embodiment. In at least one embodiment, the APU 3200 is developed by AMD Corporation, Santa Clara, California. In at least one embodiment, the APU 3200 can be configured to execute an application program, such as a CUDA program. In at least one embodiment, the APU 3200 includes, without limitation, a core complex 3210, a graphics complex 3240, a structure 3260, I / O interfaces 3270, memory controllers 3280, a display controller 3292, and a multimedia engine 3294. In at least one embodiment, the APU 3200 can include, without limitation, any number of core complexes 3210, any number of graphics complexes 3240, any number of display controllers 3292, and any number of multimedia engines 3294 in any combination.For explanatory purposes, multiple instances of the same object are referred to herein by reference symbols that identify one object, and numbers in parentheses identify one instance where necessary.

[0247] In at least one embodiment, the core complex 3210 is a CPU, the graphics complex 3240 is a GPU, and the APU 3200 is a processing unit that integrates the 3210 and 3240 on a single chip without restriction. In at least one embodiment, some tasks can be assigned to the core complex 3210, and other tasks can be assigned to the graphics complex 3240. In at least one embodiment, the core complex 3210 is configured to execute main control software associated with the APU 3200, such as an operating system. In at least one embodiment, the core complex 3210 is a master processor of the APU 3200, which controls and coordinates operations of other processors. In at least one embodiment, the core complex 3210 issues instructions that control an operation of the graphics complex 3240.In at least one embodiment, the core complex 3210 can be configured to execute host-executable code derived from the CUDA source code, and the graphics complex 3240 can be configured to execute device-executable code derived from the CUDA source code.

[0248] In at least one embodiment, the core complex 3210 includes, without limitation, cores 3220(1)-3220(4) and an L3 cache 3230. In at least one embodiment, the core complex 3210 can include, without limitation, any number of cores 3220 and any number and type of caches in any combination. In at least one embodiment, the cores 3220 are configured to execute instructions of a specific instruction set architecture (ISA). In at least one embodiment, each core 3220 is a CPU core.

[0249] In at least one embodiment, each core 3220 includes, without limitation, a retrieval / decoding unit 3222, an integer execution engine 3224, a floating-point execution engine 3226, and an L2 cache 3228. In at least one embodiment, the retrieval / decoding unit 3222 retrieves instructions, decodes such instructions, generates microoperations, and sends separate microinstructions to the integer execution engine 3224 and the floating-point execution engine 3226. In at least one embodiment, the retrieval / decoding unit 3222 can simultaneously send one microinstruction to the integer execution engine 3224 and another microinstruction to the floating-point execution engine 3226. In at least one embodiment, the integer execution engine 3224 performs integer and memory operations without limitation. In at least one embodiment, the floating-point engine 3226 performs floating-point and vector operations without restriction.In at least one embodiment, the retrieval / decoding unit 3222 sends micro-instructions to a single execution engine, which replaces both the integer execution engine 3224 and the floating-point execution engine 3226.

[0250] In at least one embodiment, each kernel 3220(i), where i is an integer representing a specific instance of kernel 3220, can access the L2 cache 3228(i) contained in kernel 3220(i). In at least one embodiment, each kernel 3220 contained in kernel complex 3210(j), where j is an integer representing a specific instance of kernel complex 3210, is connected to other kernels 3220 contained in kernel complex 3210(j) via the L3 cache 3230(j), which is also contained in kernel complex 3210(j). In at least one embodiment, the cores 3220 contained in the core complex 3210(j), where j is an integer representing a specific instance of the core complex 3210, can access the entire L3 cache 3230(j) contained in the core complex 3210(j). In at least one embodiment, the L3 cache 3230 can contain any number of slices without restriction.

[0251] In at least one embodiment, the graphics complex 3240 can be configured to perform computational operations in a highly parallel manner. In at least one embodiment, the graphics complex 3240 is configured to execute graphics pipeline operations, such as drawing commands, pixel operations, geometric calculations, and other operations associated with rendering an image on a display. In at least one embodiment, the graphics complex 3240 is configured to perform operations that are not related to graphics. In at least one embodiment, the graphics complex 3240 is configured to perform both operations that are related to graphics and operations that are not related to graphics.

[0252] In at least one embodiment, the graphics complex 3240 includes, without limitation, any number of processing units 3250 and an L2 cache 3242. In at least one embodiment, the processing units 3250 share the L2 cache 3242. In at least one embodiment, the L2 cache 3242 is partitioned. In at least one embodiment, the graphics complex 3240 includes, without limitation, any number of processing units 3250 and any number (including zero) and type of caches. In at least one embodiment, the graphics complex 3240 includes, without limitation, any amount of dedicated graphics hardware.

[0253] In at least one embodiment, each arithmetic unit 3250 includes, without limitation, any number of SIMD units 3252 and a shared memory 3254. In at least one embodiment, each SIMD unit 3252 implements a SIMD architecture and is configured to perform operations in parallel. In at least one embodiment, each arithmetic unit 3250 can execute any number of thread blocks, but each thread block is executed on a single arithmetic unit 3250. In at least one embodiment, a thread block includes, without limitation, any number of execution threads. In at least one embodiment, a workgroup is a thread block. In at least one embodiment, each SIMD unit 3252 executes a different warp. In at least one embodiment, a warp is a group of threads (e.g.,16 threads), wherein each thread in a warp belongs to a single thread block and is configured to process a different data set based on a single set of instructions. In at least one embodiment, predication can be used to disable one or more threads in a warp. In at least one embodiment, a path is a thread. In at least one embodiment, a work item is a thread. In at least one embodiment, a wavefront is a warp. In at least one embodiment, different wavefronts in a thread block can synchronize with each other and communicate via the shared memory 3254.

[0254] In at least one embodiment, the structure 3260 is a system interconnect that enables data and control transfers via the core complex 3210, the graphics complex 3240, the I / O interfaces 3270, the memory controllers 3280, the display controller 3292, and the multimedia engine 3294. In at least one embodiment, the APU 3200 can, without limitation, include any number and type of system interconnect in addition to or instead of the structure 3260, enabling data and control transfers via any number and type of directly or indirectly connected components that may be located inside or outside the APU 3200. In at least one embodiment, the I / O interfaces 3270 represent any number and type of I / O interfaces (e.g., PCI, PCI-Extended (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.).In at least one embodiment, various types of peripheral devices are coupled to I / O interfaces 3270. In at least one embodiment, peripheral devices coupled to the I / O interfaces 3270 can include, without limitation, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network cards, etc.

[0255] In at least one embodiment, the AMD92 display controller displays images on one or more display devices, such as a liquid crystal display (LCD). In at least one embodiment, the 3294 multimedia engine includes, without limitation, any quantity and type of multimedia-related circuitry, such as a video decoder, a video encoder, an image signal processor, etc. In at least one embodiment, the 3280 memory controllers enable data transfers between the 3200 APU and a unified system memory 3290. In at least one embodiment, the 3210 core complex and the 3240 graphics complex share the unified system memory 3290.

[0256] In at least one embodiment, the APU 3200 implements a memory subsystem that includes, without limitation, any number and type of memory controllers 3280 and memory devices (e.g., shared memory 3254) that can be assigned to a component or shared by multiple components. In at least one embodiment, the APU 3200 implements a cache subsystem that includes, without limitation, one or more cache memories (e.g., L2 caches 3328, L3 cache 3230, and L2 cache 3242) that can each be reserved for or shared by any number of components (e.g., cores 3220, core complex 3210, SIMD units 3252, arithmetic units 3250, and graphics complex 3240).

[0257] Fig. Figure 33 illustrates a CPU 3300 according to at least one embodiment. In at least one embodiment, the APU 3300 is developed by AMD Corporation, Santa Clara, California. In at least one embodiment, the CPU 3300 can be configured to execute an application program. In at least one embodiment, the CPU 3300 is configured to execute main control software, such as an operating system. In at least one embodiment, the CPU 3300 issues instructions that control the operation of an external GPU (not shown). In at least one embodiment, the CPU 3300 can be configured to execute host-executable code derived from the CUDA source code, and an external GPU can be configured to execute device-executable code derived from the CUDA source code.In at least one embodiment, the CPU 3300 includes without restriction any number of core complexes 3310, a structure 3360, I / O interfaces 3370 and memory controllers 3380.

[0258] In at least one embodiment, the core complex 3310 includes, without limitation, cores 3320(1)-3320(4) and an L3 cache 3330. In at least one embodiment, the core complex 3310 can include, without limitation, any number of cores 3320 and any number and type of caches in any combination. In at least one embodiment, the cores 3320 are configured to execute instructions of a specific ISA. In at least one embodiment, each core 3320 is a CPU core.

[0259] In at least one embodiment, each core 3320 includes, without limitation, a fetch / decode unit 3322, an integer execution engine 3324, a floating-point execution engine 3326, and an L2 cache 3328. In at least one embodiment, the fetch / decode unit 3322 fetches instructions, decodes such instructions, generates microoperations, and sends separate microinstructions to the integer execution engine 3324 and the floating-point execution engine 3326. In at least one embodiment, the fetch / decode unit 3322 can simultaneously send one microinstruction to the integer execution engine 3324 and another microinstruction to the floating-point execution engine 3326. In at least one embodiment, the integer execution engine 3324 performs integer and memory operations without limitation. In at least one embodiment, the floating-point engine 3326 performs floating-point and vector operations without restriction.In at least one embodiment, the retrieval / decoding unit 3322 sends micro-instructions to a single execution engine, which replaces both the integer execution engine 3324 and the floating-point execution engine 3326.

[0260] In at least one embodiment, each kernel 3320(i), where i is an integer representing a specific instance of kernel 3320, can access the L2 cache 3328(i) contained in kernel 3320(i). In at least one embodiment, each kernel 3320 contained in kernel complex 3310(j), where j is an integer representing a specific instance of kernel complex 3310, is connected to other kernels 3320 in kernel complex 3310(j) via the L3 cache 3330(j) contained in kernel complex 3310(j). In at least one embodiment, the cores 3320 contained in the core complex 3310(j), where j is an integer representing a specific instance of the core complex 3310, can access the entire L3 cache 3330(j) contained in the core complex 3310(j). In at least one embodiment, the L3 cache 3330 can contain any number of slices without restriction.

[0261] In at least one embodiment, the structure 3360 is a system interconnection that enables data and control transfers via the core complexes 3310(1)-3310(N) (where N is an integer greater than zero), I / O interfaces 3370, and memory controllers 3380. In at least one embodiment, the CPU 3300 can, without limitation, include any number and type of system interconnections in addition to or instead of the structure 3360, enabling data and control transfers via any number and type of directly or indirectly connected components, which may be located inside or outside the CPU 3300. In at least one embodiment, the I / O interfaces 3370 represent any number and type of I / O interfaces (e.g., PCI, PCI-X, PCIe, GBE, USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to the I / O interfaces 3370.In at least one embodiment, peripheral devices coupled to the I / O interfaces 3370 can include, without limitation, displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network cards, etc.

[0262] In at least one embodiment, the memory controllers 3380 enable data transfers between the CPU 3300 and a system memory 3390. In at least one embodiment, the core complex 3310 and the graphics complex 3340 share the system memory 3390. In at least one embodiment, the CPU 3300 implements a memory subsystem that includes, without limitation, any number and type of memory controllers 3380 and memory devices that can be assigned to a component or shared by several components. In at least one embodiment, the CPU 3300 implements a cache subsystem that includes, without limitation, one or more cache memories (e.g., L2 caches 3328 and L3 caches 3330) that can each be reserved for or shared by any number of components (e.g., cores 3320 and core complexes 3310).

[0263] Fig. Figure 34 illustrates an exemplary accelerator integration slice 3490 according to at least one embodiment. For the purposes of this document, a “slice” comprises a specified section of processing resources of the accelerator integration circuit. In at least one embodiment, an accelerator integration circuit provides cache management, memory access, context management, and interrupt management services on behalf of a plurality of graphics processing engines included in a graphics acceleration module. The graphics processing engines may each comprise a separate GPU. Alternatively, the graphics processing engines may comprise different types of graphics processing engines within a single GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blith engines.In at least one embodiment, a graphics acceleration module can be a GPU with multiple graphics processing engines. In at least one embodiment, graphics processing engines can be individual GPUs integrated on a common package, line card, or chip.

[0264] An effective address space 3482 of the application within a system memory 3414 stores process elements 3483. In one embodiment, process elements 3483 are stored in response to GPU calls 3481 from applications 3480 running on the processor 3407. A process element 3483 contains the process state for the corresponding application 3480. A work descriptor (WD) 3484 contained in the process element 3483 can be a single task requested by an application or a pointer to a queue of tasks. In at least one embodiment, the WD 3484 is a pointer to a task request queue in the effective address space 3482 of the application.

[0265] The 3446 graphics acceleration module and / or individual graphics processing engines can be shared by all or a subset of processes in a system. In at least one embodiment, an infrastructure for establishing process state and sending the WD 3484 to the 3446 graphics acceleration module to start a task in a virtualized environment can be included.

[0266] In at least one embodiment, a dedicated-process programming model is implementation-specific. In this model, a single process owns the 3446 graphics acceleration module or a single graphics processing engine. Because the 3446 graphics acceleration module is owned by a single process, a hypervisor initializes the accelerator integration circuitry for an owning partition, and an operating system initializes the accelerator integration circuitry for an owning process when the 3446 graphics acceleration module is allocated.

[0267] During operation, a WD retrieval unit 3491 in an accelerator integration slice 3490 retrieves the next WD 3484, which contains a specification of the work to be performed by one or more graphics processing engines of the graphics acceleration module 3446. Data from the WD 3484 can be stored in the registers 3445 and used by a memory management unit (“MMU”) 3439, an interrupt management circuit 3447, and / or a context management circuit 3448, as illustrated. In at least one embodiment, the MMU 3439 includes a segment / page scrolling circuit for accessing segment / page tables 3486 within the virtual address space 3485 of the OS. The interrupt management circuit 3447 can process interrupt events (“INT”) 3492 received by the graphics acceleration module 3446.When performing graphics operations, an effective address 3493, generated by a graphics processing engine, is translated into a real address by the MMU 3439.

[0268] In one embodiment, an identical set of registers 3445 is duplicated for each graphics processing engine and / or graphics acceleration module 3446 and can be initialized by a hypervisor or an operating system. Each of these duplicated registers can be included in the accelerator integration slice 3490. Exemplary registers that can be initialized by a hypervisor are shown in Table 1. Table 1 - Registers initialized by hypervisor 1 Slice-Steuerregister 2 Geplanter Prozessbereichszeiger für reale Adresse (RA) 3 Autoritätsmasken-Überschreibungsregister 4 Unterbrechungsvektor-Tabelleneintragsversatz 5 Unterbrechungsvektor-Tabelleneintragsbegrenzung 6 Statusregister 7 ID einer logischen Partition 8 Hypervisor-Beschleunigernutzungsaufzeichnungszeiger für realeAdresse (RA) 9 Speicherbeschreibungsregister

[0269] Examples of registers that can be initialized by an operating system are shown in Table 2. Table 2 - Registers initialized by the operating system 1 Process and thread identification 2 Context backup / restore pointer for effective address (EA) 3 Accelerator usage record pointer for virtual address (VA) 4 Memory segment table pointer for virtual address (VA) 5 Authority mask 6 Work descriptor

[0270] In one embodiment, each WD 3484 is specific to a particular graphics acceleration module 3446 and / or a particular graphics processing engine. It contains all the information required by a graphics processing engine to perform work, or it can be a pointer to a memory location where an application has set up a command queue of work to be performed.

[0271] The Fig.Figures 35A-35B illustrate exemplary graphics processing units (GPUs) according to at least one embodiment. In at least one embodiment, any of the exemplary GPUs can be manufactured using one or more IP cores. In addition to what is illustrated, other logic and circuitry can be included in at least one embodiment, including additional GPUs / cores, peripheral interface controllers, or general-purpose processor cores. In at least one embodiment, the exemplary GPUs are intended for use in a system-on-a-chip (SoC).

[0272] Fig. Figure 35A illustrates an exemplary graphics processor 3510 of an integrated circuit of a SoC, which may be manufactured using one or more IP cores according to at least one embodiment. Fig.Figure 35B illustrates an additional exemplary graphics processor 3540 of an integrated circuit of a SoC, which may be manufactured using one or more IP cores according to at least one embodiment. In at least one embodiment, the graphics processor 3510 is made of Fig. 35A is a low-performance graphics processor core. In at least one embodiment, the graphics processor 3540 is made of Fig. 35B is a higher-performance graphics processor core. In at least one embodiment, each of the graphics processors 3510, 3540 can be a variant of the graphics processor 1110. Fig. 11.

[0273] In at least one embodiment, the graphics processor 3510 includes a vertex processor 3505 and one or more fragment processors 3515A-3515N (e.g., 3515A, 3515B, 3515C, 3515D to 3515N-1 and 3515N). In at least one embodiment, the graphics processor 3510 can execute different shader programs via separate logic, such that the vertex processor 3505 is optimized to perform operations for vertex shader programs, while one or more fragment processors 3515A-3515N perform shading operations for fragments (e.g., pixels) for fragment or pixel shader programs. In at least one embodiment, the vertex processor 3505 performs a vertex processing stage of a 3D graphics pipeline and generates primitive and vertex data.In at least one embodiment, one or more fragment processors 3515A-3515N use primitive and vertex data generated by the vertex processor 3505 to create an image memory that is displayed on a display device. In at least one embodiment, one or more fragment processors 3515A-3515N are optimized to execute fragment shader programs, as provided in an OpenGL API, which can be used to perform operations similar to a pixel shader program, as provided in a Direct3D API.

[0274] In at least one embodiment, the graphics processor 3510 additionally includes one or more MMU(s) 3520A-3520B, cache(s) 3525A-3525B, and circuit interconnection(s) 3530A-3530B. In at least one embodiment, one or more MMU(s) 3520A-3520B provide a virtual-physical address mapping for the graphics processor 3510, including the vertex processor 3505 and / or the fragment processor(s) 3515A-3515N, which refers to vertex or in-memory image / texture data in addition to vertex or image / texture data stored in one or more caches 3525A-3525B. In at least one embodiment, one or more MMU(s) 3520A-3520B can be synchronized with other MMUs within the system, which includes one or more MMUs being synchronized with one or more application processor(s) 1105, image processors 1115 and / or video processors 1120. Fig.11 are associated, so that each processor 1105-1120 can participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnections 3530A-3530B enable the graphics processor 3510 to form interfaces with other IP cores within a SoC, either via an internal bus of an SoC or via a direct connection.

[0275] In at least one embodiment, the graphics processor 3540 includes one or more MMU(s) 3520A-3520B, caches 3525A-3525B and circuit interconnections 3530A-3530B of the graphics processor 3510. Fig.35A. In at least one embodiment, the 3540 graphics processor includes one or more shader core(s) 3555A-3555N (e.g., 3555A, 3555B, 3555C, 3555D, 3555E, 3555F to 3555N-1 and 3555N), providing a unified shader core architecture in which a single core or type can execute all types of programmable shader code, including shader program code to implement vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores can vary.In at least one embodiment, the graphics processor 3540 includes an intercore task manager 3545, which acts as a thread dispatcher to send execution threads to one or more shader cores 3555A-3555N, and a tiling unit 3558 to accelerate tiling operations for tile-based rendering, in which rendering operations for a scene are subdivided in image space to, for example, exploit local spatial coherence within a scene or to optimize the use of internal caches.

[0276] Fig. Figure 36A illustrates a graphics core 3600 according to at least one embodiment. In at least one embodiment, the graphics core 3600 can be integrated into the graphics processor 2510. Fig. 25 is included. In at least one embodiment, the 3600 graphics core can be a unified shader core 3555A-3555N as described in Fig.35B. In at least one embodiment, the graphics core 3600 includes a shared instruction cache 3602, a texture unit 3618, and a cache / shared memory 3620, which are common to the execution resources within the graphics core 3600. In at least one embodiment, the graphics core 3600 can include multiple slices 3601A-3601N or a partition for each core, and a graphics processor can include multiple instances of the graphics core 3600. The slices 3601A-3601N can include support logic, which includes a local instruction cache 3604A-3604N, a thread scheduler 3606A-3606N, a thread dispatcher 3608A-3608N, and a set of registers 3610A-3610N.In at least one embodiment, the slices 3601A-3601N can include a set of additional function units (“AFU”) 3612A-3612N, floating-point units (“FPU”) 3614A-3614N, integer arithmetic logic units (“ALU”) 3616-3616N, address computational units (“ACU”) 3613A-3613N, double-precision floating-point units (“DPFPU”) 3615A-3615N and matrix processing units (“MPU”) 3617A-3617N.

[0277] In at least one embodiment, the FPUs 3614A-3614N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while the DPFPUs 3615A-3615N perform double-precision (64-bit) floating-point operations. In at least one embodiment, the ALUs 3616A-3616N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision and can be configured for mixed-precision operations. In at least one embodiment, the MPUs 3617A-3617N can also be configured for mixed-precision matrix operations, including half-precision floating-point and 8-bit integer operations.In at least one embodiment, the MPUs 3617A-3617N can perform a series of matrix operations to accelerate CUDA programs, including enabling support for accelerated general matrix-to-matrix multiplication (GEMM). In at least one embodiment, the AFUs 3612A-3612N can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., sine, cosine, etc.).

[0278] Fig.Figure 36B illustrates a general-purpose graphics processing unit (GPGPU) 3630 according to at least one embodiment. In at least one embodiment, the GPGPU 3630 is highly parallel and suitable for use on a multi-chip module. In at least one embodiment, the GPGPU 3630 can be configured to enable highly parallel computing operations to be performed by an array of GPUs. In at least one embodiment, the GPGPU 3630 can be directly connected to other instances of the GPGPU 3630 to create a multi-GPU cluster to improve the execution time of CUDA programs. In at least one embodiment, the GPGPU 3630 includes a host interface 3632 to enable communication with a host processor. In at least one embodiment, the host interface 3632 is a PCIe interface.In at least one embodiment, the host interface 3632 can be a vendor-specific communication interface or communication structure. In at least one embodiment, the GPGPU 3630 receives instructions from a host processor and uses a global scheduler 3634 to distribute execution threads associated with these instructions to a set of compute clusters 3636A-3636H. In at least one embodiment, the compute clusters 3636A-3636H share a cache memory 3638. In at least one embodiment, the cache memory 3638 can serve as a higher-level cache for cache memories within the compute clusters 3636A-3636H.

[0279] In at least one embodiment, the GPGPU 3630 includes memory 3644A-3644B, which is coupled to the compute clusters 3636A-3636H via a set of memory controllers 3642A-3642B. In at least one embodiment, the memory 3644A-3644B can include various types of memory devices, including DRAM or graphics random-access memory, such as synchronous graphics random-access memory (“SGRAM”) or graphics dual-rate memory (“GDDR”).

[0280] In at least one embodiment, the computing clusters 3636A-3636H each include a set of graphics cores, such as the 3600 graphics core from Fig.36A, which can include several types of integer and floating-point logic units capable of performing computational operations with a range of precisions suitable for use with CUDA programs. For example, in at least one embodiment, at least one subset of floating-point units in each of the computation clusters 3636A-3636H can be configured to perform 16-bit or 32-bit floating-point operations, while another subset of floating-point units can be configured to perform 64-bit floating-point operations.

[0281] In at least one embodiment, multiple instances of the GPGPU 3630 can be configured to operate as a computing cluster. In at least one embodiment, the computing clusters 3636A-3636H can implement all technically feasible communication techniques for synchronization and data exchange. In at least one embodiment, multiple instances of the GPGPU 3630 communicate via the host interface 3632. In at least one embodiment, the GPGPU 3630 includes an I / O hub 3639 that couples the GPGPU 3630 to a GPU link 3640, enabling a direct connection to other instances of the GPGPU 3630. In at least one embodiment, the GPU link 3640 is coupled to a dedicated GPU-to-GPU bridge, enabling communication and synchronization between multiple instances of the GPGPU 3630.In at least one embodiment, the GPU link 3640 is coupled to a high-speed interconnect to transmit and receive data to and from other GPGPUs 3630 or parallel processors. In at least one embodiment, multiple instances of the GPGPU 3630 are located in separate data processing systems and communicate via a network device accessible through the host interface 3632. In at least one embodiment, the GPU link 3640 can be configured to allow a connection to a host processor in addition to, or as an alternative to, the host interface 3632. In at least one embodiment, the GPGPU 3630 can be configured to execute a CUDA program.

[0282] Fig.Figure 37A illustrates a parallel processor 3700 according to at least one embodiment. In at least one embodiment, various components of the parallel processor 3700 can be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (“ASICs”), or FPGAs.

[0283] In at least one embodiment, the parallel processor 3700 includes a parallel processing unit 3702. In at least one embodiment, the parallel processing unit 3702 includes an I / O unit 3704, which enables communication with other devices, including other instances of the parallel processing unit 3702. In at least one embodiment, the I / O unit 3704 can be directly connected to other devices. In at least one embodiment, the I / O unit 3704 is connected to other devices via a hub or switch interface, such as the memory hub 1205. In at least one embodiment, connections between the memory hub 1205 and the I / O unit 3704 form a communication link.In at least one embodiment, the I / O unit 3704 is connected to a host interface 3706 and a memory matrix 3716, wherein the host interface 3706 receives commands aimed at performing processing operations, and the memory matrix 3716 receives commands aimed at performing storage operations.

[0284] In at least one embodiment, when the host interface 3706 receives a command buffer via the I / O unit 3704, the host interface 3706 can direct work operations to a frontend 3708 to execute these commands. In at least one embodiment, the frontend 3708 is coupled to a scheduler 3710, which is configured to distribute commands or other work items to a processing array 3712. In at least one embodiment, the scheduler 3710 ensures that the processing array 3712 is correctly configured and in a valid state before tasks are distributed to the processing array 3712. In at least one embodiment, the scheduler 3710 is implemented via firmware logic running on a microcontroller.In at least one embodiment, the scheduler 3710, implemented by a microcontroller, is configurable to perform complex scheduling and workload distribution operations with coarse and fine granularity, enabling fast pre-allocation and context switching of threads running on the processing array 3712. In at least one embodiment, the host software can check workloads for scheduling on the processing array 3712 via one of several graphics processing doorbells. In at least one embodiment, workloads can then be automatically distributed across the processing array 3712 by the logic of the scheduler 3710 within a microcontroller that includes the scheduler 3710.

[0285] In at least one embodiment, the processing array 3712 can contain up to "N" clusters (e.g., cluster 3714A, cluster 3714B to cluster 3714N). In at least one embodiment, each cluster 3714A-3714N of the processing array 3712 can execute a large number of concurrent threads. In at least one embodiment, the scheduler 3710 can allocate work to the clusters 3714A-3714N of the processing array 3712 using various scheduling and / or workload allocation algorithms, which can vary depending on the workloads generated by each type of program or computation. In at least one embodiment, scheduling can be handled dynamically by the scheduler 3710 or can be partially assisted by the compiler logic during the compilation of the program logic configured for execution by the processing array 3712.In at least one embodiment, different clusters 3714A-3714N of the processing array 3712 can be assigned to process different types of programs or to perform different types of calculations.

[0286] In at least one embodiment, the processing array 3712 can be configured to perform various types of parallel processing operations. In at least one embodiment, the processing array 3712 is configured to perform general-purpose parallel computing operations. In at least one embodiment, the processing array 3712 can include logic for executing processing tasks, which includes filtering video and / or audio data, performing modeling operations, including physics operations, and performing data transformations.

[0287] In at least one embodiment, the processing array 3712 is configured to perform parallel graphics processing operations. In at least one embodiment, the processing array 3712 may include additional logic to support the execution of such graphics processing operations, including, but not limited to, texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, the processing array 3712 may be configured to execute graphics processing-related shader programs, such as vertex shaders, tessellation shaders, geometry shaders, and pixel shaders, without limitation. In at least one embodiment, the parallel processing unit 3702 may transfer data from system memory for processing via the I / O unit 3704.In at least one embodiment, data transmitted during processing can be stored in the chip's internal memory (e.g., parallel processor memory 3722) during processing and then written back to the system memory.

[0288] In at least one embodiment, when the parallel processing unit 3702 is used to perform graphics processing, the scheduler 3710 can be configured to divide a processing workload into approximately equal-sized tasks to enable better distribution of graphics processing operations across multiple clusters 3714A-3714N of the processing array 3712. In at least one embodiment, portions of the processing array 3712 can be configured to perform different types of processing. In at least one embodiment, a first section can be configured to perform vertex shading and topology generation, a second section can be configured to perform tessellation and geometry shading, and a third section can be configured to perform pixel shading or other screen-space operations to produce a rendered image for display.In at least one embodiment, intermediate data generated by one or more of the clusters 3714A-3714N can be stored in buffers to allow intermediate data to be transferred between the clusters 3714A-3714N for further processing.

[0289] In at least one embodiment, the processing array 3712 can receive processing tasks to be executed via the scheduler 3710, which receives processing task-defining instructions from the frontend 3708. In at least one embodiment, processing tasks can include indices of data to be processed, such as surface (patch) data, primitive data, vertex data, and / or pixel data, as well as status parameters and instructions that define how data is to be processed (e.g., which program is to be executed). In at least one embodiment, the scheduler 3710 can be configured to retrieve indices corresponding to tasks or can receive indices from the frontend 3708. In at least one embodiment, the frontend 3708 can be configured to ensure that the processing array 3712 is configured in a valid state before an incoming instruction buffer (e.g., stack buffer, push buffer, etc.) is executed.) specified workload is initiated.

[0290] In at least one embodiment, each instance of the parallel processing unit 3702 can be coupled to the parallel processor memory 3722. In at least one embodiment, the parallel processor memory 3722 can be accessed via the memory matrix 3716, which can receive memory requests from the processing array 3712 and from the I / O unit 3704. In at least one embodiment, the memory matrix 3716 can access the parallel processor memory 3722 via a memory interface 3718. In at least one embodiment, the memory interface 3718 can include several partition units (e.g., partition unit 3720A, partition unit 3720B to partition unit 3720N), each of which can be coupled to a section (e.g., memory unit) of the parallel processor memory 3722.In at least one embodiment, the number of partition units 3720A-3720N is configured to correspond to the number of storage units, such that a first partition unit 3720A has a corresponding first storage unit 3724A, a second partition unit 3720B has a corresponding storage unit 3724B, and an Nth partition unit 3720N has a corresponding Nth storage unit 3724N. In at least one embodiment, the number of partition units 3720A-3720N may not correspond to the number of storage devices.

[0291] In at least one embodiment, the memory units 3724A-3724N can include various types of memory devices, including DRAM or graphics random-accessible memory, such as SGRAM, which includes GDDR memory. In at least one embodiment, the memory units 3724A-3724N can also include 3D stacked memory, including, but not limited to, high-bandwidth memory (HBM). In at least one embodiment, rendering targets, such as image memory or texture maps, can be stored across the memory units 3724A-3724N, enabling the partition units 3720A-3720N to write sections of each rendering target in parallel to efficiently utilize the available bandwidth of the parallel processor memory 3722.In at least one embodiment, a local instance of the parallel processor memory 3722 can be excluded in favor of a unified memory design that uses system memory in conjunction with local cache memory.

[0292] In at least one embodiment, any one of the clusters 3714A-3714N of the processing array 3712 can process data written to any one of the storage units 3724A-3724N within the parallel processor memory 3722. In at least one embodiment, the memory crosspoint 3716 can be configured to transmit an output from each cluster 3714A-3714N to any partition unit 3720A-3720N or to another cluster 3714A-3714N, which can perform additional processing operations on an output. In at least one embodiment, each cluster 3714A-3714N can communicate with the memory interface 3718 through the memory crosspoint 3716 to read from or write to various external storage devices.In at least one embodiment, the memory matrix 3716 has a connection to the memory interface 3718 for communication with the I / O unit 3704, as well as a connection to a local instance of the parallel processor memory 3722, which allows the processing units within the different clusters 3714A-3714N to communicate with system memory or other memory that is not local to the parallel processing unit 3702. In at least one embodiment, the memory matrix 3716 can use virtual channels to separate traffic flows between clusters 3714A-3714N and partition units 3720A-3720N.

[0293] In at least one embodiment, multiple instances of the 3702 parallel processing unit can be provided on a single expansion card, or multiple expansion cards can be interconnected. In at least one embodiment, different instances of the 3702 parallel processing unit can be configured to work together, even if the different instances have different numbers of processing cores, different amounts of local parallel processor memory, and / or other configuration differences. In at least one embodiment, some instances of the 3702 parallel processing unit can include floating-point units with higher precision than other instances.In at least one embodiment, systems incorporating one or more instances of the 3702 parallel processing unit or the 3700 parallel processor can be implemented in a variety of configurations and form factors, including, but not limited to, desktop, laptop or portable personal computers, servers, workstations, game consoles and / or embedded systems.

[0294] Fig. Figure 37B illustrates a processing cluster 3794 according to at least one embodiment. In at least one embodiment, the processing cluster 3794 is included in a parallel processing unit. In at least one embodiment, the processing cluster 3794 is one of the processing clusters 3714A-3714N from Fig.37. In at least one embodiment, the processing cluster 3794 can be configured to execute many threads in parallel, where the term "thread" refers to an instance of a concrete program that is executed on a concrete set of input data. In at least one embodiment, output techniques for single-instruction multiple data ("SIMD") instructions are used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, single-instruction multiple thread ("SIMT") techniques are used to support the parallel execution of a large number of generally synchronized threads, using a common instruction unit configured to output instructions to a set of processing engines within each processing cluster 3794.

[0295] In at least one embodiment, the operation of the processing cluster 3794 can be controlled via a pipeline manager 3732, which distributes the processing tasks to the SIMT parallel processors. In at least one embodiment, the pipeline manager 3732 receives instructions from the scheduler 3710. Fig.37 and manages the execution of these instructions via a graphics multiprocessor 3734 and / or a texture unit 3736. In at least one embodiment, the graphics multiprocessor 3734 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, different types of SIMT parallel processors with different architectures can be included within the processing cluster 3794. In at least one embodiment, one or more instances of the graphics multiprocessor 3734 can be included within the processing cluster 3794. In at least one embodiment, the graphics multiprocessor 3734 can process data, and a data crosspoint 3740 can be used to distribute processed data to one of several possible destinations, including other shader units.In at least one embodiment, the pipeline manager 3732 can facilitate the distribution of processed data by specifying destinations for processed data to be distributed via the data crosspoint 3740.

[0296] In at least one embodiment, each graphics multiprocessor 3734 within the processing cluster 3794 can include an identical set of functional execution logic (e.g., arithmetic logic units, load / store units (LSUs), etc.). In at least one embodiment, functional execution logic can be configured pipeline-like, allowing new instructions to be issued before previous instructions have completed. In at least one embodiment, the functional execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, bit shifting, and the computation of various algebraic functions. In at least one embodiment, the same hardware of a functional unit can be used to perform different operations, and any combination of functional units can be present.

[0297] In at least one embodiment, the instructions transmitted to the processing cluster 3794 constitute a thread. In at least one embodiment, a set of threads executed across a set of parallel processing engines is a thread group. In at least one embodiment, a thread group executes a program on different input data. In at least one embodiment, each thread within a thread group can be assigned to a different processing engine within the graphics multiprocessor 3734. In at least one embodiment, a thread group can contain fewer threads than the number of processing engines within the graphics multiprocessor 3734.If a thread group contains fewer threads than the number of processing engines, in at least one embodiment, one or more of the processing engines can be inactive during the cycles in which that thread group is processed. In at least one embodiment, a thread group can also contain more threads than the number of processing engines within the 3734 graphics multiprocessor. If a thread group contains more threads than the number of processing engines within the 3734 graphics multiprocessor, processing can be performed over successive clock cycles in at least one embodiment. In at least one embodiment, multiple thread groups can be executed concurrently on a single 3734 graphics multiprocessor.

[0298] In at least one embodiment, the graphics multiprocessor 3734 includes an internal cache memory for performing load and store operations. In at least one embodiment, the graphics multiprocessor 3734 can dispense with an internal cache and use a cache memory (e.g., L1 cache 3748) within the processing cluster 3794. In at least one embodiment, each graphics multiprocessor 3734 also has access to Level 2 ("L2") caches within partition units (e.g., partition units 3720A-3720N). Fig.37A), which are shared by all processing clusters 3794 and can be used to transfer data between threads. In at least one embodiment, the graphics multiprocessor 3734 can also access the global chip-external memory, which may include one or more local parallel processor memories and / or system memories. In at least one embodiment, any memory external to the parallel processing unit 3702 can be used as global memory. In at least one embodiment, the processing cluster 3794 includes multiple instances of the graphics multiprocessor 3734 that can share common instructions and data, which may be stored in the L1 cache 3748.

[0299] In at least one embodiment, each processing cluster 3794 can include an MMU 3745 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of the MMU 3745 can be located within the memory interface 3718. Fig.37. In at least one embodiment, the MMU 3745 includes a set of page table entries (“PTEs”) used to map a virtual address to a physical address of a tile, and optionally a cache row index. In at least one embodiment, the MMU 3745 may include translation lookaside buffers (“TLBs”) or caches, which may reside within the graphics multiprocessor 3734, the L1 cache 3748, or the processing cluster 3794. In at least one embodiment, a physical address is processed to distribute surface data access locality to enable efficient request nesting between partition units. In at least one embodiment, a cache row index may be used to determine whether a request for a cache row is a hit or a miss.

[0300] In at least one embodiment, the processing cluster 3794 can be configured such that each graphics multiprocessor 3734 is coupled to a texture unit 3736 for performing texture mapping operations, e.g., determining texture sampling positions, reading texture data, and filtering texture data. In at least one embodiment, the texture data is read from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 3734 and retrieved as needed from an L2 cache, the local parallel processor memory, or system memory.In at least one embodiment, each graphics multiprocessor 3734 outputs a processed task to the data crosspoint 3740 to provide a processed task to another processing cluster 3794 for further processing or to store a processed task via the memory crosspoint 3716 in an L2 cache, local parallel processor memory, or system memory. In at least one embodiment, a pre-raster operations unit (“PreROP”) 3742 is configured to receive data from the graphics multiprocessor 3734 and forward the data to ROP units that may be located in the partition units described herein (e.g., partition units 3720A-3720N from ). Fig. 37). In at least one embodiment, PreROP 3742 can perform color mixing optimizations, organize pixel color data, and perform address translations.

[0301] Fig.Figure 37C illustrates a graphics multiprocessor 3796 according to at least one embodiment. In at least one embodiment, the graphics multiprocessor 3796 is the graphics multiprocessor 3734. Fig. 37B. In at least one embodiment, the graphics multiprocessor 3796 is coupled to the pipeline manager 3732 of the processing cluster 3794. In at least one embodiment, the graphics multiprocessor 3796 has an execution pipeline that includes, but is not limited to, an instruction cache 3752, an instruction unit 3754, an address mapping unit 3756, a register bank 3758, one or more GPGPU cores 3762, and one or more LSUs 3766. The GPGPU cores 3762 and the LSUs 3766 are coupled to the cache memory 3772 and the shared memory 3770 via a memory and cache interconnect 3768.

[0302] In at least one embodiment, the instruction cache 3752 receives a stream of instructions to be executed from the pipeline manager 3732. In at least one embodiment, the instructions are temporarily stored in the instruction cache 3752 and sent by the instruction unit 3754 for execution. In at least one embodiment, the instruction unit 3754 can send instructions as thread groups (e.g., warps), with each thread of the thread group being assigned to a different execution unit within the GPGPU cores 3762. In at least one embodiment, an instruction can access any address space—local, shared, or global—by specifying an address within a unified address space.In at least one embodiment, the address mapping unit 3756 can be used to translate addresses in a unified address space into a unique memory address that can be accessed by the LSUs 3766.

[0303] In at least one embodiment, the register bank 3758 provides a set of registers for functional units of the graphics multiprocessor 3796. In at least one embodiment, the register bank 3758 provides temporary data storage for operands associated with data paths of functional units (e.g., GPGPU cores 3762, LSUs 3766) of the graphics multiprocessor 3796. In at least one embodiment, the register bank 3758 is partitioned among the individual functional units such that each functional unit is assigned a dedicated section of the register bank 3758. In at least one embodiment, the register bank 3758 is partitioned into different thread groups executed by the graphics multiprocessor 3796.

[0304] In at least one embodiment, the GPGPU cores 3762 can each include FPUs and / or integer ALUs used to execute instructions from the graphics multiprocessor 3796. The GPGPU cores 3762 can have a similar architecture or differ architecturally. In at least one embodiment, a first section of the GPGPU cores 3762 includes a single-precision FPU and an integer ALU, while a second section of the GPGPU cores 3762 includes a double-precision FPU. In at least one embodiment, FPUs can implement the IEEE 754-2008 standard for floating-point arithmetic or enable variable-precision floating-point arithmetic.In at least one embodiment, the graphics multiprocessor 3796 may additionally include one or more fixed-function or special-function units to perform specific functions, such as rectangle copying or pixel mixing operations. In at least one embodiment, one or more of the GPGPU cores 3762 may also include fixed-function or special-function logic.

[0305] In at least one embodiment, the GPGPU cores 3762 include SIMD logic capable of executing a single instruction on multiple data records. In at least one embodiment, the GPGPU cores 3762 can physically execute SIMD4, SIMD8, and SIMD16 instructions and logically execute SIMD1, SIMD2, and SIMD32 instructions. In at least one embodiment, SIMD instructions for GPGPU cores 3762 can be generated at compile time by a shader compiler or automatically when programs written and compiled for Single-Program-Multiple-Data (SPMD) or SIMT architectures are executed. In at least one embodiment, multiple threads of a program configured for a SIMT execution model can be executed via a single SIMD instruction.In at least one embodiment, eight SIMT threads, which perform the same or similar operations, can be executed in parallel via a single SIMD8 logic unit.

[0306] In at least one embodiment, the memory and cache interconnect 3768 is an interconnect network that connects each functional unit of the graphics multiprocessor 3796 to the register bank 3758 and the shared memory 3770. In at least one embodiment, the memory and cache interconnect 3768 is a crossbar interconnect that enables the LSU 3766 to implement load and store operations between the shared memory 3770 and the register bank 3758. In at least one embodiment, the register bank 3758 can operate at the same frequency as the GPGPU cores 3762, so that data transmission between the GPGPU cores 3762 and the register bank 3758 has very low latency.In at least one embodiment, the shared memory 3770 can be used to enable communication between threads running on functional units within the graphics multiprocessor 3796. In at least one embodiment, the cache memory 3772 can, for example, be used as a data cache to temporarily store texture data communicated between functional units and the texture unit 3736. In at least one embodiment, the shared memory 3770 can also be used as a program-managed cache. In at least one embodiment, threads running on the GPGPU cores 3762 can programmatically store data within the shared memory in addition to the automatically cached data stored within the cache memory 3772.

[0307] In at least one embodiment, a parallel processor or GPGPU, as described herein, is communicatively coupled to host / processor cores to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. In at least one embodiment, a GPU can be communicatively coupled to the host processor / cores via a bus or other interconnect (e.g., a high-speed interconnect such as PCIe or NVLink). In at least one embodiment, a GPU can be integrated into the same package or chip as the cores and communicatively coupled to the cores via a processor bus / interconnect that is internal to the package or chip.In at least one embodiment, the processor cores can assign work to a GPU in the form of sequences of instructions contained in a workload, regardless of how a GPU is connected. In at least one embodiment, a GPU then uses dedicated circuitry / logic to efficiently process these instructions. GENERAL DATA PROCESSING

[0308] The following figures represent, without limitation, exemplary software constructs that can be used to implement at least one embodiment.

[0309] Fig.Figure 38 illustrates a software stack of a programming platform according to at least one embodiment. In at least one embodiment, a programming platform is a platform for deploying hardware on a computer system to accelerate computational tasks. In at least one embodiment, a programming platform may be accessible to software developers through libraries, compiler directives, and / or extensions of programming languages. In at least one embodiment, a programming platform may be, but is not limited to, CUDA, Radeon Open Compute Platform (“ROCm”), OpenCL (OpenCL™ is developed by the Khronos group), SYCL, or Intel One API.

[0310] In at least one embodiment, a software stack 3800 of a programming platform provides an execution environment for an application 3801. In at least one embodiment, the application 3801 can include any computer software that can be started on the software stack 3800. In at least one embodiment, the application 3801 can include, but is not limited to, an artificial intelligence (“AI”) / machine learning (“ML”) application, a high-performance computing (“HPC”) application, a virtual desktop infrastructure (“VDI”), or a data center workload.

[0311] In at least one embodiment, the application 3801 and the software stack 3800 run on hardware 3807. The hardware 3807 may, in at least one embodiment, include one or more GPUs, CPUs, FPGAs, AI engines, and / or other types of computing devices that support a programming platform. In at least one embodiment, such as CUDA, the software stack 3800 may be vendor-specific and compatible only with devices from a specific vendor. In at least one embodiment, such as OpenCL, the software stack 3800 may be used with devices from different vendors. In at least one embodiment, the hardware 3807 includes a host connected to one or more devices that can be accessed to execute computational tasks via calls to an application programming interface (API).A device within the hardware 3807 may, without limitation, include a GPU, an FPGA, an AI engine or another computing device and its memory (but may also include a CPU), as opposed to a host within the hardware 3807, which in at least one embodiment may include a CPU and its memory (but may also include a computing device), without limitation.

[0312] In at least one embodiment, the software stack 3800 of a programming platform includes, without limitation, a number of libraries 3803, a runtime 3805, and a device kernel driver 3806. Each of the libraries 3803 may, in at least one embodiment, contain data and programming code that can be used by computer programs and employed during software development. In at least one embodiment, the libraries 3803 may, but are not limited to, contain pre-built code and subroutines, classes, values, type specifications, configuration data, documentation, help data, and / or message templates. In at least one embodiment, the libraries 3803 include functions optimized for execution on one or more types of devices.In at least one embodiment, the libraries 3803 may include, but are not limited to, functions for performing mathematical operations, deep learning, and / or other types of operations on devices. In at least one embodiment, libraries 3903 are associated with corresponding APIs 3902, which may include one or more APIs that disclose functions implemented in libraries 3903.

[0313] In at least one embodiment, the application 3801 is written as source code which is compiled into executable code, as described below in conjunction with Fig.43 is discussed in more detail. In at least one embodiment, the executable code of application 3801 can run at least partially on an execution environment provided by the software stack 3800. In at least one embodiment, during the execution of application 3801, code may be required to run on a device as opposed to a host. In such a case, the runtime 3805 can be called in at least one embodiment to load and start the necessary code on a device. In at least one embodiment, the runtime 3805 can include any technically feasible runtime system capable of supporting the execution of application S01.

[0314] In at least one embodiment, the runtime 3805 is implemented as one or more runtime libraries associated with corresponding APIs shown as API(s) 3804. In at least one embodiment, one or more such runtime libraries may, without limitation, include functions for memory management, execution control, device management, error handling, and / or synchronization. In at least one embodiment, the memory management functions may include, but are not limited to, functions for allocating, releasing, and copying device memory, as well as for transferring data between host memory and device memory.In at least one embodiment, the execution control functions may include functions for starting a function (sometimes referred to as a "kernel" when a function is a global function callable from a host) on a device and set attribute values ​​in a buffer maintained by a runtime library for a particular function to be executed on a device, without being restricted to that device.

[0315] Runtime libraries and corresponding API(s) 3804 can be implemented in at least one embodiment in any technically feasible way. In at least one embodiment, one (or any number of) APIs can provide a set of low-level functions for fine-grained control of a device, while another (or any number of) APIs can expose a set of such higher-level functions. In at least one embodiment, a high-level runtime API can be built upon a low-level API. In at least one embodiment, one or more runtime APIs can be language-specific APIs built upon a language-independent runtime API.

[0316] In at least one embodiment, the 3806 device kernel driver is configured to facilitate communication with an underlying device. In at least one embodiment, the 3806 device kernel driver can provide low-level functionality that APIs, such as the 3804 API(s) and / or other software, can access. In at least one embodiment, the 3806 device kernel driver can be configured to compile intermediate representation (IR) code into binary code at runtime. For CUDA, in at least one embodiment, the 3806 device kernel driver can compile non-hardware-specific parallel thread execution (PTX) IR code into binary code for a specific target device at runtime (with caching of the compiled binary code), sometimes referred to as code finalization.In at least one embodiment, this can allow finalized code to run on a target device that may not have existed when the source code was originally compiled into PTX code. Alternatively, in at least one embodiment, the device source code can be compiled offline into binary code without requiring the device kernel driver 3806 to compile the IR code at runtime.

[0317] Fig. Figure 39 illustrates a CUDA implementation of the 3800 software stack. Fig.39 according to at least one embodiment. In at least one embodiment, a CUDA software stack 3900, on which an application 3901 can be started, includes CUDA libraries 3903, a CUDA runtime 3905, a CUDA driver 3907, and a device kernel driver 3908. In at least one embodiment, the CUDA software stack 3900 is executed on hardware 3909, which may include a GPU that supports CUDA and is developed by NVIDIA Corporation, Santa Clara, California.

[0318] In at least one embodiment, the application 3901, the CUDA runtime 3905, and the device kernel driver 3908 can perform similar functionalities to the application 3801, the runtime 3805, and the device kernel driver 3806, respectively, as described above in conjunction with Fig.38. In at least one embodiment, the CUDA driver 3907 includes a library (libcuda.so) that implements a CUDA driver API 3906. Similar to a CUDA runtime API 3904 implemented by a CUDA runtime library (cudart), the CUDA driver API 3906, in at least one embodiment, can, without limitation, expose functions for memory management, execution control, device management, error handling, synchronization, and / or graphics interoperability. In at least one embodiment, the CUDA driver API 3906 differs from the CUDA runtime API 3904 in that the CUDA runtime API 3904 simplifies device code management by providing implicit initialization, context management (analogous to a process), and module management (analogous to dynamically loaded libraries).In contrast to the high-level CUDA runtime API 3904, the CUDA driver API 3906 is a low-level API that, in at least one embodiment, provides more granular control of a device, particularly with respect to contexts and module loading. In at least one embodiment, the CUDA driver API 3906 can expose context management functions not disclosed by the CUDA runtime API 3904. In at least one embodiment, the CUDA driver API 3906 is also language-independent and supports, for example, OpenCL in addition to the CUDA runtime API 3904. Furthermore, in at least one embodiment, development libraries, including the CUDA runtime 3905, can be considered separate from driver components, including the user-mode CUDA driver 3907 and the kernel-mode device driver 3908 (sometimes referred to as the "display" driver).

[0319] In at least one embodiment, the CUDA libraries 3903 may include mathematical libraries, deep learning libraries, parallel algorithm libraries, and / or signal / image / video processing libraries that can utilize parallel computing applications such as the application 3901, without being limited thereto. In at least one embodiment, the CUDA libraries 3903 may include mathematical libraries such as, among others, a cuBLAS library, which is an implementation of Basic Linear Algebra Subprograms (“BLAS”) for performing linear algebra operations, a cuFFT library for calculating fast Fourier transforms (“FFTs”), and a cuRAND library for generating random numbers.In at least one embodiment, the CUDA libraries can include 3903 deep learning libraries, such as a cuDNN library of primitives for deep neural networks and a TensorRT platform for high-performance deep learning inference.

[0320] Fig. Figure 40 illustrates a ROCm implementation of the 3800 software stack. Fig. 38 according to at least one embodiment. In at least one embodiment, an ROCm software stack 4000, on which an application 4001 can be started, includes a language runtime 4003, a system runtime 4005, a thunk 4007, an ROCm kernel driver 4008, and a device kernel driver 4009. In at least one embodiment, the ROCm software stack 4000 is executed on hardware 4010, which may include a GPU that supports ROCm and is developed by AMD Corporation, Santa Clara, California.

[0321] In at least one embodiment, application 4001 can perform similar functionalities to application 3801, as described above in conjunction with Fig. 38 was discussed. Furthermore, the speech runtime 4003 and the system runtime 4005 can, in at least one embodiment, provide similar functionalities to those mentioned above in conjunction with Fig.38. Runtime 3805 discussed. In at least one embodiment, the language runtime 4003 and the system runtime 4005 differ in that the system runtime 4005 is a language-independent runtime that implements a ROCr system runtime API 4004 and uses a Heterogeneous System Architecture (“HAS”) runtime API. The HAS runtime API, in at least one embodiment, is a lean user-mode API that exposes interfaces for accessing and interacting with an AMD GPU, including, among other things, functions for memory management, execution control via architecture-defined kernel shipping, error handling, system and agent information, and runtime initialization and shutdown.In contrast to the system runtime 4005, the language runtime 4003, in at least one embodiment, is an implementation of a language-specific runtime API 4002 layered on top of the ROCr system runtime API 4004. In at least one embodiment, the language runtime API may include, but is not limited to, a language runtime API for Heterogeneous Compute Interface for Portability (“HIP”), a language runtime API for Heterogeneous Compute Compiler (“HCC”), or an OpenCL API. The HIP language, in particular, is an extension of the C++ programming language with functionally similar versions of CUDA mechanisms, and in at least one embodiment, an HIP language runtime API includes functions similar to those of the CUDA runtime API 3904 described above in conjunction with... Fig. 39 discussed, such as functions for memory management, execution control, device management, error handling and synchronization.

[0322] In at least one embodiment, the Thunk (ROCt) 4007 is an interface that can be used to interact with the underlying ROCm driver 4008. In at least one embodiment, the ROCm driver 4008 is a ROCk driver that is a combination of an AMDGPU driver and a HAS kernel driver (amdkfd). In at least one embodiment, the AMDGPU driver is a device kernel driver for GPUs developed by AMD that performs similar functions to the device kernel driver 3806 described above in conjunction with Fig. 38 was discussed. In at least one embodiment, the HAS kernel driver is a driver that allows different types of processors to share system resources more effectively via hardware features.

[0323] In at least one embodiment, various libraries (not shown) can be included in the ROCm software stack 4000 above the language runtime 4003 and provide a similarity in functionality to the CUDA libraries 3903 described above in conjunction with Fig. 38 were discussed. In at least one embodiment, various libraries may include mathematical, deep learning and / or other libraries, such as, among others, a hipBLAS library that implements functions similar to those of CUDA cuBLAS, and a rocFFT library for calculating FFTs, which is similar to CUDA cuFFT.

[0324] Fig. Figure 41 illustrates an OpenCL implementation of the 3800 software stack. Fig.38 according to at least one embodiment. In at least one embodiment, an OpenCL software stack 4100, on which an application 4101 can be started, includes an OpenCL framework 4105, an OpenCL runtime 4106, and a driver 4107. In at least one embodiment, the OpenCL software stack 4100 is executed on hardware 3909 that is not vendor-specific. Since OpenCL is supported by devices developed by different vendors, at least one embodiment may require specific OpenCL drivers to interact with hardware from such vendors.

[0325] In at least one embodiment, the application 4101, the OpenCL runtime 4106, the device kernel driver 4107, and the hardware 4108 can perform similar functionalities to the application 3801, the runtime 3805, the device kernel driver 3806, and the hardware 3807, respectively, as described above in conjunction with Fig.38. In at least one embodiment, the application 4101 further includes an OpenCL kernel 4102 with code to be executed on a device.

[0326] In at least one embodiment, OpenCL defines a "platform" that allows a host to control devices connected to it. In at least one embodiment, an OpenCL framework provides a platform layer API and a runtime API, shown as Platform API 4103 and Runtime API 4105, respectively. In at least one embodiment, Runtime API 4105 uses contexts to manage kernel execution on devices. In at least one embodiment, each identified device can be associated with a specific context that Runtime API 4105 can use for that device to, among other things, manage instruction queues, program objects, and kernel objects, and to share memory objects.In at least one embodiment, the Platform API 4103 discloses functions that, among other things, allow the use of device contexts for selecting and initializing devices, for sending work to devices via command queues, and for enabling data transfer to and from devices. Furthermore, in at least one embodiment, the OpenCL framework provides various built-in functions (not shown), including, but not limited to, mathematical functions, relational functions, and image processing functions.

[0327] In at least one embodiment, a compiler 4104 is also included in the OpenCL framework 4105. Source code can be compiled offline before application execution or online during application execution in at least one embodiment. Unlike CUDA and ROCm, OpenCL applications in at least one embodiment can be compiled online by the compiler 4104, which is included to be representative of any number of compilers that can be used to compile source code and / or IR code, such as Portable Intermediate Representation (“SPIR-V”) code, into binary code. Alternatively, in at least one embodiment, OpenCL applications can be compiled offline before such applications are executed.

[0328] Fig.Figure 42 illustrates software supported by a programming platform according to at least one embodiment. In at least one embodiment, a programming platform 4204 is configured to support various programming models 4203, middlewares and / or libraries 4202, and frameworks 4201 that an application 4200 can access. In at least one embodiment, the application 4200 can be an AI / ML application implemented, for example, using a deep learning framework such as MXNet, PyTorch, or TensorFlow, which can access libraries such as cuDNN, NVIDIA Collective Communications Library (“NCCL”), and / or NVIDIA Developer Data Loading Library (“DALI”) CUDA libraries to provide accelerated data processing on the underlying hardware.

[0329] In at least one embodiment, the programming platform 4204 can perform one of the above-mentioned functions in conjunction with Fig. 39, Fig. 40 or Fig. The programming platform 4204 can be a CUDA, ROCm, or OpenCL platform as described in Section 41. In at least one embodiment, the programming platform 4204 supports several programming models 4203, which are abstractions of an underlying computing system that allows expressions of algorithms and data structures. In at least one embodiment, the programming models 4203 can expose features of the underlying hardware to improve performance. In at least one embodiment, the programming models 4203 can include, but are not limited to, CUDA, HIP, OpenCL, C++ Accelerated Massive Parallelism (“C++AMP”), Open Multi-Processing (“OpenMP”), Open Accelerators (“OpenACC”), and / or Vulcan Compute.

[0330] In at least one embodiment, libraries and / or middleware 4202 provide implementations of abstractions of programming models 4204. In at least one embodiment, such libraries may include data and programming code that can be used by computer programs and employed during software development. In at least one embodiment, such middleware includes software that provides services for applications beyond those available from the programming platform 4204. In at least one embodiment, the libraries and / or middleware 4202 may include, but are not limited to, cuBLAS, cuFFT, cuRAND, and other CUDA libraries, or rocBLAS, rocFFT, rocRAND, and other ROCm libraries.Furthermore, the libraries and / or middlewares 4202 may, in at least one embodiment, include NCCL and ROCm Communication Collectives Library ("RCCL") libraries providing communication routines for GPU, an MIOpen library for deep learning acceleration, and / or a proprietary library for linear algebra, matrix and vector operations, geometric transformations, numerical solvers, and related algorithms.

[0331] In at least one embodiment, the application frameworks 4201 depend on the libraries and / or middlewares 4202. In at least one embodiment, each of the application frameworks 4201 is a software framework used to implement a standard structure of application software. An AI / ML application can be implemented in at least one embodiment using a framework such as a Caffe, Caffe2, TensorFlow, Keras, PyTorch, or MxNet deep learning framework.

[0332] Fig. Figure 43 illustrates how to compile code for execution on one of the programming platforms of the Fig. 38-41 according to at least one embodiment. In at least one embodiment, a compiler 4301 receives source code 4300, which includes both host code and device code. In at least one embodiment, the compiler 4301 is configured to convert the source code 4300 into host-executable code 4302 for execution on a host and device-executable code 4303 for execution on a device. The source code 4300 can be compiled either offline before the execution of an application or online during the execution of an application.

[0333] In at least one embodiment, the 4300 source code can contain code in any programming language supported by the 4301 compiler, such as C++, C, Fortran, etc. In at least one embodiment, the 4300 source code can be contained in a single source file with a mixture of host code and device code, specifying locations of the device code. In at least one embodiment, a single source file can be a .cu file containing CUDA code or a .hip.cpp file containing HIP code. Alternatively, in at least one embodiment, the 4300 source code can consist of multiple source code files instead of a single source file, in which the host code and device code are separated.

[0334] In at least one embodiment, the compiler 4301 is configured to compile the source code 4300 into host-executable code 4302 for execution on a host and into device-executable code 4303 for execution on a device. In at least one embodiment, the compiler 4301 performs operations that include parsing the source code 4300 into an abstract system tree (AST), performing optimizations, and generating executable code. In at least one embodiment in which the source code 4300 comprises a single source file, the compiler 4301 can separate the device code from the host code in such a single source file and compile the device code and the host code into device-executable code 4303 and 4302, respectively.Compile a host-executable code 4302 and link the device-executable code 4303 and the host-executable code 4302 together in a single file, as described below with reference to . Fig. 27 is discussed in more detail.

[0335] In at least one embodiment, the host-executable code 4302 and the device-executable code 4303 can be in any suitable format, such as binary code and / or IR code. In the case of CUDA, in at least one embodiment, the host-executable code 4302 can contain native object code, and the device-executable code 4303 can contain code in a PTX intermediate representation. In the case of ROCm, both the host-executable code 4302 and the device-executable code 4303 can contain target binary code in at least one embodiment.

[0336] Other variations are within the spirit of the present disclosure. Although various modifications and alternative constructions can be made to the disclosed techniques, certain illustrated embodiments are shown in the drawings and have been described in detail above. However, it is understood that the intention is not to limit the disclosure to the specific disclosed form(s), but rather, on the contrary, to cover all modifications, alternative constructions, and equivalents that fall within the spirit and scope of the disclosure as defined in the attached claims.

[0337] The use of the terms "a," "an," "the," and similar referents in the context of describing disclosed embodiments (particularly in the context of the following claims) is to be interpreted as covering both the singular and the plural unless otherwise specified herein or the context clearly contradicts this, and not as a definition of a term. The terms "comprising," "having," "including," and "containing" are to be interpreted as open terms (meaning "including but not limited to") unless otherwise specified. The term "connected" is to be interpreted as partially or completely contained within, attached to, or joined to one another when used unmodified and referring to physical connections, even if an element is inserted between them.The mention of value ranges herein is intended merely as a short procedure for individual reference to each separate value falling within the range, unless otherwise stated herein, and each separate value is included in the description as if it were reproduced individually herein. In at least one embodiment, the use of the term "set" (e.g., "a set of items") or "subset" is to be understood as a non-empty collection comprising one or more elements, unless otherwise noted or the context contradicts. Furthermore, unless otherwise stated or the context contradicts, the term "subset" of a corresponding set does not necessarily denote a proper subset of the corresponding set, but the subset and the corresponding set may be the same.

[0338] At least one embodiment of the disclosure can be described with regard to the following sentences: 1. Processor comprising one or more circuits to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain period of time has elapsed. 2. Processor according to sentence 1, wherein the first software program is restarted to be executed by one or more processors of the group of processors. 3. Processor according to sentence 1 or 2, wherein the first software program is reset to a checkpoint in order to be executed by one or more processors of the group of processors. 4. Processor according to one of sentences 1-3, wherein the one or more processors are assigned only to execute the first software program before the expiry of the time period. 5. Processor according to one of sentences 1-4, wherein the expiry of the time period includes a time period that elapses after a failure of the group of processors. 6. Processor according to one of sentences 1-5, wherein the expiry of the time period includes a predetermined time after a failure of the group of processors. 7. Processor according to one of the sentences 1-6, wherein the second software program differs from the first software program. 8. Method comprising causing one or more processors of a group of processors that have not fully executed a first software program to be prevented from being assigned to execute a second software program until at least a period of time has elapsed. 9. Method according to sentence 8, wherein the first software program is restarted to be executed by one or more processors of the group of processors. 10. Method according to sentence 8 or 9, wherein the first software program is reset to a checkpoint in order to be executed by one or more processors of the group of processors. 11. Method according to one of sentences 8-10, wherein the one or more processors are assigned only to execute the first software program before the expiry of the time period. 12. Method according to one of sentences 8-11, wherein the expiry of the time period includes a time period that elapses after a failure of the group of processors. 13. Method according to one of sentences 8-12, wherein the expiry of the time period includes a predetermined time after a failure of the group of processors. 14. Method according to one of sentences 8-13, wherein the second software program differs from the first software program. 15. System comprising one or more circuits to prevent one or more proc...

Claims

[1] Processor comprising one or more circuits to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a certain period of time has elapsed. [2] Processor according to claim 1, wherein the first software program is restarted to be executed by one or more processors of the group of processors. [3] Processor according to claim 1 or 2, wherein the first software program is reset to a checkpoint in order to be executed by one or more processors of the group of processors. [4] Processor according to one of the preceding claims, wherein the one or more processors are assigned only to execute the first software program before the expiry of the time period. [5] Processor according to any of the preceding claims, wherein the expiry of the time period comprises a time period that elapses after a failure of the group of processors. [6] Processor according to any of the preceding claims, wherein the expiry of the time period comprises a predetermined time after a failure of the group of processors. [7] Processor according to any of the preceding claims, wherein the second software program differs from the first software program. [8] Method comprising causing one or more processors of a group of processors that have not fully executed a first software program to be prevented from being assigned to execute a second software program until at least a period of time has elapsed. [9] Method according to claim 8, wherein the first software program is restarted to be executed by one or more processors of the group of processors. [10] Method according to claim 8 or 9, wherein the first software program is reset to a checkpoint in order to be executed by one or more processors of the group of processors. [11] Method according to any one of claims 8 to 10, wherein the one or more processors are assigned to execute the first software program only before the expiry of the time period. [12] Method according to any one of claims 8 to 11, wherein the elapse of the time period comprises a time period that elapses after a failure of the group of processors. [13] Method according to any one of claims 8 to 12, wherein the elapse of the time period comprises a predetermined time after a failure of the group of processors. [14] Method according to any one of claims 8 to 13, wherein the second software program differs from the first software program. [15] System comprising one or more circuits to prevent one or more processors of a group of processors that have not fully executed a first software program from being assigned to execute a second software program until at least a period of time has elapsed. [16] System according to claim 15, wherein the first software program is restarted to be executed by one or more processors of the group of processors. [17] System according to claim 15 or 16, wherein the first software program is reset to a checkpoint in order to be executed by one or more processors of the group of processors. [18] System according to one of claims 15 to 17, wherein the one or more processors are assigned only to execute the first software program before the expiry of the time period. [19] System according to any one of claims 15 to 18, wherein the elapse of the time span comprises a time span that elapses after a failure of the group of processors. [20] System according to any one of claims 15 to 19, wherein the expiry of the time period comprises a predetermined time after a failure of the group of processors.

Images