Online fault detection in safety-critical systems using isolation and process migration

Abstract

This document discloses devices, systems, and techniques for testing the processing units of a computer system. A request is received to initiate a test process for each set of processing units. A first processing unit contains a first virtual processor that performs first operations. A second processing unit contains a second virtual processor that performs second operations. The execution of the first operations is transferred from the first virtual processor to the second virtual processor. The execution of the test process is initiated at the first processing unit while the second virtual processor performs the first and second operations. Upon detection that the execution of the test process is complete, the execution of the first and second operations is transferred back to the first virtual processor.The execution of the test process is initiated on the second processing unit, while the first virtual processor performs the first and second operations.

Description

TECHNICAL AREA

[0001] At least one embodiment relates to testing the processing units of a computer system. For example, operations to be executed can be transferred from a first virtual processor to a second virtual processor before a test process is performed on a first processing unit containing the first virtual processor. After completion of the test process, the execution of the operations can be transferred from the second virtual processor to the first virtual processor (or another virtual processor) before the test process is performed on a second processing unit containing the second virtual processor. BACKGROUND

[0002] A multi-core processor is a processor (e.g., a microprocessor) on a single integrated circuit with two or more separate processing units called cores. Each core of a multi-core processor can read and execute program instructions (e.g., concurrently). Multi-core processors may be included in systems where failure or malfunction could have catastrophic consequences. Testing multi-core processors in such systems can be critical for ensuring the system's safety and effectiveness. Coordinating multi-core processor testing in such systems while maintaining consistent processor availability for instruction execution, which is essential for system functionality, can be challenging. BRIEF DESCRIPTION OF THE INVENTION

[0003] The invention is defined by the claims. To illustrate the invention, aspects and embodiments that may or may not fall within the scope of the claims are described here.

[0004] Herein, devices, systems, and techniques for testing the processing units of a computer system are disclosed. A request is received to initiate a test process for each of a set of processing units. A first processing unit contains a first virtual processor that performs first operations. A second processing unit contains a second virtual processor that performs second operations. The execution of the first operations is transferred from the first virtual processor to the second virtual processor. The execution of the test process is initiated on the first processing unit while the second virtual processor performs the first and second operations. Upon detection that the execution of the test process is complete, the execution of the first and second operations is transferred back to the first virtual processor.The execution of the test process is initiated on the second processing unit, while the first virtual processor performs the first and second operations.

[0005] The revelation extends to all new aspects or characteristics described and / or illustrated herein.

[0006] Further features of the disclosure are characterized by the independent and dependent claims.

[0007] Any feature of one aspect of the disclosure can be applied in any suitable combination to other aspects of the disclosure. In particular, procedural aspects can be applied to apparatus or system aspects, and vice versa.

[0008] Furthermore, features implemented in hardware can be implemented in software and vice versa. Any reference to software and hardware features here should be interpreted accordingly.

[0009] Each system or device feature described here can also be provided as a process feature, and vice versa. System and / or device aspects that are functionally described (including means plus functional features) can alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and associated main memory.

[0010] It is also understood that certain combinations of the various features described and defined in each aspect of the revelation can be implemented and / or provided and / or used independently of one another.

[0011] The disclosure also provides computer programs and computer program products comprising software code designed to perform one of the methods described herein when executed on a data processing device and / or to embody one of the device and system features described herein, including one or all component steps of a method.

[0012] The disclosure also includes a computer or computer system (including networked or distributed systems) with an operating system that supports a computer program for carrying out the procedures described herein and / or for embodying the device or system features described herein.

[0013] The disclosure also provides a computer-readable medium on which one or more of the aforementioned computer programs are stored.

[0014] The revelation also provides a signal that carries one or more of the aforementioned computer programs.

[0015] The disclosure extends to methods and / or devices and / or systems as described herein with reference to the accompanying drawings.

[0016] Aspects and embodiments of the disclosure will now be described purely by way of example with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various embodiments according to the present disclosure are described with reference to the drawings, in which: Fig. 1 a block diagram of an exemplary system architecture according to at least one embodiment; Fig. 2 is a block diagram of an exemplary processing unit test machine according to at least one embodiment; Fig. Figure 3 illustrates a flowchart of an exemplary procedure for online fault detection in safety-critical systems using isolation and process migration according to at least one embodiment; Fig. 4A-4E illustrate examples of testing processing units of a computer system according to at least one embodiment; Fig. Figures 5A-5B illustrate additional examples of testing processing units of a computer system according to at least one embodiment; Fig. 6A illustrates one or more exemplary hardware structures for inference and / or training logic according to at least one embodiment; Fig. 6B illustrates one or more exemplary hardware structures for inference and / or training logic according to at least one embodiment; Fig. Figure 7 illustrates an exemplary data center system according to at least one embodiment; Fig. Figure 8 illustrates a computer system according to at least one embodiment; Fig. Figure 9 illustrates a computer system according to at least one embodiment; Fig. 10 illustrates at least parts of a graphics processor according to one or more embodiments; Fig. 11 illustrates at least parts of a graphics processor according to one or more embodiments; Fig. Figure 12 is an exemplary data flow diagram for an advanced computing pipeline according to at least one embodiment; Fig. Figure 13 is a system diagram for an example system for training, designing, instantiating, and deploying machine learning models in an advanced computational pipeline according to at least one embodiment; and Fig. 14A and Fig. Figure 14B illustrates a data flow diagram for a process for training a machine learning model and a client-server architecture for improving annotation tools with pre-trained annotation models according to at least one embodiment. DETAILED DESCRIPTION

[0018] A safety-critical system refers to a system whose failure or malfunction could have catastrophic consequences (e.g., serious injury, loss, or significant property damage, etc.). Examples of safety-critical systems include autonomous driving systems, aircraft and / or air traffic control systems, some medical systems, etc. Testing can be performed on the hardware (e.g., processing units, etc.) and / or software of safety-critical systems to ensure the safety and effectiveness of such systems. In some cases, testing can be performed while the system is in operation. For example, a fault detection process can be performed to detect faults in processing units (e.g., central processing units (CPUs), graphics processing units (GPUs), etc.) and / or other hardware components of an autonomous driving system while the autonomous driving system is being operated (e.g., by a driver and / or a passenger).

[0019] Some computer systems, including safety-critical systems, may contain multiple processing units capable of performing operations concurrently. In some cases, the performance of a computer system can be measured based on the availability of processing units to execute operations during a specific period and / or the number of operations the processing units can perform during that period.A computer system with processing units that are frequently available to execute operations for one or more applications and / or can reliably and successfully perform a high number of operations within a given timeframe can be considered a high-performance computer system. Conversely, a computer system with processing units that are frequently unavailable to execute operations for one or more applications and / or can only reliably and successfully perform a small number of operations within a given timeframe can be considered a low-performance computer system. Since a given processing unit cannot execute more than one operation at a time, a processing unit performing one or more test operations (e.g., as part of a security test) is unavailable to perform operations for an application running on the system, which impacts overall performance (e.g.,efficiency, throughput, latency, etc.) of the system.

[0020] In conventional systems, entire groups or clusters of processing units (e.g., processing unit cores or processor cores) are taken offline (i.e., made unavailable for application operations) when a test process is running on the system, and only brought back online (i.e., made available for application operations) once the entire test process is complete. To illustrate: A computer system may contain one or more processor cores that perform operations for an application running on the computer system. During a test period, processing units of the processor core are taken offline until the test process is complete. The execution of operations for the application is therefore temporarily suspended at the processor core, reducing the number of operations running on the system during a given period, which impacts the overall performance of the system.

[0021] Some systems can contain multiple processor cores, each executing operations assigned to one or more applications. In some cases, operations running on a first processor core can be temporarily transferred to a second processor core for execution while the first core's processing units are being tested. However, the second processor core's processing units may also be performing additional operations during the test period (e.g., based on a system partitioning scheme) and will therefore execute operations transferred from the first processor core as well as the additional operations partitioned to the second processor core during the test period.Since the processing capacity of the second processor core is limited, the number of operations it can perform during the test period is restricted, and performance during the execution of these operations and any additional operations during the test period may decrease. As a result, the overall performance (e.g., efficiency, throughput, etc.) of the system may decline. Such a performance degradation can affect the overall security of the system and / or the user experience.

[0022] Embodiments of the present disclosure address the aforementioned and other shortcomings by providing techniques for online fault detection in safety-critical systems using isolation and process migration. An operating system (or other component) of a computer system can receive a request to initiate a test process for a cluster of processing units (e.g., a processor core) of the computer system. For illustration, the computer system may be contained within an autonomous driving system. In some cases, the request may be received according to a test protocol to test processing units of the system at specific time intervals (e.g., every 100 milliseconds), based on a safety standard for the autonomous driving system. In some cases, resources of the computer system may be partitioned and / or abstracted to be accessed by one or more virtualized systems (e.g., virtual machines ...virtual machines, containers, etc., are used to run applications for the autonomous driving system. Processing units can be partitioned and / or abstracted into one or more virtual processors, each performing operations for a corresponding application of the autonomous driving system. For example, a first virtual processor hosted by a first processing unit can perform a first set of operations for one or more applications, and a second virtual processor hosted by a second processing unit performs a second set of operations for the same one or more applications.

[0023] In response to receiving the request to initiate the test process, the operating system (or another component) of the computer system can cause the execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor while the test process is performed on the first processing unit. The operating system can initiate this transfer by sending a command to a virtual system manager (such as a hypervisor) to transfer the execution of the first set of operations to the second virtual processor on the second processing unit. Once the execution of the first set of operations has been transferred to the second virtual processor, the operating system can initiate the execution of the test process on the first processing unit.The first processing unit may be offline or otherwise unavailable to perform other operations while the test process is running. The second virtual processor, running on the second processing unit, can execute the first set of operations and / or the second set of operations while the test process is running on the first processing unit.

[0024] When the operating system detects that the test process has finished executing on the first processing unit, it can transfer the execution of the first and second sets of operations from the second virtual processor to the first virtual processor (which, for example, is running on the first processing unit). The operating system can initiate the execution of the test process on the second processing unit while the first virtual processor is executing the first and / or second sets of operations. The second processing unit may be offline or otherwise unavailable to perform other operations during the execution of the test process, as described above.

[0025] In some cases, the first and second processing units may be contained within a shared group or cluster of processing units (e.g., a shared processor core) with other processing units. Once the test process is complete in the second processing unit, the execution of operations from another processing unit (e.g., a third processing unit) of the processor core can be transferred to the second virtual processor for execution while the test process is performed for the other processing unit. The execution of operations by the processor core can be transferred to virtual processors hosted by the respective processing units in the manner described above (e.g., using a round-robin approach) until the test process has been completed for each of the processor core's processing units.In some cases, the test process is a fault detection process. If it is detected that the test process has completed for each processing unit of the processor core, the operating system (or another component of the computer system) can determine, based on the result of the test process, whether a fault has occurred in any of the processing units.

[0026] Aspects and embodiments of the present disclosure provide techniques for isolating a processing unit (e.g., of a processor core) of a system and testing it according to a test process, while other processing units of the system continue to perform operations during the test process. By testing processing units of a processor core according to the techniques described above, each processing unit of the processor core can be taken offline and tested, while virtual processors hosted by other processing units of the processor core perform operations that were originally partitioned for the offline processing unit. Accordingly, only one unit of the processor core is taken offline at a time during the test period, and not every processing unit of the processor core.The time a given processing unit is taken offline during a test process is shorter than the total time required to execute the test process, thus correspondingly reducing the time that the respective processing unit is unavailable for executing application operations. For example, the total time required to execute a test process might be 100 milliseconds (e.g., according to a security specification for the computer system). Instead of taking each processing unit of the processor core offline for the entire 100 milliseconds, each processor core is taken offline for 25 milliseconds, while other processing units of the processor core continue to perform operations for applications running on the computer system.Accordingly, the time that processing units are available in the computer system and the total number of operations performed by the processing units of the computer system are increased, which improves the overall performance (e.g., efficiency, throughput, latency, etc.) of the computer system.

[0027] The systems and methods described here can be used for a wide variety of purposes, including, but not limited to, systems or applications for: online multiplayer games, machine control, machine locomotion, machine guidance, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, data center processing, conversational AI, light transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation for 3D assets, digital twinning systems, cloud computing, and / or other suitable applications.

[0028] The embodiments presented here can encompass a wide variety of systems, such as systems for participating in online games, automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine), systems implemented with a robot, aviation systems, media systems, boat systems, intelligent area surveillance systems, systems for performing deep learning operations, systems for performing simulation operations, systems implemented using an edge device, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations, and systems for performing light transport simulations.Systems for collaborative content creation for 3D assets, systems for generating or managing digital twinning representations of physical objects, systems that are implemented at least partially using cloud computing resources, and / or other types of systems.

[0029] Fig. Figure 1 is a block diagram of an exemplary system architecture 100 according to at least one embodiment. The system architecture 100 (here also referred to as the "system") may include a computer device 102, one or more client devices 106AN (here collectively and individually referred to as client devices 106), and / or one or more data storage devices 112 (here collectively and individually referred to as data storage devices 112), each of which is connected via a network 110. It is noted that the system 100 may additionally or alternatively include other components (e.g., one or more servers, etc.) that are connected to the computer device 102, the client device 106, the data storage device 112, etc., via the network 110. In implementations, the network 110 may be a public network (e.g., the Internet), a private network (e.g., a private network), or a private network (e.g., a private network).a local area network (LAN) or wide area network (WAN)), a wired network (e.g. an Ethernet network), a wireless network (e.g. an 802.11 network or a Wi-Fi network), a cellular network (e.g. a Long Term Evolution (LTE) network), routers, hubs, switches, server computers and / or a combination thereof.

[0030] In some embodiments, the data store 112 is a persistent storage device capable of storing both data and data structures for identifying, organizing, and indexing the data. The data store 112 can be hosted by one or more storage devices, such as main memory, magnetic or optical storage disks, tapes or hard disks, NAS, SAN, etc. In some implementations, the data store 112 can be a network-connected file server, while in other embodiments, the data store 112 can be a different type of persistent storage device, such as an object-oriented database, a relational database, etc., which can be hosted by the computer device 102 or by one or more different machines connected to the computer device 102 via the network 110.

[0031] The computer device 102 can be a desktop computer, a laptop computer, a smartphone, a tablet computer, a server, or any suitable computer device capable of performing the techniques described herein. In some embodiments, the computer device 102 can be a computer device of a cloud computing platform. For example, the computer device 102 can be a server machine of a cloud computing platform or a component thereof. In such embodiments, the computer device 102 can be connected via the network 110 to one or more edge devices (not shown). An edge device refers to a computer device that enables communication between computer devices at the boundary (e.g., interface) between two networks.For example, an edge device can be connected to the computer device 102, the client device 106, and / or the data storage 112 via network 110 and to one or more endpoint devices (not shown) via another network. In such an example, the edge device can facilitate communication between the computer device 102, the client device 106, and / or the data storage devices 112 and the one or more endpoint devices. In other or similar embodiments, the computer device 102 can be an edge device or a component thereof.For example, the computer device 102 can enable communication between data storage devices 112 and / or a client device 106 connected to the computer device 102 via the network 110, and the client device 106 (or one or more other user devices and / or other computer devices) connected to the computer device 102 via a different network.

[0032] The client device 106 can include any computer device 102 that enables users to access functions of an application. For example, the client device 106 can be devices or a component thereof, such as, but not limited to: televisions, smartphones, mobile phones, PDAs (Personal Digital Assistants), portable media players, netbooks, laptops, e-book readers, tablet PCs, desktop computers, set-top boxes, game consoles, autonomous vehicles, surveillance devices, and the like. In some embodiments, the computer device 102 can be an edge device that connects the client device 106 to the data storage 112 and / or other components of the system 100. In other or similar embodiments, the computer device 102 can not connect the client device 106 to the data storage 112 and / or other components and instead provide the client device 106 with data supplied by the computer device 102 (e.g.,from the data storage device 112). In additional or alternative embodiments, the computer device 102 and the client device 106 can be the same device and / or share the same or similar components.

[0033] The client device 106 may, in some embodiments, contain one or more hardware components. As in Fig. As illustrated in Figure 1, the client device 106A can contain one or more processors 120 (hereinafter referred to collectively and individually as processors 120), a memory 124, one or more input / output (I / O) devices 126, and / or other components. The processor 120 can contain one or more processing units 122. A processing unit refers to a component that performs logical and / or arithmetic operations on data. In some embodiments, processing units 122 can contain one or more central processing units (CPUs) and / or one or more graphics processing units (GPUs). A GPU can contain any processing unit specifically designed to accelerate graphics rendering (for example, for applications 132 running through a client device 106). As shown in Figure 1, the processor 120 can contain one or more processing units 122. Fig. As illustrated in Figure 1, the processor 120 can contain multiple processing units. In some embodiments, the processor 120 can be, or be equivalent to, a multi-core processor. A multi-core processor refers to a processor on a single integrated circuit with two or more separate processing units. Each processing unit of a multi-core processor can read and execute instructions as described herein. It should be noted that, although some embodiments describe the processor 120 as a multi-core processor, the embodiments of the present disclosure can be applied to any type of computer architecture.

[0034] In some embodiments, each physical processing unit 122 of the processor 120 can be assigned to a logical processing unit. A logical processing unit can be defined as a logical partition of a physical processing unit 122 to support parallel processing by the physical processing unit 122. A logical processing unit can contain a virtual construct of an operating system (OS) of the client device 106 for managing and scheduling tasks on physical processing units 122. In some cases, a logical processing unit is also referred to as a thread (e.g., execution thread).

[0035] The main memory 124 may contain one or more memory devices (not shown) capable of storing data and / or instructions that the processor 120 can access (e.g., via a bus, etc.). In some embodiments, the main memory 124 may contain volatile memory devices and / or non-volatile memory devices. For example, the main memory 124 may contain or otherwise correspond to a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, or another storage device. The I / O device 126 may contain any device that facilitates the transfer of data between one or more components of the client device 106 (e.g., processor 120, main memory 124, etc.).) and / or between one or more components of the client device 106 and another component of the system 100. For example, the I / O device 126 may contain a network interface card (NIC), an audiovisual device (e.g., a monitor, speakers, etc.), a storage device, a keyboard, a mouse, etc.

[0036] As stated above, one or more physical devices (e.g., processor 120, memory 124, I / O device 126, etc.) can be used to support one or more virtualized systems 130A-N (hereafter referred to collectively and individually as virtualized system 130) of system 100. The virtualized system 130 (also referred to as virtual system) can contain one or more parts of a virtual machine, a container, a process, and / or any other type of virtualized system. One or more applications 132 (or application instances) can run on a virtualized system 130 under a guest operating system (OS) (not shown).A virtualization manager 134 (sometimes called a hypervisor), running under an operating system of a client device 106, can abstract physical components of the client device 106 and present this abstraction to virtualized systems 130 as one or more virtual devices. For example, the virtualization manager 134 can abstract the processing unit 122A of the processor 120 and present this abstraction to the virtualized system 130A as an abstracted (or virtual) processing unit. The guest operating system can then cause instructions relating to the application 132A to be executed using the virtual processing unit.In some embodiments, the virtualization manager 134 can abstract the processing unit 122A for the virtualized system 130A by scheduling time windows on the processing unit 122A, instead of dedicating the processing unit 122A to the virtualized system 130A. In one or more embodiments, the virtualization manager 134 can abstract one or more logical processing units (e.g., threads) as described above and present the abstracted logical processing units to the virtualized system 130A as a virtual processing unit, as described above.

[0037] In other or similar embodiments, the virtualization manager 134 can abstract a portion of the main memory 124 and present this abstraction to a virtualized system 130 as abstracted main memory. The virtualization manager 134 can abstract the main memory 124 by using a page table to translate the memory accesses for the abstracted main memory to physical memory addresses of the main memory 124. During the runtime of the application 132, which uses the virtualized system 130, the virtualization manager 134 can intercept virtual memory access operations (e.g., read operations, write operations, etc.) and, using the page table, translate a virtual memory address associated with the intercepted operations into a physical memory address in main memory 124.In additional or alternative embodiments, the virtualization manager 134 can abstract one or more I / O devices 126 on the client device 106 and present this abstraction to the virtualized system 130 as one or more corresponding abstracted I / O devices. The virtualization manager 134 can abstract an I / O device 126 by allocating specific port ranges of an interface slot of the I / O device 126 to the virtualized system 130 and presenting the allocated port ranges as an abstracted I / O device. A guest operating system can use an abstracted processing unit, abstracted memory, and / or one or more abstracted I / O devices to support the execution of one or more applications 132 on a virtualized system 130.

[0038] As in Fig. As illustrated in Figure 1, the system 100 can include a processing unit test machine 151 configured to facilitate the testing of one or more processing units 122 of a processor 120 of a client device 106. In some embodiments, testing a processing unit 122 can involve performing operations of a test process. The operations can include one or more logical operations that cause a processing unit 122 under test to perform operations to access and / or process data and provide feedback (e.g., a result) of the operations. The processing unit test machine 151 can analyze the provided feedback to determine the performance of the processing unit 122 (e.g., the efficiency of the processing unit 122, the availability of the processing unit 122, etc.).In some embodiments, the test process may correspond to a test protocol designed for or otherwise provided to the client device 106. For example, the client device 106 may be contained within or otherwise access a safety-critical system (e.g., an autonomous driving system, etc.). The test process may include one or more operations that test the performance of the processing units 122 of the client device 106 (e.g., according to the safety and effectiveness standards applicable to the safety-critical system). In some embodiments, the processing unit test machine 151 may perform the test process while the processor 120 is online (e.g., made available for the operation of the application 132). Further details regarding the processing unit test machine 151 and the execution of a test process for an online processor 120 are provided herein.

[0039] In some embodiments, the processing unit test machine 151 may be located on the computer device 102. As stated above, the computer device 102 may contain computer resources (e.g., of a server computer, etc.) that the client device 106 can access (e.g., via the network 110). In other or similar embodiments, the processing unit test machine 151 may be a component of the virtualization manager 134 of the client device 106. In still other or similar embodiments, the processing unit test machine 151 may be a component of another machine or component of the client device 106 and / or any other device or system of the system architecture 100, as described herein.

[0040] In some embodiments, the computer device 102 and / or the client device 106 may include a processing unit allocation machine 151. The processing unit allocation machine 151 may be configured to allocate the execution of one or more parts of an application to different processing units of the client device 106. As described above, in some embodiments the client device 106 may include one or more iGPUs and one or more dGPUs. The processing unit allocation machine 151 may allocate one or more parts of an application (e.g., all parts of an application) to be executed using the one or more iGPUs and / or the one or more dGPUs.It should be noted that although some embodiments of the present disclosure relate to the assignment of an application to execution using iGPUs or dGPUs of the client device 106, embodiments of the present disclosure can also be applied to the assignment of a portion of the application to execution using the iGPUs or dGPUs or the client device 106 (or another type of processing unit on the client device 106). For example, according to embodiments of the present disclosure, the processing unit evaluation machine 151 can assign one or more operations of the application to execution using an iGPU of the client device 106 and one or more additional operations of the application to execution using a dGPU of the client device 106.

[0041] In some implementations, the computer device 102, the client device 106, and / or the data storage 112 may be: one or more computer devices (such as a rackmount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, etc.), data storage (such as hard disks, memory, databases), networks, software components, and / or hardware components that can be used to enable the allocation of the execution of an application 132 using different processing units of the client device 106. It should be noted that in some other implementations, the functions of the computer device 102 and / or the client device 106 may be provided by a smaller number of machines.For example, in some implementations, the computer device 102 and / or the client device 106 may be integrated into a single machine, while in other implementations, the computer device 102 and / or the client device 106 may be integrated into multiple machines. Furthermore, in some implementations, one or more of the computer devices 102 may be integrated into the client device 106. Generally, functions described in implementations as being performed by the computer device 102 and / or the client device 106 may optionally also be performed on one or more edge devices (not shown) and / or other client devices (not shown). Additionally, the functionality attributed to a particular component may be performed by different or multiple components working together.The computer device 102 and / or the client device 106 can also be accessed as a service provided to other systems or devices via suitable application programming interfaces.

[0042] Fig. Figure 2 is a block diagram of an exemplary processing unit test machine 151 according to at least one embodiment. As shown in Fig. As illustrated in Figure 2, the processing unit test machine 151 can include a scheduler component 210, an execution transfer component 212, a test performance component 214, and / or a test result component 216. As described above, the processing unit test machine 151 can be located on the client device 106. In other or similar embodiments, the processing unit allocation machine 151 can be located on the computer device 102 and / or on another component of the system 100. The processing unit test machine 151 can be connected to a memory 250. In some embodiments, the memory 250 can contain one or more areas of the data storage 112.In other or similar embodiments, the main memory 250 can contain areas of another main memory which can be accessed by one or more client devices 106, one or more computer devices 102, and / or one or more other components of the system 100. Further details of the components of the processing unit allocation machine 151 are given here with reference to . Fig. 3 described.

[0043] As described above, the client device 106 can contain one or more processors 106. A processor 120 can, in some embodiments, be a multi-core processor or another such processor containing multiple processing units 122. Fig. Figure 4A illustrates an exemplary processor 120 according to at least one embodiment of the present disclosure. As in Fig. As illustrated in Figure 4A, the processor 120 can contain multiple processing units 122 (here referred to as PUs). In an illustrative example, the processor 120 can contain eight PUs (e.g., PUs 122A-122H). It should be noted that the processor 120 can contain any number of PUs 122. Each PU 122 can be partitioned into or otherwise allocated to one or more logical processing units (LPUs) 402. As stated above, an LPU 402 can correspond to an execution strand for operations performed using the processing unit 122. It should be noted that Fig. While 4A represents each PU 122 of the processor 120 as partitioned into a single LPU 402 (e.g., PU 122A is partitioned into LPU 402A, PU 122B is partitioned into LPU 402B, etc.), each PU 122 can be partitioned into any number of LPU 402s. It should also be noted that in additional or alternative embodiments, LPU 402s can be assigned to multiple PU 122s.

[0044] As in relation to Fig. As described in Figure 1, the virtualization manager 134 can abstract computer resources (e.g., PU 122, LPU 402, etc.) of a client device 106 and present the abstracted resources as virtualized resources to a virtualized system 130 running on a client device 106. For example, the virtualization manager 134 can abstract one or more PUs 122 and / or LPUs 402 and present the abstracted PUs as virtual processing units (VPUs) 404 to a virtualized system 130. According to the description in Fig. In the example shown in Figure 4A, the virtualization manager can abstract 134 PUs 122A-F and / or LPUs 402A-F and present the abstracted PUs 122 and / or LPUs 402 as VPUs of the virtualized system 130A. Additionally or alternatively, the virtualization manager can abstract 134 PUs 122G-H and / or LPUs 402G-H and present the abstracted PUs 122G-H and / or LPUs 402G-H as VPUs of the virtualized system 130B. As described above, a virtualized system 130 can execute operations of one or more applications 132 (or application instances) over virtual resources that are abstracted for the virtual system. Each application 132 can contain one or more application threads 404 or otherwise be assigned to them, corresponding to tasks that affect application 132 (or application instance).In some embodiments, VPUs of a virtualized system 130 can execute operations assigned to a task of a specific application thread 404. In such embodiments, the sub-LPUs 402 and / or PUs 122 assigned to the VPUs execute operations assigned to the application thread 404. According to the [reference], Fig. In the example shown in 4A, VPUs assigned to LPU 402A and / or PU 122A can be allocated to perform operations assigned to thread 404A, VPUs assigned to LPU 402B and / or PU 122B can be allocated to perform operations assigned to thread 404B, and so on.

[0045] Fig. Figure 3 illustrates a flowchart of an exemplary method 300 for online fault detection in safety-critical systems using isolation and process migration according to at least one embodiment. In at least one embodiment, the method 300 can be performed by a computer device 102, a client device 106, one or more edge devices, one or more endpoint devices, or another computer device, or a combination of several computer devices. The method 300 can be performed by processing logic of one or more processing units (e.g., CPUs and / or GPUs) that can contain (or communicate with) one or more memory devices. In at least one embodiment, the method 300 can be performed by multiple processing threads (e.g.,Processing of the procedure can be carried out by CPU threads and / or GPU threads, with each thread executing one or more individual functions, routines, subroutines, or operations of the procedure. In at least one embodiment, processing threads implementing the procedure 300 can be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, processing threads implementing the procedure 300 can be executed asynchronously. Various operations of the procedure 300 can be performed in a different order than that shown in the diagram. Fig. The procedures are performed in the sequence shown in point 3. Some operations of the procedures can be performed simultaneously with other operations. In at least one embodiment, one or more of the operations described in relation to Fig. The 3 operations shown and / or described are not always performed.

[0046] In block 310, the processing logic receives a request to initiate a test process for a set of processing units. In some embodiments, the processing unit test machine 151 can receive the request to initiate the test process. In some embodiments, the request can be received by a component of the computer device 102 and / or the client device 106 according to a test protocol for PUs 122 of the client device 106. In other or similar embodiments, the request can be received by another device of the system 100 (e.g., assigned to a system administrator). In some embodiments, the test process can be a fault detection process. Based on the performance of the test process on each of the set of PUs, the processing unit test machine 151 can determine whether a fault has occurred on a particular PU.A fault refers to an error or failure that occurs during the execution of one or more operations by the processing unit (PU). Examples of a fault include, but are not limited to, a failure by the PU to initiate and / or complete the execution of one or more operations, an incorrect or unexpected result from the execution of one or more operations, and so forth. In some embodiments, the test process may include one or more test instructions 252 that correspond to test operations to be executed by PUs 122 during the test process. The test operations may include operations that have been specifically selected or designed to detect a fault in a PU. In some embodiments, a system administrator or operator of the processing unit test machine 151 may provide test instructions 252 (e.g., via a user device assigned to the system administrator or operator).In other or similar embodiments, the processing unit test machine 151 (or another component of the system 100) can determine and / or generate test instructions based on the historical activity of the processor 120 and / or the client device 106 in the system 100. As in . Fig. As illustrated in Figure 2, test instructions 252 can be stored in the main memory 250 in some embodiments.

[0047] The scheduler component 210 of the processing unit test machine 151 can schedule the execution of one or more operations of test instructions 252 by one or more processing units (PUs) 122 of the processor 120. In some embodiments, test instructions 252 can specify one or more PUs 122 that are subject to the test process. In other or similar embodiments, the scheduler component 210 can determine the PUs 122 that are subject to the test process based on a test protocol for the system 100 and / or the client device 106, as described above. PUs 122 that are subject to a test process can be online during the period in which the request to execute the test process is received in some embodiments. In some embodiments, the scheduler 210 can identify a PU 122 to initiate the test process. In such scenarios, the PU 122 can be taken offline (e.g.,(for the execution of operations related to threads 404 of one or more applications 132 are not made available), while other processing units 122 of the processing unit remain online, as described herein. In some embodiments, the scheduler 210 can identify the PU 122 to initiate the test process based on the test instructions 252 and / or the test protocol, as described above. In other or similar embodiments, the scheduler 210 can access an operation queue associated with each PU 122 subject to the test process and identify the PU 122 whose operation queue meets one or more criteria. For example, the scheduler 210 can identify the PU 122 that has an operation queue containing a number of operations to be executed that is below a threshold and / or lower than the number of operations in the operation queues of other PUs 122.In another example, Scheduler 210 can identify PU 122, which has an operation queue that does not contain any operations critical to the safety or functionality of System 100. In response to the identification of PU 122 for initiating the test process, Scheduler 210 can send a notification about the identified PU 122 to Execution Transfer Component 212. As in... Fig. As illustrated in Figure 4B, the Scheduler 210 can identify the PU 122H to initiate the test process.

[0048] With reference to Fig. 3. The processing logic in block 312 causes the first set of operations to be executed by a first virtual processor of a first processing unit, which are then to be transferred to a second virtual processor of a second processing unit. As described above, the scheduler 210 can identify PU 122H to initiate the test process and can provide an indication of PU 122H to the execution transfer component 212. In response to receiving the indication, the execution transfer component 212 can cause the execution of the operations of thread 404 (e.g., thread 404AB) by a VPU assigned to PU 122H to be transferred for execution by another VPU assigned to a different PU 122. In some embodiments, the execution transfer component 212 can identify the other VPU based on the VPUs abstracted for the virtualized system 130B.For example, the execution transfer component 212 can determine (e.g., based on information associated with virtualized system 130B) that virtualized system 130B contains VPUs abstracted from LPUs 402H and 402G. The execution transfer component 212 can determine that LPU 402H is associated with PU 122H (e.g., the PU 122 under test) and therefore select LPU 402G (e.g., associated with PU 122G, which is not currently being tested) to transfer the execution of operations associated with thread 404AB. In some embodiments, the execution transfer component 212 can determine that virtualized system 130B does not contain any additional VPUs abstracted from LPUs 402 that are associated with PUs 122 that are not being tested.In such embodiments, the execution transfer component 212 can identify VPUs of another virtualized system 130A supported by the processor 120 to transfer the execution of operations assigned to thread 404AB. In some embodiments, the execution transfer component 212 can identify VPUs of the virtualized system 130A for transfer by determining which VPUs are abstracted from PUs that meet one or more criteria (e.g., having a number of operations in an operation queue that is below the threshold and / or lower than the number of operations in other operation queues, etc.).In other or similar embodiments, the execution transfer component 212 can identify VPUs of the virtualized system 130A for transfer by determining which VPU is next in a sequence of PUs to be tested according to the test instructions 252 and / or the test protocol. In still other or similar embodiments, the execution transfer component 212 can identify VPUs of the virtualized system 130A using other techniques.

[0049] According to the Fig. In the example illustrated in Figure 4B, the execution transfer component 212 can identify or otherwise select VPUs assigned to LPU 402G and / or PU 122G to execute the operations belonging to thread 404AB while the test process for PU 122H is running. In some embodiments, the execution transfer component 212 can cause the execution of the operations belonging to thread 404AB (e.g., the first set of operations) to be transferred from the VPUs assigned to LPU 402H (e.g., the first virtual processor) to the VPUs assigned to LPU 402G (e.g., the second virtual processor) by sending an instruction to the virtualization manager 134. Upon receiving the instruction, the virtualization manager 134 can stop the execution of operations on VPUs from LPU 402H (e.g., by issuing a termination instruction to the VPUs).In additional or alternative embodiments, the virtualization manager 134 and / or the execution transfer component 212 can receive a status of operations being executed via the VPUs and / or a status of operations in an operation queue associated with the VPUs and, in some embodiments, store the received status in memory 250 as execution status 254. Execution status 254 can indicate the operations being executed via VPUs of LPU 402H, a current or previous result of operations being executed via the VPUs and / or in the operation queue, a sequence of operations in the operation queue for the VPUs, etc. The virtualization manager 134 can update an operation queue associated with the VPUs of LPU 402G to contain one or more operations with respect to thread 404AB based on the received execution status 254.In some embodiments, the virtualization manager 134 can update the operation queue associated with the VPUs of LPU 402G to include the operation that was being executed on the VPUs of LPU 402H when the execution of operations on the VPUs was stopped. The virtualization manager 134 can additionally or alternatively update the operation queue associated with the VPUs of LPU 402G to include operations from the queue associated with the VPUs of LPU 402G. In some embodiments, the operation queue associated with the VPUs of LPU 402G can also include operations related to thread 404AA, as described herein. The Virtualization Manager 134 can update the operation queue assigned to the VPUs of LPU 402G so that an order of operations belonging to threads 404AA and 404AB corresponds to an operation priority protocol assigned to processor 120 (e.g.The operations are ordered based on criticality, the time period in which the operations were received from application 132, etc. When transferring the execution of operations belonging to thread 404AB from VPUs of LPU 402H to VPUs of LPU 402G, the VPUs of LPU 402G can execute operations belonging to threads 404Ab and 404AA.

[0050] Returning to Fig. In block 314, the processing logic initiates the execution of the test process on the first processing unit, while the second virtual processor executes the first set of operations (e.g., with respect to thread 404AB) and a second set of operations (e.g., with respect to thread AA). The test performance component 214 can initiate the test process on PU 122H by executing the test instructions 252 on PU 122H. In some embodiments, the test result component 216 can receive one or more results of the test process. A result of the test process can contain one or more signals that were received or otherwise obtained before, during, or after the execution of the test instructions 252 for PU 122H. The signals can include signals output by PU 122H during the execution of the test (e.g., to retrieve data from memory 124, to send or receive data via I / O device 126, etc.).) and / or signals acquired by one or more sensors for PU 122H (e.g., temperature sensors, voltage sensors, etc.). In additional or alternative embodiments, a result of the test process may include a value, a data element, or other such output of one or more functions of the test process operations.

[0051] In block 316, the processing logic detects that the execution of the test process on the first test unit is complete. In some embodiments, the test performance component 214 and / or the test result component 216 can detect that the execution of the test process on the first test unit (e.g., PU 122H) is complete by determining that PU 122H has successfully executed each operation of the test instructions 252. In other or similar embodiments, the test performance component 214 and / or the test result component 216 can detect that the execution of the test process is complete by determining that PU 122H has finished executing the operations of the test instructions 252 and / or is unable to complete the execution of the operations.In some embodiments, the test result component 216 can generate test results 256 that indicate the results of the test process based on the test results described in relation to Block 314. The test results 256 can indicate whether the test process was completed successfully (e.g., each operation of the test instructions 252 was performed and an expected result of the test instructions 252 was obtained), was not completed successfully (e.g., each operation of the test instructions 252 was performed and an unexpected result of the test instructions 252 was obtained), and / or was not completed (e.g., each operation of the test instructions 252 was not performed). In some embodiments, the test result component 216 can generate test results 252 based on results for the test process that is performed for each PU 122 to be tested (e.g., PU 122H, PU 122G, etc.).In some embodiments, the test result component 216 can deliver test results 256 to a device assigned to a system administrator for presentation to the system administrator. In other or similar embodiments, the test result component 216 can deliver test results 256 to another component of the system 100 (e.g., located on the computer device 102 or elsewhere within the system 100). In some embodiments, the other component can perform one or more operations to correct one or more defects indicated by the test results 256.

[0052] With reference to Fig. 3 causes the processing logic in block 318 to execute the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor. As described above, the execution transfer component 212 of the processing unit test machine 151 can select or otherwise identify VPUs to transfer the execution of operations belonging to thread 404AA and / or 404AB while the test process for PU 122G is being performed. The execution transfer component 212 can select and / or identify the VPUs according to the embodiments described above. According to the Fig. In the example illustrated in Figure 4C, the execution transfer component 212 can select VPUs from LPU 402H to transfer the execution of operations belonging to Thread 404AA and Thread 404AB. The execution transfer component 212 can initiate the execution of operations on VPUs for LPU 402H by sending an instruction to the virtualization manager 134 to transfer the execution of the operations, as described above.

[0053] In block 320, the processing logic initiates the execution of the test process in the second processing unit, while the first virtual processor executes the first set of operations and the second set of operations. As in Fig. As illustrated in Figure 4C, the test performance component 214 can execute the test instructions 252 on PU 122G, as described above. VPUs of LPU 402H can execute the operations of threads 404AA and 404AB while the test instructions 252 are being executed, as described above. The test result component 216 can obtain results of the test process performed on PU 122G and generate test results 256 according to the embodiments described above.

[0054] The processing unit test machine 151 can perform the test process for each PU 122 of the processor 120 that is to be tested according to the received requirement. As described above, in some embodiments of VPUs of a virtualized system 130, the processing unit test machine 151 can transfer operations performed to other VPUs of the virtualized system 130. For example, as in Fig. Figure 4D illustrates the transfer of operations relating to thread 404F of virtualized system 130A to other VPUs associated with virtualized system 130A (e.g., VPUs of LPU 402E). In other or similar embodiments, the processing unit test machine 151 can transfer operations from VPUs of one virtualized system 130 to VPUs of another virtualized system 130. In another example, as in Fig. As illustrated in Figure 4D, the execution of operations affecting thread 404F of virtualized system 130A can be transferred to VPUs of LPU 402G, which are assigned to virtualized system 130B. The selection of the VPUs for transferring the execution of the operations can be made according to the embodiments described above and / or according to an isolation protocol assigned to each virtualized environment 130 of the client device 106. In some embodiments, the execution of operations transferred to VPUs during the test process can be maintained until the test process is complete. For example, operations affecting thread 404AA can be executed using VPUs of LPU 402H until the test process is complete.After completion of the test process, the execution of operations relating to thread 404AA can be transferred back to VPUs of LPU 402G according to the embodiments described above. In other or similar embodiments, the processing unit test machine 151 can transfer the execution of operations belonging to thread 404AA back to VPUs of LPU 402G after completion of the test process on PU 122G (e.g., but before completion of the test process on other PUs 122 of processor 120).

[0055] In some embodiments, a virtualized system 130 of a client device 106 can be associated with an isolation protocol that prohibits the execution of operations associated with applications running on VPUs of the virtualized system 130 by VPUs of other virtualized systems 130 of the client device 106. For example, an isolation protocol associated with virtualized system 130A can prohibit the execution of operations associated with threads 404A-404F that are intended to be executed using VPUs of virtualized system 130A. In such embodiments, the processing unit test machine 151 can identify VPUs of each virtualized system 130 of the client device 106 in order to initiate the test process, as described above, and initiate the test process on each of the identified VPUs (e.g., concurrently). For example, as in Fig. Figure 4E illustrates the processing unit test machine 151 initiating tests for PU 122H (supporting virtualized system 130B) and PU 122F (supporting virtualized system 130A) simultaneously, in accordance with embodiments of the present disclosure.

[0056] Fig. Figures 5A-5B illustrate additional examples of testing processing units of a computer system according to at least one embodiment. In some embodiments, PUs 122 of a processor 120 can be organized into cores 502, as shown in Fig. 5A is shown. For example, core 502A may contain PUs 122AA-DA, core 502B may contain PUs 122AB-DB, and / or core 502C may contain PUs 122AC-DC. Each PU 122 of a core 502 may, in some embodiments, share a common integrated circuit. As with respect to Fig. As shown in Figures 5A-5B, the processing unit test machine 151 can select a PU 122 to initiate a test for each core and perform the test process on each core according to the techniques described above. In some embodiments, the processing unit test machine 151 can perform the test process on the PUs 122 of each core 502 of the processor 120 simultaneously, as shown in Figures 5A-5B. Fig. 5B illustrates this.

[0057] As described above, the test process for PUs 122 can be performed by transferring the execution of operations by VPUs supported by PUs 122 to other PUs 122 (e.g., those not being tested) until each PU 122 has been tested. However, in additional or alternative embodiments, the test process can be performed by transferring the VPUs between PUs. For example, the test processing unit 151 can receive a request to initiate a test process as described above. According to the test instructions 252 for the test process, the scheduler 210 can identify PU 122H to initiate the test process. The execution transfer component 212 can transfer (e.g., migrate) the VPU supported by PU 122H from PU 122H to PU 122G.While the operations of the test process are being carried out at PU 122H, PU 122G can support VPUs running Thread 404AA and Thread 404AB, as described herein. After the test process is complete at PU 122H, VPUs located at PU 122G can be transferred (i.e., migrated) from PU 122G to PU 122H or another PU 122, as described herein.

[0058] Fig. Figure 6A illustrates one or more hardware structures 615 for the inference and / or training logic to perform inference and / or training operations associated with one or more embodiments. Details of the inference and / or training logic are given below in conjunction with Fig. 6A and / or 6B specified.

[0059] In at least one embodiment, the one or more hardware structures 615 may, without limitation, include a code and / or data memory 601 to store forward and / or output weights and / or input / output data and / or other parameters to configure neurons or layers of a neural network that is trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the training logic may include or be coupled to a code and / or data memory 601 to store graph code or other software that controls the timing and / or order in which weight and / or other parameter information is loaded to configure logic, including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, a code, such asA graph code, based on the architecture of a neural network to which the code corresponds, transmits weight or other parameter information to the processor ALU(s). In at least one embodiment, the code and / or data memory 601 stores weight parameters and / or input / output data of each layer of a neural network that is trained during the forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments, or used in conjunction with one or more embodiments. In at least one embodiment, each part of the code and / or data memory 601 may be contained in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or the system working memory of a processor.

[0060] In at least one embodiment, each part of the coding memory and / or data memory 601 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data memory 601 can be a cache memory, a dynamic randomly addressable memory (DRAM), a static randomly addressable memory (SRAM), a non-volatile memory (e.g., flash memory), or another type of memory.In at least one embodiment, the choice of whether the code and / or data storage 601 is internal or external, for example to the processor, or comprises DRAM, SRAM, flash memory or another type of memory, may depend on the available memory on or off the chip, latency requirements of the training and / or inference functions performed, batch size of the data used in the inference and / or training of a neural network, or a combination of these factors.

[0061] In at least one embodiment, the hardware structure 615 can, without limitation, include a code and / or data memory 605 for storing backward and / or output weights and / or input / output data corresponding to neurons or layers of a neural network that is trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the code and / or data memory 605 stores weighting parameters and / or input / output data of each layer of a neural network that is trained or used in conjunction with one or more embodiments during backward propagation of input / output data and / or weighting parameters during training and / or inference using aspects of one or more embodiments.In at least one embodiment, the training logic can include or be coupled to a code and / or data memory 605 to store graph code or other software that controls the timing and / or sequence in which weight and / or other parameter information is loaded to configure logic, including integer and / or floating-point units (collectively, arithmetic logic units, ALUs). In at least one embodiment, a code, such as a graph code, based on a neural network architecture to which the code corresponds, loads weight or other parameter information into the processor ALUs. In at least one embodiment, each part of the code and / or data memory 605 can be contained in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or the system working memory of a processor.In at least one embodiment, each part of the coding memory and / or data storage 605 can be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 605 can be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or another type of memory. In at least one embodiment, the choice of whether the code and / or data storage 605 is internal or external to, for example, the processor, or whether it comprises DRAM, SRAM, flash memory, or another type of memory, can depend on the available on-chip or off-chip memory, the latency requirements of the training and / or inference functions performed, the batch size of the data used in the inference and / or training of a neural network, or a combination of these factors.

[0062] In at least one embodiment, the code and / or data memory 601 and the code and / or data memory 605 can be separate memory structures. In at least one embodiment, the code and / or data memory 601 and the code and / or data memory 605 can be the same memory structure. In at least one embodiment, the code and / or data memory 601 and the code and / or data memory 605 can be partly the same memory structure and partly separate memory structures. In at least one embodiment, any part of the code and / or data memory 601 and the code and / or data memory 605 can be contained in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or the system memory of a processor.

[0063] In at least one embodiment, the hardware structures 615 can, without limitation, include one or more arithmetic logic units (“ALU(s)”) 610, including integer and / or floating-point units, to perform logical and / or mathematical operations that are at least partially based on or specified by a training and / or inference code (e.g., graph code), the result of which can generate activations (e.g., output values ​​of layers or neurons within a neural network) that are stored in an activation memory 620 and are functions of input / output and / or weight parameter data that are stored in the code and / or data memory 601 and / or code and / or data memory 605.In at least one embodiment, the activations stored in the activation memory 620 are generated according to linear algebraic and / or matrix-based mathematics, which is performed by ALU(s) 610 in response to execution instructions or other code, wherein weight values ​​stored in code and / or data memory 605 and / or code and / or data memory 601 are used as operands together with other values, such as bias values, gradient information, pulse values ​​or other parameters or hyperparameters, some or all of which may be stored in the code and / or data memory 605 or in the code and / or data memory 601 or in another memory on or off the chip.

[0064] In at least one embodiment, the ALU(s) 610 are contained within one or more processors or in other hardware logic devices or circuits, while in another embodiment, the ALU(s) 610 may be external to a processor or other hardware logic device or circuit that uses them (e.g., a coprocessor). In at least one embodiment, the ALU(s) 610 may be contained within the execution units of a processor or otherwise in a bank of ALUs that the execution units of a processor can access either within the same processor or distributed across different processors of different types (e.g., CPUs, GPUs, fixed functional units, etc.).In at least one embodiment, the code and / or data memory 601, the code and / or data memory 605, and the activation memory 620 can be located on the same processor or other hardware logic unit or circuit, while in another embodiment they can be located on different processors or other hardware logic devices or circuits, or in a combination of identical and different processors or other hardware logic devices or circuits. In at least one embodiment, any part of the activation memory 620 can be contained in another on-chip or off-chip data memory, including the L1, L2, or L3 cache or the system working memory of a processor.Furthermore, the inference and / or training code can be stored together with other code that a processor or other hardware logic or circuitry can access and that is retrieved and / or processed via the retrieval, decoding, scheduling, execution, elimination, and / or other logical circuitry of a processor.

[0065] In at least one embodiment, the activation memory 620 can be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or another type of memory. In at least one embodiment, the activation memory 620 can be located wholly or partially inside or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the activation memory 620 is internal or external to, for example, the processor, or whether it comprises DRAM, SRAM, flash memory, or another type of memory, can depend on the available on-chip or off-chip memory, the latency requirements of the training and / or inference functions performed, the batch size of the data used in the inference and / or training of a neural network, or a combination of these factors. In at least one embodiment, the one or more in Fig. 6A illustrates hardware structures 615 in conjunction with an application-specific integrated circuit (“ASIC”), such as Google’s Tensorflow® Processing Unit, a Graphcore™ Inference Processing Unit (“IPU”), or an Intel Corp. Nervana® (e.g., “Lake Crest”) processor. In at least one embodiment, the one or more hardware structures and / or the inference and / or training logic of Fig. 6A in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware such as data processing unit (DPU) hardware or field programmable gate arrays (FPGAs).

[0066] Fig. Figure 6B illustrates the hardware structure(s) 615 for inference and / or training logic according to at least one or more embodiments. In at least one embodiment, the one or more hardware structures 615 can, without limitation, include hardware logic in which computer resources are dedicated or otherwise used exclusively in connection with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, the one or more hardware structures 615 and / or the inference and / or training logic of Fig. 6B can be used in conjunction with an application-specific integrated circuit (ASIC), such as Google's Tensorflow® Processing Unit, a Graphcore™ Inference Processing Unit (IPU), or an Intel Corp. Nervana® (e.g., "Lake Crest") processor. In at least one embodiment, the hardware structure(s) 615 and / or the inference and / or training logic of Fig. 6B in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware, such as data processing unit (DPU) hardware, or with field-programmable gate arrays (FPGAs). In at least one embodiment, the inference and / or training logic includes, without limitation, a code and / or data memory 601 and a code and / or data memory 605, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, pulse values, and / or other parameter or hyperparameter information. In at least one embodiment, Fig. In the embodiment illustrated in Figure 6B, each of the code and / or data stores 601 and code and / or data stores 705 is assigned to a dedicated computing resource, such as computing hardware 602 and computing hardware 606, respectively. In at least one embodiment, each of the computing hardware 602 and computing hardware 606 comprises one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on information stored in the code and / or data store 601 and code and / or data store 605, respectively, and whose result is stored in the activation memory 620.

[0067] In at least one embodiment, each of the code and / or data stores 601 and 605 and the corresponding computing hardware 602 and 606, respectively, correspond to different layers of a neural network, such that the resulting activation from a "memory / computing pair 601 / 602" consisting of code and / or data store 601 and computing hardware 602 is provided as input for the "memory / computing pair 605 / 606" consisting of code and / or data store 605 and computing hardware 606, in order to reflect the conceptual organization of a neural network. In at least one embodiment, each of the memory / computing pairs 601 / 602 and 605 / 606 can correspond to more than one neural network layer. In at least one embodiment, additional memory / computing pairs (not shown) may be included following or in parallel to the memory / computing pairs 601 / 602 and 605 / 606 in the inference and / or training logic.

[0068] Fig. Figure 7 shows an example of a data center 700 in which at least one embodiment can be used. In at least one embodiment, the data center 700 comprises a data center infrastructure layer 710, a framework layer 720, a software layer 730, and an application layer 1240.

[0069] In at least one embodiment, as in Fig. As shown in Figure 7, the data center infrastructure layer 710 can include a resource orchestrator 712, clustered computing resources 714 and node computing resources (“node CR s”) 616(1)-616(N), where “N” is any positive integer. In at least one embodiment, the node CRs 616(1)-616(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including accelerators, field-programmable gate arrays (FPGAs), data processing units, graphics processing units, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power supply modules, and / or cooling modules, etc. In at least one embodiment, one or more node CRs may be located among the node CRs.s 616(1)-616(N) a server with one or more of the computer resources mentioned above.

[0070] In at least one embodiment, the grouped computing resources 714 can contain separate groupings of node CRs located in one or more racks (not shown) or in many racks in data centers at different geographic locations (also not shown). Separate groupings of node CRs within the grouped computing resources 714 can include grouped compute, network, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node CRs, including CPUs or processors, can be grouped within one or more racks to provide computing resources to support one or more workloads.In at least one embodiment, one or more racks can also contain any number of power supply modules, cooling modules and network switches in any combination.

[0071] In at least one embodiment, the resource orchestrator 712 can configure or otherwise control one or more node CRs 616(1)-616(N) and / or grouped computing resources 714. In at least one embodiment, the resource orchestrator 712 can include a software design infrastructure (“SDI”) management entity for the data center 700. In at least one embodiment, the resource orchestrator can include hardware, software, or a combination thereof.

[0072] In at least one embodiment, as in Fig. As shown in Figure 7, the framework layer 720 comprises a job scheduler 722, a configuration manager 724, a resource manager 726, and a distributed file system 728. In at least one embodiment, the framework layer 720 can include a framework to support software 732 of software layer 730 and / or one or more applications 742 of application layer 740. In at least one embodiment, the software 732 or the one or more applications 742 can each be web-based service software or applications such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 720 can be a type of free and open-source software web application framework such as Apache Spark™ (hereinafter “Spark”), which can utilize a distributed file system 728 for processing large amounts of data (e.g., “Big Data”), without being limited to it.In at least one embodiment, the job scheduler 722 can include a Spark driver to facilitate the scheduling of workloads supported by different layers of the data center 700. In at least one embodiment, the configuration manager 724 can be able to configure different layers, such as the software layer 730 and the framework layer 720, including Spark and the distributed file system 728 to support the processing of large amounts of data. In at least one embodiment, the resource manager 726 can be able to manage clustered or grouped computing resources that are allocated or will be allocated to support the distributed file system 728 and the job scheduler 722. In at least one embodiment, the clustered or grouped computing resources can include the grouped computing resource 714 on the infrastructure layer 710 of the data center.In at least one embodiment, the resource manager 726 can be coordinated with the resource orchestrator 712 to manage these allocated or assigned computer resources.

[0073] In at least one embodiment, the software contained in software layer 730 may include software 732 that is used by at least sections of the node CRs 616(1)-616(N), the grouped computing resources 714, and / or the distributed file system 728 of framework layer 720. The one or more types of software may include, among others, web page search software, email virus scanning software, database software, and streaming video content software.

[0074] In at least one embodiment, the applications 742 contained in the application layer 740 may include one or more types of applications used by at least sections of the node CRs 616(1)-616(N), the grouped computing resources 714, and / or the distributed file system 728 of the framework layer 720. One or more types of applications may include, but are not limited to, any number of a genome application, a cognitive computing application, and a machine learning application, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0075] In at least one embodiment, the configuration manager 724, the resource manager 726, and / or the resource orchestrator 712 can implement any number and type of self-modifying actions based on any set and type of data acquired in any technically feasible way. In at least one embodiment, self-modifying actions can relieve a data center operator of the data center 700 of potentially making poor configuration decisions and potentially avoiding underutilized and / or poorly functioning parts of a data center.

[0076] In at least one embodiment, the Data Center 700 may contain tools, services, software, or other resources for training one or more machine learning models or for predicting or deriving information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by calculating weight parameters according to a neural network architecture using software and computing resources described above in relation to the Data Center 700.In at least one embodiment, trained machine learning models corresponding to one or more neural networks can be used to derive or predict information, utilizing the resources described above in relation to the data center 700 by using weighting parameters calculated by one or more training techniques described here.

[0077] In at least one embodiment, the data center can use CPUs, application-specific integrated circuits (ASICs), GPUs, DPUs, FPGAs, or other hardware to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above can be configured as a service to allow users to train or infer information, such as image capture, speech capture, or other artificial intelligence services.

[0078] The inference and / or training logic is used to perform inference and / or training operations that are associated with one or more embodiments. Details regarding the inference and / or training logic are available in conjunction with Fig. 6A and / or 6B are specified. In at least one embodiment, the inference and / or training logic in the system of Fig. 7 for inference or prediction operations that are based at least partially on weighting parameters calculated using neural network training operations, neural network functions and / or neural network architectures or neural network use cases described herein.

[0079] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0080] Fig. Figure 8 is a block diagram showing an exemplary computer system, which may be a system with interconnected devices and components, a system-on-a-chip (“SoC”), or a combination thereof, 800, formed with a processor that may include execution units for executing an instruction according to at least one embodiment. In at least one embodiment, the computer system 800 may, without limitation, include a component, such as a processor 802, to employ execution units, including logic, for performing algorithms for processing data according to the present disclosure, as in the embodiment described herein. In at least one embodiment, the computer system 800 may include processors, such as…The PENTIUM® processor family, Xeon™, Itanium®, XScale™ and / or StrongARM™, Intel® Core™, or Intel® Nervana™ microprocessors available from Intel Corporation in Santa Clara, California, USA, although other systems (including PCs with other microprocessors, technical workstations, set-top boxes, and the like) may also be used. In at least one embodiment, the Computer System 800 may run a version of the WINDOWS operating system available from Microsoft Corporation in Redmond, Washington, USA, although other operating systems (e.g., UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0081] Embodiments can be used in other devices such as handheld devices and embedded applications. Some examples of handheld devices are mobile phones, Internet Protocol devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, embedded applications can include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip, network computers (“NetPCs”), set-top boxes, network hubs, wide-area network (“WAN”) switches, edge devices, Internet of Things (“IoT”) devices, or any other system capable of carrying out one or more instructions according to at least one embodiment.

[0082] In at least one embodiment, the computer system 800 can, without limitation, include a processor 802, which can, without limitation, include one or more execution units 808 for training and / or inferring a machine learning model according to the techniques described herein. In at least one embodiment, the computer system 800 is a single-processor desktop or server system, but in another embodiment, the computer system 800 can be a multiprocessor system. In at least one embodiment, the processor 802 can, without limitation, be a complex instruction set computer (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing a combination of instruction sets, or any other processing device, such as...a digital signal processor. In at least one embodiment, the processor 802 can be connected to a processor bus 810, which can transmit data signals between the processor 802 and other components in the computer system 800.

[0083] In at least one embodiment, the processor 802 can, without limitation, include an internal level 1 cache (“L1”) 804. In at least one embodiment, the processor 802 can have a single internal cache or multiple levels of an internal cache. In at least one embodiment, the cache can be located outside the processor 802. Other embodiments can also include a combination of both internal and external caches, depending on the specific implementation and requirements. In at least one embodiment, a register file 806 can contain different types of data in different registers, including, without limitation, integer registers, floating-point registers, status registers, and instruction pointer registers.

[0084] In at least one embodiment, the execution unit 808, which, without limitation, contains logic for performing integer and floating-point operations, is also located in the processor 802. In at least one embodiment, the processor 802 can also contain a microcode ("ucode") read-only memory ("ROM") that stores microcode for specific macro instructions. In at least one embodiment, the execution unit 808 can contain logic for processing a packed instruction set 809. In at least one embodiment, by incorporating a packed instruction set 809 into an instruction set of a general-purpose processor 802, together with associated instruction execution circuitry, operations used by many multimedia applications can be performed using packed data in a general-purpose processor 802.In one or more embodiments, many multimedia applications can be accelerated and run more efficiently by utilizing the full width of a processor's data bus for performing operations on packed data, thereby eliminating the need to transfer smaller data units across the processor's data bus to perform one or more operations on one data element at a time.

[0085] In at least one embodiment, the execution unit 808 can also be used in microcontrollers, embedded processors, graphics devices, DSPs, and other types of logic circuits. In at least one embodiment, the computer system 800 can, without limitation, include a memory 820. In at least one embodiment, the memory 820 can be implemented as a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, or another type of memory device. In at least one embodiment, the memory 820 can store instructions 819 and / or data 821 represented by data signals that can be executed by the processor 802.

[0086] In at least one embodiment, a system logic chip can be connected to the processor bus 810 and the main memory 820. In at least one embodiment, the system logic chip can, without limitation, include a main memory controller hub (“MCH”) 816, and the processor 802 can communicate with the MCH 816 via the processor bus 810. In at least one embodiment, the MCH 816 can provide a high-bandwidth main memory path 818 to the main memory 820 for instruction and data storage and for storing graphics instructions, data, and textures. In at least one embodiment, the MCH 816 can route data signals between the processor 802, the main memory 820, and other components in the computer system 800, and can bridge data signals between the processor bus 810, the main memory 820, and a system I / O 822.In at least one embodiment, a system logic chip can provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH 816 can be coupled to the main memory 820 via a high-bandwidth memory path 818, and the graphics / video card 812 can be coupled to the MCH 816 via an Accelerated Graphics Port (“AGP”) connection 814.

[0087] In at least one embodiment, the computer system 800 can use system I / O 822, which is a proprietary hub interface bus, to connect the MCH 816 to the I / O controller hub (“ICH”) 830. In at least one embodiment, the ICH 830 can provide direct connections to some I / O devices via a local I / O bus. In at least one embodiment, a local I / O bus can, without limitation, include a high-speed I / O bus for connecting peripheral devices to the main memory 820, the chipset, and the processor 802.Examples may include, without limitation, an audio controller 829, a firmware hub (“flash BIOS”) 828, a wireless transceiver 826, a data storage device 824, a legacy I / O controller 823 incorporating a user input and keyboard interface 825, a serial expansion port 827, such as the Universal Serial Bus (“USB”), and a network controller 834, which in some embodiments may include a data processing unit. The data storage device 824 may include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage devices.

[0088] Illustrated in at least one embodiment Fig. 8 a system comprising interconnected hardware devices or “chips”, whereas in other embodiments Fig. Figure 8 illustrates an exemplary system on a chip (“SoC”). In at least one embodiment, the devices can be interconnected by proprietary connections, standardized connections (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of the Computer System 800 are interconnected via Compute Express Link (CXL) connections.

[0089] The inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the inference and / or training logic 615 are provided below in conjunction with Fig. 6A and / or 6B are specified. In at least one embodiment, the inference and / or training logic 615 can be implemented in the system of Fig. 8 for inference or prediction operations that are based at least partially on weighting parameters calculated using neural network training operations, neural network functions and / or neural network architectures or neural network use cases described herein.

[0090] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0091] Fig. Figure 9 is a block diagram showing an electronic device 900 for using a processor 910 according to at least one embodiment. In at least one embodiment, the electronic device 900 can be, for example, and without limitation, a notebook, a tower server, a rack server, a blade server, a laptop, a desktop computer, a tablet, a mobile device, a telephone, an embedded computer, an edge device, an IoT device, or any other suitable electronic device.

[0092] In at least one embodiment, the System 900 can, without limitation, comprise a Processor 910 that is communicatively connected to any number or type of components, peripherals, modules, or devices. In at least one embodiment, the Processor 910 is coupled via a bus or interface, such as a 1°C bus, a System Management Bus (“SMBus”), a Low Pin Count (LPC) bus, a Serial Peripheral Interface (“SPI”), a High Definition Audio (“HDA”) bus, a Serial Advance Technology Attachment (“SATA”) bus, a Universal Serial Bus (“USB”) (versions 1, 2, 3), or a Universal Asynchronous Receiver / Transmitter (“UART”) bus. In at least one embodiment, the following is illustrated: Fig. 9 a system comprising interconnected hardware devices or “chips”, whereas in other embodiments Fig. 9 can illustrate an exemplary system on a chip (“SoC”). In at least one embodiment, the Fig. The devices shown in section 9 are connected to each other by proprietary connections, standardized connections (e.g., PCIe), or a combination thereof. In at least one embodiment, one or more components of Fig. 9 are interconnected via Compute Express Link (CXL) connections.

[0093] In at least one embodiment, Fig. 9 a display 924, a touchscreen 925, a touchpad 930, a near field communications unit (NFC) 945, a sensor hub 940, a thermal sensor 946 and an Express chipset (EC) 935, a Trusted Platform Module (TPM) 938, a BIOS / firmware / flash memory (BIOS, FW Flash) 922, a DSP 960, a drive 920 such as a solid state disk (SSD) or a hard disk drive (HDD), a wireless local area network unit (WLAN) 950, a Bluetooth unit 952, a wireless wide area network unit (WWAN) 956, a global positioning system (GPS) 955, a camera (USB 3.0 camera) 954, such as For example, a USB 3.0 camera, and / or a Low Power Double Data Rate (“LPDDR”) memory unit (“LPDDR3”) 915, which is implemented, for example, in the LPDDR3 standard. These components can each be implemented in any suitable way.

[0094] In at least one embodiment, other components can be communicatively connected to the processor 910 via the components mentioned above. In at least one embodiment, an accelerometer 941, an ambient light sensor (ALS) 942, a compass 943, and a gyroscope 944 can be communicatively connected to the sensor hub 940. In at least one embodiment, the thermal sensor 939, a fan 937, a keyboard 936, and a touchpad 930 can be communicatively coupled to the EC 935. In at least one embodiment, loudspeakers 963, headphones 964, and a microphone (mic) 965 can be communicatively coupled to an audio unit (audio codec and Class-D amplifier) ​​962, which in turn can be communicatively coupled to the DSP 960. In at least one embodiment, the audio unit 964 may, for example and without limitation, comprise an audio encoder / decoder (“codec”) and a class-D amplifier.In at least one embodiment, the SIM card (“SIM”) 957 can be communicatively coupled with the WWAN unit 956. In at least one embodiment, components such as the WLAN unit 950 and the Bluetooth unit 952, as well as the WWAN unit 956, can be implemented in a Next Generation Form Factor (“NGFF”).

[0095] The inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the inference and / or training logic 615 are provided below in conjunction with Fig. 6A and / or 6B are specified. In at least one embodiment, the inference and / or training logic 615 can be implemented in the system of Fig. 9 for inference or prediction operations that are based at least partially on weighting parameters calculated using neural network training operations, neural network functions and / or neural network architectures or neural network use cases described herein.

[0096] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0097] Fig. Figure 10 is a block diagram of a processing system according to at least one embodiment. In at least one embodiment, the system 1000 comprises one or more processors 1002 and one or more graphics processors 1008 and can be a single-processor desktop system, a multi-processor workstation system, or a server system with a large number of processors 1002 or processor cores 1007. In at least one embodiment, the system 1000 is a processing platform integrated into an integrated system-on-a-chip (SoC) circuit for use in mobile, portable, or embedded devices.

[0098] In at least one embodiment, the system 1000 may comprise or be integrated with a server-based gaming platform, a gaming console (including a gaming and media console), a mobile gaming console, a handheld gaming console, or an online gaming console. In at least one embodiment, the system 1000 is a mobile phone, a smartphone, a tablet computer device, or a mobile internet device. In at least one embodiment, the processing system 1000 may also comprise, be coupled to, or be integrated with a wearable device, such as a wearable device with a smartwatch, smart glasses, an augmented reality device, or a virtual reality device. In at least one embodiment, the processing system 1000 is a television device or a set-top box with processors 1002 and a graphical interface generated by graphics processors 1008.

[0099] In at least one embodiment, the processors 1002 each comprise processor cores 1007 for processing instructions that, upon execution, perform operations for system and application software. In at least one embodiment, each of the processor cores 1007 is configured to process a specific instruction set 1009. In at least one embodiment, the instruction set 1009 may include complex instruction set computing (CISC), reduced instruction set computing (RISC), or very long instruction word (VLIW) computing. In at least one embodiment, the processor cores 1007 may each process a different instruction set 1009, which may contain instructions to facilitate the emulation of other instruction sets. In at least one embodiment, the processor core 1007 may also include other processing devices, e.g.,a digital signal processor (DSP).

[0100] In at least one embodiment, the processors 1002 include a cache memory 1004. In at least one embodiment, the processor 1002 can have a single internal cache or multiple levels of an internal cache. In at least one embodiment, the cache memory is shared by different components of the processors 1002. In at least one embodiment, the processor 1002 also uses an external cache (e.g., an Ebeme-3 (L3) cache or Last Level Cache (LLC)) (not shown), which can be shared by the processor cores 1007 using known cache coherence techniques. In at least one embodiment, the register file 1006 is additionally included in the processors 1002, which can contain different types of registers for storing different data types (e.g., integer registers, floating-point registers, status registers, and an instruction pointer register).In at least one embodiment, the register file can contain 1006 general-purpose registers or other registers.

[0101] In at least one embodiment, one or more processors 1002 are coupled to one or more interface buses 1010 to transmit communication signals such as address, data, or control signals between the processors 1002 and other components in the system 1000. In at least one embodiment, the interface bus 1010 can be a processor bus, for example, a version of a Direct Media Interface (DMI) bus. In at least one embodiment, the one or more interfaces 1010 are not limited to a DMI bus and can include one or more Peripheral Component Interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, the one or more processors 1002 comprise an integrated memory controller 1016 and a platform controller hub 1030.In at least one embodiment, the main memory controller 1016 facilitates communication between a main memory device and other components of the system 1000, while the platform controller hub (PCH) 1030 provides connections to I / O devices via a local I / O bus.

[0102] In at least one embodiment, the memory device 1020 can be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or another memory device with suitable performance to serve as process memory. In at least one embodiment, the memory device 1020 can operate as system memory for the system 1000 to store data 1022 and instructions 1021 for use when one or more processors 1002 are executing an application or process.In at least one embodiment, the memory controller 1016 is also coupled to an optional external graphics processor 1012, which can communicate with one or more graphics processors 1008 in the processors 1002 to perform graphics and media operations. In at least one embodiment, a display device 1011 can be connected to the one or more processors 1002. In at least one embodiment, a display device 1011 can comprise one or more internal displays, such as in a mobile electronic device or a laptop device, or an external display connected via a display interface (e.g., DisplayPort, etc.). In at least one embodiment, the display device 1011 can include a head-mounted display (HMD), such as...a stereoscopic display device for use in Virtual Reality (VR) applications or Augmented Reality (AR) applications.

[0103] In at least one embodiment, the platform controller hub 1030 enables the connection of peripheral devices to the memory device 1020 and the processors 1002 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include, among others, an audio controller 1046, a network controller 1034, a firmware interface 1028, a wireless transceiver 1026, touch sensors 1025, and a data storage device 1024 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 1024 can be connected via a storage interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCI Express).In at least one embodiment, the touch sensors 1025 can comprise touchscreen sensors, pressure sensors, or fingerprint sensors. In at least one embodiment, the wireless transceiver 1026 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a cellular network transceiver such as a 3G, 4G, or Long Term Evolution (LTE) transceiver. In at least one embodiment, the firmware interface 1028 allows communication with the system firmware and can, for example, be a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, the network controller 1034 can enable a network connection to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to the interface buses 1010.In at least one embodiment, the audio controller 1046 is a multi-channel high-definition audio controller. In at least one embodiment, the system 1000 includes an optional legacy I / O controller 1040 for connecting legacy devices (e.g., Personal System 2 (PS / 2)) to the system. In at least one embodiment, the platform controller hub 1030 can also be connected to one or more universal serial bus (USB) controllers 1042, which connect input devices such as keyboard and mouse combinations 1043, a camera 1044, or other USB input devices.

[0104] In at least one embodiment, an instance of the memory controller 1016 and the platform controller hub 1130 can be integrated into a discrete external graphics processor, such as the external graphics processor 1011. In at least one embodiment, the platform controller hub 1130 and / or memory controller 1016 can be external to one or more processors 1002. In at least one embodiment, the system 1000 can, for example, include an external memory controller 1016 and platform controller hub 1130 configured as a memory controller hub and peripheral controller hub within a system chipset that communicates with the one or more processors 1002.

[0105] The inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the inference and / or training logic 615 are provided below in conjunction with Fig. 6A and / or 6B are specified. In at least one embodiment, parts or all of the inference and / or training logic 615 can be integrated into the graphics processor 1008. For example, in at least one embodiment, the training and / or inference techniques described herein can use ALUs contained in a graphics processor. Furthermore, in at least one embodiment, the inference and / or training operations described herein can be performed with an ALU other than that specified in Fig. 6A or Fig. The logic illustrated in Figure 6B can be carried out. In at least one embodiment, weighting parameters can be stored in on-chip or off-chip memory and / or in registers (shown or not shown) that configure the ALUs of a graphics processor to execute one or more of the machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0106] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0107] Fig. Figure 11 is a block diagram of a processor 1100 comprising one or more processor cores 1102A-1102N, an integrated memory controller 1113, and an integrated graphics processor 1108, according to at least one embodiment. In at least one embodiment, the processor 1100 may include additional cores up to and including the additional core 1102N, which is represented by dashed, lined boxes. In at least one embodiment, each of the processor cores 1102A-1102N contains internal cache units 1104A-1104N. In at least one embodiment, each processor core also has access to one or more shared cache units 1106.

[0108] In at least one embodiment, the internal cache units 1104A-1104N and the shared cache units 1106 constitute a cache memory hierarchy within the processor 1100. In at least one embodiment, cache memory units 1104A-1104N can comprise at least one level of instruction and data cache within each processor core and one or more levels of shared mid-level cache, such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, with a highest level of cache prior to external memory being classified as LLC. In at least one embodiment, the cache coherence logic maintains coherence between the different cache units 1106 and 1104A-1104N.

[0109] In at least one embodiment, the processor 1100 can also include a set of one or more bus controller units 1116 and a system agent core 1110. In at least one embodiment, one or more bus controller units 1116 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. In at least one embodiment, the system agent core 1110 provides management functions for various processor components. In at least one embodiment, the system agent core 1110 includes one or more integrated memory controllers 1113 to manage access to various external memory devices (not shown).

[0110] In at least one embodiment, one or more of the processor cores 1102A-1102N include support for simultaneous multithreading. In at least one embodiment, the system agent core 1110 includes components for coordinating and operating the cores 1102A-1102N during multithreading. In at least one embodiment, the system agent core 1110 may additionally include a power control unit (PCU) containing logic and components for controlling one or more power states of processor cores 1102A-1102N and the graphics processor 1108.

[0111] In at least one embodiment, the processor 1100 additionally comprises the graphics processor 1108 for performing graphics processing operations. In at least one embodiment, the graphics processor 1108 is coupled with shared cache units 1106 and a system agent core 1110, which includes one or more integrated memory controllers 1113. In at least one embodiment, the system agent core 1110 also includes a display controller 1111 for controlling the output of the graphics processor to one or more coupled displays. In at least one embodiment, the display controller 1111 can also be a separate module connected to the graphics processor 1108 via at least one connection, or it can be integrated into the graphics processor 1108.

[0112] In at least one embodiment, a ring-based connection unit 1112 is used to connect internal components of the processor 1100. In at least one embodiment, an alternative connection unit can be used, such as a point-to-point connection, a switched connection, or other techniques. In at least one embodiment, the graphics processor 1108 is connected to the ring connection 1112 via an I / O connection 1113.

[0113] In at least one embodiment, the I / O connection 1113 represents at least one of several types of I / O connections, including an on-package I / O connection, which facilitates communication between different processor components and an embedded high-performance memory module 1118, such as an eDRAM module. In at least one embodiment, each of the processor cores 1102A-1102N and the graphics processor 1108 use the embedded memory modules 1118 as a shared last-level cache.

[0114] In at least one embodiment, the processor cores 1102A-1102N are homogeneous cores executing a common instruction set architecture. In at least one embodiment, the processor cores 1102A-1102N are heterogeneous with respect to the instruction set architecture (ISA), wherein one or more processor cores 1102A-1102N execute a common instruction set, while one or more other cores of the processor cores 1102A-1102N execute a subset of a common instruction set or a different instruction set. In at least one embodiment, the processor cores 1102A-1102N are heterogeneous with respect to the microarchitecture, wherein one or more cores with a relatively higher power consumption are coupled with one or more cores with a lower power consumption. In at least one embodiment, the processor 1100 can be implemented on one or more chips or as an integrated SoC circuit.

[0115] The inference and / or training logic 615 is used to perform inference and / or training operations associated with one or more embodiments. Details regarding the inference and / or training logic 615 are provided below in conjunction with Fig. 6A and / or 6B. In at least one embodiment, parts or all of the inference and / or training logic 615 can be integrated into the processor 1100. In at least one embodiment, the training and / or inference techniques described herein can, for example, use one or more ALUs located in the graphics processor 1108, in the one or more graphics cores 1102A-1102N, or in other components in Fig. 11 are included. Furthermore, in at least one embodiment, the inference and / or training operations described herein can be performed with a different method than the one described in Fig. 6A or Fig. The logic illustrated in Figure 6B can be performed. In at least one embodiment, weighting parameters can be stored in on-chip or off-chip memory and / or in registers (shown or not shown) that configure the ALUs of a 1100 graphics processor to perform one or more of the machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0116] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0117] Fig. Figure 12 is an exemplary data flow diagram for a process 1200 for generating and deploying an image processing and inference pipeline according to at least one embodiment. In at least one embodiment, the process 1200 can be used with imaging devices, processing devices, and / or other types of devices in facilities 1202. The process 1200 can be executed in a training system 1204 and / or a deployment system 1206. In at least one embodiment, the training system 1204 can be used to perform the training, deployment, and implementation of machine learning models (e.g., neural networks, object detection algorithms, computer vision algorithms, etc.) for use in the deployment system 1206.In at least one embodiment, the provisioning system 1206 can be configured to offload processing and computing resources to a distributed computing environment in order to reduce the infrastructure requirements of the facility 1202. In at least one embodiment, one or more applications in a pipeline can use or call services (e.g., inference, visualization, computation, AI, etc.) of the provisioning system 1206 during application execution.

[0118] In at least one embodiment, some of the applications used in advanced processing and inference pipelines may employ machine learning models or other AI to perform one or more processing steps. In at least one embodiment, machine learning models may be trained in facility 1202 using data 1208 (such as imaging data) generated in facility 1202 (and stored on one or more Picture Archiving and Communication System (PACS) servers in facility 1202), they may be trained using imaging or sequencing data 1208 from one or more other facilities, or a combination thereof.In at least one embodiment, the training system 1204 can be used to provide applications, services and / or other resources for generating functional, deployable machine learning models for the deployment system 1206.

[0119] In at least one embodiment, the model registration 1224 can be supported by an object store that can support versioning and object metadata. In at least one embodiment, the object store can, for example, be connected to cloud storage (e.g., Cloud 1126 from Fig. 12) A compatible application programming interface (API) accessible from a cloud platform. In at least one embodiment, machine learning models in model registry 1224 can be uploaded, listed, modified, or deleted by developers or partners of a system that interacts with an API. In at least one embodiment, an API can provide access to procedures that allow users with appropriate credentials to associate models with applications, so that models can be executed as part of the execution of containerized instantiations of applications.

[0120] In at least one embodiment, the training pipeline 1204 ( Fig. 12) include a scenario in which the device 1202 trains its own machine learning model or has an existing machine learning model that needs to be optimized or updated. In at least one embodiment, imaging data 1208 generated by one or more imaging devices, sequencing devices, and / or other types of devices can be received. In at least one embodiment, after receiving the imaging data 1208, the AI-assisted annotation 1210 can be used to assist in generating annotations that correspond to the imaging data 1208 and are used as basic truth data for a machine learning model. In at least one embodiment, the AI-assisted annotation 1210 can include one or more machine learning models (e.g., convolutional neural networks (CNNs)) that can be trained to generate annotations for certain types of imaging data 1208 (e.g.,(from certain devices). In at least one embodiment, the AI-assisted annotations 1210 can then be used directly or adapted or fine-tuned with the aid of an annotation tool to generate basic truth data. In at least one embodiment, AI-assisted annotations 1210, labeled clinical data 1212, or a combination thereof can be used as basic truth data for training a machine learning model. In at least one embodiment, a trained machine learning model can be designated as an output model 1216 and can be used by the delivery system 1206 as described herein.

[0121] In at least one embodiment, a training pipeline 1204 ( Fig. 12) include a scenario in which the facilities 1202 require a machine learning model for use in performing one or more processing tasks for one or more applications in the deployment system 1206, but the facilities 1202 may not currently have such a machine learning model (or may not have had a model that is optimized, efficient, or effective for such purposes). In at least one embodiment, an existing machine learning model can be selected from a model registry 1224. In at least one embodiment, the model registry 1224 can contain machine learning models that have been trained to perform a variety of different inferences from imaging data.In at least one embodiment, the machine learning models in model registration 1224 may have been trained on imaging data from facilities other than facilities 1202 (e.g., from facilities located at other sites). In at least one embodiment, machine learning models may have been trained on imaging data from one site, two sites, or any number of sites. In at least one embodiment, if training is performed on imaging data from a particular site, the training may take place at that site or at least in a manner that protects the confidentiality of the imaging data or restricts the transmission of imaging data outside the site. In at least one embodiment, once a machine learning model has been trained—or partially trained—at a site, it may be added to model registration 1224.In at least one embodiment, a machine learning model can then be retrained or updated at any number of other facilities, and a retrained or updated model can be made available in the model registry 1224. In at least one embodiment, a machine learning model can then be selected from the model registry 1224—and designated as output models 1216—and used in the deployment system 1206 to perform one or more processing tasks for one or more applications of a deployment system.

[0122] In at least one embodiment, training pipeline 1204 ( Fig. 12) A scenario may include facility 1202, which requires a machine learning model for use in performing one or more processing tasks for one or more applications in the deployment system 1206. However, facilities 1202 may not currently have such a machine learning model (or may not have had a model that is optimized, efficient, or effective for such purposes). In at least one embodiment, a machine learning model selected from the model registry 1224 may not be matched or optimized for the imaging data 1208 generated in facilities 1202 because of differences in the populations, the robustness of the training data used to train a machine learning model, the variety of anomalies in the training data, and / or other problems with the training data.In at least one embodiment, the AI-assisted annotation 1210 can be used to assist in the creation of annotations corresponding to the imaging data 1208, which can be used as baseline truth data for the initial training or updating of a machine learning model. In at least one embodiment, labeled data 1212 can be used as baseline truth data for training a machine learning model. In at least one embodiment, the initial training or updating of a machine learning model can be referred to as model training 1214. In at least one embodiment, the model training 1214 (e.g., AI-assisted annotations 1210, labeled clinical data 1212, or a combination thereof) can be used as baseline truth data for the initial training or updating of a machine learning model.In at least one embodiment, a trained machine learning model can be designated as output model 1216 and can be used by the deployment system 1206 as described here.

[0123] In at least one embodiment, the deployment system 1206 can include software 1218, services 1220, hardware 1222, and / or other components, features, and functions. In at least one embodiment, the deployment system 1206 can include a software "stack" such that the software 1218 is built on top of the services 1220 and can use the services 1220 to perform some or all of the processing tasks, and the services 1220 and the software 1218 can be built on top of the hardware 1222 and use the hardware 1222 to perform processing, storage, and / or other computational tasks of the deployment system 1206. In at least one embodiment, the software 1218 can include any number of different containers, each container being capable of instantiating an application.In at least one embodiment, each application can perform one or more processing tasks in an advanced processing and inference pipeline (e.g., inference, object detection, feature detection, segmentation, image enhancement, calibration, etc.). In at least one embodiment, an advanced processing and inference pipeline can be defined based on the selection of various containers desired or required for processing imaging data 1208, in addition to containers that receive and configure imaging data for use by the individual containers and / or for use by the facility 1202 after processing by a pipeline (e.g., to convert outputs back into a usable data type). In at least one embodiment, a combination of containers within the software 1218 (e.g.,a pipeline) can be referred to as a virtual instrument (as described in more detail here), and a virtual instrument can use services 1220 and hardware 1222 to perform some or all of the processing tasks of applications instantiated in containers.

[0124] In at least one embodiment, a data processing pipeline can receive input data (e.g., imaging data 1208) in a specific format in response to an inference request (e.g., a request from a user of the deployment system 1206). In at least one embodiment, the input data can represent one or more images, videos, and / or other data representations generated by one or more imaging devices. In at least one embodiment, the data can be preprocessed as part of the data processing pipeline to prepare the data for processing by one or more applications.In at least one embodiment, post-processing can be performed on the output of one or more inference tasks or other processing tasks of a pipeline to prepare output data for a subsequent application and / or to prepare output data for transmission and / or use by a user (e.g., in response to an inference request). In at least one embodiment, inference tasks can be performed by one or more machine learning models, e.g., by trained or deployed neural networks, which may include the output models 1216 of the training system 1204.

[0125] In at least one embodiment, the tasks of the data processing pipeline can be encapsulated in one or more containers, each representing a discrete, fully functional instantiation of an application and a virtualized computing environment capable of referencing machine learning models. In at least one embodiment, containers or applications can be published in a private (e.g., access-restricted) area of ​​a container registry (described in more detail herein), and trained or deployed models can be stored in the model registry 1224 and assigned to one or more applications. In at least one embodiment, images of applications (e.g.,Container images) may be available in a container registry, and once a user has selected them from a container registry for deployment in a pipeline, an image can be used to create a container for instantiating an application for use by a user's system.

[0126] In at least one embodiment, developers (e.g., software developers, clinicians, physicians, etc.) can develop, publish, and store applications (e.g., as containers) for performing image processing and / or inference on provided data. In at least one embodiment, development, publication, and / or storage can be performed using a Software Development Kit (“SDK”) associated with a system (e.g., to ensure that a developed application and / or container is compatible with a system). In at least one embodiment, a developed application can be tested locally (e.g., in a first facility, with data from a first facility) using an SDK that can support at least some of the services 1220 as a system (e.g., the system 1200 from Fig. 12) Since DICOM objects can contain between one and hundreds of image or other data types, and due to the diversity of the data, in at least one embodiment a developer may be responsible for managing the extraction and preparation of the incoming data (e.g., defining constructs, incorporating preprocessing into an application, etc.). In at least one embodiment, after being validated by System 1200 (e.g., for accuracy), an application may be available in a container register for selection and / or implementation by a user to perform one or more processing tasks with respect to data in a user's facility (e.g., a second facility).

[0127] In at least one embodiment, developers can then make applications or containers available over a network for access and use by users of a system (e.g., System 1200 from Fig. 12) release. In at least one embodiment, finished and validated applications or containers can be stored in a container registry, and associated machine learning models can be stored in the model registry 1224. In at least one embodiment, a requesting entity—making an inference or image processing request—can search a container registry and / or model registry 1224 for an application, container, dataset, machine learning model, etc., select a desired combination of elements to include in the data processing pipeline, and submit an image processing request.In at least one embodiment, a query may contain input data (and in some examples, associated patient data) required to execute a query, and / or a selection of applications and / or machine learning models to be executed when processing a query. In at least one embodiment, a query may then be forwarded to one or more components of the provisioning system 1206 (e.g., a cloud) to perform the data processing pipeline. In at least one embodiment, the processing by the provisioning system 1206 may include referencing selected elements (e.g., applications, containers, models, etc.) from a container registry and / or model registry 1224. In at least one embodiment, once generated by a pipeline, the results may be returned to a user for reference (e.g.,for display in a viewing application suite running on a local workstation or terminal on-site).

[0128] In at least one embodiment, the services 1220 can be used to support the processing or execution of applications or containers in pipelines. In at least one embodiment, the services 1220 can include computing services, artificial intelligence (AI) services, visualization services, and / or other service types. In at least one embodiment, the services 1220 can provide functions common to one or more applications in the software 1218, so that the functions can be abstracted into a service that can be called or used by the applications. In at least one embodiment, the functionality provided by the services 1220 can run dynamically and more efficiently while scaling well by enabling the applications to process data in parallel (e.g., via a parallel computing platform 1230). Fig. 12)). In at least one embodiment, not every application that uses the same functionality offered by a service 1220 needs to have its own instance of the service 1220; rather, the service 1220 can be shared by different applications. In at least one embodiment, the services may include an inference server or inference engine that can be used to perform acquisition or segmentation tasks, as non-limiting examples. In at least one embodiment, a model training service may be included that provides the ability to train and / or retrain machine learning models. In at least one embodiment, a data augmentation service may also be included that provides GPU-accelerated data extraction (e.g., DICOM, RIS, CIS, REST-compliant, RPC, raw data, etc.), resizing, scaling, and / or other augmentations.In at least one embodiment, a visualization service can be used that can add image rendering effects—such as ray tracing, rasterization, noise reduction, sharpening, etc.—to enhance the realism of two-dimensional (2D) and / or three-dimensional (3D) models. In at least one embodiment, services for virtual instruments can be included that provide beamforming, segmentation, inference, imaging, and / or support for other applications within virtual instrument pipelines.

[0129] In at least one embodiment, where a service 1220 contains an AI service (e.g., an inference service), one or more machine learning models can be executed by calling an inference service (e.g., an inference server) (e.g., as an API call) to execute machine learning models or their processing as part of application execution. In at least one embodiment, where another application contains one or more machine learning models for segmentation tasks, an application can call an inference service to execute machine learning models for performing one or more of the processing operations associated with the segmentation tasks.In at least one embodiment, the software 1218, which implements an advanced processing and inference pipeline that includes a segmentation application and an anomaly detection application, can be streamlined because each application can call the same inference service to perform one or more inference tasks.

[0130] In at least one embodiment, the hardware 1222 can include GPUs, CPUs, DPUs, graphics cards, an AI / deep learning system (e.g., an AI supercomputer such as NVIDIA's DGX), a cloud platform, or a combination thereof. In at least one embodiment, different types of hardware 1222 can be used to provide efficient, purposeful support for software 1218 and services 1220 in the deployment system 1206. In at least one embodiment, GPU processing can be implemented for on-premises processing (e.g., in the facility 1202), in an AI / deep learning system, in a cloud system, and / or in other processing components of the deployment system 1206 to improve the efficiency, accuracy, and effectiveness of image processing and generation.In at least one embodiment, the software 1218 and / or the services 1220 can be optimized for GPU processing with regard to deep learning, machine learning, and / or high-performance computing, as non-limiting examples. In at least one embodiment, at least part of the computing environment of the deployment system 1206 and / or the training system 1204 can be run in a data center on one or more supercomputers or high-performance computers with GPU-optimized software (e.g., a hardware and software combination of NVIDIA's DGX system). In at least one embodiment, the hardware 1222 can include any number of GPUs that can be used for parallel data processing as described herein.In at least one embodiment, the cloud platform may also include GPU processing for GPU-optimized execution of deep learning tasks, machine learning tasks, or other computational tasks. In at least one embodiment, the cloud platform may further include DPU processing to transfer data received over a network and / or via a network controller or other network interface directly to one or more GPUs (e.g., main memory). In at least one embodiment, the cloud platform (e.g., NVIDIA's NGC) may run as a hardware abstraction and scaling platform using one or more AI / deep learning supercomputers and / or GPU-optimized software (e.g., as on NVIDIA's DGX systems). In at least one embodiment, the cloud platform may include an application container clustering system or an orchestration system (e.g.,Integrate Kubernetes across multiple GPUs to enable seamless scaling and load balancing.

[0131] Fig. Figure 13 is a system diagram for an example system 1300 for generating and providing an image distribution pipeline according to at least one embodiment. In at least one embodiment, the system 1300 can be used to perform the process 1200 of Fig. 12 and / or other processes that include extended processing and inference pipelines. In at least one embodiment, the system 1300 can include a training system 1204 and a deployment system 1206. In at least one embodiment, the training system 1204 and the deployment system 1206 can be implemented using software 1218, services 1220, and / or hardware 1222, as described herein.

[0132] In at least one embodiment, System 1300 (e.g., Training System 1204 and / or Deployment System 1206) can be implemented in a cloud computing environment (e.g., using Cloud 1326). In at least one embodiment, System 1300 can be implemented locally in a healthcare facility or as a combination of cloud and local computing resources. In at least one embodiment, access to the APIs in Cloud 1326 can be restricted to authorized users by implemented security measures or protocols. In at least one embodiment, a security protocol can include web tokens that can be signed by an authentication service (e.g., AuthN, AuthZ, Gluecon, etc.) and appropriately authorized.In at least one embodiment, the APIs of virtual instruments (as described herein) or other instances of System 1300 can be restricted to a set of public IPs that have been verified or authorized for interaction.

[0133] In at least one embodiment, various components of System 1300 can communicate with each other over a variety of different network types, including, but not limited to, local area networks (LANs) and wide area networks (WANs) using wired and / or wireless communication protocols. In at least one embodiment, communication between devices and components of System 1300 (e.g., for sending inference requests, receiving results of inference requests, etc.) can occur via data bus(s), wireless data protocols (Wi-Fi), wired data protocols (e.g., Ethernet), etc.

[0134] In at least one embodiment, the training system 1204 can execute training pipelines 1304, similar to those described here in relation to Fig. 12 are described. In at least one embodiment, in which one or more machine learning models in the deployment pipelines 1310 are to be used by the deployment system 1206, the training pipelines 1304 can be used to train or retrain one or more (e.g., pre-trained) models and / or to implement pre-trained models 1306 (e.g., without the need for retraining or updating). In at least one embodiment, one or more output models 1216 can be generated as a result of training the training pipelines 1304. In at least one embodiment, the training pipelines 1304 can contain any number of processing steps, such as the conversion or interpretation of imaging data (or other input data).In at least one embodiment, different training pipelines 1304 can be used for different machine learning models used by the deployment system 1206. In at least one embodiment, training pipelines 1304 similar to a first example described with reference to can be used for a first machine learning model. Fig. As described in section 12, training pipelines 1304, which are a second, in relation to Fig. The 12 described examples can be used for a second machine learning model, and training pipelines 1304, which resemble a third, in terms of Fig. Similar to the example described in section 12, the tasks can be used for a third machine learning model. In at least one embodiment, any combination of tasks within the training system 1204 can be used, depending on the requirements of the respective machine learning model. In at least one embodiment, one or more of the machine learning models can already be trained and ready for use, so that the machine learning models do not undergo any processing by the training system 1204 and can be implemented by the deployment system 1206.

[0135] In at least one embodiment, the one or more output models 1216 and / or the one or more pretrained models 1306 may contain all types of machine learning models, depending on the implementation or embodiment. In at least one embodiment, and without limitation, the machine learning models used by the system 1300 may include machine learning models that employ linear regression, logistic regression, decision trees, support vector machines (SVMs), Naive Bayes, k-nearest neighbor (Knn), k-mean clustering, random forest, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (for example, auto-encoders, convolutional, recurrent, perceptrons, long / short-term memory (LSTM), Hopfield, Boltzmann, deep belief, deconvolutional, generative adversarial, liquid state machines, etc.), and / or other types of machine learning models.

[0136] In at least one embodiment, the training pipelines 1304 may contain an AI-supported annotation, as described herein with respect to at least Fig. 12B is described in more detail. In at least one embodiment, labeled data 1212 (e.g., conventional annotations) can be generated by any number of techniques. In at least one embodiment, labels or other annotations can be generated in a drawing program (e.g., an annotation program), a CAD program, a labeling program, another type of program suitable for generating annotations or labels for the basic truth, and / or, in some examples, drawn by hand. In at least one embodiment, the basic truth data can be generated synthetically (e.g., from computer models or renderings), generated in reality (e.g., designed and generated from real data), machine-generated (e.g., using feature analysis and learning to extract features from the data and then generate labels), or labeled by humans (e.g.,(a labeling expert determines the position of the labels) and / or a combination thereof. In at least one embodiment, for each instance of imaging data 1208 (or other data types used by machine learning models), there can be corresponding basic truth data generated by the training system 1204. In at least one embodiment, the AI-assisted annotation can be performed as part of the deployment pipelines 1310; either in addition to or instead of the AI-assisted annotation included in the training pipelines 1304. In at least one embodiment, the system 1300 can include a multi-layered platform that contains a software layer (e.g., software 1218) with diagnostic applications (or other application types) capable of performing one or more medical imaging and diagnostic functions. In at least one embodiment, the system 1300 can communicate with (e.g.,The system may be coupled (via encrypted connections) to PACS server networks of one or more institutions. In at least one embodiment, the System 1300 can be configured to access and reference data from PACS servers to perform operations such as training machine learning models, deploying machine learning models, image processing, inference, and / or other operations.

[0137] In at least one embodiment, a software layer can be implemented as a secure, encrypted, and / or authenticated API through which applications or containers can be called from one or more external environments (e.g., facility 1202). In at least one embodiment, applications can then call or execute one or more services 1220 to perform computational, AI, or visualization tasks associated with the respective applications, and the software 1218 and / or the services 1220 can utilize the hardware 1222 to perform processing tasks effectively and efficiently.

[0138] In at least one embodiment, the deployment system 1206 can execute the deployment pipelines 1310. In at least one embodiment, the deployment pipelines 1310 can contain any number of applications that can be applied sequentially, non-sequentially, or otherwise to imaging data (and / or other data types) generated by imaging devices, sequencing devices, genomics devices, etc.—including AI-assisted annotation, as described above. In at least one of the embodiments described herein, a deployment pipeline 1310 for a single device can be referred to as a virtual instrument for a device (e.g., a virtual ultrasound instrument, a virtual CT scan instrument, a virtual sequencing instrument, etc.).In at least one embodiment, there can be more than one provisioning pipeline 1310 for a single device, depending on what information is desired from the data generated by a device. In at least one embodiment where the detection of anomalies from an MRI device is desired, there can be a first provisioning pipeline 1310, and if image enhancement from the output of an MRI device is desired, there can be a second provisioning pipeline 1310.

[0139] In at least one embodiment, an image-generating application may include a processing task that involves the use of a machine learning model. In at least one embodiment, a user may use their own machine learning model or select a machine learning model from model registration 1224. In at least one embodiment, a user may implement their own machine learning model or select a machine learning model for inclusion in an application to perform a processing task. In at least one embodiment, applications may be selectable and customizable, and by defining constructs of applications, the provisioning and implementation of applications for a particular user is presented as a more seamless user experience.In at least one embodiment, the deployment pipelines 1310 can be made even more user-friendly, enabling simpler integration and delivering more accurate, efficient, and timely results by utilizing other features of the system 1300—such as the services 1220 and the hardware 1222. One or more embodiments of the application can be a game, video streaming application, machine control application, machine motion application, machine driving application, synthetic data generation application, model training application, perception application, augmented reality application, virtual reality application, mixed reality application, robotics application, security and monitoring application, an application for autonomous or semi-autonomous machines, a deep learning application, an environmental simulation application, a data center processing application, a conversational AI application, a light transport simulation application (e.g.,Ray tracing, path tracing, etc.), a collaborative content creation application for 3D assets, an application for a digital twinning system, a cloud computing application and / or another type of application or service are implemented or include these.

[0140] In at least one embodiment, the deployment system 1206 may include a user interface 1314 (e.g., a graphical user interface, a web interface, etc.) that can be used to select applications for inclusion in one or more deployment pipelines 1310, to arrange applications, to modify or change applications or parameters or constructs thereof, to use and interact with the deployment pipeline(s) 1310 during setup and / or deployment, and / or to otherwise interact with the deployment system 1206.In at least one embodiment, which is not illustrated with respect to the training system 1204, the user interface 1314 (or another user interface) can be used to select models for use in the deployment system 1206, to select models for training or new training in the training system 1204 and / or to otherwise interact with the training system 1204.

[0141] In at least one embodiment, the pipeline manager 1312 can be used in addition to an application orchestration system 1328 to manage the interaction between applications or containers of the deployment pipeline(s) 1310 and services 1220 and / or hardware 1222. In at least one embodiment, the pipeline manager 1312 can be configured to allow application-to-application, application-to-service 1220, and / or application-to-service or hardware 1222 interaction. In at least one embodiment, although illustrated as included in software 1218, this is not intended to be restrictive, and in some examples (e.g., as in Fig. (As illustrated in Figure 11), the pipeline manager 1312 can be included in the services 1220. In at least one embodiment, the application orchestration system 1328 (e.g., Kubernetes, Docker, etc.) can include a container orchestration system that can group applications into containers as logical units for coordination, management, scaling, and deployment. In at least one embodiment, by assigning applications from the one or more deployment pipelines 1310 (e.g., a reconstruction application, a segmentation application, etc.) to individual containers, each application can be run in a self-contained environment (e.g., at the kernel level) to increase speed and efficiency.

[0142] In at least one embodiment, each application and / or container (or an image thereof) can be developed, modified, and deployed individually (e.g., a first user or developer can develop, modify, and deploy a first application, and a second user or developer can develop, modify, and deploy a second application independently of the first user or developer), thus enabling users to focus on the task of a single application and / or container without being hindered by the tasks of another application or container. In at least one embodiment, communication and collaboration between different containers or applications can be supported by the pipeline manager 1312 and the application orchestration system 1328.In at least one embodiment, where an expected input and / or output of each container or application is known by a system (for example, based on constructs of applications or containers), the application orchestration system 1328 and / or the pipeline manager 1312 can facilitate communication between the individual applications or containers and the sharing of resources between them. In at least one embodiment, since one or more applications or containers in the one or more delivery pipelines 1310 can share the same services and resources, the application orchestration system 1328 can orchestrate, utilize, and determine the sharing of services or resources between and among the different applications or containers.In at least one embodiment, a scheduler can be used to track the resource requirements of applications or containers, the current or planned use of those resources, and resource availability. In at least one embodiment, a scheduler can thus allocate resources to different applications and distribute resources between and among applications with respect to system requirements and availability. In some examples, a scheduler (and / or another component of the Application Orchestration System 1328) can determine resource availability and distribution based on constraints imposed on a system (e.g., user constraints), such as Quality of Service (QoS), urgency of data output needs (e.g., to determine whether processing should be performed in real time or with a delay), and so on.

[0143] In at least one embodiment, the services 1220, which are used by applications or containers in the deployment system 1206, can include compute services 1316, AI services 1318, visualization services 1320, and / or other service types. In at least one embodiment, applications can call (e.g., execute) one or more of the services 1220 to perform processing operations for an application. In at least one embodiment, the compute services 1316 can be used by applications to perform supercomputing or other high-performance computing (HPC) tasks. In at least one embodiment, the one or more compute services 1316 can be used to perform parallel processing (e.g., with a parallel computing platform 1330) to process data by one or more applications and / or one or more tasks of a single application substantially simultaneously.In at least one embodiment, the parallel computing platform 1330 (e.g., NVIDIA's CUDA) can enable general-purpose computing on GPUs (GPGPU) (e.g., GPUs 1322). In at least one embodiment, a software layer of the parallel computing platform 1330 can enable access to virtual instruction sets and parallel computing elements of GPUs to execute computing cores. In at least one embodiment, the parallel computing platform 1330 can include main memory, and in some embodiments, main memory can be shared between multiple containers and / or between different processing tasks within a single container.In at least one embodiment, inter-process communication (IPC) calls can be generated for multiple containers and / or for multiple processes within a container to use the same data from a shared memory segment of the Parallel Platform 1330 (e.g., when several different stages of an application or multiple applications process the same information). In at least one embodiment, the same data can be used at the same location in memory for any number of processing tasks (e.g., at the same time, at different times, etc.), instead of creating a copy of the data and moving the data to different locations in memory (e.g., a read / write operation).In at least one embodiment, when data is used to generate new data as a result of processing, this information can be stored in a new data location and exchanged between different applications. In at least one embodiment, the data location and the location of updated or modified data can be part of a definition of how a payload is to be understood within containers.

[0144] In at least one embodiment, AI services 1318 can be used to perform inference services for the execution of machine learning models associated with applications (e.g., with the task of performing one or more processing tasks of an application). In at least one embodiment, the AI ​​services 1318 can utilize the AI ​​system 1324 to execute one or more machine learning models (e.g., neural networks such as CNNs) for segmentation, reconstruction, object detection, feature detection, classification, and / or other inference tasks. In at least one embodiment, the applications of the one or more deployment pipelines 1310 can use one or more output models 1216 of the training system 1204 and / or other application models to draw inferences from imaging data.In at least one embodiment, two or more examples of inference using the application orchestration system 1328 (e.g., a scheduler) may be available. In at least one embodiment, a first category may contain a high-priority / low-latency path that can achieve higher service-level agreements, e.g., for performing inferences on urgent requests during an emergency or for a radiologist during diagnosis. In at least one embodiment, a second category may contain a standard-priority path that can be used for requests that are not urgent or where the analysis can be performed at a later time. In at least one embodiment, the application orchestration system 1328 may distribute resources (e.g., services 1220 and / or hardware 1222) based on priority paths for different inference tasks of AI services 1318.

[0145] In at least one embodiment, shared memory can be provided for the AI ​​services 1318 within the system 1300. In at least one embodiment, the shared memory can function as a cache (or another type of memory device) and be used to process inference requests from applications. In at least one embodiment, when an inference is submitted, it can be received by a number of API instances of the provisioning system 1206, and one or more instances can be selected (e.g., for best fit, load balancing, etc.) to process a request.In at least one embodiment, a query can be entered into a database to process a query; a machine learning model can be found from the model registry 1224 if it is not already in a cache; a validation step can ensure that a suitable machine learning model is loaded into a cache (e.g., shared memory); and / or a copy of a model can be stored in a cache. In at least one embodiment, a scheduler (e.g., the pipeline manager 1312) can be used to initiate an application referenced in a query if an application is not already running or if there are not enough instances of an application. In at least one embodiment, an inference server can be started if no inference server has yet been started to execute a model.Any number of inference servers can be started per model. In at least one embodiment, models can be cached in a pull model where inference servers are clustered, if load balancing is advantageous. In at least one embodiment, inference servers can be statically loaded onto corresponding distributed servers.

[0146] In at least one embodiment, inference can be performed using an inference server running in a container. In at least one embodiment, an instance of an inference server can be associated with a model (and optionally with multiple versions of a model). In at least one embodiment, a new instance can be loaded if an instance of an inference server does not exist when a request to perform inferences on a model is received. In at least one embodiment, a model can be passed to an inference server when the inference server is started, so that the same container can be used to handle different models, as long as the inference server is running as a separate instance.

[0147] In at least one embodiment, an inference request for a specific application can be received during the execution of that application, and a container (e.g., hosting an instance of an inference server) can be loaded (if not already loaded), and a start procedure can be invoked. In at least one embodiment, the preprocessing logic within a container can load, decode, and / or perform additional preprocessing on the incoming data (e.g., using CPU(s) and / or GPU(s) and / or DPU(s)). In at least one embodiment, once the data is prepared for inference, a container can perform the necessary inference on the data. In at least one embodiment, this can involve a single inference call for a single image (e.g., an X-ray of the hand) or require inference on hundreds of images (e.g., a CT scan of the chest).In at least one embodiment, an application can summarize the results before completion, which can include, without limitation, a single confidence score, pixel-level segmentation, voxel-level segmentation, the creation of a visualization, or the generation of text summarizing the results. In at least one embodiment, different models or applications can be assigned different priorities. For example, some models can have a real-time priority (TAT < 1 min), while others can have a lower priority (e.g., TAT < 12 min). In at least one embodiment, the model's execution time can be measured from the requesting institution or entity and can include the time spent traversing the partner network as well as execution in an inference service.

[0148] In at least one embodiment, the transmission of requests between the 1220 services and the inference applications can be hidden behind a Software Development Kit (SDK), and robust transport can occur via a queue. In at least one embodiment, a request is queued via an API for a specific application / tenant ID combination, and an SDK pulls a request from the queue and forwards it to an application. In at least one embodiment, a queue name can be specified in an environment from which an SDK retrieves it. In at least one embodiment, asynchronous communication via a queue can be beneficial because it allows each instance of an application to begin work as soon as it is available. Results can be transferred back via a queue to ensure that no data is lost.In at least one embodiment, queues can also provide the ability to segment work, since the highest-priority work can go to a queue with the most connected instances of an application, while the lowest-priority work can go to a queue with a single connected instance, which processes the tasks in the order they are received. In at least one embodiment, an application can run on a GPU-accelerated instance generated in Cloud 1326, and an inference service can perform inferences on a GPU.

[0149] In at least one embodiment, visualization services 1320 can be used to generate visualizations for displaying the outputs of applications and / or the delivery pipeline(s) 1310. In at least one embodiment, GPUs 1322 can be used by visualization services 1320 to generate visualizations. In at least one embodiment, rendering effects, such as ray tracing, can be implemented by visualization services 1320 to generate higher-quality visualizations. In at least one embodiment, the visualizations can include, without limitation, 2D image renderings, 3D volume renderings, 3D volume reconstructions, 2D tomography layers, virtual reality displays, augmented reality displays, etc. In at least one embodiment, virtualized environments can be used to provide a virtual interactive display or environment (e.g.,to create a virtual environment for interaction by users of a system (e.g., doctors, nurses, radiologists, etc.). In at least one embodiment, the visualization services 1320 may include an internal visualizer, kinematics, and / or other rendering or image processing capabilities or functions (e.g., ray tracing, rasterization, internal optics, etc.).

[0150] In at least one embodiment, the hardware 1222 may include GPUs 1322, the AI ​​system 1324, the cloud 1326, and / or any other hardware used to run the training system 1204 and / or the deployment system 1606. In at least one embodiment, GPUs 1322 (e.g., NVIDIA's TESLA and / or QUADRO GPUs) may include any number of GPUs that can be used to perform processing tasks of compute services 1316, AI services 1318, visualization services 1320, other services, and / or any features or functions of the software 1218. With regard to AI services 1318, GPUs 1322 can be used, for example, to perform preprocessing of imaging data (or other data types used by machine learning models), postprocessing of machine learning model outputs, and / or inference (e.g., to run machine learning models).In at least one embodiment, the Cloud 1326, the AI ​​system 1324, and / or other components of the System 1300 can utilize GPUs 1322. In at least one embodiment, the Cloud 1326 can include a GPU-optimized platform for deep learning tasks. In at least one embodiment, the AI ​​system 1324 can utilize GPUs, and the Cloud 1326—or at least a portion thereof tasked with deep learning or inferencing—can be run using one or more AI systems 1324. Although the Hardware 1222 is illustrated as discrete components, this is not intended as a limitation, and all components of the Hardware 1222 can be combined with or utilized by other components of the Hardware 1222.

[0151] In at least one embodiment, the AI ​​system 1324 may include a specially designed computing system (e.g., a supercomputer or an HPC) configured for inferencing, deep learning, machine learning, and / or other artificial intelligence tasks. In at least one embodiment, the AI ​​system 1324 (e.g., NVIDIA's DGX) may include GPU-optimized software (e.g., a software stack) that can run on a variety of GPUs 1322, in addition to DPUs, CPUs, RAM, memory, and / or other components, features, or functions. In at least one embodiment, one or more AI systems 1324 may be deployed in the cloud 1326 (e.g., in a data center) to perform some or all of the AI-based processing tasks of the system 1300.

[0152] In at least one embodiment, Cloud 1326 can include a GPU-accelerated infrastructure (e.g., NVIDIA's NGC) that provides a GPU-optimized platform for performing processing tasks of System 1300. In at least one embodiment, Cloud 1326 can include one or more AI systems 1324 to perform one or more AI-based tasks of System 1300 (e.g., as a hardware abstraction and scaling platform). In at least one embodiment, Cloud 1326 can be integrated with the application orchestration system 1328, which utilizes multiple GPUs to enable seamless scaling and load balancing between and among the applications and services 1220. In at least one embodiment, Cloud 1326 can be tasked with performing at least some of the services 1220 of System 1300, including the compute services 1316, the AI ​​services 1318, and / or the visualization services 1320, as described herein.In at least one embodiment, the Cloud 1326 can perform small and large batch inferences (e.g., NVIDIA's TENSOR RT), provide an accelerated parallel computing API and platform 1330 (e.g., NVIDIA's CUDA), run an application orchestration system 1328 (e.g., KUBERNETES), provide a graphics rendering API and platform (e.g., for ray tracing, 2D graphics, 3D graphics, and / or other rendering techniques to produce higher-quality cinematic content), and / or provide other functions for the System 1300.

[0153] Fig. Figure 14A illustrates a data flow diagram for a process 1400 for training, retraining, or updating a machine learning model according to at least one embodiment. In at least one embodiment, the process 1400 can be combined with the system 1300 as a non-limiting example. Fig. 13. In at least one embodiment, the process 1400 can utilize the services 1220 and / or hardware 1222 of the system 1300 described herein. In at least one embodiment, the refined models 1412 generated by process 1400 can be executed by deployment system 1206 for one or more containerized applications in deployment pipelines 1310.

[0154] In at least one embodiment, the model training 1214 can include retraining or updating an initial model 1404 (e.g., a pre-trained model) using new training data (e.g., new input data, such as the customer data set 1406, and / or new truth data associated with the input data). In at least one embodiment, retraining or updating the initial model 1404 can involve resetting or deleting one or more output or loss layers of the initial model 1404 and / or replacing them with one or more updated or new output or loss layers. In at least one embodiment, the initial model 1404 can already have fine-tuned parameters (e.g., weights and / or distortions) left over from previous training, so that the training or retraining 1214 does not take as long or require as much computation as training a new model.In at least one embodiment, during model training 1214, the parameters can be updated and retuned for a new data set based on loss calculations that correspond to the accuracy of the one or more output or loss layers when generating predictions for a new customer data set 1406 (e.g., the image data 1208 from . Fig. 12) are assigned.

[0155] In at least one embodiment, the one or more pre-trained models 1306 can be stored in a data storage device or a registry (e.g., model registry 1224 of Fig. 12) are stored. In at least one embodiment, the pre-trained models 1306 may have been trained at least partially in one or more facilities other than the facility performing the process 1400. To protect the privacy and rights of patients, subjects, or customers of different facilities, in at least one embodiment the pre-trained models 1306 may have been trained on-site using customer or patient data generated on-site. In at least one embodiment the pre-trained models 1306 may be trained using the cloud 1326 and / or other hardware 1222, but confidential, privacy-protected patient data must not be transferred to, used by, or accessible to components of the cloud 1326 (or other off-site hardware).In at least one embodiment, where a pre-trained Model 1306 is trained with patient data from more than one facility, the pre-trained Model 1306 may have been individually trained for each facility before being trained with patient or customer data from another facility. In at least one embodiment, for example, where customer or patient data has been released for privacy reasons (e.g., by a waiver, for experimental purposes, etc.), or where customer or patient data is included in a public dataset, customer or patient data from any number of facilities may be used to train the pre-trained Model 1306 on-premises and / or off-premises, for example, in a data center or other cloud computing infrastructure.

[0156] In at least one embodiment, when selecting applications for use in deployment pipelines 1310, a user can also select machine learning models to be used for specific applications. In at least one embodiment, a user may not have a model to use, so a user can select a pre-trained model 1306 for use with an application. In at least one embodiment, the pre-trained model 1306 may not be optimized to produce accurate results for a user's facility's customer dataset 1406 (e.g., based on patient diversity, demographics, the type of medical imaging equipment used, etc.).In at least one embodiment, prior to the provision of a pre-trained model 1306 in a provisioning pipeline 1310 for use with one or more applications, the pre-trained model 1306 can be updated, retrained and / or fine-tuned for use in a suitable facility.

[0157] In at least one embodiment, a user can select the pre-trained model 1306 to be updated, retrained, and / or fine-tuned, and the pre-trained model 1306 can be designated as the initial model 1404 for a training system 1204 in process 1400. In at least one embodiment, a customer dataset 1406 (e.g., imaging data, genomic data, sequencing data, or other data types generated by devices in a facility) can be used to perform model training 1214 (which may include transfer learning without restriction) on the initial model 1404 to produce a refined model 1412. In at least one embodiment, the training system 1204 can generate basic truth data corresponding to the customer dataset 1406.In at least one embodiment, the basic truth data can be generated at least partially by clinicians, scientists, physicians, practitioners in an institution (e.g. as labeled clinical data 1212 in . Fig. 12).

[0158] In at least one embodiment, the AI-powered annotation 1210 can be used in some examples to generate basic truth data. In at least one embodiment, the AI-powered annotation 1210 (e.g., implemented with an AI-powered annotation SDK) can utilize machine learning models (e.g., neural networks) to generate suggested or predicted basic truth data for a customer dataset. In at least one embodiment, a user can use annotation tools 1410 within a user interface (a graphical user interface (GUI)) on the computer device 1408.

[0159] In at least one embodiment, the user 1410 can interact with a GUI via the computer device 1408 to edit or fine-tune (auto-)annotations. In at least one embodiment, a polygon editing function can be used to move vertices of a polygon to more precise or refined positions.

[0160] In at least one embodiment, once basic truth data is associated with the customer dataset 1406, basic truth data (e.g., from AI-assisted annotation, manual labeling, etc.) can be used during model training 1214 to generate a refined model 1412. In at least one embodiment, the customer dataset 1406 can be applied to the initial model 1404 any number of times, and the basic truth data can be used to update the parameters of the initial model 1404 until an acceptable level of accuracy is achieved for the refined model 1412. In at least one embodiment, once generated, the refined model 1412 can be deployed in one or more deployment pipelines 1210 in a facility to perform one or more processing tasks related to medical imaging data.

[0161] In at least one embodiment, the refined model 1412 can be uploaded to a model registry 1224 alongside the already pre-trained models 1206, so that it can be selected by another facility. In at least one embodiment, this process can be carried out in any number of facilities, so that the refined model 1412 can be further refined any number of times on new datasets to create a more universal model.

[0162] Fig. Figure 14B is an exemplary illustration of a client-server architecture 1432 for improving annotation tools with pre-trained annotation models, according to at least one embodiment. In at least one embodiment, the AI-supported annotation tools 1436 can be instantiated based on a client-server architecture 1432. In at least one embodiment, annotation tools 1436 can, for example, help radiologists in imaging applications to identify organs and anomalies. In at least one embodiment, imaging applications can include software tools that help the user 1410, as a non-restrictive example, to identify a few extreme points on a particular organ of interest in the raw images 1434 (e.g., in a 3D MRI or CT scan) and to obtain automatically annotated results for all 2D slices of a particular organ.In at least one embodiment, the results can be stored in a data store as training data 1438 and used (for example, and without limitation) as basic truth data for training. In at least one embodiment, when the computer device 1408 sends extreme points for AI-assisted annotation 1210, a deep learning model can receive this data as input and return inference results of a segmented organ or abnormality. In at least one embodiment, pre-instantized annotation tools, such as the AI-assisted annotation tool 1436B in , can be used. Fig.14B, can be extended to a server, such as an annotation assistant server 1440, via API calls (e.g., API call 1444), which may contain a set of pre-trained models 1442 stored, for example, in an annotation model registry. In at least one embodiment, an annotation model registry can store pre-trained models 1442 (e.g., machine learning models, such as deep learning models) that are pre-trained to perform AI-assisted annotation for a specific organ or anomaly. These models can be further updated using training pipelines 1304. In at least one embodiment, the pre-installed annotation tools can be improved over time as new labeled clinical data 1212 are added.

[0163] The disclosure of this application also includes the following numbered clauses: Clause 1. Procedure comprising the following: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units performs a first set of operations through a first virtual processor, and a second processing unit of the plurality of processing units performs a second set of operations through a second virtual processor; To cause the execution of the first set of operations by the first virtual processor to be transferred from the first processing unit to the second processing unit; Initiating the execution of the test process on the first processing unit, while the second processing unit executes the first set of operations through the first virtual processor and the second set of operations through the second virtual processor; In response to the detection that the execution of the test process on the first processing unit is complete, cause the execution of the first set of operations by the first virtual processor and the second set of operations by the second virtual processor to be transferred from the second processing unit to the first processing unit; and Initiating the execution of the test process on the second processing unit, while the first processing unit performs the first set of operations through the first virtual processor and the second set of operations through the second virtual processor. Clause 2. Method according to Clause 1, wherein a third processing unit of the plurality of processing units hosts a third virtual processor which performs a third set of operations, and wherein the method further comprises: In response to the detection that the execution of the test process on the second processing unit is complete, cause the execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating the execution of the test process on the third processing unit while the first virtual processor is performing at least one of the first or second sets of operations and the second virtual processor is performing the third set of operations. Clause 3. Procedure according to Clause 1 or Clause 2, wherein the test process is a fault detection process, and wherein the procedure further comprises: determining, based on a performance of the test process of at least one processing unit of the plurality of processing units, whether a fault has occurred in the at least one processing unit; and in response to the determination that a fault has occurred in the at least one processing unit, transmitting a warning indicating the fault. Clause 4. Method according to any of the preceding clauses, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units. Clause 5. Method according to Clause 4, wherein the plurality of processing units further comprises a fourth processing unit and a fifth processing unit which are components of a second processor core of the system, wherein the fourth processing unit performs a fourth set of operations assigned to a fourth virtual processor, and the fifth processing unit performs a fifth set of operations assigned to a fifth virtual processor, and wherein the method further comprises: In response to the request to initiate the test process, cause the execution of the fourth set of operations to be transferred from the fourth virtual processor to the fifth virtual processor; and Initiating the execution of the test process on the fourth processing unit, while the test process is simultaneously being carried out using at least one of the first processing unit or the second processing unit of the first processor core. Clause 6. Method according to any of the preceding clauses, wherein the first virtual processor and the second virtual processor are assigned to a virtual computer system running on the plurality of processing units. Clause 7. Procedure according to Clause 6, wherein the virtual computer system comprises at least one virtual machine or container. Clause 8. A procedure according to any of the preceding clauses, wherein initiating the execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor comprises sending a first instruction to a virtual system manager associated with the first virtual processor and the second virtual processor to transfer the execution of the first set of operations from the first virtual processor to the second virtual processor, and wherein initiating the execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor includes sending a second instruction to the virtual system manager to transfer the execution of the first set of operations and the second set of operations from the first virtual processor to the second virtual processor. Clause 9. Procedure according to one of the preceding clauses, wherein the plurality of processing units comprises at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for conducting digital twinning operations; a system for performing light transport simulations; a system for conducting collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system that is implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system for presenting content from virtual reality, augmented reality or mixed reality; a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources. Clause 10. System, which includes the following: a processing device for performing operations, comprising the following: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units comprises a first virtual processor that performs a first set of operations, and a second processing unit of the plurality of processing units comprises a second virtual processor that performs a second set of operations; To cause the execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor; Initiating the execution of the test process on the first processing unit, while the second virtual processor, running on the second processing unit, executes the first set of operations and the second set of operations; In response to the detection that the execution of the test process on the first processing unit is complete, cause the execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor; and Initiating the execution of the test process on the second processing unit, while the first virtual processor running on the first processing unit performs the first set of operations and the second set of operations. Clause 11. System according to Clause 10, wherein a third processing unit of the plurality of processing units hosts a third virtual processor which performs a third set of operations, and wherein the operations further comprise: In response to the detection that the execution of the test process on the second processing unit is complete, cause the execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating the execution of the test process on the third processing unit while the first virtual processor is performing at least one of the first or second sets of operations and the second virtual processor is performing the third set of operations. Clause 12. System according to Clause 10 or Clause 11, wherein the test process is a fault detection process, and wherein the operations further comprise: determining, based on a performance of the test process of at least one processing unit of the plurality of processing units, whether a fault has occurred in the at least one processing unit; and in response to the determination that a fault has occurred in the at least one processing unit, sending a warning indicating the fault. Clause 13. System according to any of Clauses 10-12, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units. Clause 14. System according to Clause 13, wherein the plurality of processing units further comprises a fourth processing unit and a fifth processing unit, which are components of a second processor core of the system, wherein the fourth processing unit performs a fourth set of operations assigned to a fourth virtual processor, and the fifth processing unit performs a fifth set of operations assigned to a fifth virtual processor, and wherein the operations further comprise: In response to the request to initiate the test process, cause the execution of the fourth set of operations to be transferred from the fourth virtual processor to the fifth virtual processor; and Initiating the execution of the test process on the fourth processing unit, while the test process is simultaneously being carried out using at least one of the first processing unit or the second processing unit of the first processor core. Clause 15. System according to any of Clauses 10-14, wherein the first virtual processor and the second virtual processor are assigned to a virtual computer system running on the plurality of processing units. Clause 16. System according to Clause 15, wherein the virtual computer system comprises at least one virtual machine or container. Clause 17. Processor comprising the following: a large number of processing units; and a controller for the following: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units comprises a first virtual processor that performs a first set of operations, and a second processing unit of the plurality of processing units comprises a second virtual processor that performs a second set of operations; To cause the execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor; Initiating the execution of the test process on the first processing unit, while the second virtual processor, running on the second processing unit, executes the first set of operations and the second set of operations; In response to the detection that the execution of the test process on the first processing unit is complete, cause the execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor; and Initiating the execution of the test process on the second processing unit, while the first virtual processor running on the first processing unit performs the first set of operations and the second set of operations. Clause 18. Processor according to Clause 17, wherein a third processing unit of the plurality of processing units hosts a third virtual processor which performs a third set of operations, and wherein the controller further performs the following: In response to the detection that the execution of the test process on the second processing unit is complete, cause the execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating the execution of the test process on the third processing unit while the first virtual processor is performing at least one of the first or second sets of operations and the second virtual processor is performing the third set of operations. Clause 19. Processor according to Clause 18, wherein the test process is a fault detection process, and wherein the controller further performs the following: Determine, based on the performance of the test process of at least one processing unit of the plurality of processing units, whether an error has occurred in the at least one processing unit; and in response to the determination that an error has occurred in the at least one processing unit, send a warning indicating the error. Clause 20. Processor according to Clause 18 or Clause 19, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units.

[0164] Such components can be used to generate synthetic data that mimic error cases in a network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0165] Other variations are within the scope of this disclosure. While the disclosed techniques are suitable for various modifications and alternative designs, certain illustrated embodiments are shown in the drawings and have been described in detail above. It should be noted, however, that the disclosure is not intended to be limited to any particular form or forms, but rather, on the contrary, to cover all modifications, alternative designs, and equivalents that are within the scope and meaning of the disclosure, as defined in the accompanying claims.

[0166] The use of the terms "a" and "an" and "the" and similar references in connection with the description of disclosed embodiments (particularly in connection with the following claims) is to be interpreted as covering both the singular and the plural, unless otherwise specified herein or clearly contradicted by the context, and not as a definition of a term. The terms "comprise," "have," "including," and "contain" are to be interpreted as open terms (i.e., "including but not limited to"), unless otherwise specified. "Connected," when used unchanged and referring to physical connections, is to be interpreted as being partially or completely contained in, attached to, or connected to something, even if something is in between.The enumeration of value ranges in this document serves only as a shorthand for individual reference to each value falling within the range, unless otherwise specified in this document, and each individual value shall be incorporated into the specification as if it were listed here individually. In at least one embodiment, the use of the term "set" (e.g., "a set of elements") or "subset," unless otherwise specified or contradicted by the context, is to be understood as a non-empty collection comprising one or more members. Furthermore, the term "subset" of a corresponding set does not necessarily denote a proper subset of the corresponding set unless otherwise specified or contradicted by the context; however, "subset" and "corresponding set" may be the same.

[0167] Subjunctive language, such as phrases of the form "at least one of A, B, and C" or "at least one of A, B, and C," is, unless expressly stated otherwise or clearly contradicted by the context, understood in the context as it is generally used to indicate that an element, term, etc., can be either A, B, or C, or a non-empty subset of the set of A, B, and C. For example, in an illustrative example of a set with three elements, the subjunctive phrases "at least one of A, B, and C" and "at least one of A, B, and C" refer to one of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such subjunctive language is generally not intended to imply that certain embodiments require at least one of A, at least one of B, and at least one of C. Furthermore, the term "plurality" denotes a state of plurality (e.g.“A plurality of items” means multiple items unless otherwise stated or contradicted by the context. In at least one embodiment, the number of items in a plurality is at least two, but may be higher if either explicitly stated or indicated by the context. Furthermore, unless otherwise stated or evident from the context, the phrase “based on” means “at least partly based on” and not “exclusively based on”.

[0168] The processes described herein may be executed in any order unless otherwise specified herein or clearly indicated otherwise by the context. In at least one embodiment, a process such as the processes described herein (or variations and / or combinations thereof) is executed under the control of one or more computer systems configured with executable instructions and is implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or combinations thereof. In at least one embodiment, the code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions that can be executed by one or more processors.In at least one embodiment, a computer-readable storage medium is a non-volatile computer-readable storage medium that excludes volatile signals (e.g., a propagating transient electrical or electromagnetic transmission) but includes non-volatile data storage circuitry (e.g., buffers, caches, and queues) in transceivers of volatile signals. In at least one embodiment, code (e.g., executable code or source code) is stored on a set of one or more non-volatile computer-readable storage media containing executable instructions (or other working memory for storing executable instructions) which, when executed (i.e., as a result of execution) by one or more processors of a computer system, cause the computer system to perform the operations described herein.In at least one embodiment, a set of non-volatile, computer-readable storage media comprises several non-volatile, computer-readable storage media, and one or more individual non-volatile storage media lack the entire code, while several non-volatile, computer-readable storage media collectively store the entire code. In at least one embodiment, executable instructions are executed such that different instructions are executed by different processors.

[0169] Accordingly, computer systems in at least one embodiment are configured to implement one or more services that individually or collectively perform the operations of processes described herein, and such computer systems are configured with appropriate hardware and / or software that enables the execution of operations. Furthermore, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, it is a distributed computer system comprising several devices that operate differently, such that the distributed computer system performs the operations described herein, and a single device does not perform all operations.

[0170] The use of examples or illustrative language (e.g., "like") in this document is solely for the purpose of better illustrating the embodiments of the disclosure and does not constitute a limitation of the scope of the disclosure unless otherwise stated. No wording in the description should be interpreted as indicating that an unclaimed element is essential for exercising the disclosure.

[0171] All references contained herein, including publications, patent applications and patents, are hereby incorporated by reference to the same extent as if each reference had been individually and specifically indicated as incorporated by reference and set forth herein in its entirety.

[0172] In the description and claims, the terms "coupled" and "connected," along with their derivatives, may be used. It is understood that these terms are not synonymous. Rather, in certain examples, "connected" or "coupled" may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" may also mean that two or more elements are not in direct contact with each other but nevertheless cooperate or interact.

[0173] Unless expressly stated otherwise, it is hereby noted that terms such as "processing", "computing", "calculating", "determining" or similar terms throughout this specification refer to actions and / or processes of a computer or computer system or similar electronic computing device that manipulate and / or convert data represented as physical, e.g. electronic, quantities within the registers and / or memories of the computer system into other data represented in a similar manner as physical quantities within the memories, registers or other such devices for storing, transmitting or displaying information of the computer system.

[0174] Similarly, the term "processor" can refer to any device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. A "computer platform" can include one or more processors. As used here, "software" processes can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Furthermore, each process can refer to multiple processes for executing instructions sequentially or in parallel, continuously or intermittently.In at least one embodiment, the terms “system” and “method” are used interchangeably here, insofar as the system may contain one or more methods and methods may be considered as a system.

[0175] This document may refer to the acquisition, obtaining, receiving, or inputting of analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data may be performed in various ways, for example, by receiving data as parameters of a function call or a call to an application programming interface. In at least one embodiment, processes for obtaining, acquiring, receiving, or inputting analog or digital data may be performed by transmitting data over a serial or parallel interface.In at least one embodiment, processes for obtaining, acquiring, receiving, or inputting analog or digital data can be performed by transmitting data over a computer network from the providing entity to the acquiring entity. In at least one embodiment, reference can also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, processes for providing, outputting, transmitting, sending, or presenting analog or digital data can be performed by transmitting data as input or output parameters of a function call, as parameters of an application programming interface, or as a mechanism for communication between processes.

[0176] Although the descriptions contained herein present exemplary embodiments of the described techniques, other architectures can be used to implement the described functionality and are included within the scope of this disclosure. Furthermore, although specific distributions of responsibilities may be defined above for the purpose of description, various functions and responsibilities may be distributed and divided in different ways depending on the circumstances.

[0177] Although the subject matter was described in language relating to structural features and / or methodological actions, it should be understood that the subject matter claimed in the attached claims is not necessarily limited to certain described features or actions. Rather, certain features and actions are disclosed as exemplary forms of implementing the claims.

[0178] It is understood that the aspects and embodiments described above are purely exemplary and that changes in detail can be made within the scope of the claims.

[0179] Each device, method and feature disclosed in the description and (optionally) in the claims and drawings may be represented independently or in any suitable combination.

[0180] Reference numerals in the claims are for illustrative purposes only and do not restrict the scope of the claims.

Claims

[1] Procedure comprising: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units executes a first set of operations through a first virtual processor and a second processing unit of the plurality of processing units executes a second set of operations through a second virtual processor; Causing execution of the first set of operations by the first virtual processor to be transferred from the first processing unit to the second processing unit; initiating execution of the test process at the first processing unit while the second processing unit executes the first set of operations through the first virtual processor and the second set of operations through the second virtual processor; in response to detecting that execution of the test process at the first processing unit is complete, causing execution of the first set of operations by the first virtual processor and the second set of operations by the second virtual processor to be transferred from the second processing unit to the first processing unit; and Initiating execution of the test process at the second processing unit while the first processing unit executes the first set of operations through the first virtual processor and the second set of operations through the second virtual processor. [2] The method of claim 1, wherein a third processing unit of the plurality of processing units hosts a third virtual processor that performs a third set of operations, and wherein the method further comprises: in response to detecting that execution of the test process at the second processing unit is complete, causing execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating execution of the test process at the third processing unit while the first virtual processor is executing at least one of the first or second sets of operations and the second virtual processor is executing the third set of operations. [3] A method according to claim 1 or claim 2, wherein the testing process is a fault detection process, and wherein the method further comprises: Determining, based on a performance of the test process of at least one processing unit of the plurality of processing units, whether an error has occurred in the at least one processing unit; and in response to determining that an error has occurred in the at least one processing unit, transmitting an alert indicating the error. [4] The method of any preceding claim, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units. [5] The method of claim 4, wherein the plurality of processing units further comprises a fourth processing unit and a fifth processing unit that are components of a second processor core of the system, the fourth processing unit executing a fourth set of operations associated with a fourth virtual processor and the fifth processing unit executing a fifth set of operations associated with a fifth virtual processor, and the method further comprising: in response to the request to initiate the test process, causing execution of the fourth set of operations to be transferred from the fourth virtual processor to the fifth virtual processor; and Initiating execution of the test process on the fourth processing unit while simultaneously performing the test process using at least one of the first processing unit or the second processing unit of the first processor core. [6] The method of any preceding claim, wherein the first virtual processor and the second virtual processor are associated with a virtual computer system executing on the plurality of processing units. [7] The method of claim 6, wherein the virtual computer system comprises at least one virtual machine or container. [8] The method of any preceding claim, wherein causing execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor comprises sending a first instruction to a virtual system manager associated with the first virtual processor and the second virtual processor to transfer execution of the first set of operations from the first virtual processor to the second virtual processor, and wherein causing execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor comprises sending a second instruction to the virtual system manager to transfer execution of the first set of operations and the second set of operations from the first virtual processor to the second virtual processor. [9] A method according to any one of the preceding claims, wherein the plurality of processing units comprises at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for conducting digital twinning operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system for presenting virtual reality, augmented reality or mixed reality content; a system that is at least partially implemented in a data center; or a system implemented at least in part using cloud computing resources. [10] System comprising: a processing device for performing operations comprising: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units comprises a first virtual processor executing a first set of operations and a second processing unit of the plurality of processing units comprises a second virtual processor executing a second set of operations; causing execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor; initiating execution of the test process on the first processing unit while the second virtual processor running on the second processing unit executes the first set of operations and the second set of operations; in response to detecting that execution of the test process at the first processing unit is complete, causing execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor; and Initiating execution of the test process on the second processing unit while the first virtual processor running on the first processing unit executes the first set of operations and the second set of operations. [11] The system of claim 10, wherein a third processing unit of the plurality of processing units hosts a third virtual processor that performs a third set of operations, and wherein the operations further comprise: in response to detecting that execution of the test process at the second processing unit is complete, causing execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating execution of the test process at the third processing unit while the first virtual processor is executing at least one of the first or second sets of operations and the second virtual processor is executing the third set of operations. [12] The system of claim 10 or claim 11, wherein the testing process is a fault detection process, and wherein the operations further comprise: Determining, based on a performance of the test process of at least one processing unit of the plurality of processing units, whether an error has occurred in the at least one processing unit; and in response to determining that an error has occurred in the at least one processing unit, sending an alert indicating the error. [13] The system of any of claims 10-12, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units. [14] The system of claim 13, wherein the plurality of processing units further comprises a fourth processing unit and a fifth processing unit that are components of a second processor core of the system, the fourth processing unit performing a fourth set of operations associated with a fourth virtual processor and the fifth processing unit performing a fifth set of operations associated with a fifth virtual processor, and the operations further comprising: in response to the request to initiate the test process, causing execution of the fourth set of operations to be transferred from the fourth virtual processor to the fifth virtual processor; and Initiating execution of the test process on the fourth processing unit while simultaneously performing the test process using at least one of the first processing unit or the second processing unit of the first processor core. [15] The system of any of claims 10-14, wherein the first virtual processor and the second virtual processor are associated with a virtual computer system executing on the plurality of processing units. [16] The system of claim 15, wherein the virtual computer system comprises at least one virtual machine or container. [17] Processor comprising: a variety of processing units; and a controller for the following: Receiving a request to initiate a test process for each of a plurality of processing units, wherein a first processing unit of the plurality of processing units comprises a first virtual processor executing a first set of operations and a second processing unit of the plurality of processing units comprises a second virtual processor executing a second set of operations; causing execution of the first set of operations to be transferred from the first virtual processor to the second virtual processor; initiating execution of the test process on the first processing unit while the second virtual processor running on the second processing unit executes the first set of operations and the second set of operations; in response to detecting that execution of the test process at the first processing unit is complete, causing execution of the first set of operations and the second set of operations to be transferred from the second virtual processor to the first virtual processor; and Initiating execution of the test process on the second processing unit while the first virtual processor running on the first processing unit executes the first set of operations and the second set of operations. [18] The processor of claim 17, wherein a third processing unit of the plurality of processing units hosts a third virtual processor that performs a third set of operations, and wherein the controller further performs: in response to detecting that execution of the test process at the second processing unit is complete, causing execution of the third set of operations to be transferred from the third virtual processor to the second virtual processor; and Initiating execution of the test process at the third processing unit while the first virtual processor is executing at least one of the first or second sets of operations and the second virtual processor is executing the third set of operations. [19] The processor of claim 18, wherein the testing process is an error detection process, and wherein the controller further performs: Determining, based on a performance of the test process of at least one processing unit of the plurality of processing units, whether an error has occurred in the at least one processing unit; and in response to determining that an error has occurred in the at least one processing unit, sending an alert indicating the error. [20] The processor of claim 18 or claim 19, wherein the first processing unit and the second processing unit are components of a first processor core of a system comprising the plurality of processing units.

Images