Efficiently free up computing resources across memory contexts and processor pipelines

Abstract

Systems and techniques for efficiently releasing computing resources across memory contexts and processor pipelines are disclosed. These techniques include acquiring a first lock and providing a first task to be performed by a first processor. These techniques also include receiving a first signal from the first processor that the first lock can be released. These techniques further include providing the first signal to a second processor, and having the second processor release the first lock.

Description

Technical Field

[0001] At least one embodiment relates to a working node graph, and more specifically, to releasing a lock associated with a first node from a second node. Background Technology

[0002] Parallel processing units (such as graphics processing units) can execute multiple tasks in parallel. In some cases, the tasks to be performed by a parallel processing unit can be represented as a node graph. Each node can represent a unit of work to be executed on the parallel processing unit. Nodes may depend on other nodes, which can indicate the order of execution and control the data flow between nodes. Producer nodes can produce data for consumer nodes. In some cases, a node can be both a consumer node (e.g., using data generated by a predecessor node in the graph) and a producer node (e.g., producing data used by the next node in the graph). In some cases, data from producer nodes can be stored in queues associated with consumer nodes. In some cases, multiple producer nodes can store data in the same queue for a specific consumer node. In some cases, multiple consumer nodes can use data from the same queue. Attached Figure Description

[0003] Figure 1 This is a block diagram of an exemplary system for efficiently releasing computing resources across memory contexts and processor pipelines, according to at least one embodiment.

[0004] Figure 2 It is a block diagram of an exemplary graphical pipeline according to at least one embodiment.

[0005] Figure 3 This is a flowchart illustrating an example of releasing computing resources across a memory context according to at least one embodiment.

[0006] Figure 4 This is a flowchart of an exemplary method for efficiently releasing computing resources across memory contexts and processor pipelines according to at least one embodiment.

[0007] Figure 5 This is a block diagram illustrating an exemplary computer system according to at least one embodiment of the present disclosure.

[0008] Figure 6A The inference and / or training logic according to at least one embodiment of this disclosure is illustrated.

[0009] Figure 6B The inference and / or training logic according to at least one embodiment is illustrated.

[0010] Figure 7 Training and deployment of a neural network according to at least one embodiment are illustrated.

[0011] Figure 8 This is an example data flow diagram of an advanced computing pipeline according to at least one embodiment.

[0012] Figure 9 This is a system diagram of an example system for training, tuning, instantiating, and deploying machine learning models in an advanced computing pipeline, according to at least one embodiment. Detailed Implementation

[0013] Some nodes in the node graph can be executed using the computational pipeline of a parallel processing unit. Some nodes in the node graph can be executed using the graph pipeline of a parallel processing unit. The computational pipeline can execute in a first context with a first address space, and the graph pipeline can execute in a second context with a second address space.

[0014] In some embodiments, compute nodes (e.g., nodes executed using a compute pipeline) can generate data for use by graphics nodes. For example, a compute node can generate a record that includes graphics data and provide that record as input to the graphics node.

[0015] Before a compute node can generate an output record, it may need to acquire one or more locks (e.g., reference counts) to ensure that the output record has available resources (e.g., sufficient space in the output queue). In some embodiments, the lock is a reference count representing the number of "free" or "unused" entries in the queue. Acquiring a lock can decrease the reference count, resulting in fewer remaining "free" entries in the queue. Releasing a lock can increase the reference count, resulting in additional "free" entries appearing in the queue.

[0016] After acquiring one or more locks (e.g., after reducing the number of remaining reference counts), a compute node can generate output records and store them in a queue of a graph node (e.g., the output queue of a compute node can be the same as the input queue of a graph node).

[0017] A graph node can begin execution in the graph pipeline and can access data in records from the compute nodes. After accessing data in a record, a graph node can signal that one or more locks in the compute node are released. However, because the graph pipeline and the compute pipeline run in different contexts with different address spaces, releasing one or more locks from the graph context to the compute context can involve significant memory latency and reduce throughput.

[0018] In some cases, one or more semaphores can be used to indicate when a graph operation has finished accessing data in a compute node record. However, acquiring and releasing semaphores before releasing one or more locks on a compute node can introduce memory latency.

[0019] In some cases, a separate process can be initiated within the compute pipeline to monitor memory locations to indicate that a graph job has completed accessing data in a compute node record. For example, a graph node might be configured to write a value to a specific memory location after consuming data from a compute node record. This separate process can monitor (e.g., periodically poll) that specific memory location until it sees a value from the graph node. Once the graph node has written the value to that specific memory location, the separate process can release one or more locks on the compute node because it is executing within the compute pipeline context. However, both of these solutions introduce significant latency when accessing memory.

[0020] The aspects of this disclosure address the aforementioned and other shortcomings by providing a method for efficiently releasing computational resources across memory contexts and processor pipelines. In some embodiments, a producer node can acquire one or more locks (e.g., decrement one or more reference counts) to ensure that there are resources available for output records in the record queue. After acquiring one or more locks, the compute node can generate an output record and store it in a queue of the graph node. The graph node can begin execution in the graph pipeline and can access data in the records from the compute nodes. After the graph node has accessed the data in the record, it can signal that one or more locks on the compute node are released.

[0021] In some embodiments, the signal to the compute node is generated by a first processor executing the graphics pipeline. This signal can be provided to one or more processing circuits (e.g., cross-context resource release processing circuits) that can "forward" the signal to a second processor executing the compute pipeline. When the second processor receives the signal from the graphics pipeline (e.g., via one or more processing circuits), one or more locks previously acquired before generating an output record can be released.

[0022] Because the signal to release the lock travels from the graphics pipeline to the compute pipeline without passing through memory, the lock can be released quickly without incurring the memory latency associated with semaphore communication. Furthermore, no additional process is required to monitor memory locations. Therefore, a graphics pipeline running in the first memory context can cause a compute pipeline running in the second memory context to release resources (e.g., locks, reference counts, etc.).

[0023] Compared with the prior art, the advantages of the disclosed embodiments include, but are not limited to, efficient release of computing resources across memory contexts and processor pipelines, thereby improving computing resource utilization and throughput.

[0024] Figure 1 This is a block diagram of an exemplary system 102 for efficiently freeing up computing resources across memory contexts and processor pipelines according to at least one embodiment. System 102 may include one or more host processors 104 and host memory 106. Host processor 104 may include one or more central processing units (CPUs), graphics processing units (GPUs), accelerators, physical processing units (PPUs), data processing units (DPUs), etc. Host memory 106 may include read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), double data rate (DDRSDRAM), or DRAM (RDRAM), etc.

[0025] System 102 may further include a parallel processing unit 108, which may include a host interface 110, memory 112, cross-context resource release processing circuitry 114, one or more processors 118, and one or more processors 122. A host processor 104 may be connected to the parallel processing unit 108 via the host interface 110. In some embodiments, the host processor 104 is a CPU that provides one or more parallel processing tasks to the parallel processing unit 108 for execution via the host interface 110. In some embodiments, the parallel processing tasks provided to the parallel processing unit 108 are graphics rendering tasks. In some embodiments, the parallel processing tasks provided to the parallel processing unit 108 are computational tasks. In some embodiments, the parallel processing unit 108 performs a combination of graphics rendering tasks and computational tasks.

[0026] In some embodiments, task-related data may be stored in memory 112. In some embodiments, one or more tasks are provided to the execution pipeline via cross-context resource release processing circuitry 114. For example, cross-context resource release processing circuitry 114 may receive execution instructions from host interface 110 and assign work to graphics pipeline 116 and / or compute pipeline 120 based on the received instructions.

[0027] In some embodiments, the parallel processing unit 108 can be used to execute work defined in a node graph. Each node can represent a unit of work to be executed by processor 118 or processor 122. Nodes may depend on other nodes, which can indicate the order of execution and control the data flow between nodes. Producer nodes can produce data for consumer nodes to use. In some cases, data from producer nodes can be stored in queues associated with consumer nodes. The queues associated with each node can be stored in the memory 112 of the parallel processing unit 108.

[0028] In some embodiments, the work defined in the node graph is graphics rendering. In some embodiments, the work defined in the node graph is artificial intelligence and / or machine learning work, such as training an AI model and / or performing inference on the AI ​​model. About Figure 6A , Figure 6B , Figure 7 , Figure 8 and Figure 9 This allows for a more detailed description of the training and use of artificial intelligence models.

[0029] Some nodes in the node working graph can be executed using the computation pipeline 120 of the parallel processing unit 108. Some nodes in the node graph can be executed using the graphics pipeline 116 of the parallel processing unit 108. The computation pipeline 120 can be executed in a first context having a first address space, and the graphics pipeline 116 can be executed in a second context having a second address space.

[0030] In some embodiments, compute nodes (e.g., nodes executed using compute pipeline 120) can generate data for use by graphics nodes (e.g., nodes executed using graphics pipeline 116). For example, a compute node can generate a record containing graphics data and store that record in memory 112. A graphics node can execute in graphics pipeline 116 and access the graphics data of the record stored in memory 112. In some embodiments, a graphics node can only be a leaf node of a worker node graph.

[0031] Before a compute node can generate an output record, it may need to acquire one or more locks (e.g., reduce one or more reference counts) to ensure that the output record has available resources (e.g., there is enough space in the output queue). After acquiring one or more locks (e.g., after reducing the number of remaining reference counts), the compute node can generate the output record and store it in the graph node's queue (e.g., the compute node's output queue may be the same as the graph node's input queue).

[0032] A graph node can begin execution in the graph pipeline 116 and can access data in records from the compute nodes. After a graph node has accessed data in a record, it can signal that one or more locks in the compute nodes are released.

[0033] In some embodiments, a signal from a compute node is generated by a first processor (e.g., processor 118) executing the graphics pipeline 116. This signal may be provided to one or more processing circuits (e.g., cross-context resource release processing circuitry 114) that may "forward" the signal to a second processor, such as processor 122 executing the compute pipeline 120. When processor 122 in compute pipeline 120 receives a signal from processor 118 in graphics pipeline 116 (e.g., via cross-context resource release processing circuitry 114), one or more locks previously acquired before generating output records may be released.

[0034] After one or more locks are released, the compute node can acquire one or more locks again and can generate a second output record containing the data for a second graphics task to be rendered by the graphics pipeline 116. After processing the data in the second record, the graphics pipeline 116 can allow one or more locks in the compute pipeline 120 to be released again.

[0035] Figure 2 This is a block diagram of an exemplary graphics pipeline 202 according to at least one embodiment. The graphics pipeline 202 may include one or more subsystems, such as a world-space pipeline 206 and / or a screen-space pipeline 208. The graphics pipeline 202 may receive input 204, process input 204 using one or more subsystems of the graphics pipeline 202, and generate output 210. For example, input 204 may be provided to the world-space pipeline 206. The output of the world-space pipeline 206 may be provided to the screen-space pipeline 208, which may output the final output 210.

[0036] In some embodiments, the world space pipeline 206 may include multiple components for transforming and preparing vertex data for rendering. The world space pipeline can operate on vertex data contained in one or more coordinate systems. Vertex data in the same coordinate system may have a common origin.

[0037] World space pipeline 206 may include one or more components for vertex attribute acquisition (VAF) to load vertex data (e.g., position, normal, texture, coordinates, color, etc.) from memory. World space pipeline 206 may also include one or more components for vertex transformation and generation (VTG) to transform the loaded vertex data. In some embodiments, VTG components may include vertex shaders, tessellation shaders, geometry shaders, magnification shaders, and / or mesh shaders. For example, VTG components may transform vertex positions from their local object space to world space. World space pipeline 206 may also include components for vertex post-processing and pruning (VPC) to ensure that only the visible portions of the geometry are processed by further components. In some embodiments, multiple VPC components execute in parallel.

[0038] In some embodiments, the screen space pipeline 208 may include multiple components for transforming three-dimensional (3D) data from one or more world space coordinate systems into a two-dimensional (2D) image to be displayed on the screen. For example, the screen space pipeline 208 may include one or more components (e.g., "settings modules") for preparing geometric primitives to be rasterized by transforming, cropping, and / or mapping the geometric primitives to the viewport. The screen space pipeline 208 may also include one or more components for rasterization (e.g., "rasterization") to convert primitives into fragments and interpolate attributes. The screen space pipeline 208 may also include one or more components for the pixel shader (PS) to calculate the final color and effects for each fragment from the raster. The screen space pipeline 208 may also include one or more components for raster operations (ROPs) to perform depth testing, blending, stencil testing, and / or generate the final output. In some embodiments, multiple ROP components execute in parallel. In some embodiments, the output 210 is written to the frame buffer before display.

[0039] Figure 3This is a flowchart 300 illustrating an example of releasing computational resources across a memory context according to at least one embodiment. Flowchart 300 may begin at the start of a graphics pipeline. At block 302, input data may be processed in the world-space pipeline. After the world-space pipeline has been executed, at decision block 304, if data from a generated record from a compute node has been accessed (e.g., “consumed”), then at block 310, a signal may be sent to the compute node to release one or more locks associated with the generated record. For example, if the data in the record includes vertices and primitives, this data may be processed in the world-space pipeline but may not be needed in the screen-space pipeline. Therefore, the signal used to release one or more locks may be released before the screen-space pipeline in block 312 is executed. After the signal is provided, the output of the world-space pipeline may be processed by the screen-space pipeline at block 312.

[0040] If the data from the records generated by the compute node has not been fully consumed in decision box 304, then the data can be processed by the screen space pipeline in box 306. After the screen space pipeline completes execution, if a signal to release one or more locks has not been provided previously, that signal can be provided in box 308.

[0041] Therefore, in some cases, a signal to release one or more locks can be provided during the partial completion of the graphics pipeline, while in other cases, a signal to release one or more locks can be provided after the graphics pipeline is fully completed.

[0042] Figure 4 This is a flowchart of an exemplary method for efficiently releasing computing resources across memory contexts and processor pipelines according to at least one embodiment.

[0043] Method 400 can be executed using one or more processing units (e.g., CPU, GPU, accelerator, physical processing unit (PPU), data processing unit (DPU), etc.), which may include one or more memory devices (or communicate with them). In at least one embodiment, method 400 can be executed using one or more processing devices. In at least one embodiment, method 400 can be executed using... Figure 1The processing unit of system 102 executes method 400. In at least one embodiment, the processing unit executing method 400 may execute instructions stored on a non-transitory computer-readable storage medium. In at least one embodiment, multiple processing threads (e.g., CPU threads and / or GPU threads) may be used to execute method 400, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In at least one embodiment, the processing threads implementing method 400 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads implementing method 400 may execute asynchronously with each other. Figure 4 Compared to the order shown, the various operations of method 400 can be executed in a different order. Some operations of method 400 can be executed concurrently with other operations. In at least one embodiment, Figure 4 One or more of the operations shown may not always be performed.

[0044] Now for reference Figure 4 At block 402, the processing unit executing method 400 may acquire a first lock in a first memory context. For example, the first lock may be acquired in a computational pipeline executed in the first memory context. At block 404, the processing unit may provide a first task to be executed by a first processor. The first processor may execute in a second memory context. For example, the first processor may execute a graphics pipeline in the second memory context.

[0045] In some embodiments, at block 406, the processing unit can access first data associated with the first lock. For example, the first lock may be associated with an output record from a producer node in a worker node graph. Nodes executing in the graphics pipeline can access data stored in the output record. In some embodiments, the worker may be a graphics rendering job, and the data associated with the first lock may be graphics rendering data. For example, the data may include vertices and / or primitives to be rendered by the graphics pipeline.

[0046] At block 408, the processing unit may receive a first signal from the first processor indicating that the first lock can be released. In some embodiments, the first processor receives the first signal after partially completing the first task. For example, the first signal may be provided after the world space pipeline of the graphics pipeline has completed. In some embodiments, the first processor receives the first signal after fully completing the first task. For example, the first signal may be provided after both the world space pipeline and the screen space pipeline of the graphics pipeline have completed.

[0047] At block 410, the processing unit may provide a first signal to a second processor. The second processor may execute in a first memory context. For example, the second processor may execute a computational pipeline in the first memory context. At block 412, the processing unit may release the first lock via the second processor. In some embodiments, releasing the first lock is performed in response to receiving a plurality of signals. The plurality of signals may include the first signal. The plurality of signals may include signals from various components of the graphics pipeline. For example, a world-space pipeline may include a plurality of components executing in parallel, and releasing the first lock may be performed in response to receiving a signal from each of the components in the world-space pipeline. In some embodiments, a screen-space pipeline may include a plurality of components executing in parallel, and releasing the first lock may be performed in response to receiving a signal from each of the components in the screen-space pipeline.

[0048] In some embodiments, the processing unit may acquire the second lock in a third memory context and provide a second operation for execution by a third processor. The third processor may execute in the second memory context. The processing unit may receive a second signal from the third processor indicating that the second lock can be released, and may provide the second signal to a fourth processor. The fourth processor may execute in the third memory context. The processing unit may release the second lock through the fourth processor.

[0049] Figure 5 This is a block diagram illustrating an exemplary computer system according to at least one embodiment of the present disclosure. Computer system 500 may correspond to reference... Figure 1 System 102 is described. Computer system 500 can operate as a server or endpoint machine in an endpoint server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. This machine can be a television, personal computer (PC), tablet computer, set-top box (STB), personal digital assistant (PDA), mobile phone, network device, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify the actions the machine should take. Furthermore, although only one machine is shown in the figure, the term "machine" should also be understood to include any set of machines that, individually or jointly, execute a set (or more) of instructions to perform one or more methods discussed herein.

[0050] Example computer system 500 includes a processing device (processor) 502, main memory 504 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), double data rate (DDRSDRAM) or DRAM (RDRAM), etc.), static memory 506 (e.g., flash memory, static random access memory (SRAM), etc.) and data storage device 516, which communicate with each other via bus 528.

[0051] Processor (processing device) 502 represents one or more general-purpose processing devices, such as microprocessors, central processing units, etc., and may include processing logic 522. More specifically, processor 502 may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor that implements other instruction sets or combinations of instruction sets. Processor 502 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. Processor 502 is configured to execute instructions 526 (e.g., for generating threat indicator alerts) to perform the operations described herein.

[0052] The computer system 500 may also include a network interface device 508. The computer system 500 may also include a video display unit 510 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an input device 512 (e.g., a keyboard, alphanumeric keypad, motion-sensing input device, touchscreen), a cursor control device 514 (e.g., a mouse), and a signal generation device 518 (e.g., a speaker). In some embodiments, the computer system 500 may not include the video display unit 510, the input device 512, and / or the cursor control device 514 (e.g., a headless configuration).

[0053] Data storage device 516 may include a non-transitory machine-readable storage medium 524 (also referred to as a computer-readable storage medium) storing one or more sets of instructions 526 (e.g., efficiently releasing computing resources across memory contexts and processor pipelines) embodying one or more methods or functions described herein. Instructions 526 may also reside wholly or at least partially in main memory 504 and / or processor 502, and be executed by computer system 500, which also constitute machine-readable storage media. These instructions may also be sent or received on network 520 via network interface device 508.

[0054] In one embodiment, instruction 526 includes instructions for efficiently releasing computing resources across memory contexts and processor pipelines. While computer-readable storage medium 524 (machine-readable storage medium) is shown as a single medium in the exemplary embodiment, the terms "computer-readable storage medium" and "machine-readable storage medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) storing one or more sets of instructions. The terms "computer-readable storage medium" and "machine-readable storage medium" should also be understood to include any medium capable of storing, encoding, or carrying a set of machine-executable instructions and enabling the machine to perform one or more methods as described in this disclosure. Therefore, the terms "computer-readable storage medium" and "machine-readable storage medium" should be understood to include, but are not limited to, solid-state memory, optical media, and magnetic media.

[0055] Reasoning and training logic Figure 6A Inference and / or training logic 615 is shown for performing inference and / or training operations associated with one or more embodiments.

[0056] In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, code and / or data storage 601 for storing forward and / or output weights and / or input / output data, and / or other parameters of neurons or layers of a neural network trained for and / or used for inference in one or more embodiments. In at least one embodiment, the training logic 615 may include graph code or other software for controlling timing and / or sequence (or coupled to code and / or data storage 601 for storing graph code or other software for controlling timing and / or sequence), wherein weight and / or other parameter information is loaded to configure the processing unit, eight logic units, integer and / or floating-point units (collectively referred to as an arithmetic logic unit (ALU) or simple circuit). In at least one embodiment, code (such as graph code) loads weight or other parameter information into the processor ALU based on the architecture of the neural network corresponding to the code. In at least one embodiment, code and / or data storage 601 is stored in the weight parameters and / or input / output data of each layer of a neural network trained or used in conjunction with one or more embodiments during the forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of code and / or data storage 601 may be included within other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0057] In at least one embodiment, any portion of the code and / or data storage 601 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 601 may be a cache memory, dynamic random-addressable memory (“DRAM”), static random-addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice of whether the code and / or data storage 601 is internal or external to the processor, for example, or composed of DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip or off-chip storage space, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0058] In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, code and / or data storage 605 for storing backpropagation and / or output weights and / or input / output data corresponding to neurons or layers of a neural network trained and / or used for inference in one or more embodiments. In at least one embodiment, during training and / or inference using one or more embodiments, the code and / or data storage 605 stores weight parameters and / or input / output data for each layer of a neural network trained or used in one or more embodiments during backpropagation of input / output data and / or weight parameters. In at least one embodiment, the training logic 615 may include graph code or other software for controlling timing and / or sequence (or coupled to the code and / or data storage 605 storing graph code or other software for controlling timing and / or sequence), wherein weight and / or other parameter information is loaded to configure processing units, including logic units comprising integer and / or floating-point units (collectively, arithmetic logic units (ALUs)).

[0059] In at least one embodiment, code (such as graph code) causes the architecture of the neural network corresponding to that code to load weights or other parameter information into the processor ALU. In at least one embodiment, any portion of the code and / or data storage 605 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of the code and / or data storage 605 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 605 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the choice between the code and / or data storage 605 being internal or external to the processor, for example, whether it consists of DRAM, SRAM, flash memory, or some other type of storage, depends on whether the available storage is on-chip or off-chip, the latency requirements of the training and / or inference functions being performed, the data batch size used in the inference and / or training of the neural network, or some combination of these factors.

[0060] In at least one embodiment, code and / or code and / or data storage 601 and code and / or data storage 605 may be separate storage structures. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 may be combined storage structures. In at least one embodiment, code and / or data storage 601 and code and / or data storage 605 may be partially combined and partially separated. In at least one embodiment, any portion of code and / or data storage 601 and code and / or data storage 605 may be included with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0061] In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, one or more arithmetic logic units (“ALUs”) 610 (including integer and / or floating-point units) for performing logical and / or mathematical operations at least in part based on or instructed by training and / or inference code (e.g., graph code), the results of which may produce activations (e.g., output values ​​from layers or neurons within a neural network) stored in activation storage 620, which are functions of input / output and / or weight parameter data stored in code and / or data storage 601 and / or code and / or data storage 605. In at least one embodiment, activation is activated in response to execution instructions or other code, linear algebraic and / or matrix-based mathematical generation performed by ALU 610, and the activation is stored in activation storage 620. The weight values ​​stored in code and / or data storage 605 and / or code and / or data storage 601 are used as operands with other values, such as bias values, gradient information, momentum values, or other parameters or hyperparameters. Any or all of these can be stored in code and / or data storage 605 or code and / or code and / or data storage 601 or another on-chip or off-chip memory.

[0062] In at least one embodiment, one or more processors or other hardware logic devices or circuits include one or more ALUs 610, while in another embodiment, one or more ALUs 610 may be located outside the processor or other hardware logic device or the circuitry that uses them (e.g., a coprocessor). In at least one embodiment, one or more ALUs 610 may be included within an execution unit of a processor, or otherwise included in a group of ALUs accessible by the execution unit of the processor, which may be within the same processor or distributed among different processors of different types (e.g., a central processing unit, a graphics processing unit, a fixed-function unit, etc.). In at least one embodiment, code and / or data storage 601, code and / or data storage 605, and activation storage 620 may share a processor or other hardware logic device or circuitry, while in another embodiment, they may be located in different processors or other hardware logic devices or circuitry, or in some combination of the same and different processors or other hardware logic devices or circuitry. In at least one embodiment, any portion of activation storage 620 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Furthermore, inference and / or training code may be stored together with other code accessible to the processor or other hardware logic or circuitry, and may be retrieved and / or processed using the processor’s fetch, decode, schedule, execute, exit, and / or other logic circuitry.

[0063] In at least one embodiment, the active memory 620 may be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other memory. In at least one embodiment, the active memory 620 may be wholly or partially located inside or outside one or more processors or other logic circuits. In at least one embodiment, the choice of whether the active memory 620 is internal to or external to the processor may depend on the available on-chip or off-chip storage, the latency requirements for training and / or inference functions, the batch size of data used in inference and / or training the neural network, or some combination of these factors. For example, it may include DRAM, SRAM, flash memory, or some other memory type.

[0064] In at least one embodiment, Figure 6A The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as a TensorFlow® processing unit from Google, an inference processing unit (IPU) from Graphcore™, or a Nervana® processor from Intel (e.g., “Lake Crest”). In at least one embodiment, Figure 6A The inference and / or training logic 615 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware, or other hardware such as field programmable gate array (“FPGA”).

[0065] Figure 6B An inference and / or training logic 615 according to at least one embodiment is illustrated. In at least one embodiment, the inference and / or training logic 615 may include, but is not limited to, hardware logic, wherein computational resources are dedicated or otherwise uniquely used in conjunction with weight values ​​or other information corresponding to one or more layers of neurons within a neural network. In at least one embodiment, Figure 6B The inference and / or training logic 615 shown can be used in conjunction with an application-specific integrated circuit (ASIC), such as a TensorFlow® processing unit from Google, an inference processing unit (IPU) from Graphcore™, or a Nervana® (e.g., “Lake Crest”) processor from Intel. In at least one embodiment, Figure 6BThe inference and / or training logic 615 shown can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware (e.g., field-programmable gate array (FPGA)). In at least one embodiment, the inference and / or training logic 615 includes, but is not limited to, code and / or data storage 601 and code and / or data storage 605, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. Figure 6B In at least one embodiment shown, each of code and / or data storage 601 and code and / or data storage 605 is associated with dedicated computing resources (e.g., computing hardware 602 and computing hardware 606). In at least one embodiment, each of computing hardware 602 and computing hardware 606 includes one or more ALUs that perform mathematical functions (e.g., linear algebraic functions) only on the information stored in code and / or data storage 601 and code and / or data storage 605, respectively, and the results of the function execution are stored in activation memory 620.

[0066] In at least one embodiment, each of the code and / or data storage 601 and 605 and the corresponding computing hardware 602 and 606 corresponds to a different layer of the neural network, such that activations obtained from one storage / computation pair 601 / 602 of the code and / or data storage 601 and computing hardware 602 provide input as input to the next storage / computation pair 605 / 606 of the code and / or data storage 605 and computing hardware 606, in order to reflect the conceptual organization of the neural network. In at least one embodiment, each storage / computation pair 601 / 602 and 605 / 606 may correspond to more than one neural network layer. In at least one embodiment, additional storage / computation pairs (not shown) may be included in the inference and / or training logic 615 after or in parallel with the storage / computation pairs 601 / 602 and 605 / 606.

[0067] Neural network training and deployment Figure 7Training and deployment of a deep neural network according to at least one embodiment are illustrated. In at least one embodiment, an untrained neural network 706 is trained using a training dataset 702. In at least one embodiment, the training framework 704 is the PyTorch framework, while in other embodiments, the training framework 704 is TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training frameworks. In at least one embodiment, the training framework 704 trains the untrained neural network 706 and enables it to be trained using the processing resources described herein to generate a trained neural network 708. In at least one embodiment, the weights may be randomly selected or pre-trained using a deep belief network. In at least one embodiment, training may be performed in a supervised, partially supervised, or unsupervised manner.

[0068] In at least one embodiment, supervised learning is used to train an untrained neural network 706, wherein the training dataset 702 includes inputs paired with desired outputs for input, or wherein the training dataset 702 includes inputs with known outputs and the neural network 706 is manually graded output. In at least one embodiment, the untrained neural network 706 is trained in a supervised manner, and inputs from the training dataset 702 are processed, and the resulting output is compared with a set of expected or desired outputs. In at least one embodiment, the error is then propagated back through the untrained neural network 706. In at least one embodiment, a training framework 704 adjusts the weights controlling the untrained neural network 706. In at least one embodiment, the training framework 704 includes tools for monitoring the degree to which the untrained neural network 706 converges to a model (e.g., a trained neural network 708) adapted to generate the correct answer (e.g., result 714) based on input data (e.g., a new dataset 712). In at least one embodiment, the training framework 704 repeatedly trains the untrained neural network 706 while adjusting the weights to improve the output of the untrained neural network 706 using a loss function and tuning algorithms (e.g., stochastic gradient descent). In at least one embodiment, the training framework 704 trains an untrained neural network 706 until the untrained neural network 706 reaches the desired accuracy. In at least one embodiment, the trained neural network 708 can then be deployed to perform any number of machine learning operations.

[0069] In at least one embodiment, unsupervised learning is used to train an untrained neural network 706, wherein the untrained neural network 706 attempts to train itself using unlabeled data. In at least one embodiment, the unsupervised learning training dataset 702 will include input data without any associated output data or "ground truth" data. In at least one embodiment, the untrained neural network 706 can learn groupings within the training dataset 702 and can determine how each input relates to the untrained dataset 702. In at least one embodiment, unsupervised training can be used to generate a self-organizing graph in the trained neural network 708, which is capable of performing operations useful for reducing the dimensionality of the new dataset 712. In at least one embodiment, unsupervised training can also be used to perform anomaly detection, which allows the identification of data points in the new dataset 712 that deviate from the normal patterns of the new dataset 712.

[0070] In at least one embodiment, semi-supervised learning can be used, a technique in which the training dataset 702 includes a mixture of labeled and unlabeled data. In at least one embodiment, the training framework 704 can be used to perform incremental learning, for example, through transfer learning techniques. In at least one embodiment, incremental learning enables the trained neural network 708 to adapt to a new dataset 712 without forgetting the knowledge injected into the trained neural network 708 during initial training.

[0071] Reference Figure 8 , Figure 8 This is an example data flow diagram of process 800 for generating and deploying a processing and inference pipeline according to at least one embodiment. In at least one embodiment, process 800 may be deployed to perform game name recognition analysis and inference on user feedback data at one or more facilities 802, such as a data center.

[0072] In at least one embodiment, process 800 may be executed within training system 804 and / or deployment system 806. In at least one embodiment, training system 804 may be used to train, deploy, and implement machine learning models (e.g., neural networks, object detection algorithms, computer vision algorithms, etc.) for use with deployment system 806. In at least one embodiment, deployment system 806 may be configured to offload processing and computing resources in a distributed computing environment to reduce infrastructure requirements of facility 802. In at least one embodiment, deployment system 806 may provide a pipeline platform for selecting, customizing, and implementing virtual instruments for use with computing devices at facility 802. In at least one embodiment, virtual instruments may include software-defined applications for performing one or more processing operations on feedback data. In at least one embodiment, one or more applications in the pipeline may use or invoke services of deployment system 806 (e.g., inference, visualization, computation, AI, etc.) during application execution.

[0073] In at least one embodiment, some applications used in the advanced processing and inference pipeline may use machine learning models or other AI to perform one or more processing steps. In at least one embodiment, the machine learning model may be trained at facility 802 using feedback data 808 stored at facility 802 (e.g., imaging data) or feedback data 808 from another or more facilities, or a combination thereof. In at least one embodiment, training system 804 may be used to provide applications, services, and / or other resources to generate a deployable machine learning model for the work of deployment system 806.

[0074] In at least one embodiment, the model registry 824 may be supported by an object storage system that supports version control and object metadata. In at least one embodiment, it may be available from within a cloud platform via, for example, cloud storage (e.g., Figure 9 The system uses a Cloud 926-compatible application programming interface (API) to access object storage. In at least one embodiment, machine learning models within the model registry 824 can be uploaded, listed, modified, or deleted by the developer or partner of the system interacting with the API. In at least one embodiment, the API can provide access to methods that allow users with appropriate credentials to associate models with applications, enabling the models to be executed as part of the containerized instantiation of the application.

[0075] In at least one embodiment, training pipeline 904 ( Figure 9The scenario may include where facility 802 is training its own machine learning model or has an existing machine learning model that needs optimization or updating. In at least one embodiment, feedback data 808 may be received from various channels, such as forums, web forms, etc. In at least one embodiment, once feedback data 808 is received, AI-assisted annotation 810 may be used to help generate annotations corresponding to the feedback data 808 for use as ground truth data for the machine learning model. In at least one embodiment, AI-assisted annotation 810 may include one or more machine learning models (e.g., convolutional neural networks (CNNs)) that can be trained to generate annotations corresponding to certain types of feedback data 808 (e.g., from certain devices) and / or certain types of anomalies in the feedback data 808. In at least one embodiment, AI-assisted annotation 810 may then be used directly or may be adjusted or fine-tuned using annotation tools to generate ground truth data. In at least one embodiment, in some examples, labeled data 812 may be used as ground truth data for training the machine learning model. In at least one embodiment, AI-assisted annotation 810, labeled data 812, or a combination thereof may be used to train the machine learning model (e.g., via...). Figure 8 and / or Figure 9 The model training (814) uses ground-based real-world data. In at least one embodiment, the trained machine learning model may be referred to as output model 816 and may be used by deployment system 806, as described herein.

[0076] In at least one embodiment, training pipeline 904 ( Figure 9This can include situations where facility 802 requires a machine learning model to perform one or more processing tasks for deploying one or more applications in system 806, but facility 802 may not currently have such a machine learning model (or may not have an efficient or effective model optimized for this purpose). In at least one embodiment, an existing machine learning model can be selected from model registry 824. In at least one embodiment, model registry 824 can include machine learning models trained to perform various inference tasks on imaging data. In at least one embodiment, the machine learning model in model registry 824 can be trained on imaging data from a different facility (e.g., a remote facility) instead of facility 802. In at least one embodiment, the machine learning model may have already been trained on imaging data from one location, two locations, or any number of locations. In at least one embodiment, when training on imaging data from a specific location (which may be in the form of feedback data 808), training can be performed at that location, or at least in a manner that protects the confidentiality of the imaging data or restricts the transfer of the imaging data off-site (e.g., in compliance with HIPAA regulations, privacy regulations, etc.). In at least one embodiment, once a model has been trained or partially trained at one location, a machine learning model can be added to a model registry 824. In at least one embodiment, the machine learning model can then be retrained or updated at any number of other facilities, and the retrained or updated model can be used in the model registry 824. In at least one embodiment, a machine learning model (referred to as output model 816) can then be selected from the model registry 824 and used in a deployment system 806 to perform one or more processing tasks for one or more applications of the deployment system.

[0077] In at least one embodiment, training pipeline 904 ( Figure 9This can be used in scenarios including facility 802, which requires a machine learning model to perform one or more processing tasks for deploying one or more applications in system 806, but facility 802 may not currently have such a machine learning model (or may not have an optimized, efficient, or effective model for this purpose). In at least one embodiment, the machine learning model selected from model registry 824 may not be fine-tuned or optimized for the feedback data 808 generated at facility 802 due to population differences, genetic variations, robustness, anomalous diversity of the training data, and / or other problems with the training data used to train the machine learning model. In at least one embodiment, AI-assisted annotation 810 can be used to help generate annotations corresponding to the feedback data 808 for use as ground-based data for retraining or updating the machine learning model. In at least one embodiment, labeled data 812 can be used as ground-based data for training the machine learning model. In at least one embodiment, retraining or updating the machine learning model can be referred to as model training 814. In at least one embodiment, model training 814 may include data (e.g., AI-assisted annotations 810, labeled data 812, or a combination thereof) that can be used as ground-based data to retrain or update the machine learning model.

[0078] In at least one embodiment, deployment system 806 may include software 818, service 820, hardware 822, and / or other components, features, and functions. In at least one embodiment, deployment system 806 may include a software "stack" such that software 818 can be built on top of service 820 and can be used to perform some or all of the processing tasks, and service 820 and software 818 can be built on top of hardware 822 and can be used to perform processing, storage, and / or other computing tasks of deployment system 806.

[0079] In at least one embodiment, software 818 may include any number of different containers, each of which may perform an instantiation of an application. In at least one embodiment, each application may perform one or more processing tasks (e.g., inference, object detection, feature detection, segmentation, image enhancement, calibration, etc.) in an advanced processing and inference pipeline. In at least one embodiment, for each type of computing device, there may be any number of containers that may perform data processing tasks on feedback data 808 (or other data types, such as those described herein). In at least one embodiment, in addition to the containers that receive and configure imaging data for use by each container and / or for use by facility 802 after processing through the pipeline, the advanced processing and inference pipeline may also be defined based on the selection of different containers desired or required for processing feedback data 808 (e.g., to convert output back to a usable data type for storage and display in facility 802). In at least one embodiment, a combination of containers within software 818 (e.g., constituting a pipeline) may be referred to as a virtual instrument (as described in more detail herein), and the virtual instrument may utilize service 820 and hardware 822 to perform some or all of the processing tasks of the application instantiated in the container.

[0080] In at least one embodiment, data may be preprocessed as part of a data processing pipeline to prepare it for processing by one or more applications. In at least one embodiment, post-processing may be performed on the output of one or more inference tasks or other processing tasks in the pipeline to prepare output data for the next application and / or to prepare output data for user transmission and / or use (e.g., as a response to an inference request). In at least one embodiment, the inference task may be performed by one or more machine learning models, such as trained or deployed neural networks, which may include the output model 816 of the training system 804.

[0081] In at least one embodiment, the tasks of the data processing pipeline can be encapsulated in one or more containers, each container representing a discrete, fully functional instantiation of an application and a virtualized computing environment capable of referencing machine learning models. In at least one embodiment, containers or applications can be published to a private (e.g., limited access) area of ​​a container registry (described in more detail herein), and trained or deployed models can be stored in a model registry 824 and associated with one or more applications. In at least one embodiment, an image of the application (e.g., a container image) can be used in the container registry, and once a user selects an image from the container registry for deployment in the pipeline, that image can be used to generate containers for instantiation of the application for use by the user's system.

[0082] In at least one embodiment, the developer can develop, publish, and store an application (e.g., as a container) for performing processing and / or inference on provided data. In at least one embodiment, a software development kit (SDK) associated with the system can be used to perform development, publication, and / or storage (e.g., to ensure the developed application and / or container conforms to or is compatible with the system). In at least one embodiment, the developed application can be tested locally using the SDK (e.g., at a first facility, on data from a first facility), the SDK serving as a system (e.g.,...). Figure 9 System 900 may support at least some services 820. In at least one embodiment, once verified by system 900 (e.g., for accuracy, etc.), the application becomes available in the container registry for users (e.g., hospitals, clinics, laboratories, healthcare providers, etc.) to select and / or implement one or more processing tasks on data at the user's facility (e.g., a second facility).

[0083] In at least one embodiment, the developer can then share the application or container over a network for the system (e.g., Figure 9 The system 800 allows user access and use of the data processing pipeline. In at least one embodiment, a completed and validated application or container may be stored in a container registry, and the associated machine learning model may be stored in a model registry 824. In at least one embodiment, a requesting entity (which provides an inference or image processing request) may browse the container registry and / or model registry 824 for applications, containers, datasets, machine learning models, etc., select the desired combination of elements to include in the data processing pipeline, and submit a processing request. In at least one embodiment, the request may include input data necessary to execute the request, and / or may include a selection of the application and / or machine learning model to be executed when the request is processed. In at least one embodiment, the request may then be passed to one or more components of the deployment system 806 (e.g., the cloud) to perform processing in the data processing pipeline. In at least one embodiment, the processing performed by the deployment system 806 may include referencing elements (e.g., applications, containers, models, etc.) selected from the container registry and / or model registry 824. In at least one embodiment, once results are generated through the pipeline, the results may be returned to the user for reference (e.g., for viewing in a suite of viewing applications executed on a local machine, local workstation, or terminal).

[0084] In at least one embodiment, service 820 may be utilized to assist in processing or executing applications or containers in the pipeline. In at least one embodiment, service 820 may include computing services, collaborative content creation services, simulation services, artificial intelligence (AI) services, visualization services, and / or other service types. In at least one embodiment, service 820 may provide functionality common to one or more applications in software 818, thus abstracting functionality into services that can be invoked or utilized by applications. In at least one embodiment, the functionality provided by service 820 can operate dynamically and more efficiently, while also allowing applications to process data in parallel (e.g., using...). Figure 9 The parallel computing platform 930 in the system can be scaled well. In at least one embodiment, it is not required that each application providing the same functionality as the shared service 820 must have a corresponding instance of the service 820, but rather that the service 820 can be shared between and among various applications. In at least one embodiment, as a non-limiting example, the service may include an inference server or engine that can be used to perform detection or segmentation tasks. In at least one embodiment, a model training service may be included, which can provide the ability to train and / or retrain machine learning models.

[0085] In at least one embodiment, where service 820 includes an AI service (e.g., an inference service), as part of application execution, one or more machine learning models associated with an application for anomaly detection (e.g., tumors, growth abnormalities, scarring, etc.) can be executed by invoking (e.g., as an API call) the inference service (e.g., an inference server) to execute one or more machine learning models or their processing. In at least one embodiment, where another application includes one or more machine learning models for a segmentation task, the application can invoke the inference service to execute the machine learning models for performing one or more processing operations associated with the segmentation task. In at least one embodiment, the software 818 implementing the high-level processing and inference pipeline can be pipelined because each application can invoke the same inference service to execute one or more inference tasks.

[0086] In at least one embodiment, hardware 822 may include a GPU, CPU, data processing unit (DPU), AI / deep learning system (e.g., an AI supercomputer, such as NVIDIA's DGX). TM(Supercomputer system), cloud platform, or a combination thereof. In at least one embodiment, different types of hardware 822 may be used to provide efficient, specially built support for software 818 and services 820 in deployment system 806. In at least one embodiment, GPU processing may be used to perform local processing (e.g., at facility 802) within the AI / deep learning system, in the cloud system, and / or in other processing components of deployment system 806 to improve the efficiency, accuracy, and performance of game name recognition.

[0087] In at least one embodiment, as a non-limiting example, regarding deep learning, machine learning and / or high-performance computing, simulation and visual computing, software 818 and / or services 820 may be optimized for GPU processing. In at least one embodiment, at least some of the computing environment of the deployment system 806 and / or training system 804 may have GPU-optimized software (e.g., NVIDIA DGX). TM The system's hardware and software combination is executed in a data center or one or more supercomputers or high-performance computing systems. In at least one embodiment, as described herein, hardware 822 may include any number of GPUs that can be invoked to perform data processing in parallel. In at least one embodiment, the cloud platform may also include GPU-optimized execution for deep learning tasks, machine learning tasks, or other computing tasks. In at least one embodiment, AI / deep learning supercomputers and / or GPU-optimized software (e.g., such as NVIDIA's DGX) may be used. TM The system provides a hardware abstraction and scaling platform to execute cloud platforms (e.g., NVIDIA's NGC). TM In at least one embodiment, the cloud platform can integrate application container cluster systems or coordination systems (e.g., KUBERNETES) across multiple GPUs to achieve seamless scaling and load balancing.

[0088] Figure 9 This is a system diagram of an example system 900 for generating and deploying a deployment pipeline according to at least one embodiment. In at least one embodiment, system 900 can be used to implement Figure 8 The process 800 and / or other processes include advanced processing and inference pipelines. In at least one embodiment, system 900 may include training system 804 and deployment system 806. In at least one embodiment, training system 804 and deployment system 806 may be implemented using software 818, service 820 and / or hardware 822, as described herein.

[0089] In at least one embodiment, system 900 (e.g., training system 804 and / or deployment system 806) may be implemented in a cloud computing environment (e.g., using cloud 926). In at least one embodiment, system 900 may be implemented locally (regarding facilities) or as a combination of cloud computing resources and local computing resources. In at least one embodiment, access to the API in cloud 926 may be restricted to authorized users by establishing security measures or protocols. In at least one embodiment, the security protocol may include a network token, which may be signed by an authentication service (e.g., AuthN, AuthZ, Gluecon, etc.) and may carry appropriate authorization. In at least one embodiment, the API of the virtual instrument (described herein) or other instances of system 900 may be restricted to a set of public Internet Service Providers (ISPs) that have been audited or authorized for interaction.

[0090] In at least one embodiment, the various components of system 900 may communicate with each other and among themselves using any of a variety of different network types, including but not limited to local area networks (LANs) and / or wide area networks (WANs) via wired and / or wireless communication protocols. In at least one embodiment, communication between facilities and components of system 900 (e.g., for sending inference requests, for receiving the results of inference requests, etc.) may be transmitted via one or more data buses, wireless data protocols (e.g., Wi-Fi), wired data protocols (e.g., Ethernet), etc.

[0091] In at least one embodiment, similar to the description herein. Figure 8 As described, training system 804 can execute training pipeline 904. In at least one embodiment, where deployment system 806 uses one or more machine learning models in deployment pipeline 910, training pipeline 904 can be used to train or retrain one or more (e.g., pre-trained) models, and / or implement one or more pre-trained models 906 (e.g., without retraining or updating). In at least one embodiment, as a result of training pipeline 904, output model 816 can be generated. In at least one embodiment, training pipeline 904 can include any number of processing steps, AI-assisted annotation 810, tagging or annotating feedback data 808 to generate labeled data 812, selecting a model from a model registry, model training 814, training, retraining, or updating the model, and / or other processing steps. In at least one embodiment, DICOM adapter 902a can be used to access DICOM data. In at least one embodiment, different training pipelines 904 can be used for different machine learning models used by deployment system 806. In at least one embodiment, similar to the... Figure 8The training pipeline 904 described in the first example can be used for the first machine learning model, similar to the one about Figure 8 The training pipeline 904 described in the second example can be used for a second machine learning model, similar to the one described above. Figure 8 The training pipeline 904 of the third example described can be used for a third machine learning model. In at least one embodiment, any combination of tasks within the training system 804 can be used according to the requirements of each respective machine learning model. In at least one embodiment, one or more machine learning models may have already been trained and are ready for deployment, so the training system 804 may not perform any processing on the machine learning models, and the machine learning models may be implemented by the deployment system 806.

[0092] In at least one embodiment, one or more output models 816 and / or pre-trained models 906 may comprise any type of machine learning model. In at least one embodiment, and not limited thereto, the machine learning model used by system 900 may include models using linear regression, logistic regression, decision trees, support vector machines (SVM), Naive Bayes, k-nearest neighbors (Knn), k-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., autoencoders, convolutions, recursion, perceptrons, long / short-term memory (LSTM), Bi-LSTM, Hopfield, Boltzmann, deep belief, deconvolution, generative adversarial, liquid state machines, etc.), and / or other types of machine learning models.

[0093] In at least one embodiment, training pipeline 904 may include AI-assisted annotation. In at least one embodiment, labeled data 812 (e.g., conventional annotation) can be generated using any number of techniques. In at least one embodiment, in some examples, labels or other annotations can be generated by drawing programs (e.g., annotation programs), computer-aided design (CAD) programs, tagging programs, another type of application suitable for generating annotations or labels for ground reality, and / or can be hand-drawn. In at least one embodiment, ground reality data can be synthetically generated (e.g., generated from computer models or renderings), realistically generated (e.g., designed and generated from real-world data), automatically generated by machines (e.g., extracting features from data using feature analysis and learning, and then generating labels), manually annotated (e.g., taggers or annotation experts, defining the placement of labels), and / or combinations thereof. In at least one embodiment, for each instance of feedback data 808 (or other data types used by the machine learning model), there may be corresponding ground reality data generated by training system 804. In at least one embodiment, AI-assisted annotation can be performed as part of deployment pipeline 910; supplementing or replacing AI-assisted annotation included in training pipeline 904. In at least one embodiment, system 900 may include a multi-layer platform, which may include a software layer (e.g., software 818) of a diagnostic application (or other application type) that can perform one or more medical imaging and diagnostic functions.

[0094] In at least one embodiment, the software layer may be implemented as a secure, encrypted, and / or certified API that can invoke (e.g., call) an application or container from an external environment (e.g., facility 802). In at least one embodiment, the application may then invoke or execute one or more services 820 to perform computational, AI, or visualization tasks associated with their respective applications, and the software 818 and / or service 820 may utilize the hardware 822 to perform processing tasks efficiently and effectively.

[0095] In at least one embodiment, deployment system 806 may execute deployment pipeline 910. In at least one embodiment, deployment pipeline 910 may include any number of applications, which may be sequential, non-sequential, or otherwise applied to feedback data (and / or other data types), including AI-assisted annotations, as described above. In at least one embodiment, as described herein, deployment pipeline 910 for an individual device may be referred to as a virtual instrument for the device. In at least one embodiment, for a single device, there may be more than one deployment pipeline 910, depending on the desired information from the data generated from the device.

[0096] In at least one embodiment, the applications that can be used to deploy pipeline 910 may include any application that can perform processing tasks on feedback data or other data from the device. In at least one embodiment, since various applications can share common image operations, in some embodiments, data augmentation libraries (e.g., as one of services 820) may be used to accelerate these operations. In at least one embodiment, to avoid bottlenecks in traditional processing methods that rely on CPU processing, parallel computing platform 930 may be used for GPU acceleration of these processing tasks.

[0097] In at least one embodiment, deployment system 806 may include user interface 914 (e.g., graphical user interface, web interface, etc.) which may be used to select applications to be included in deployment pipeline 910, deploy applications, modify or change applications or their parameters or configurations, use and interact with deployment pipeline 910 during setup and / or deployment, and / or otherwise interact with deployment system 806. In at least one embodiment, although not shown with respect to training system 804, UI 914 (or different user interfaces) may be used to select models to be used in deployment system 806, to select models to be trained or retrained in training system 804, and / or to otherwise interact with training system 804.

[0098] In at least one embodiment, in addition to the application coordination system 928, a pipeline manager 912 may be used to manage interactions between applications or containers deployed through the pipeline 910 and services 820 and / or hardware 822. In at least one embodiment, the pipeline manager 912 may be configured to facilitate interactions from application to application, from application to service 820, and / or from application or service to hardware 822. In at least one embodiment, although shown as included in software 818, this is not intended to be limiting, and in some examples, the pipeline manager 912 may be included in service 820. In at least one embodiment, the application coordination system 928 (e.g., Kubernetes, DOCKER, etc.) may include a container coordination system that can group applications into containers as logical units for coordination, management, scaling, and deployment. In at least one embodiment, by associating applications from the deployment pipeline 910 (e.g., rebuilding applications, splitting applications, etc.) with individual containers, each application can execute in a self-contained environment (e.g., at the kernel level) to improve speed and efficiency.

[0099] In at least one embodiment, each application and / or container (or its image) can be developed, modified, and deployed independently (e.g., a first user or developer can develop, modify, and deploy a first application, and a second user or developer can develop, modify, and deploy a second application separate from the first user or developer). This allows focus on the tasks of a single application and / or container without being hindered by the tasks of other applications or containers. In at least one embodiment, the pipeline manager 912 and the application coordination system 928 can facilitate communication and collaboration between different containers or applications. In at least one embodiment, the application coordination system 928 and / or the pipeline manager 912 can facilitate communication and resource sharing between and within each application or container, provided that the expected inputs and / or outputs of each container or application are known to the system (e.g., based on the construction of the application or container). In at least one embodiment, since one or more applications or containers in the deployment pipeline 910 can share the same services and resources, the application coordination system 928 can coordinate, load balance, and determine the sharing of services or resources between and within the various applications or containers. In at least one embodiment, the scheduler can be used to track the resource requirements of applications or containers, the current or planned use of these resources, and resource availability. Therefore, in at least one embodiment, taking into account the system's needs and availability, the scheduler can allocate resources to different applications and distribute resources between and among applications. In some examples, the scheduler (and / or other components of the application coordination system 928) can determine resource availability and distribution based on constraints imposed on the system (e.g., user constraints), such as Quality of Service (QoS), the urgency of data output (e.g., to determine whether to perform real-time processing or delayed processing), etc.

[0100] In at least one embodiment, service 820, utilized and shared by applications or containers in deployment system 806, may include computing service 916, collaborative content creation service 917, AI service 918, simulation service 919, visualization service 920, and / or other service types. In at least one embodiment, an application may invoke (e.g., execute) one or more services 820 to perform processing operations for the application. In at least one embodiment, an application may utilize computing service 916 to perform supercomputing or other high-performance computing (HPC) tasks. In at least one embodiment, one or more computing services 916 may be utilized to perform parallel processing (e.g., using parallel computing platform 930) to process data substantially simultaneously through one or more applications and / or one or more tasks of a single application. In at least one embodiment, parallel computing platform 930 (e.g., NVIDIA's CUDA) ®General-purpose computing can be implemented on a GPU (GPGPU) (e.g., GPU / graphics 922). In at least one embodiment, the software layer of the parallel computing platform 930 can provide access to the GPU's virtual instruction set and parallel computing elements to execute a computing kernel. In at least one embodiment, the parallel computing platform 930 may include memory, and in some embodiments, memory may be shared between and within multiple containers, and / or between and within different processing tasks within a single container. In at least one embodiment, inter-process communication (IPC) calls can be generated for multiple containers and / or multiple processes within containers to use the same data from a shared memory segment of the parallel computing platform 930 (e.g., where multiple different stages of one or more applications are processing the same information). In at least one embodiment, instead of copying data and moving it to different locations in memory (e.g., read / write operations), the same data in the same location in memory can be used for any number of processing tasks (e.g., at the same time, at different times, etc.). In at least one embodiment, this information about the new location of the data can be stored and shared between various applications because the data as a result of processing is used to generate new data. In at least one embodiment, the location of the data, and the location of the updated or modified data, can be part of the definition of how the payload in the container is understood.

[0101] In at least one embodiment, AI service 918 may be used to perform an inference service for executing a machine learning model associated with the application (e.g., a task to perform one or more processing tasks of the application). In at least one embodiment, AI service 918 may utilize AI system 924 to execute a machine learning model (e.g., a neural network such as a CNN) for segmentation, reconstruction, object detection, feature detection, classification, and / or other inference tasks. In at least one embodiment, the application deploying pipeline 910 may use one or more output models 816 from training system 804 and / or other models from the application to perform inference on imaging data (e.g., DICOM data, RIS data, CIS data, REST-compliant data, RPC data, raw data, etc.). For example, DICOM adapter 902b may be used to access DICOM data. In at least one embodiment, two or more examples of inference using application coordination system 928 (e.g., a scheduler) may be available. In at least one embodiment, a first category may include a high-priority / low-latency path that can implement a higher service level protocol, such as for performing inference on urgent requests in emergency situations or for radiologists during diagnostic procedures. In at least one embodiment, the second category may include standard priority paths, which can be used for requests that may not be urgent or for situations where analysis can be performed at a later time. In at least one embodiment, the application coordination system 928 may allocate resources (e.g., services 820 and / or hardware 822) based on priority paths for different inference tasks of the AI ​​service 918.

[0102] In at least one embodiment, shared memory may be installed in AI service 918 within system 900. In at least one embodiment, shared memory may operate as a cache (or other storage device type) and may be used to process inference requests from applications. In at least one embodiment, when an inference request is submitted, a set of API instances of deployment system 806 may receive the request and may select one or more instances (e.g., for best fit, for load balancing, etc.) to process the request. In at least one embodiment, to process the request, the request may be fed into a database, and if not already in the cache, a machine learning model may be located from model registry 824. A verification step may ensure that an appropriate machine learning model is loaded into the cache (e.g., shared memory), and / or a copy of the model may be saved to the cache. In at least one embodiment, if the application is not already running or there are not enough instances of the application, a scheduler (e.g., the scheduler of pipeline manager 912) may be used to start the application referenced in the request. In at least one embodiment, if an inference server has not yet been started to execute the model, an inference server may be started. In at least one embodiment, any number of inference servers may be started for each model. In at least one embodiment, in a pull model that clusters inference servers, the model can be cached whenever load balancing is favorable. In at least one embodiment, the inference servers can be statically loaded into the corresponding distributed servers.

[0103] In at least one embodiment, an inference server running in a container can be used to perform inference. In at least one embodiment, an instance of the inference server can be associated with a model (and optionally multiple versions of the model). In at least one embodiment, if an instance of the inference server does not exist when a request to perform inference on the model is received, a new instance can be loaded. In at least one embodiment, when the inference server is started, a model can be passed to the inference server, allowing the same container to be used to serve different models, as long as the inference server runs as different instances.

[0104] In at least one embodiment, during application execution, an inference request for a given application can be received, and a container (e.g., an instance of a hosted inference server) can be loaded (if not already loaded), and a launcher can be invoked. In at least one embodiment, preprocessing logic within the container can (e.g., using a CPU and / or GPU) load, decode, and / or perform any additional preprocessing on the incoming data. In at least one embodiment, once the data is ready for inference, the container can infer the data as needed. In at least one embodiment, this can include a single inference call for an image (e.g., a hand X-ray) or can request inference for hundreds of images (e.g., a chest CT scan). In at least one embodiment, the application can summarize the results before completion, which may include, but is not limited to, a single confidence score, pixel-level segmentation, voxel-level segmentation, generating visualizations, or generating text to summarize the results. In at least one embodiment, different priorities can be assigned to different models or applications. For example, some models may have a real-time (turnaround time less than 1 minute) priority, while other models may have a lower priority (e.g., turnaround time less than 10 minutes). In at least one embodiment, model execution time can be measured from the requesting agency or entity, and may include cooperative network traversal time and inference service execution time.

[0105] In at least one embodiment, the transfer of requests between service 820 and the inference application can be hidden behind a software development kit (SDK) and robust transfer can be provided via queues. In at least one embodiment, requests are placed in queues via an API for individual application / tenant ID combinations, and the SDK pulls requests from the queues and provides them to the application. In at least one embodiment, the name of the queue can be provided in the environment where the SDK picks up requests. In at least one embodiment, asynchronous communication via queues may be useful because it allows any instance of the application to pick up work as it becomes available. In at least one embodiment, results can be sent back via queues to ensure no data loss. In at least one embodiment, queues can also provide the ability to partition work, as the highest priority work can go into a queue connected to a majority of instances of the application, while the lowest priority work can go into a queue connected to a single instance that processes tasks in the order they are received. In at least one embodiment, the application can run on a GPU-accelerated instance generated in cloud 926, and the inference service can perform inference on the GPU.

[0106] In at least one embodiment, visualization service 920 can be used to generate visualizations for viewing the output of application and / or deployment pipeline 910. In at least one embodiment, visualization service 920 can utilize GPU / graphics 922 to generate visualizations. In at least one embodiment, visualization service 920 can implement rendering effects such as ray tracing or other light transport simulation techniques to generate higher quality visualizations. In at least one embodiment, visualizations can include, but are not limited to, 2D image rendering, 3D volume rendering, 3D volume reconstruction, 2D tomographic slicing, virtual reality display, augmented reality display, etc. In at least one embodiment, a virtualized environment can be used to generate virtual interactive displays or environments (e.g., virtual environments) for system users (e.g., doctors, nurses, radiologists, etc.) to interact with. In at least one embodiment, visualization service 920 can include an internal visualizer, film and / or other rendering or image processing capabilities or functions (e.g., ray tracing, rasterization, internal optics, etc.).

[0107] In at least one embodiment, hardware 822 may include a GPU / graphics 922, an AI system 924, a cloud 926, and / or any other hardware for executing training system 804 and / or deployment system 806. In at least one embodiment, the GPU / graphics 922 (e.g., NVIDIA's TESLA) ® The GPU / graphics 922 may include any number of GPUs that can be used to perform processing tasks of any feature or function of computing service 916, collaborative content creation service 917, AI service 918, simulation service 919, visualization service 920, other services, and / or software 818. For example, for AI service 918, the GPU / graphics 922 may be used to perform preprocessing on imaging data (or other data types used by machine learning models), postprocessing on the output of machine learning models, and / or to perform inference (e.g., to execute machine learning models). In at least one embodiment, the cloud 926, AI system 924, and / or other components of system 900 may use the GPU / graphics 922. In at least one embodiment, the cloud 926 may include a GPU-optimized platform for deep learning tasks. In at least one embodiment, the AI ​​system 924 may use GPUs, and one or more AI systems 924 may be used to execute the cloud 926 (or the task is at least part of deep learning or inference). Similarly, although hardware 822 is shown as a discrete component, this is not intended to be limiting, and any component of hardware 822 can be combined with or utilized by any other component of hardware 822.

[0108] In at least one embodiment, the AI ​​system 924 may include a specially built computing system (e.g., a supercomputer or HPC) configured for inference, deep learning, machine learning, and / or other artificial intelligence tasks. In at least one embodiment, in addition to a CPU, RAM, memory, and / or other components, features, or functions, the AI ​​system 924 (e.g., NVIDIA's DGX) TM It may also include GPU-optimized software (e.g., a software stack) that can use multiple GPUs / graphics 922 to perform GPU-optimized tasks. In at least one embodiment, one or more AI systems 924 may be implemented in the cloud 926 (e.g., in a data center) to perform some or all of the AI-based processing tasks of system 900.

[0109] In at least one embodiment, cloud 926 may include GPU-accelerated infrastructure (e.g., NVIDIA's NGC). TM The cloud 926 can provide a GPU-optimized platform for performing processing tasks of system 900. In at least one embodiment, cloud 926 may include AI system 924 for performing one or more AI-based tasks of system 900 (e.g., as a hardware abstraction and scaling platform). In at least one embodiment, cloud 926 may be integrated with application coordination system 928 utilizing multiple GPUs to achieve seamless scaling and load balancing between and within applications and services 820. In at least one embodiment, as described herein, cloud 926 may be responsible for performing at least some services 820 of system 900, including computing service 916, AI service 918, and / or visualization service 920. In at least one embodiment, cloud 926 may perform large and small batch inference (e.g., performing NVIDIA TensorRT). TM ), providing an accelerated parallel computing platform 930 (e.g., NVIDIA's CUDA). ® The application coordination system 928 (e.g., KUBERNETES) provides graphics rendering APIs and platforms (e.g., for ray tracing, 2D graphics, 3D graphics, and / or other rendering techniques to produce higher quality cinematic effects), and / or may provide other functionalities for system 900. In at least one embodiment, the parallel computing platform 930 may include APIs.

[0110] In at least one embodiment, to protect patient confidentiality (e.g., in cases of off-site use of patient data or records), the cloud 926 may include a registry—such as a deep learning container registry. In at least one embodiment, the registry may store containers for instantiating applications that can perform preprocessing, post-processing, or other processing tasks on patient data. In at least one embodiment, the cloud 926 may receive data, including patient data and sensor data in containers, perform the requested processing only on the sensor data in those containers, and then output and / or visualize the results to appropriate parties and / or devices (e.g., local medical devices for visualization or diagnosis), without extracting, storing, or otherwise accessing the patient data. In at least one embodiment, the confidentiality of patient data is maintained in accordance with HIPAA and / or other data regulations.

[0111] Other variations are within the spirit of this disclosure. Therefore, although the disclosed technology is readily adaptable to various modifications and alternative constructions, certain embodiments thereof are illustrated in the accompanying drawings and have been described in detail above. However, it should be understood that the disclosure is not intended to be limited to one or more specific forms disclosed, but rather, it is intended to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of this disclosure as defined in the appended claims.

[0112] Unless otherwise stated or obviously contradicted by the context, the terms “a,” “an,” and “the,” and similar references, used in the context of describing the disclosed embodiments (particularly in the context of the appended claims), should be interpreted as encompassing both singular and plural forms, rather than as definitions of the terms. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (meaning “including, but not limited to”). The term “connection” (referring to a physical connection when unmodified) should be interpreted as partially or wholly contained, attached to, or joined together, even with some intervention. Unless otherwise indicated herein, references to numerical ranges herein are intended only as a way of abbreviating each individual value falling within that range, and each individual value is incorporated into the specification as if it were separately described herein. In at least one embodiment, unless otherwise indicated or contradicted by the context, the use of the terms “set” (e.g., “item set”) or “subset” should be interpreted as a non-empty set comprising one or more members. Furthermore, unless otherwise indicated or contradicted by the context, the term “subset” of the corresponding set does not necessarily mean an appropriate subset of the corresponding set, but rather that the subset and the corresponding set can be equal.

[0113] Unless otherwise explicitly stated or clearly contradicted by the context, connective phrases such as “at least one of A, B, and C” or “at least one of A, B, and C” are understood in the context to generally refer to items, terms, etc., which can be A or B or C, or any non-empty subset of the set A, B, and C. For example, in an illustrative example of a set with three members, the connective phrases “at least one of A, B, and C” and “at least one of A, B, and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Therefore, such connective language is generally not intended to imply that some embodiments require the presence of at least one of A, at least one of B, and at least one of C. Additionally, unless otherwise stated or contradicted by the context, the term “multiple” indicates a plural state (e.g., “multiple items” means multiple items). In at least one embodiment, the number of items in the multiple items is at least two, but may be more if explicitly indicated or indicated by the context. Furthermore, unless otherwise stated or clearly understood from the context, the phrase “based on” means “at least partially based on” or “at least based on” rather than “based on only”.

[0114] Unless otherwise indicated herein or clearly contradicted by the context, the operations of the processes described herein may be performed in any suitable order. In at least one embodiment, processes such as those described herein (or variations thereof and / or combinations thereof) are executed under the control of one or more computer systems configured with executable instructions and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that is executed jointly on one or more processors via hardware or a combination thereof. In at least one embodiment, the code is stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-transitory computer-readable storage medium that excludes transient signals (e.g., propagating transient electrical or electromagnetic transmissions) but includes non-transitory data storage circuitry (e.g., buffers, caches, and queues) within transient signal transceivers. In at least one embodiment, code (e.g., executable code or source code) is stored on one or more non-transitory computer-readable storage media (or other memory for storing executable instructions) on which executable instructions are stored, which, when executed by one or more processors of a computer system (i.e., as a result of execution), cause the computer system to perform the operations described herein. In at least one embodiment, the set of non-transitory computer-readable storage media comprises multiple non-transitory computer-readable storage media, and one or more of the individual non-transitory storage media lack all the code, but the multiple non-transitory computer-readable storage media collectively store all the code. In at least one embodiment, the executable instructions are executed such that different instructions are executed by different processors; for example, the non-transitory computer-readable storage media store the instructions, and the main central processing unit (“CPU”) executes some instructions while the graphics processing unit (“GPU”) executes other instructions. In at least one embodiment, different components of the computer system have separate processors, and the different processors execute different subsets of the instructions.

[0115] Therefore, in at least one embodiment, the computer system is configured to implement one or more services that perform the operations of the processes described herein, either individually or collectively, and such a computer system is configured with suitable hardware and / or software to enable the implementation of the operations. Furthermore, the computer system implementing at least one embodiment of this disclosure is a single device, and in another embodiment it is a distributed computer system comprising multiple devices operating in different ways, such that the distributed computer system performs the operations described herein, and that a single device does not perform all the operations.

[0116] The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate embodiments of this disclosure and does not constitute a limitation on the scope of the disclosure unless otherwise required. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of the disclosure.

[0117] All references cited in this article, including publications, patent applications and patents, are incorporated herein by reference as if each reference were individually and specifically indicated to be incorporated herein by reference and the entire contents of which are described herein.

[0118] The terms “coupled” and “connected”, and their derivatives, may be used in the specification and claims. It should be understood that these terms may not be intended to be synonyms with each other. Rather, in certain examples, “connected” or “coupled” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. “Coupled” may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other.

[0119] Unless otherwise expressly stated, in some embodiments, it will be understood that throughout the specification, terms such as “processing,” “computing,” “determining,” etc., refer to the actions and / or processes of a computer or computing system or similar electronic computing device that process and / or convert data represented as physical quantities (e.g., electrons) in the registers and / or memory of the computing system into other data represented as physical quantities in the memory, registers, or other such information storage, transmission, or display devices of the computing system.

[0120] Similarly, the term "processor" can refer to any device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or a GPU. A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Likewise, each process can refer to multiple processes that execute instructions sequentially or intermittently, sequentially, or in parallel. In at least one embodiment, the terms "system" and "method" are used interchangeably herein, provided that a system can embody one or more methods, and a method can be considered a system.

[0121] In this document, reference may be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in various ways, such as by receiving data as a parameter to a function call or a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference may also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be implemented by transmitting data as an input or output parameter to a function call, an application programming interface, or an inter-process communication mechanism.

[0122] While the description herein illustrates exemplary embodiments of the described technologies, other architectures may be used to implement the described functionality and are intended to fall within the scope of this disclosure. Furthermore, although specific assignments of responsibilities have been defined above for descriptive purposes, various functions and responsibilities may be assigned and divided in different ways depending on the circumstances.

[0123] Furthermore, although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter claimed in the appended claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claims.

Claims

1. A method, the method comprising: Acquire the first lock in the first memory context; Provide a first task to be executed by a first processor, wherein the first processor executes the task in a second memory context; Receive a first signal from the first processor indicating that the first lock can be released; The first signal is provided to the second processor, wherein the second processor executes within the first memory context; and The second processor releases the first lock.

2. The method of claim 1, further comprising the first processor accessing first data associated with the first lock.

3. The method according to claim 2, wherein, The first data associated with the first lock is graphics rendering data, and the first operation is a graphics rendering operation based on the graphics rendering data.

4. The method according to claim 1, wherein, The first signal is received after the first processor section has completed the first operation.

5. The method according to claim 1, wherein, The first signal is received after the first processor has completed all of the first tasks.

6. The method according to claim 1, wherein, The release of the first lock by the second processor is performed in response to receiving a plurality of signals, wherein the plurality of signals includes the first signal.

7. The method according to claim 1, further comprising: Acquire the second lock in the third memory context; Provide a second task to be executed by a third processor, wherein the third processor executes the task within the second memory context; Receive a second signal from the third processor indicating that the second lock can be released; The second signal is provided to a fourth processor, wherein the fourth processor executes within the third memory context; and The second lock is released by the fourth processor.

8. A system comprising: Memory; and A processing circuit coupled to the memory, the processing circuit being used for: Acquire the first lock in the first context of the memory; Provide a first task to be executed by a first processor, wherein the first processor executes the task in a second context of the memory; Receive from the first processor a first signal indicating that the first lock can be released; and The first signal is provided to a second processor, wherein the second processor executes in the first context of the memory, and wherein the first lock is released by the second processor.

9. The system according to claim 8, wherein, The first processor accesses the first data associated with the first lock.

10. The system according to claim 9, wherein, The first data associated with the first lock is graphics rendering data, and the first operation is a graphics rendering operation based on the graphics rendering data.

11. The system according to claim 8, wherein, The first signal is received after the first processor section has completed the first operation.

12. The system according to claim 8, wherein, The first signal is received after the first processor has fully completed the first task.

13. The system according to claim 8, wherein, The first lock is released by the second processor in response to receiving a plurality of signals, wherein the plurality of signals includes the first signal.

14. The system according to claim 8, wherein the processing circuit is further configured to: Acquire the second lock in the third context of the memory; Provide a second task to be executed by a third processor, wherein, The third processor executes in the second context of the memory; Receive a second signal from the third processor indicating that the second lock can be released; as well as The second signal is provided to a fourth processor, which executes in the third context of the memory, and wherein the second lock is released by the fourth processor.

15. A system comprising: First processor and second processor; as well as A processing circuit coupled to the first processor and the second processor, the processing circuit being used for: Acquire the first lock in the first memory context; Provide a first task to be executed by the first processor, wherein the first processor executes the task in a second memory context; Receive from the first processor a first signal indicating that the first lock can be released; and The first signal is provided to the second processor, wherein the second processor executes in the first memory context, and wherein the first lock is released by the second processor.

16. The system according to claim 15, wherein, The first processor accesses the first data associated with the first lock.

17. The system according to claim 16, wherein, The first data associated with the first lock is graphics rendering data, and the first operation is a graphics rendering operation based on the graphics rendering data.

18. The system according to claim 15, wherein, The first signal is received after the first processor section has completed the first operation.

19. The system according to claim 15, wherein, The first signal is received after the first processor has fully completed the first task.

20. The system according to claim 15, wherein, The first lock is released in response to receiving a plurality of signals, wherein the plurality of signals includes the first signal.

Images