Apparatus, system, and method for designing data path circuits.

Abstract

To provide apparatuses, systems, and techniques for designing a data path circuit such as a parallel prefix circuit with reinforcement learning.SOLUTION: A method provided herein may comprise: receiving a first design state of a data path circuit; inputting the first design state of the data path circuit into a machine learning model; and performing reinforcement learning using the machine learning model to output a final design state of the data path circuit, where the final design state of the data path circuit has decreased area, power consumption and / or delay as compared to conventionally designed data path circuits.SELECTED DRAWING: Figure 9A

Description

Technical Field

[0001] At least one embodiment relates to the use of machine learning to perform and facilitate circuit design. For example, at least one embodiment relates to techniques for data path circuit design that utilize reinforcement learning, by means of various novel techniques described herein.

Background Art

[0002] Many types of circuits can include a data path or data path circuit, such as an arithmetic logic unit or multiplier that can perform data processing operations. For example, a data path circuit can include a parallel prefix circuit (e.g., a gray to binary converter, adder, decrementer, incrementer, priority encoder, etc.) that processes or synthesizes data on the data path. Prefix calculation is a very useful and fundamental operation used in various applications such as image processing, cryptography, processor allocation, biological sequence comparison, binary addition, silicon compiler design, job scheduling, loop parallelization, polynomial evaluation, and sorting.

[0003] The design of a data path circuit aims to reduce the delay (e.g., the amount of time required for the data path circuit to output a value when given an input) and area (e.g., the amount of space required by the data path circuit), while avoiding an increase in the power consumption of the circuit. As the delay of the data path circuit is reduced, the area and power consumption of the data path circuit can be affected. For example, as the delay of the data path circuit decreases, the area of the data path circuit may increase. Therefore, the design of a data path circuit aims to optimize the delay, area, and power consumption of the data path circuit.

[0004] Optimizing prefix circuits is difficult because the large design space of prefix circuits increases exponentially with input length, making them cumbersome to count. As a result, exhaustive search methods do not scale beyond small input lengths. Several conventional prefix circuit structures have been proposed that strike trade-offs between logic level, maximum fan-out, and routing tracks. Another set of algorithms optimizes prefix circuit size and level properties. However, prefix circuit level and maximum fan-out properties do not map to circuit area, power, and delay due to physical design complexities such as capacitive loading and congestion. Conventional methods for data path circuit design do not always adequately optimize the delay, area, and power consumption of data path circuits. [Overview of the project] [Means for solving the problem]

[0005] Various embodiments of this disclosure will be described with reference to the drawings. [Brief explanation of the drawing]

[0006] [Figure 1] This is a diagram of an exemplary system architecture, based on at least several embodiments. [Figure 2] This figure shows an exemplary system architecture for reinforcement learning, based on at least several examples. [Figure 3] This figure shows exemplary prefix graph modification, based on at least several examples. [Figure 4] This figure shows an exemplary grid representation of a parallel prefix graph, based on at least several examples. [Figure 5] This figure shows exemplary data path circuit design calculations based on at least several examples. [Figure 6] This figure shows exemplary constraints on data path circuit design, based on at least several examples. [Figure 7]This figure illustrates an exemplary method for designing data path circuits using reinforcement learning, based on at least several embodiments. [Figure 8] This figure shows an exemplary system architecture based on at least several examples. [Figure 9A] This is a flowchart illustrating an exemplary method for designing data path circuits using reinforcement learning, based on at least several examples. [Figure 9B] This is a flowchart illustrating an exemplary method for designing data path circuits using reinforcement learning, based on at least several examples. [Figure 10A] This figure shows inference and / or training logic in at least some embodiments. [Figure 10B] This figure shows the inference and / or training logic in at least some embodiments. [Figure 11] This figure shows an exemplary data center system, based on at least several embodiments. [Figure 12] This figure shows a computer system according to at least some embodiments. [Figure 13] This figure shows a computer system according to at least some embodiments. [Figure 14] This figure shows at least a portion of a graphics processor, according to at least some embodiments. [Figure 15] This figure shows at least a portion of a graphics processor, according to at least some embodiments. [Figure 16] This is an exemplary data flow diagram for an advanced computing pipeline, based on at least several examples. [Figure 17] This is a system diagram for an exemplary system for training, adapting, instantiating, and deploying machine learning models in an advanced computing pipeline, based on at least several examples. [Figure 18A]This is a data flow diagram for the process of training a machine learning model, based on at least several examples. [Figure 18B] This figure shows a client-server architecture for extending annotation tools with pre-trained annotation models, based on at least several examples. [Modes for carrying out the invention]

[0007] Several fundamental digital design building blocks, such as adders, priority encoders, incrementers, decrementers, and Gray binary code converters, can be reduced to prefix summation and implemented as data path circuits, such as prefix circuits (e.g., parallel prefix circuits). Optimizing data path circuits, such as prefix circuits, for area, delay, and power can be important in digital hardware design. The embodiments described herein provide systems and methods for designing data path circuits (e.g., parallel prefix circuits) optimized for area, power, and / or delay using reinforcement learning.

[0008] Some memory systems store or communicate data for a host system, for example. A memory system may include data paths that communicate data from one component of the memory system to another. In some embodiments, a memory system may process or synthesize data in data path circuits. For example, a memory system may include prefix circuits for processing and synthesizing data, such as adders, decrementers, incrementers, Gray binary converters, priority encoders, etc. For example, an adder may be used to compute an address or table index for a process in the memory system. In one example, a prefix circuit may be used in a parallel configuration to reduce the time required to perform prefix computation, for example, as a parallel prefix circuit.

[0009] Each data path circuit may have an associated area (e.g., the size of the data path circuit in a memory system), power consumption (e.g., the amount of power consumed by the data path circuit during operation), and delay (e.g., the amount of time it takes to generate an output from a given number of inputs). To increase the performance of the circuit (e.g., for a memory system), data path circuits may be designed to reduce area, power consumption, and / or delay. However, in some examples, reducing one property of a data path circuit may affect another property of the data path circuit; for example, reducing the delay of a data path circuit may result in a larger area of ​​the data path circuit or the data path circuit consuming more power. Therefore, in embodiments, data path circuits may be designed to strike a trade-off between reduced area, power consumption, and / or delay, and may be designed or optimized, for example, for the smallest amount of area for each delay or power consumption.

[0010] In some cases, conventional methods for designing data path circuits are ineffective in designing optimized data path circuits. For example, some conventional methods propose prefix circuit structures that optimize logic levels, maximum fan-out, and routing tracks. However, optimizing logic levels, fan-out, and routing tracks may fail to optimize for area, delay, and / or power due to the complexity of the physical design of the prefix structure, for example, due to capacitive loading or congestion. Some conventional methods for designing data path circuits may include utilizing heuristic rules for circuit design, which either attempt to predict physical metrics about the circuit design or perform random modifications to the circuit design. Circuit designs generated using such heuristics may be evaluated using inaccurate analytical models. These analytical models and heuristic rules may be ineffective in producing optimal circuits because they rely on hand-crafted heuristics or are limited by analytical evaluation metrics. For example, prefix circuits designed using analytical evaluation metrics will degrade in quality (e.g., experiencing increased delay, area, or power consumption) when the prefix circuits undergo physical synthesis, for instance, when the analytical model is converted to a predicted physical model through simulation. Because physical synthesis is more powerful than analytical evaluation, conventional methods may not scale well for physical synthesis. Consequently, conventional circuit design techniques for data path circuits are not optimized and reduce the performance of systems using these data path circuits (e.g., memory systems).

[0011] Aspects of this disclosure address the above and other deficiencies by designing data path circuits (e.g., parallel prefix circuits) using reinforcement learning with a machine learning model. For example, a first processing device (e.g., running an agent) can provide a machine learning model with an initial design for a data path circuit. The machine learning model can modify the design to generate a second design for the data path circuit. In some examples, modifying the design can include modifying a prefix graph associated with the data path circuit, for example, modifying nodes in a prefix graph representing the data path circuit, where each node represents one or more components of the data path circuit. For example, the machine learning model can modify the prefix graph represented by the initial design by adding or removing nodes to generate a second prefix graph representing the second design. After modification, the first or second processing device can process the first and second designs to determine one or more metrics associated with each design. For example, the first or second processing device can determine area, power, and / or delay associated with the initial and second designs generated by the machine learning model. In some embodiments, a second machine learning model is used to predict the area, power, and / or delay associated with the initial circuit design and the second circuit design. In some embodiments, a circuit synthesis tool is used to predict the area, power, and / or delay between the initial circuit design and the second circuit design.

[0012] A first or second processing device can generate a reward (e.g., parameters) that represents a net change in area, delay, and / or power based on the transition from the initial design to the second design. The reward can be used to update the training of a machine learning model, for example, the machine learning model can undergo reinforcement learning based on the received reward. For example, the machine learning model can receive a reward, determine that the previous change caused a decrease in the area, delay, and / or power of the data path circuit, and update the weights of one or more nodes of the machine learning model. The updated machine learning model can then modify the second design of the data path circuit to generate a third data path circuit. This process can be repeated, and reinforcement learning of the data path circuit design using the machine learning model can continue until it is determined that no further improvements have been output (e.g., no additional modifications to the design that reduce the delay, area, and / or power of the data path circuit), or until it is determined that the delay, area, and / or power of the data path circuit satisfy the target delay, area, and / or power for the data path circuit, for example, until the data path circuit design is optimized for the respective delay, area, and / or power constraints.

[0013] The embodiments avoid the use of handcrafted heuristics (e.g., heuristics for pruning) applied in conventional circuit design techniques. In the embodiments, a machine learning model may be trained to perform circuit design for data path circuits via reinforcement learning techniques, as described herein. That is, the machine learning model learns to modify the data path circuit in a way that optimizes the current design of the data path circuit, for example, to modify the data path circuit to reduce the area, power consumption, and / or delay of the data path circuit. For example, the machine learning model may be trained on receiving a reward indicating one or more improved or reduced circuit design optimization metric values, determining whether a previous modification of the data path circuit design resulted in an improved or reduced circuit design optimization metric value, and then adjusting the nodes of the machine learning model based on that reward. In some embodiments, the machine learning model may be trained using model (based / none), value / policy based, or on / off policy reinforcement learning techniques. In other embodiments, the machine learning model may utilize deep neural networks (e.g., convolutional neural networks, transformers, graph neural networks, etc.) and / or decision trees. By utilizing reinforcement learning, the design of data path circuits can be more completely optimized than other solutions, for example, the delay, area, and / or power of the data path circuit can be reduced compared to data path circuits designed using conventional methods. The embodiment produces a circuit design that has less area and power for a given delay, and less delay for a given area and power, compared to previous methods in the circuit design of data path circuits. Thus, the overall performance of a system such as a memory system can be increased by the embodiment of this disclosure, because the designed data path circuit can consume less power, have less delay, and / or consume less area compared to conventionally designed data path circuits.

[0014] Figure 1 shows a computer system 100 according to at least one embodiment. In at least one embodiment, the computer system 100 may be a system, a SOC, or any combination thereof, with interconnected devices and components. In at least one embodiment, the computer system 100 is formed together with a processor 102 which may include an execution unit for executing instructions. In at least one embodiment, the computer system 100 may include components such as the processor 102 for employing an execution unit which includes logic for implementing algorithms for processing data. In at least one embodiment, the computer system 100 may include a processor such as the PENTIUM® processor family, Xeon®, Itanium®, XScale®, and / or StrongARM®, Intel® Core®, or Intel® Nervana® microprocessors, available from Intel Corporation in Santa Clara, California, but other systems (including PCs with other microprocessors, engineering workstations, set-top boxes, etc.) may also be used.

[0015] In at least one embodiment, the computer system 800 may be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants ("PDAs"), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (DSP), a SoC, a network computer ("NetPC"), a set-top box, a network hub, a wide area network ("WAN") switch, or any other system capable of executing one or more instructions. In one embodiment, the computer system 100 may be used with devices such as a graphics processing unit (GPU), a network adapter, a central processing unit, and network devices such as a switch (for example, a high-speed direct GPU-to-GPU interconnect such as an NVIDIA GH100 NVLINK or an NVIDIA Quantum 2 64 Ports InfiniBand NDR Switch).

[0016] In at least one embodiment, the computer system 100 may include, but is not limited to, a processor 102, which may include, but is not limited to, one or more execution units 107 that can be configured to run a Compute Unified Device Architecture ("CUDA") program (CUDA® is a registered trademark developed by NVIDIA Corporation in Santa Clara, California). In at least one embodiment, the CUDA program is at least part of a software application written in the CUDA programming language. In at least one embodiment, the computer system 100 is a single-processor desktop or server system. In at least one embodiment, the computer system 100 may be a multi-processor system. In at least one embodiment, the processor 102 may include, but is not limited to, a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing a combination of instruction sets, or any other processor device such as a digital signal processor. In at least one embodiment, the processor 102 may be coupled to a processor bus 110, which may transmit data signals between the processor 102 and other components in the computer system 100.

[0017] In at least one embodiment, the processor 102 may include, but is not limited to, a level 1 ("L1") internal cache memory ("cache") 104. In at least one embodiment, the processor 102 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory may reside outside the processor 102. In at least one embodiment, the processor 102 may also include a combination of both internal and external caches. In at least one embodiment, the register file 106 may store different types of data in various registers, including, but is not limited to, integer registers, floating-point registers, status registers, and instruction pointer registers.

[0018] In at least one embodiment, the processor 102 may include a data path circuit 108. In some examples, the data path circuit 108 may be an example of a parallel prefix circuit. For example, the data path circuit 108 may be an adder, incrementer, decrementer, priority encoder, and / or Gray binary converter, possibly including connected logic. In some examples, the data path circuit 108 may also be located among other components of the computer system 100. In some embodiments, the data path circuit 108 may consume power, occupy an area, and have a delay. In some embodiments, the delay of the data path circuit 108 may be inversely related to the clock frequency of the components of the computer system 108, for example, the delay of the data path circuit 108 may be used to set the clock frequency for the components of the computer system 108. In some embodiments, the design of the data path circuit 108 may be carried out via reinforcement learning using a machine learning model trained over time to reduce or optimize the area, power consumption, and / or delay of the data path circuit 108. For example, during the design of a data patch circuit 108, a machine learning model can modify the design of the data path circuit 108 and determine whether the modification resulted in a reduction in the area, power, or delay of the given data path circuit 108. The computer system 100 can then benefit from the optimized data path circuit designed using reinforcement learning. That is, the machine learning model can be updated based on whether the modifications made to the design of the data path circuit resulted in a reduction in the area, delay, and / or power of the data path circuit 108. Over several iterations, the machine learning model can be trained to select the modification that results in the best reduction in the area, power consumption, and / or delay of the data path circuit. In some embodiments, the machine learning model can be used until the area, power consumption, and / or delay of the data path circuit 108 meet a target metric, for example, until the area, power consumption, and / or delay of the data path circuit 108 meet the target area, power consumption, and / or delay.By utilizing reinforcement learning to design the data path circuit 108, the design of the data path circuit 108 can be optimized, and the performance of the computer system 100 can be improved.

[0019] In at least one embodiment, but not limited to, an execution unit 107 containing logic for performing integer and floating-point arithmetic may also be present in the processor 102. The processor 102 may also include a microcode ("u-code") read-only memory ("ROM") for storing microcode for several macro instructions. In at least one embodiment, the execution unit 102 may include logic for handling a packed instruction set 109. In at least one embodiment, by including the packed instruction set 109, along with the associated circuit elements for executing the instructions, in the instruction set of the general-purpose processor 102, arithmetic used by many multimedia applications can be performed using packed data in the general-purpose processor 102. In at least one embodiment, many multimedia applications can be accelerated and run more efficiently by using the full width of the processor's data bus to perform arithmetic on packed data, which may eliminate the need to transfer smaller units of data across the processor's data bus to perform one or more arithmetic operations, one data element at a time.

[0020] In at least one embodiment, the execution unit may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuits. In at least one embodiment, the computer system 100 may include, but is not limited to, memory 120. In at least one embodiment, memory 120 may be implemented as a DRAM device, SRAM device, flash memory device, or other memory device. Memory 120 may store (one or more) instructions 119 and / or data 121, which are represented by data signals that can be executed by the processor 102.

[0021] In at least one embodiment, a system logic chip may be coupled to the processor bus 110 and the memory 120. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub ("MCH") 116, and the processor 102 may communicate with the MCH 116 via the processor bus 110. In at least one embodiment, the MCH 116 may provide a high-bandwidth memory path 118 to the memory 120 for instruction and data storage, as well as for the storage of graphics commands, data, and textures. In at least one embodiment, the MCH 116 may direct data signals between the processor 102, the memory 120, and other components in the computer system 100, and bridge data signals between the processor bus 110, the memory 120, and the system I / O 122. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH116 may be coupled to memory 120 through a high-bandwidth memory path 118, and the graphics / video card 112 may be coupled to the MCH116 via an Accelerated Graphics Port ("AGP") interconnect 114.

[0022] In at least one embodiment, the computer system 100 may use a system I / O 122, which is a proprietary hub interface bus for coupling the MCH 116 to an I / O controller hub ("ICH") 130. In at least one embodiment, the ICH 130 may provide direct connectivity to several I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripherals to memory 120, a chipset, and a processor 102. An example may include, but is not limited to, an audio controller 129, a firmware hub ("Flash BIOS") 128, a wireless transceiver 126, data storage 124, a legacy I / O controller 123 including a user input interface 125 and a keyboard interface, a serial expansion port 127 such as USB, and a network controller 134. The data storage 124 may include a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0023] Figure 2 shows an exemplary system 200 for performing reinforcement learning to generate an improved design of a data path circuit, in at least one embodiment. In some embodiments, the system is a Q network or includes a Q network. In some embodiments, system 200 is a deep Q network or includes a deep Q network.

[0024] Reinforcement learning (RL) is a class of algorithms applicable to sequential decision-making tasks. RL utilizes the Markov Decision Process (MDP) formalism where an agent 202 attempts to optimize a function in its environment 204. The MDP can be completely described by a state space S (with states s ∈ S), an action space A (a ∈ A), a transition function T: S × A → S, and a reward function

Number

[0025] The Q-network can be trained via a process called Q-learning. Q-learning is a reinforcement learning process that causes the model to ultimately execute a sequence of actions that will generate the maximum total reward. This total reward is also called the Q-value. The function for calculating the Q-value can be as follows. Q(s t , a t ) ← r t + γ max a Q(s t+1 , a)

[0026] The above equation is for state s t (206) and action a t The Q value resulting from performing (210) is the immediate reward r(s t ,a t )(214)+Next state s t+1 We show that this is the highest possible Q value from the state in (212), where γ is a discount factor that controls the contribution of future rewards. The recursive definition of the Q function is Q(s t+1 ,a t+1 The expression ) can be extended to future states as follows: Q(s t ,a t )=r t +γr t+1 +···+γ n-1 r t+n-1 +γ n Q(s t+n ,a t+n ) The machine learning model 240 of agent 202 will be updated by performing the following update steps: Q(s t ,a t It learns to predict ).

number

[0027] In the embodiment, state action pairs (s) under policy π t ,a t The Q value of ) is as follows, state s t Action a in 206 t 210 is taken, and if future actions are taken using policy π, it is defined to be the expected gain.

number

[0028] In the example, the discount coefficient γ∈[0,1] balances the short-term reward and the long-term reward. The Q learning algorithm can start agent 202 with a random policy and environment (s t ,a t ,r t ,s t+1 Using the experience gathered during its interaction with 204, it iterates toward the optimal policy by updating Q with a learning rate α ∈ [0,1].

number

[0029] The policy for Q Learning Agent 202 is π(·│s t ) = argmaxQ(s t It can be expressed as ,a). In one embodiment, a ∈-greedy policy is used in which a random action a with probability ∈ is selected to increase exploration in the state space. In one embodiment, ∈ is annealed to 0 during training and is 0 when evaluation is performed. In one embodiment, multiple explorations may be performed in parallel with a range of ∈ values.

[0030] Deep Q-learning is essentially an extension of Q-learning that implements one or more machine learning models (e.g., machine learning model 240), such as neural networks, to approximate the Q-value described above. In deep Q-learning, one or more artificial neural networks (e.g., machine learning model 240) may be used to approximate the Q-value function described above. Artificial neural networks generally include feature representation components with classifier or regression layers that map features to a desired output space. A convolutional neural network (CNN) hosts, for example, multiple layers of convolutional filters. In the lower layers, pooling may be performed to address nonlinearity, and a multilayer perceptron is usually added on top to map the top layer features extracted by the convolutional layers to decisions (e.g., modifications to the design state of a prefix circuit). Deep learning is a class of machine learning algorithms that use a cascade of multiple layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. Deep neural networks can learn in a supervised (e.g., classification) manner and / or an unsupervised (e.g., pattern analysis) manner. A deep neural network contains a hierarchy of layers, where different layers learn different levels of representation corresponding to different levels of abstraction. In deep learning, each level learns to transform its input data into a slightly more abstract and complex representation. In particular, the deep learning process can learn on its own which features should be optimally placed at which level. In "deep learning," "deep" refers to the number of layers to which the data is transformed. More precisely, a deep learning system has a substantial credit assignment path (CAP) depth. A CAP is a chain of transformations from input to output. A CAP explains the latent causal relationships between input and output. In the case of a feedforward neural network, the CAP depth can be the network depth, which can be the number of hidden layers + 1.In the case of a recurrent neural network where a signal can propagate through layers two or more times, the CAP depth is potentially unlimited.

[0031] In the embodiment, the deep Q-network (DQN) is (s t+1 Training can be stabilized by using a second target network to estimate the Q-value of (a'). The second target network may be updated at a lower frequency than the first network. In one embodiment, the DQN may sample an experience-replay buffer. In one embodiment, the first machine learning model is used to determine the prediction, and the second neural network is used to determine the target. In one embodiment, the second neural network may have the same architecture as the first neural network. However, in one embodiment, the second neural network may have frozen parameters, while the first neural network may have variable parameters. In one embodiment, the second neural network is updated at a lower frequency than the first neural network. In one embodiment, a double DQN algorithm is used, which may further improve training by reducing overestimation in the DQN.

[0032] In some embodiments, the system 200 may include agents (e.g., actors, circuit modifiers, etc.) 202 and environments (e.g., simulation environments, circuit synthesizers, etc.) 204. In some embodiments, the agent 202 may include one or more machine learning models 240. The machine learning models 240 may be, for example, deep neural networks (e.g., convolutional neural networks, transformers, graph neural networks, etc.) or decision trees. For example, the machine learning model 240 may be a neural network of a deep Q network.

[0033] In some embodiments, system 200 may be used to design data path circuits 108 as described with reference to Figure 1. In embodiments, the optimization of data path circuits (e.g., prefix circuits) is framed as an RL task by creating MDPs for their construction. For example, the RL system 200 may be trained to select a design for a parallel prefix adder (e.g., to design an area-delay minimized Pareto frontier for an adder containing possible connected logic). In some embodiments, agent 202 is configured to modify the design of the data path circuits and train a machine learning model 240 based on that modification. In some embodiments, agent 202 may run on a processing device such as a graphical processing unit (GPU) or a central processing unit (CPU). In some embodiments, system 200 may include multiple agents 202 that can operate in parallel and share learning. Each agent 202 may run on the same or different processing devices and / or on the same or different cores of the processing devices. Each agent may perform modifications to the data path circuit in parallel; for example, multiple agents 202 may modify the data path circuit simultaneously. This can reduce the amount of time required to find the target or optimal data path circuit design. In some embodiments, one or more agents 202 may receive the design state 206 of the data path circuit. Upon receiving the design state 206, one or more agents 202 may modify the design state 206 via an action 210. Each agent 202 may output a different action 210 in some embodiments. In some embodiments, the action 210 may be determined for agent 202 by a machine learning model 240 (e.g., a deep neural network). Agent 202 may also be configured to output the modified state of the data path circuit to the environment 204.

[0034] In the embodiment, the design of a data path circuit 108 (for example, a prefix circuit such as a parallel prefix adder) can be represented using a prefix graph. A prefix graph is a directed acyclic graph (DAG) where edges may represent signals or signal pairs and nodes may represent logical operators. For example, parallel prefix computation can be represented as a directed acyclic parallel prefix graph, where any computation unit z i:j However, it is a graph node that performs a single operation on two inputs, namely z i:j =z i:k °z k-1:j Here, ° represents operations such as addition, carry, and difference.

[0035] Therefore, agent 202 can receive a prefix graph representing the initial design state 206 of the data path circuit. Agent 202 can then modify the prefix graph (for example, using a machine learning model 240) and output a new prefix graph representing the modified state of the data path circuit via action 210. In embodiments, modifying the prefix graph may include adding, removing, or moving nodes in the prefix graph.

[0036] Figure 3 shows an exemplary prefix graph modification in at least some embodiments. The data path circuit may be represented by prefix graphs 305, 310, where prefix graph 305 represents the initial state of the circuit design for the data path circuit (e.g., an adder), and prefix graph 310 represents the modified or updated state of the data path circuit. In the prefix problem, n inputs x n-1 ,x n-2, ...x0 and any associative operator ° result in n outputs y i =x i °x i-1Used to calculate °...°x0, i=0,...,n-1. Therefore, each output y i This includes all inputs x with the same or lower index (j ≤ i). j It depends on.

[0037] In one example, the calculation of an N-input prefix sum can be performed in several ways depending on the associativity of the operators. For example, two ways in which a 4-input prefix sum can be calculated are as follows: y0=x0,y1=x1°y0,y2=x2°y1,y3=x3°y2 y0=x0, y1=x1°y0, y2=x2°y1, z 3:2 =x3°x2,y3=z 3:2 °y1

[0038] In this example, the additional term z 3:2 Introducing this breaks the dependency of y3 on y2, allowing it to be computed in parallel with y2, and thus making this term a parallel prefix. An example is x i °x i-1 °...°x j z i:j This shows the output y. i is, z i:0 It can be rewritten as input x i is, z i:i It can be rewritten as follows: both y0 and x0 are z 0:0 Please note that this may be possible.

[0039] Parallel prefix computation can be represented as a directed acyclic parallel prefix graph, where every computation unit z i:j However, it is a graph node that performs a single operation on two inputs, namely z i:j =z i:k °z k-1:j In the example, the calculation node z i:jThe most significant and least significant bits (MSB, LSB) of a node can be (i,j). Using this notation, node (i,k) can be the superparent of (i,j), and node (k-1;j) can be the subparent of (i,j). The prefix graphs corresponding to the four-input prefix sum calculation in the above example are shown in Figure 4 as prefix graph 305 and prefix graph 310. In both graphs, the superparent and subparent of node (2,0) are (2,2) and (1,0), respectively.

[0040] In at least one embodiment, every valid N-input prefix graph has an input node (i,i), an output node (i,0), and an input / output node (0,0) for 1 ≤ i ≤ N-1. Furthermore, in at least one embodiment, every non-input node has exactly one up parent (up) and one down parent (lp), and thus the following: LSB(node) = LSB(lp(node)) LSB(lp(node))≦MSB(lp(node)) MSB(lp(node)) = LSB(up(node)) - 1 LSB(up(node))≦MSB(up(node)) MSB(up(node)) = MSB(node)

[0041] In one example, a data path circuit may receive inputs 315-a to 315-d and produce outputs 325-a to 325-d. In some examples, each input 315 and output 325 may represent a signal received or produced by the data path circuit, or a wire coming into the data path circuit. In some examples, each input 315 may represent an element (or some data) configured to be synthesized by the data path circuit. For example, in an embodiment where the data path circuit is a binary adder, each input may represent a bit of the input for which the adder circuit will perform a calculated sum; for example, input 315-a may represent the 0th bit or position 1, and input 315-d may represent the 3rd bit or position 8 of the number for which the adder circuit will find its sum. Thus, each output 325 may represent a value or bit generated by the data path circuit during its operation. In some embodiments, each output 325 may be generated from each preceding input; for example, output 325-c may be generated from inputs 315-a to 315-c, and output 325-d may be generated from inputs 315-a to 315-d.

[0042] In some embodiments, each prefix graph may also contain one or more nodes. For example, prefix graph 305 may contain nodes 320-a to 320-c. In some embodiments, each node may represent or be associated with one or more components of a data path circuit that performs one or more operations. For example, node 320-a may represent one or more logic gates (e.g., AND gates, NOR gates, XOR gates, etc.) of data path circuit 108. In some embodiments, node 320 may also represent a buffer or other type of gate.

[0043] In some embodiments, the area of ​​a data path circuit may relate to the number of nodes 320 in the prefix graph representing the data path circuit; for example, a larger number of nodes 320 results in a larger area associated with the data path circuit. For instance, the area of ​​a data path circuit represented by prefix graph 305 may be smaller than the area of ​​a data path circuit represented by prefix graph 310, for example, prefix graph 305 has fewer nodes 320. In some embodiments, the prefix graph can also indicate the delay associated with the data path circuit. For example, the delay of a data path circuit may relate to the longest path required for an input 315 to be generated as an output 325. For example, input 315-a passes through three nodes 320 before being used in prefix graph 305 to generate output 325-d, while each input 315 in prefix graph 310 passes through at most two nodes 320. In such embodiments, the delay of the data path circuit associated with prefix graph 310 may be less than the delay of the data path circuit 108 associated with prefix graph 305.

[0044] Referring to Figures 2 and 3, in some examples, each design state 206 of a data path circuit may be represented by a unique prefix graph. In such embodiments, agent 202 can receive a prefix graph representing the current design state 206 of the data path circuit 108. In some embodiments, as described in detail below and referring to Figure 4, a grid representation of the prefix graph is received. In one example, if prefix graph 305 represents the initial design state 206 of the data path circuit, agent 202 can receive prefix graph 305 (or a grid representation of the prefix graph) from environment 204 before making any modifications to it.

[0045] In some embodiments, agent 202 may be configured to take action 210 to modify node 320 in the prefix graph representing the current design state 206 of the data path circuit. In at least one embodiment, agent 202 may add node 320 or remove node 320 from the prefix graph representing the current design state 206 of the data path circuit. For example, agent 202 may add node 320-d to prefix graph 305. In some embodiments, a machine learning model 240 may determine which node 320 should be removed or where node 320 should be added to the prefix graph. In some embodiments, the machine learning model 240 receives an input grid representation of the prefix graph and outputs a grid representation of the modified prefix graph with nodes added or removed.

[0046] In some embodiments, agent 202 includes a graph effectiveness determinator 244. Alternatively, the graph effectiveness determinator 244 may be included in the environment 204. The graph effectiveness determinator 244 may assess the effectiveness of the updated prefix graph (e.g., output by a machine learning model 240). If the updated prefix graph is invalid, the graph effectiveness determinator 244 may perform one or more further modifications to the prefix graph to make it valid. Such modifications may be the addition or removal of one or more nodes from the prefix graph. In some embodiments, if the graph effectiveness determinator 244 determines that action 210 results in an invalid state (e.g., an invalid prefix graph), the graph effectiveness determinator 244 may again modify the design state 206. That is, the graph effectiveness determinator 244 may verify the updated state following the action. In some embodiments, the graph validity determiner 244 can verify the state by adding or removing invalid nodes and / or by ensuring that each node conforms to the rules specified below.

[0047] In the embodiment, as described above, the effective N-input prefix graph has input nodes (i,i), output nodes (i,0), and an input / output node (0,0) for 1 ≤ i ≤ N-1. For example, in prefix graph 310, input nodes (1,1)~(3,3) correspond to inputs 315-b~315-d at indices 1~3, output nodes (1,0)~(3,0) correspond to nodes 320-a~320-c, nodes 320-a~320-c supply outputs 325-b~325-d at indices 1~3, and the input / output node (0,0) corresponds to input 315-a at index 0, and input 315-a also supplies output 325-a at index 0.

[0048] In some embodiments, a valid prefix graph may be one from which each non-input node (e.g., each node other than (0,0), (1,1), (2,2), etc.) has exactly two parents, namely a high-level parent and a low-level parent, from which each non-input node directly receives values. These parents may be another non-input node 320 or an input node 315. That is, a prefix graph from which a non-input node 320 has one parent or three or more parents is invalid. In some embodiments, a valid prefix graph also has each node 320 having a sequentially continuous range of input indices from which each node 320 directly or indirectly receives values. For example, node 320-b is a valid node with the range (2,0) so as to receive values ​​from inputs at indices 0, 1, and 2. In the (MSB, LSB) notation for nodes, the most significant bit (MSB) represents the upper end or first element of the node's range, and the least significant bit (LSB) represents the lower end or second element of the node's range. Node 320 with range (0,2) is invalid because its range increases from MSB to LSB. In some embodiments, a prefix graph with range (6,3) but node 320 that does not receive a value from the input at index 5 is invalid, for example, because its range is not continuous because node 320 receives input at indices 3, 4, and 6, but not at 5.

[0049] Furthermore, each non-input node can directly receive values ​​from exactly one super-parent and one super-parent that are contiguous. For example, if node 320 has the range (3,1) and a super-parent with the range (3,3), then node 320 must also have a super-parent with the range (2,1). That is, the super-parent range of node 320 must have the same MSB as node 320's range (e.g., 3), and the super-parent range of node 320 must have the same LSB as node 320's range (e.g., 1), and at the same time, it must also be contiguous and include the input at index 2, for example. Thus, prefix graphs can follow the above rules regarding super-parents and super-parents.

[0050] In the embodiment, the action space A for an N-input prefix graph consists of two actions (addition or deletion) for any non-input / output node, for example, where LSB ∈ [1, N-2] and MSB ∈ [LSB+1, N-1]. Thus, |A| = (N-1) × (N-2) / 2. The environment evolution through T can maintain a valid prefix graph by 1) applying a verification procedure (perhaps performed by the graph validity determiner 244) after actions that may add or delete additional nodes to maintain validity, and 2) prohibiting redundant actions that would be reversed by the verification procedure.

[0051] In one embodiment, during verification, the node's superior parent, up(node), is an existing node with the same MSB and the next highest LSB. In one embodiment, the node's inferior parent, lp(node), is calculated using the node and its superior parent according to the following: (MSB lp(node) ,LSB lp(node) )=(LSB up(node) -1, LSB node )

[0052] In one embodiment, an invalid condition occurs when a node does not have a lower parent. Under such conditions, the graph validity determiner 244 performs a verification procedure to add the missing lower parent node.

[0053] In one embodiment, system 200 (e.g., environment 204 or agent 202) maintains a list of all nodes in the active prefix graph, nodelist. In one embodiment, the action of adding a node that already exists in nodelist is redundant and prohibited. In one embodiment, system 200 (e.g., environment 204 or agent 202) maintains a minimum list of nodes from nodelist that are not subparents of other nodes, minlist. In one embodiment, the action of deleting a node is limited to nodes in minlist, and in some cases, nodes that either do not exist or are subparents of other nodes may be deleted. For such corrections, deleted nodes may be re-added during verification.

[0054] Environment 204 receives an action 210 (e.g., prefix graph update) and determines the next design state 212 and reward 214 associated with the action 210 for the current design state 206. Environment 204 may start in an initial state S0, which may be randomly selected in some embodiments. In some embodiments, Environment 204 may run on one or more processing devices, such as a CPU or GPU. In some embodiments, System 200 may include multiple environments 204, each of which may receive a different action (e.g., a different modified state of the data path circuit). In one embodiment, multiple environments 204 may simultaneously receive modified states of the data path circuit output by different agents 202 to reduce the amount of time it takes to find a target data path circuit design. In at least one embodiment, Environment 204 may update or modify the design state 206 based on an action 210 selected by a machine learning model 240. In one embodiment, the environment 204 can generate a prefix graph for the state 212 after an action 210. For example, the environment can generate prefix graph 310 when action 210 specifies that a node 320 with the value (3,2) is added to prefix graph 305. In some embodiments, agent 202 can receive a prefix graph associated with the modified state. For example, agent 202 can receive prefix graph 310, or a grid representation of the prefix graph.

[0055] For any action 210 (for example, for any design state of the data path circuit), the prefix graph assessor 242 of environment 204 may estimate one or more parameters of the data path circuit. Such estimated parameters may include, for example, the area of ​​the data path circuit, the power consumption of the data path circuit, and / or the delay of the data path circuit. The prefix graph assessor 242 may then compare the determined parameters to one or more targets and / or constraints for the data path circuit. Furthermore, the prefix graph assessor 242 may estimate (or may have previously estimated) similar parameters for previous states of the data path circuit (for example, design state 206). The prefix graph assessor 242 may further compare the parameters for previous states of the data path circuit to constraints and / or targets. The prefix graph assessor 242 may compare the parameters of the initial design state 206, and / or the distance between the parameters and the target, with the parameters of the updated design state 212, and / or the distance between the parameters and the target. Based on such a comparison, the prefix graph assessor 242 may output a reward 214. For example, if the parameters associated with the updated design state are closer to the target than the parameters associated with the previous design state, a positive reward 214 may be output. Conversely, if the parameters associated with the updated design state are further from the target than the parameters associated with the previous design state, a negative reward 214 may be output. The environment 204 may also output the next design state 212 to be input to the agent 202.

[0056] If the environment 204 determines that the next design state 212 is valid following an action 210 to modify the current design state 206, or if it verifies the modified design state, the environment can calculate the next reward 214. In some embodiments, the reward 214 predicts the net change in area, power consumption, and / or delay of the data path circuit 108 as a result of action 210. That is, the environment 204 can calculate the delay, area, and power for the initial state (or current design state 206) and the delay, area, and power for the next design state 212, and determine the difference between the two to calculate the next reward 214. In one embodiment, the environment 204 can determine the reward as described with reference to Figure 8. In some embodiments, the environment 204 can determine the reward via a second machine learning model as described with reference to Figure 7. That is, the environment 204 may include a second machine learning model that predicts the changes in area, delay, and power of the data path circuit following action 210. In some examples, the environment 204 can determine the reward 208 (or the next reward 214), as described with reference to Figure 5.

[0057] For example, environment 204 can determine the area, delay, and / or power consumed by a data path circuit 108 having a design represented by a prefix graph 305. In some examples, the prefix graph assessor 242 of environment 204 can utilize a synthesis tool 505 to calculate the area, delay, and / or power of a physical data path circuit given a prefix graph (e.g., prefix graph 305). In some examples, the synthesis tool 505 can determine the type of logic gate to use, the size of the logic gate to use, the connectivity of the logic gate, and whether buffers or other components will optimize the prefix graph. In some examples, generating a potential physical data path circuit from a prefix graph via the synthesis tool 505 can increase or decrease the area, delay, or power consumption of the data path circuit. That is, generating a potential physical data path circuit can cause changes and modifications from the prefix graph to a specific circuit implementation form due to physical and manufacturing constraints. For example, if a node in a prefix graph is an input to four other nodes (e.g., four other nodes receive values ​​from that node), the synthesis tool 505 may insert buffers when generating potential physical data path circuits, causing an increase in area.

[0058] In some embodiments, the prefix graph assessor 242 of the environment 204 can also determine the area, delay, and / or power consumed by a data path circuit having a design represented by a prefix graph (e.g., prefix graph 310) output by agent 202 (e.g., the result of modification and verification of the input prefix graph). To determine the area, delay, and / or power of the data path circuit, the environment can use the synthesis tool 505 to generate a predicted physical data path circuit from the prefix graph. For each calculation (e.g., for the calculation of an initial prefix circuit such as prefix circuit 305, and for the calculation of a modified or updated prefix circuit such as prefix circuit 310), the prefix graph assessor 242 can determine the area / delay curve for graph 510. That is, the environment 204 can determine a curve 515 representing the delay of the data path circuit associated with the initial prefix graph (e.g., prefix graph 305) for each area. Similarly, environment 204 can determine curves 520 that represent the delays of the data path circuits associated with the updated prefix graphs (e.g., prefix graph 310) for each area.

[0059] In some embodiments, the environment can calculate the reward for action 210 by determining the difference in delay and area based on weight constraints described with reference to Figure 6. For example, if the weight is 1, the reward may be calculated by taking the difference between point 530 and point 535 based on the weight curve described with reference to Figure 6, for example. In this embodiment, the modification from prefix graph 305 to prefix graph 310 via action 210 reduced area and delay. Thus, the environment 204 can calculate the next reward 214 showing the amount of area and delay reduced via action 210. In other embodiments, the environment 204 can calculate the next reward 214 showing an increased area, a decreased area, an increased delay, a decreased delay, or a combination thereof. Area versus delay is shown in Figure 5, but the environment 204 can determine the difference in area, power, delay, or any combination thereof when calculating the next reward 214. In some embodiments, the environment 204 can send the next reward to agent 202, for example, machine learning model 240.

[0060] Agent 202 may be configured to receive the next reward 214. In some embodiments, Agent 202 can determine whether the action taken 210 optimized the data path circuit, for example, whether the action 210 caused a reduction in the area, power, and / or delay of the data path circuit. In some embodiments, Agent 202 can use reinforcement learning to train a machine learning model 240. That is, Agent 202 can train a machine learning model 240 based on the next reward 214 received. In one embodiment, Agent 202 trains (one or more) machine learning models 240 based on the reward 214 and the previously output action 210. Training may be carried out by defining an error (for example, based on the action and reward) and using techniques such as stochastic gradient descent and backpropagation to adjust the network weights across all its layers and nodes so that the error is minimized.

[0061] In one example, if the next reward 214 indicates a reduction in area, power, or latency, the machine learning model 240 may continue to take the same action 210 as the action that resulted in the reduction. If the next reward 214 indicates an increase in area, power, or latency, the machine learning model 240 may be trained to take other actions 210 by adjusting the node weights in the machine learning model. In some embodiments, the machine learning model may be trained using model (based / none), value / policy based, or on / off policy reinforcement learning techniques.

[0062] As described above, in embodiments in which the machine learning model 240 is used as a component of a deep Q network, the machine learning model 240 can receive the input of the design state 206 and predict the area and delay reduction for the action 210. For example, the machine learning model 240 can receive the input of the state s t Given the input 206, the following can be predicted: ∀a∈A:[Q area (s t ,a),Q delay (st ,a)]

[0063] The input to a machine learning model can be an N×N×4 tensor, where the four channels encode the node features as follows: 1) If it is a node (MSB, LSB) in nodelist, it is 1; otherwise, it is 0. 2) If it's a node (MSB, LSB) in the minlist, it's 1; otherwise, it's 0. 3) The level of the node (MSB, LSB) in nodelist, otherwise it is 0. 4) This is the fan-out of the node (MSB, LSB) in nodelist, otherwise it is 0. Here, nodelist is all nodes 320 in the active prefix graph, minlist is all nodes 320 in nodelist that are not subparents of other nodes, node level is the topology depth from the input node in the prefix graph (e.g., the number of nodes 320 between it and each input 315), and node fanout refers to the number of nodes 320 that are subordinate to it. Over time, a machine learning model 240 via reinforcement learning techniques is trained to take actions 210 that best optimize the design of the data path circuit 108, for example, reducing the area, power, and / or delay of the data path circuit 108.

[0064] In some embodiments, agent 202 may be configured to separately train different instances of the machine learning model 240 for each specific circuit that agent 202 will optimize. For example, agent 202 may train the machine learning model 240 for a 32-bit adder circuit and separately train another instance of the machine learning model 240 for a 64-bit adder circuit. In some embodiments, different instances of the machine learning model 240 may be trained for each of their respective properties, for example, for each target delay time of the data path circuit to reduce the area as much as possible. By leveraging reinforcement learning with the machine learning model, the design of the data path circuit may be more optimized than other solutions, for example, the delay, area, and / or power of the data path circuit may be reduced. Thus, the overall performance of any system (e.g., a memory system) incorporating the data path circuit can be increased, as the data path circuit consumes less power, has less delay, and occupies less area in the memory system.

[0065] In some embodiments, agent 202 and / or environment 204 can use reinforcement learning to train a machine learning model 240 to take actions 210 that optimize the area, delay, and / or power of the data path circuit 108, as described herein. In some embodiments, the machine learning model 240 can determine whether to add or remove a node 320 based on programmed constraints or values. For example, the machine learning model 240 may be constrained by the maximum circuit width of the data path circuit, the maximum arrival time, the weights between area and delay (or between area and power, between power and delay, between power and area, or any combination thereof), a technology library, synthesis tools, circuit generator options, and / or other target parameters for the data path circuit. Such constraints may, in embodiments, be provided by environment 204 and used to determine the reward 214.

[0066] In one example, the machine learning model 240 may be trained not to add or remove nodes that would cause the delay of the data path circuit to exceed the maximum circuit delay. In some embodiments, the machine learning model 240 may be constrained by the arrival time of the input 315 of the data path circuit, for example, some delay associated with receiving the input 315 rather than the delay associated with the data path circuit itself. In some embodiments, the machine learning model 240 may be constrained by target parameters. For example, the machine learning model 240 may be trained to modify (e.g., design) the data path circuit until it reaches a given area, power consumption, and / or delay. In another example, the machine learning model 240 may be trained to modify (e.g., design) the data path circuit to optimize it for a weighted equilibrium between improvements for area, improvements for power consumption, and improvements for delay, using weight parameters. In other embodiments, the machine learning model 240 may be trained to modify the data path circuit for a given area, a given power consumption, and / or a given delay.

[0067] In some embodiments, the machine learning model 240 may be constrained by the circuit generator option. That is, there may be different ways of generating data path circuits for the same parallel prefix graph, and for example, the predicted physical implementation form of the data path circuit may vary based on which option is selected to predict the physical implementation form. For example, for an adder data path circuit, the prefix adder circuit generally performs the following function: O(n+1) = A(n) + B(n) + C(1) Here, "O" is an output with length n+1 bits, "A" and "B" are binary inputs with length "n" bits, and "C" is an input with length 1 bit. In some embodiments, the circuit generator can have different options that produce different functions and results. For example, the following functions are possible: 1) No carry-in (e.g., no "C" value): O(n+1)=A(n)+B(n), 2) No carry-out: O(n) = A(n) + B(n) + C(1), 3) No carry-in or carry-out: O(n) = A(n) + B(n) 4) When one or more bits of the input "A / B / C" are fixed at values ​​of 1 and 0. In some embodiments, system 200 can be selected from any of the options described above. In such embodiments, agent 202 or environment 204 can generate different types of circuits from the same parallel prefix graph based on which option is selected. For example, the same prefix graph can produce different areas, delays, power consumption, or any combination thereof based on the selected settings for the circuit generator. Thus, agent 202 is trained based on the specific settings and options selected for the data path circuit. In some embodiments, prefix adder generation can also be influenced by the selection of recurrence (e.g., Weinberger or Ring) or the selection of bitwise carry propagation logic (XOR or OR). In some embodiments, the various options for the circuit generator described above may apply to adder circuits but not to other data path circuits, such as priority encoder circuits.

[0068] In some embodiments, the system 200 may choose to include connected logic in the data path circuit 108. For example, the environment 204 may use a circuit generator that generates prefix circuits corresponding to a prefix graph, along with additional circuit logic connected to the inputs and / or outputs of the prefix circuits. In such an example, agent 202 is trained based on specific additional circuit logic connected to the prefix circuits. In such an example, prefix graph assessor 242 may assess the area, power consumption, and / or delay of the prefix circuits and the combined additional circuit logic.

[0069] In other embodiments, the machine learning model 240 may be constrained by a weighted equilibrium between optimizing area, power consumption, and / or delay. For example, the prefix graph assessor may use a weighted equilibrium between estimated area, power consumption, and / or delay when calculating the reward 214. That is, as described above, modifying a data path circuit to reduce the area, power, or delay associated with the data path circuit may result in an increase in different properties of the data path circuit; for example, reducing delay may result in an increase in the area of ​​the data path circuit. For example, modifying prefix graph 305 to generate prefix graph 310 may result in a decrease in the delay of the data path circuit but an increase in the area of ​​the data path circuit. Thus, the machine learning model 240 may be trained to optimize the data path circuit according to the respective weights assigned to delay and area.

[0070] In one example, Figure 6 shows possible weighted values ​​on the curve between delay and area for a data path circuit. In some embodiments, the weighted value 610 represents a weight of 1. In such embodiments, a reward may be determined that causes the machine learning model 240 to attempt to optimize (minimize) only the delay, for example, reinforcement learning trains the machine learning model 240 to take action 210 that reduces the delay without regard to the resulting decrease or increase in the area of ​​the data path circuit. In some embodiments, the weighted value 602 represents a weight of 0. In such embodiments, the machine learning model 240 is trained to optimize (minimize) only the area of ​​the data path circuit, for example, reinforcement learning trains the machine learning model 240 to take action 210 that reduces the area without regard to the resulting decrease or increase in the delay of the data path circuit. The weighted value 608 may represent a greater optimization for delay than for area, and the weighted value 604 may represent a greater optimization for area than for delay. In some embodiments, a weighted value of 606 can represent equal optimization of delay and area.

[0071] In some embodiments, the following formula can be used to determine how much the machine learning model 240 and / or environment 204 should be optimized between delay and area.

number

[0072] In some embodiments, a grid representation is used to represent a prefix graph, as shown in Figure 4. The use of a grid representation of a prefix graph allows the prefix graph to be processed by a machine learning model. A grid representation is a concise representation of a prefix graph in a grid (for example, in a two-dimensional grid). Each row and each column of the grid representation of a prefix graph may be associated with a different input of the prefix graph, and each intersection of rows and columns may be associated with a node of the prefix graph. In one embodiment, the state space S of system 200 consists of all valid N input prefix graphs. An N input graph can be represented by an N × N grid with rows representing the MSB and columns representing the LSB. Note that in embodiments, input nodes (MSB=LSB) are on the diagonal, output nodes are on the first column (LSB=0), and locations above the diagonal (LSB>MSB) may not contain nodes. In the embodiment, the remaining (N-1)(N-2) / 2 locations, which may or may not have non-input / output nodes, are in the N-input prefix graph.

number

[0073] Referring to Figures 3 and 4, each value in the grid can represent an input 315 (e.g., an input node) or a potential node 320 on the prefix graph 305. In such embodiments, each node 320 may have a range (e.g., a position) on the grid 405 that corresponds to its location on the prefix graph 305. Each range may include a row index as its first element and a column index as its second element. For example, node 320-a may be represented by range 420-a(1,0), and node 320-a receives a first input (e.g., 1 or 315-b) and a second input (e.g., 0 or 315-a).

[0074] As will be explained with reference to Figure 3, each output 325 of the prefix graph receives a value directly or indirectly from any previous input 315. A valid prefix graph will have nodes 320 associated with a continuous range, for example, (2,0) or (3,1), where node 320 receives inputs at indices 0-2 or 1-3, respectively. In such an embodiment, ranges where the first element is smaller than the second element are not possible; for example, (0,3) is not a possible range for a node.

[0075] In some embodiments, the machine learning model 240 can receive a grid representation of the initial state of the data path circuit and can select nodes 320 for addition or removal from the prefix graph. The machine learning model 240 can then output a grid representation of the nodes for addition or removal. For example, the machine learning model 240 for a Q network can populate the output grid representation with Q values ​​at any node location. The Q value at any location will correspond to the Q value for the node 320 corresponding to that location for an add action (if the node does not exist) or a remove action (if the node exists). In some embodiments, the machine learning model 240 can output multiple grid representations of the same dimension. For example, if the machine learning model 240 is a Q network optimized for the area and delay of the data path circuit 108, the machine learning model 240 can output Q area Grid representation and Q delay A grid representation of this can be output. In such an example, the action with the highest weighted Q value would be selected to either add node 320 or remove node 320 from the prefix graph.

[0076] In some embodiments, the machine learning model 240 can receive a grid representation of the current design state 206. For example, the machine learning model 240 can receive a generated grid 405 when the current design state 206 is represented by a prefix graph 305. In some embodiments, the machine learning model 240 can modify the prefix graph 305 by selecting nodes to be added or removed from the grid 405. In one embodiment, the machine learning model 240 can choose to add node 420-d (e.g., (3,2)) to grid 405 in order to generate grid 410. In at least some embodiments, grid 410 can represent a modification to the prefix graph 305. For example, by adding range 420-d to the grid for the prefix graph, the prefix graph 305 may be modified to generate a grid for prefix graph 310 with the additional node 320-d.

[0077] In some embodiments, agent 202 (for example, agent 202's machine learning model 240) can output an action. The output action may be an action to update the prefix graph (for example, a new node to add, or an existing node to delete). In one example, a grid representation to which a node should be added at (3,2) is output. Environment 204 receives the output action 210 (for example, the add action along with the node location (3,2) in the grid representation). The environment may be configured to operate on the prefix graph, on the grid representation of the prefix graph, or on other representations of the prefix graph. In some embodiments, environment 204 receives the node location on the grid representation of the prefix graph (for example, (3,2)), updates the grid from the previous state (for example, grid 405) to the grid (for example, grid 410), and then generates the prefix graph 310.

[0078] As described elsewhere in this disclosure, the machine learning model 240 may be trained using reinforcement learning so that the actions taken 210 optimize the data path circuit with respect to area, power consumption, and / or delay. That is, the machine learning model 240 may be trained so that it reduces delay, area, power, and / or any combination thereof according to weighted constraints, for example, so that it decides to add a node 320-d having the range (3,2) to optimize one or more properties associated with the data path circuit. As described elsewhere, in some embodiments, several parameters in the reinforcement learning algorithm, such as a discount factor, may be configured to train the machine learning model 240 to select a sequence of actions that optimize one or more properties associated with the data path circuit 108 over time.

[0079] Figure 7 shows an exemplary Figure 700 of a data path circuit design using reinforcement and machine learning in at least one embodiment. The reinforcement learning shown in Figure 700 may be implemented by processing logic comprising hardware, software, firmware, or any combination thereof. In at least one embodiment, the operation shown in Figure 700 includes operations performed by agent 202, machine learning model 240, and / or environment 204, as described with reference to Figure 2. Although shown in a specific sequence or order, the order of processes may be modified unless otherwise specified. Thus, the illustrated embodiment should be understood as an example only, the illustrated processes may be performed in a different order, and some processes may be performed in parallel. Furthermore, one or more processes may be omitted in various embodiments. Thus, not all processes are required in every embodiment.

[0080] In operation 705, agent 202 may receive the current design state 206 of the data path circuit. In some examples, agent 202 may receive a prefix graph corresponding to the current design state 206 of the data path circuit. In at least one embodiment, agent 202 may receive or generate a grid representation of the prefix graph as described with reference to Figure 4. In some embodiments, agent 202 may send the grid representation of the current design state 206 to the machine learning model 240. In some embodiments, agent 202 may also receive a reward (e.g., reward 208 as described with reference to Figure 2). In such embodiments, agent 202 may train the machine learning model 240 using reinforcement techniques based on the received reward. For example, if the reward indicates that a previous action taken by the machine learning model 240 resulted in a reduction of area, delay, or power associated with the data path circuit, the agent may train the machine learning model 240 to continue taking similar actions.

[0081] In operation 710, the machine learning model 240 can receive a grid representation of the parallel prefix circuit. In some embodiments, the machine learning model 240 can select an action to modify the grid representation of the prefix graph. In some embodiments, the machine learning model 240 can select an action to add or remove nodes from the prefix graph. For example, the machine learning model can select a range on the grid representation that corresponds to adding or removing nodes on the prefix graph, as described with reference to Figures 2 and 4.

[0082] In operation 715, agent 202 can send an action (for example, adding a node at (3,2)) to environment 204.

[0083] In operation 720, the environment 204 can calculate the next reward 214 based on modifications to the grid representation of the prefix graph. For example, the environment 204 can modify the grid representation based on an action received from agent 202, and then modify the prefix graph accordingly. In some embodiments, the environment 204 can first determine whether the received action causes a design state 206 to become active or deactivate, as described with reference to Figure 2. In some embodiments, if the environment 204 determines that the action causes the state to become deactivate, the environment 204 can verify the state by adding or removing an invalid node from the prefix graph. In some embodiments, after determining that the modified state after applying the action received from agent 202 is active, or after verifying the modified state after applying the action received from agent 202, the environment 204 can generate a prefix graph representing the next design state 212. In some embodiments, the environment 204 can calculate the area, delay, and / or power for the current design state 206 and the modified next design state 212 after generating a prefix graph representing the next design state 212. In one embodiment, the environment 204 can calculate the next reward 214 by performing circuit synthesis for each of the design state 206 and the next design state 212. In such embodiments, the environment 204 can determine the area, delay, and / or power for the design state 206 and the next design state 212, determine the difference between the two, and then determine the next reward 214 as described with reference to Figure 5.

[0084] In some embodiments, the environment 204 may include a metrics predictor model (e.g., a second machine learning model or a second model) for calculating the next reward 214. In such embodiments, the metrics predictor model may be trained to predict delay, area, and / or power for each state of the data path circuit, for example, for design state 206 and for the next design state 212. For example, initially, the environment 204 can determine the area, power, and / or delay of the data path circuit modified by using circuit synthesis. A database may store each state generated by the environment and the area, power, and / or delay metrics associated with each state. For example, the database may store design state 206, the next design state 212, and the area, power, and / or delay for design state 206 and the next design state 212. The metrics predictor model may be trained to process the data stored in the database and predict the area, power, and delay for each state. In other words, a metric predictor model can be trained to predict the area, delay, and / or power associated with a state, based on receiving a state as input and processing data stored in a database, for example, by processing previous states and their respective delays, areas, and powers. In an embodiment, the metric predictor model can be trained to receive a grid representation of a prefix graph as input. Thus, when the metric predictor model is trained, the environment can feed the metric predictor model a prefix graph (or grid representation) associated with a given state. In such an embodiment, the metric predictor model can predict the delay, area, and / or power for each state. To calculate a reward, the environment 204 can find the difference between the predicted delays, areas, and / or powers output by the metric predictor model. For example, the environment 204 can provide the metric predictor model with a design state 206 and the next design state 212.In such embodiments, the metric predictor model can predict the delay, area, and / or power for design state 206 and for the next design state 212. The environment 204 can then find the difference between the delay, area, and / or power for the next design state 212 and the delay, area, and / or power for design state 206 in order to determine the next reward 214. In some embodiments, the environment 204 can then send the reward 214 back to agent 202. In such embodiments, agent 202 can utilize the next reward 214 to train a machine learning model 240 via reinforcement learning techniques as described herein. In some embodiments, using machine learning to predict area, delay, and power can require fewer resources and consume less time than performing circuit synthesis.

[0085] Figure 8 shows an exemplary system 800 that performs reinforcement learning using a machine learning model 240, according to at least one embodiment. In some embodiments, system 800 may include agents (e.g., actors, circuit modifiers, etc.) 202 and environments (e.g., simulation environments, circuit synthesizers, etc.) 204, as described with reference to Figure 2. In some embodiments, system 800 may include a parallel circuit synthesizer 810, a database 810, and an optimizer 825. In some embodiments, agent 202 may include the machine learning model 240, as described with reference to Figure 2. In some examples, the optimizer 825 may be included in agent 202. In at least one embodiment, system 200 may be used to design a data path circuit 108, as described with reference to Figure 1.

[0086] As illustrated with reference to Figure 2, agent 202 may be configured to select an action to correct the state of the data path circuit. In some embodiments, environment 204 may be configured to apply the correction, verify the corrected state if necessary, and return the next state (e.g., new state 805) to agent 202. In some examples, environment 204 may also convert the state generated after applying the action 210 taken by agent 202 into a prefix graph, for example, as illustrated with reference to Figure 4, the grid generated by applying the action from agent 202 into a prefix graph. In some embodiments, system 800 may continue this process until the design of the data path circuit is optimized, as illustrated with reference to Figure 2. In some embodiments, system 800 may include multiple agents 202 and multiple environments 204 to carry out the process.

[0087] In at least one embodiment, system 800 can calculate a reward for the transition from the initial state 805 to the new state 805, independently of applying the action 805 from agent 202. That is, system 800 demonstrates an alternative method for utilizing reinforcement learning to design the data path circuit 108 compared to the methods shown in Figures 2 and 7.

[0088] For example, in one embodiment, agent 202 can send an action to be taken (e.g., adding or removing a node) to both environment 204 and database 820. Similarly, environment 204 can send its current state to agent 202, as well as to database 820 and parallel circuit synthesis 810. In such an embodiment, environment 204 and agent 202 can cycle through actions and states without waiting for circuit synthesis 810 to convert the prefix graph into a predicted physical data path circuit to determine the reward, for example, agent 202 and environment 204 can cycle through states and actions in a shorter duration.

[0089] In some embodiments, the parallel circuit synthesizer 810 may include one or more CPUs that synthesize prefix graphs into predicted physical data path circuits. In such embodiments, the parallel circuit synthesizer 810 may also calculate a reward for each state 805 received from the environment 204, for example, for each prefix graph associated with each state 805. That is, the parallel circuit synthesizer 810 may determine the area, power, and delay for each prefix graph and state 805 received from the environment 204. In some embodiments, the parallel circuit synthesizer 810 may calculate multiple rewards 815 and synthesize multiple prefix graphs simultaneously, for example, each CPU included in the parallel circuit synthesizer 810 may calculate a reward 815 for a different state 805. In some examples, utilizing the parallel circuit synthesizer 810 can reduce the time required to calculate the reward 815 for each state 805. In some embodiments, the parallel circuit synthesizer 810 may send the calculated rewards 815 for each state 805 to the database 820.

[0090] In some embodiments, the database 820 may be configured to store states 805, actions 210, and rewards 815, for example, rewards 815 for each action 210 taken on the transition from a first state 805 to a second state 805. In some embodiments, the database may send states 805, actions 210, and rewards 815 to the database, for example, a first state 805, a second state 805, each action 210 used to proceed from the first state 805 to the second state 805, and rewards calculated for modification. In some embodiments, the optimizer 825 may access the database for states 805, actions 210, and rewards 815.

[0091] In some embodiments, the optimizer 825 can receive a state 805, an action 210, and a reward 815 from a database. For example, the optimizer 825 can receive a first state 805, a second state 805, each action 210 used to move from the first state 805 to the second state 805, and a reward 815 calculated for the correction. The optimizer 825 can determine whether the action 210 taken resulted in a reduction in the area, delay, or power of the data path circuit. In some examples, the optimizer 825 can train a machine learning model 240 in response to its determination of whether the action 210 taken reduced the area, power, and / or delay of the data path circuit 108. For example, if the optimizer 825 determines that action 210 failed to reduce the area, delay, and / or power of the data path circuit, the optimizer 825 can update or train the machine learning model 240 to avoid taking such action 210 again. Thus, the machine learning model 240 may be trained using reinforcement learning. In such an embodiment, the updated machine learning model 240 may take different actions 210, causing, for example, agent 202 to take different actions 210 in response to being trained or updated. In this, initially, agent 202 may take multiple actions 802 using an initial machine learning algorithm. While agent 202 cycles through multiple actions, the parallel circuit synthesizer 810 can simultaneously compute a reward 815 for each action 210 taken. Since synthesis takes longer than performing an action 210, agent 202 may continue to cycle through several actions 210 before any reward 815 is computed. In some embodiments, when the reward 815 is calculated, the database can send the information to the optimizer 825. Thus, the machine learning model 240 is updated, which can cause the agent to take action 210 using the updated machine learning algorithm.Agent 202 can then cycle through action 210 using the updated machine learning algorithm until an additional reward 815 is calculated and used by the optimizer 825 to update the machine learning algorithm a second time. System 800 can use this method to continue calculating rewards independently of Agent 202 and the environment 204, for example, by periodically updating the machine learning model 240 while action 210 cycles through until the optimal target parameters for the data path circuit 108 are met.

[0092] Figure 9A shows a flowchart of Method 900 for designing data path circuits using reinforcement learning. Method 900 can be implemented by processing logic comprising hardware, software, firmware, or any combination thereof. In at least one embodiment, Method 900 is implemented by System 200 as described with reference to Figure 2. Although shown in a specific sequence or order, the order of processes can be modified unless otherwise specified. Thus, the illustrated embodiments should be understood as examples only, the illustrated processes may be implemented in different orders, and some processes may be implemented in parallel. Furthermore, one or more processes may be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other diagrams illustrating methods for designing data path circuits are possible.

[0093] In operation 905, the processing logic can receive a first design state of the data path circuit. In one embodiment, the data path circuit may comprise a parallel prefix circuit. In at least one embodiment, the processing logic can generate a first parallel prefix graph representing the first design state in response to receiving the first design state. That is, the processing logic can use the first parallel prefix graph to represent the first design state of the parallel prefix circuit. In at least one embodiment, the processing logic can generate a grid representation of the first parallel prefix graph.

[0094] In operation 910, the processing logic can input a first design state of the data path circuit to the machine learning model. In at least one embodiment, the processing logic generates a grid representation of the first parallel prefix graph and then inputs that grid representation to the machine learning model.

[0095] In operation 915, the processing logic performs reinforcement learning using a machine learning model to cause the machine learning model to output the final design state of the data path circuit. The final design state may be achieved after multiple iterations of reinforcement learning, where, for each iteration, a different design state is generated and assessed, and where, for each iteration, the machine learning model is trained to produce a new design state that is an improvement over the previous design state. In some embodiments, the final design state of the data patch circuit is associated with a final parameter value that is closer to the target parameter value associated with the first design state. That is, the processing logic can use reinforcement learning to optimize the design of the data path circuit as described with reference to Figure 2. In some embodiments, the first parameter value and the final parameter value may represent the area associated with the data path circuit, the delay associated with the data path circuit, the predicted power consumption associated with the data path circuit, or any combination thereof. Thus, the final design state of the data patch circuit is associated with a final parameter value that has an area, delay, or power consumption smaller than the first parameter value. In some embodiments, the first and second parameter values ​​can represent weighted values, for example, predictions of weighted values ​​as described with reference to Figure 6. In some embodiments, the machine learning model outputs actions that cause the processing logic to construct a grid representation of the final prefix graph representing the final design state. That is, the machine learning model can output actions, and the processing logic can use these actions to generate a grid representation and convert the grid representation into a prefix graph representing the final design state. In some embodiments, the machine learning model can iteratively modify the design state of the data path circuit from a first design state to a final design state, where, during each iteration, the machine learning model removes or adds nodes to the graph of the data path circuit, and each node in the graph is associated with one or more components of the data path circuit, for example, one or more logic gates of the data path circuit.

[0096] Figure 9B shows a flowchart of Method 902 for designing data path circuits using reinforcement learning. Method 902 can be implemented by processing logic comprising hardware, software, firmware, or any combination thereof. In at least one embodiment, Method 902 is implemented by System 200 as described with reference to Figure 2. Although shown in a specific sequence or order, the order of processes can be modified unless otherwise specified. Thus, the illustrated embodiment should be understood as an example only, the illustrated processes may be implemented in a different order, and some processes may be implemented in parallel. Furthermore, one or more processes may be omitted in various embodiments. Thus, not all processes are required in every embodiment. Other diagrams illustrating methods for designing data path circuits are possible.

[0097] In operation 920 of method 902, the processing logic can receive the design state of the data path circuit. In some embodiments, the processing logic can receive a first design state of the data path circuit. In some embodiments, the processing logic can generate a parallel prefix graph representing the design state, for example, a first parallel prefix graph representing the first design state. In some embodiments, the processing logic can generate a grid representation of the parallel prefix graph, for example, a first grid representation of the first parallel prefix graph. In some embodiments, the processing logic can input the grid representation into a machine learning model.

[0098] In operation 925, the processing logic may use a machine learning model to process the design state of a data path circuit to generate a next design state, for example, a second design state. In some embodiments, the machine learning model outputs actions that the environment applies to the design state of the data path circuit to generate a second design state, for example, a modified design state. In some embodiments, the machine learning model may output nodes in the grid representation that should be added to or removed from the first design state to generate the second design state. That is, the machine learning model may output actions that the processing logic can use to construct a grid representation of a second graph associated with the second design state, for example, a second grid representation of a second parallel prefix graph associated with the next design state. In some embodiments, the processing logic can convert the second grid representation of the graph into a second parallel prefix graph.

[0099] In operation 928, the processing logic may determine whether the second design state is valid. If the second design state is invalid, the method proceeds to operation 929. If it is determined that the second design state is valid, the method proceeds to operation 930. In operation 929, the processing logic verifies the parallel prefix graph by modifying the second design state of the data path circuit to create an updated second design state that is valid, for example by adding and / or removing additional nodes. In some embodiments, the processing logic may convert the first and second design states into data path circuit implementation forms. The method then proceeds to operation 930.

[0100] In operation 930, the processing logic may determine a first parameter value for a first design state and a second parameter value for a second design state, for example, a first parameter value for a first design state and a second parameter value for a second design state. In some embodiments, the processing logic may determine area, delay, and power associated with a design state and the next design state, for example, the first and second parameter values ​​may represent area, delay, and power. The processing logic may then determine a first parameter value based on the first area, delay, and / or power (for example, based on a first weighted combination of these values) and a second parameter value based on the second area, delay, and / or power (for example, based on a first weighted combination of these values).

[0101] In some embodiments, processing logic can process a first design state using a second model (e.g., a second machine learning model or a metric predictor model), where the second model outputs a first parameter value associated with the first design state. Similarly, processing logic can process a second design state using a second model, where the second model outputs a second parameter value associated with the second design state. For example, a metric predictor model can process the first and second design states to output a first parameter value and a second parameter value. In such embodiments, the processing device can send a grid representation of a first graph of the first design state as a first input to the second model to receive a first parameter value, and a grid representation of a second graph of the second design state as a second input to the second model to receive a second parameter value. For example, a metric predictor model can output a first parameter value and a second parameter value based on having received the grid of the first graph and the grid representation of the second graph. In some embodiments, a second model may receive a circuit implementation of a first design state as a first input to determine first parameter values, and a circuit implementation of a second design state as a second input to determine second parameter values. For example, a metric predictor model may receive a first parallel prefix graph and a second parallel prefix graph. In some embodiments, processing logic may process the first design state and the second design image using a circuit synthesis tool. In such embodiments, the circuit synthesis tool may output first parameter values ​​and second parameter values. In such embodiments, the circuit synthesis tool may process the first design state while a machine learning model outputs a second design state of the data path circuit.

[0102] In operation 935, the processing logic may update the machine learning model, for example, based on whether the modifications selected by the machine learning model reduced or increased the area, delay, power consumption, or any combination thereof associated with the data path circuit. After updating the machine learning model, in operation 938, the processing logic may determine whether one or more termination criteria have been met. For example, termination criteria may be met after a threshold number of design state iterations, after one or more target objectives (e.g., for area, power, and / or delay) have been met, or after a threshold number of design state iterations have been performed without further improvements to the parameter values ​​(one or more). If the termination criteria are met, the method may proceed to operation 945, in which a final design state for the data path circuit may be selected. The final design state may be, for example, the most recent design state, or a design state encountered during an iteration that obtained the best data path circuit metrics (e.g., best area, delay, power consumption, or any combination thereof) or parameters closest to the target. If the termination criterion is not met, in operation 940, the processing logic may select the next design state to be input to the machine learning model. The next design state may be, for example, the most recent design state or another design state encountered during the iteration. The method may return to operation 925, in which the next design state is processed using the machine learning model. The processing logic may utilize an updated machine learning model. For example, the processing logic may use the updated machine learning model to process a design state or the next design state, for example, the processing logic may use the machine learning model to process a first design state or a second design state. In some embodiments, the updated machine learning model outputs a third design state of the data path circuit, which is a modification of the first or second design state. In at least one embodiment, the processing logic may determine a third parameter value associated with the third design state, the third parameter value being closer to the target than the first or second parameter value. The process may be repeated for a fourth, fifth, and subsequent design state until the termination criterion is met.

[0103] Figure 10A shows the inference and / or training logic 1015 used to perform the inference and / or training operations associated with one or more embodiments. Further details regarding the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B.

[0104] In at least one embodiment, the inference and / or training logic 1015 may include, but not limited to, code and / or data storage 1001 for storing forward and / or output weights and / or input / output data, and / or other parameters, for constituting neurons or layers of a neural network used for training and / or inference in one or more embodiments. In at least one embodiment, the training logic 1015 may include, or be coupled to, code and / or data storage 1001 for storing graph code or other software for controlling timing and / or sequence, and weight and / or other parameter information should be loaded into the code and / or data storage 1001 to constitute logic including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, the code, such as graph code, loads weight or other parameter information into the processor ALU based on the architecture of the neural network to which the code corresponds. In at least one embodiment, the code and / or data storage 1001 stores the weight parameters and / or input / output data of each layer of the neural network being trained or used in conjunction with one or more embodiments during the forward propagation of input / output data and / or weight parameters during training and / or inference using the embodiments of one or more embodiments. In at least one embodiment, any portion of the code and / or data storage 1001 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0105] In at least one embodiment, any portion of the code and / or data storage 1001 may be inside or outside one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or code and / or data storage 1001 may be cache memory, dynamic randomly addressable memory ("DRAM"), static randomly addressable memory ("SRAM"), non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the selection of whether the code and / or code and / or data storage 1001 is inside or outside the processor, or whether it consists of DRAM, SRAM, flash, or some other type of storage, may depend on available storage, on-chip vs. off-chip, latency requirements of the training and / or inference functions being performed, batch size of data used in neural network inference and / or training, or any combination of these factors.

[0106] In at least one embodiment, the inference and / or training logic 1015 may include code and / or data storage 1005 for storing backward and / or output weights and / or input / output data corresponding to neurons or layers of a neural network used to train and / or infer in one or more embodiments. In at least one embodiment, the code and / or data storage 1005 stores weight parameters and / or input / output data for each layer of the neural network used to train or in conjunction with one or more embodiments during backward propagation of input / output data and / or weight parameters during training and / or inference using one or more embodiments. In at least one embodiment, the training logic 1015 may include, or be coupled to, code and / or data storage 1005 for storing graph code or other software for controlling timing and / or sequence, and weight and / or other parameter information should be loaded into the code and / or data storage 1005 to constitute logic including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, the code, such as graph code, loads weight or other parameter information into the processor ALU based on the architecture of the neural network to which the code corresponds. In at least one embodiment, any portion of the code and / or data storage 1005 may be included with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of the code and / or data storage 1005 may be inside or outside one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 1005 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage.In at least one embodiment, the selection of whether the code and / or data storage 1005 is, for example, internal or external to the processor, or whether it consists of DRAM, SRAM, flash or some other type of storage, may depend on the available storage, on-chip vs. off-chip, latency requirements of the training and / or inference functions being performed, the batch size of the data used in the neural network inference and / or training, or any combination of these factors.

[0107] In at least one embodiment, the code and / or data storage 1001 and the code and / or data storage 1005 may be separate storage structures. In at least one embodiment, the code and / or data storage 1001 and the code and / or data storage 1005 may be the same storage structure. In at least one embodiment, the code and / or data storage 1001 and the code and / or data storage 1005 may be partially the same storage structure and partially separate storage structures. In at least one embodiment, any part of the code and / or data storage 1001 or code and / or data storage 1005 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory.

[0108] In at least one embodiment, the inference and / or training logic 1015 may include, but not limited to, one or more arithmetic logic units ("ALUs") 1010, including integer and / or floating-point units, for performing logical and / or mathematical operations that are at least partially based on or shown by training and / or inference code (e.g., graph code), the result of which activations (e.g., output values ​​from layers or neurons in a neural network) stored in activation storage 1020, and these activations are functions of input / output and / or weight parameter data stored in code and / or data storage 1001 and / or code and / or data storage 1005. In at least one embodiment, the activation stored in activation storage 1020 is generated according to linear algebra and / or matrix-based mathematics performed by (one or more) ALU 1010 in response to the execution of an instruction or other code, and the weight values ​​stored in code and / or data storage 1005 and / or code and / or data storage 1001 are used as operands along with other values ​​such as bias values, gradient information, momentum values, or other parameters or hyperparameters, any or all of which may be stored in code and / or data storage 1005 or code and / or data storage 1001, or in other on-chip or off-chip storage.

[0109] In at least one embodiment, the (one or more) ALU1010 are contained within one or more processors or other hardware logic devices or circuits, while in another embodiment, the (one or more) ALU1010 may be outside the processor or other hardware logic devices or circuits (e.g., coprocessors) that use them. In at least one embodiment, the ALU1010 may be contained within an execution unit of a processor, or otherwise contained within a bank of ALUs accessible by execution units of a processor, either within the same processor or distributed across different types of processors (e.g., a central processing unit, a graphics processing unit, a fixed-function unit, etc.). In at least one embodiment, the code and / or data storage 1001, the code and / or data storage 1005, and the activation storage 1020 may reside on the same processor or other hardware logic device or circuitry, while in another embodiment, they may be in different processors or other hardware logic devices or circuits, or in some combination of the same processor or other hardware logic device or circuitry and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of the activation storage 1020 may be included with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Furthermore, the inference and / or training code may be stored with other code accessible from the processor or other hardware logic or circuitry, and may be fetched and / or processed using the processor's fetch, decode, schedule, execute, retirement, and / or other logic circuits.

[0110] In at least one embodiment, the activated storage 1020 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage. In at least one embodiment, the activated storage 1020 may be entirely or partially located within or outside one or more processors or other logic circuits. In at least one embodiment, the selection of whether the activated storage 1020 is, for example, inside or outside the processor, or whether it consists of DRAM, SRAM, flash, or some other storage type may depend on the available storage, on-chip vs. off-chip, latency requirements of the training and / or inference functions being performed, batch size of data used in neural network inference and / or training, or any combination of these factors. In at least one embodiment, the inference and / or training logic 1015 shown in Figure 10A may be used in conjunction with an application-specific integrated circuit ("ASIC"), such as a Tensorflow® processing unit from Google, an inference processing unit (IPU) from Graphcore®, or a Nervana® (e.g., "Lake Crest") processor from Intel Corp. In at least one embodiment, the inference and / or training logic 1015 shown in Figure 10A may be used in conjunction with other hardware, such as a central processing unit ("CPU") hardware, a graphics processing unit ("GPU") hardware, a data processing unit ("DPU") hardware, or a field-programmable gate array ("FPGA").

[0111] Figure 10B shows the inference and / or training logic 1015 in at least one or more embodiments. In at least one embodiment, the inference and / or training logic 1015 may include, but is not limited to, hardware logic in which computational resources are dedicated or, otherwise, used only in conjunction with weight values ​​or other information corresponding to one or more layers of neurons in a neural network. In at least one embodiment, the inference and / or training logic 1015 shown in Figure 10B may be used in conjunction with application-specific integrated circuits (ASICs), such as a Tensorflow® processing unit from Google, an Inference Processing Unit (IPU) from Graphcore®, or a Nervana® (e.g., "Lake Crest") processor from Intel Corp. In at least one embodiment, the inference and / or training logic 1015 shown in Figure 10B may be used in conjunction with other hardware such as a central processing unit (CPU) hardware, a graphics processing unit (GPU) hardware, a data processing unit ("DPU") hardware, or a field-programmable gate array (FPGA). In at least one embodiment, the inference and / or training logic 1015 may include, but are not limited to, code and / or data storage 1001 and code and / or data storage 1005, which may be used to store code (e.g., graph code), weight values, and / or other information including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. In at least one embodiment shown in Figure 10B, each of the code and / or data storage 1001 and code and / or data storage 1005 is associated with a dedicated computing resource, such as computing hardware 1002 and computing hardware 1006, respectively.In at least one embodiment, each of the computation hardware 1002 and computation hardware 1006 comprises one or more ALUs that perform mathematical functions, such as linear algebraic functions, only on information stored in code and / or data storage 1001 and code and / or data storage 1005, respectively, and the results are stored in activation storage 1020.

[0112] In at least one embodiment, each of the code and / or data storages 1001 and 1005 and the corresponding compute hardware 1002 and 1006 correspond to different layers of a neural network, so that the activation resulting from one “storage / compute pair 1001 / 1002” of the code and / or data storage 1001 and compute hardware 1002 is provided as input to the “storage / compute pair 1005 / 1006” of the code and / or data storage 1005 and compute hardware 1006 in order to mirror the conceptual organization of the neural network. In at least one embodiment, the storage / compute pairs 1001 / 1002 and 1005 / 1006 may correspond to two or more neural network layers. In at least one embodiment, additional storage / computation pairs (not shown) may be included in the inference and / or training logic 1015 after or in parallel with the storage / computation pairs 1001 / 1002 and 1005 / 1006.

[0113] Figure 11 shows an exemplary data center 1100 in which at least one embodiment may be used. In at least one embodiment, the data center 1100 includes a data center infrastructure layer 1110, a framework layer 1120, a software layer 1130, and an application layer 1240.

[0114] In at least one embodiment, as shown in Figure 11, the data center infrastructure layer 1110 may include a resource orchestrator 1112, grouped computing resources 1114, and node computing resources ("node CRs") 1116(1) to 1116(N), where "N" represents any positive integer. In at least one embodiment, the node CRs 1116(1) to 1116(N) may include, but are not limited to, any number of central processing units ("CPUs") or other processors (including accelerators, field-programmable gate arrays (FPGAs), data processing units, graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output ("NW I / O") devices, network switches, virtual machines ("VMs"), power modules, and cooling modules, etc. In at least one embodiment, one or more nodes CR from among nodes CR1116(1) to 1116(N) may be servers having one or more of the computing resources described above.

[0115] In at least one embodiment, the grouped computing resources 1114 may include separate groupings of node CRs housed in one or more racks (not shown), or many racks housed in a data center at various geographical locations (also not shown). A separate grouping of node CRs within the grouped computing resources 1114 may include grouped compute resources, network resources, memory resources, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several node CRs, including CPUs or processors, may be grouped in one or more racks to provide compute resources to support one or more workloads. In at least one embodiment, one or more racks may also include any number of power modules, cooling modules, and network switches in any combination.

[0116] In at least one embodiment, the resource orchestrator 1112 may configure or otherwise control one or more nodes CR1116(1) to 1116(N) and / or a grouped computing resource 1114. In at least one embodiment, the resource orchestrator 1112 may include a software design infrastructure ("SDI") management entity for the data center 1100. In at least one embodiment, the resource orchestrator may include hardware, software, or any combination thereof.

[0117] In at least one embodiment, as shown in Figure 11, the framework layer 1120 includes a job scheduler 1122, a configuration manager 1124, a resource manager 1126, and a distributed file system 1128. In at least one embodiment, the framework layer 1120 may include a framework for supporting software 1132 of the software layer 1130, and / or one or more applications 1142 of the application layer 1140. In at least one embodiment, the software 1132 or (one or more) applications 1142 may include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure, respectively. In at least one embodiment, the framework layer 1120 may be a type of free and open-source software web application framework, such as Apache Spark® ("Spark"), which can leverage the distributed file system 1128 for large-scale data processing (e.g., "big data"). In at least one embodiment, the job scheduler 1122 may include a Spark driver to facilitate scheduling of workloads supported by various layers of the data center 1100. In at least one embodiment, the configuration manager 1124 may be able to configure different layers, such as the software layer 1130 and the framework layer 1120, which includes Spark and a distributed file system 1128 to support large-scale data processing. In at least one embodiment, the resource manager 1126 may be able to manage clustered or grouped computing resources that are mapped or allocated to support the distributed file system 1128 and the job scheduler 1122. In at least one embodiment, the clustered or grouped computing resources may include grouped computing resources 1114 in the data center infrastructure layer 1110.In at least one embodiment, the resource manager 1126 may work in conjunction with the resource orchestrator 1112 to manage these mapped or allocated computing resources.

[0118] In at least one embodiment, the software 1132 contained within the software layer 1130 may include software used by nodes CR1116(1) to 1116(N), grouped computing resources 1114, and / or at least a portion of the distributed file system 1128 of the framework layer 1120. One or more types of software may include, but are not limited to, internet web page search software, email virus scanning software, database software, and streaming video content software.

[0119] In at least one embodiment, one or more applications 1142 contained within the application layer 1140 may include one or more types of applications used by nodes CR1116(1) to 1116(N), grouped computing resources 1114, and / or at least a portion of the distributed file system 1128 of the framework layer 1120. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive compute, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), or other machine learning applications used in conjunction with one or more embodiments.

[0120] In at least one embodiment, any of the configuration manager 1124, resource manager 1126, and resource orchestrator 1112 may implement any number and type of self-correcting actions based on any amount and type of data obtained in any technically feasible manner. In at least one embodiment, the self-correcting actions may relieve the data center operator of data center 1100 of the task of determining potentially faulty configurations and potentially avoiding underutilized and / or underperforming portions of the data center.

[0121] In at least one embodiment, the data center 1100 may include tools, services, software, or other resources for training one or more machine learning models or for predicting or inferring information using one or more machine learning models according to one or more embodiments described herein. For example, in at least one embodiment, a machine learning model may be trained by computing weight parameters according to a neural network architecture using the software and computing resources described above with respect to the data center 1100. In at least one embodiment, a trained machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to the data center 1100 by using weight parameters computed through one or more training techniques described herein.

[0122] In at least one embodiment, the data center may use a CPU, application-specific integrated circuit (ASIC), GPU, DPU, FPGA, or other hardware to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above may be configured as services that enable users to train information or perform inference on information, such as image recognition, speech recognition, or other artificial intelligence services.

[0123] The inference and / or training logic 1015 is used to perform inference and / or training operations associated with one or more embodiments. Details relating to the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B. In at least one embodiment, the inference and / or training logic 1015 may be used in the system of Figure 11 for inference or prediction operations, at least in part, based on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0124] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0125] Figure 12 is a block diagram showing an exemplary computer system, which may be a system 1200 having interconnected devices and components, a system-on-a-chip (SOC), or any combination thereof, formed together with a processor that may include an execution unit for executing instructions, according to at least one embodiment. In at least one embodiment, the computer system 1200 may include components such as a processor 1202 for employing an execution unit including logic for implementing algorithms for process data, as described herein, but not limited to embodiments described herein. In at least one embodiment, the computer system 1200 may include a processor such as the PENTIUM® processor family, Xeon®, Itanium®, XScale®, and / or StrongARM®, Intel® Core®, or Intel® Nervana® microprocessors available from Intel Corporation in Santa Clara, California, but other systems (including PCs with other microprocessors, engineering workstations, set-top boxes, etc.) may also be used. In at least one embodiment, the computer system 1200 may run a version of the WINDOWS® operating system available from Microsoft Corporation in Redmond, Washington, but other operating systems (e.g., UNIX® and Linux®), embedded software, and / or graphical user interfaces may also be used.

[0126] The embodiments may be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol devices, digital cameras, personal digital assistants ("PDAs"), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor ("DSP"), a system-on-a-chip, a network computer ("NetPC"), a set-top box, a network hub, a wide-area network ("WAN") switch, an edge device, an Internet of Things ("IoT") device, or any other system capable of carrying out one or more instructions according to at least one embodiment.

[0127] In at least one embodiment, the computer system 1200 may include, but is not limited to, a processor 1202, which may include, but is not limited to, one or more execution units 1208 for performing machine learning model training and / or inference by the techniques described herein. In at least one embodiment, the computer system 1200 is a single-processor desktop or server system, but in another embodiment, the computer system 1200 may be a multi-processor system. In at least one embodiment, the processor 1202 may include, but is not limited to, a complex instruction set computer ("CISC") microprocessor, a reduced instruction set computing ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, a processor implementing a combination of instruction sets, or any other processor device, such as a digital signal processor. In at least one embodiment, the processor 1202 may be coupled to a processor bus 1210, and the processor bus 1210 may transmit data signals between the processor 1202 and other components in the computer system 1200.

[0128] In at least one embodiment, the processor 1202 may include, but is not limited to, a level 1 ("L1") internal cache memory ("cache") 1204. In at least one embodiment, the processor 1202 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory may reside outside the processor 1202. Other embodiments may also include a combination of both internal and external caches, depending on the specific implementation and requirements. In at least one embodiment, the register file 1206 may store different types of data in various registers, including, but is not limited to, integer registers, floating-point registers, status registers, and instruction pointer registers.

[0129] In at least one embodiment, but not limited to, an execution unit 1208 containing logic for performing integer and floating-point arithmetic may also be present in the processor 1202. In at least one embodiment, the processor 1202 may also include a microcode ("u-code") read-only memory ("ROM") for storing microcode for several macro instructions. In at least one embodiment, the execution unit 1208 may include logic for handling a packed instruction set 1209. In at least one embodiment, by including the packed instruction set 1209, along with the associated circuit elements for executing the instructions, in the instruction set of the general-purpose processor 1202, arithmetic used by many multimedia applications can be performed using packed data in the general-purpose processor 1202. In one or more embodiments, many multimedia applications may be accelerated and run more efficiently by using the full width of the processor's data bus to perform arithmetic on packed data, which may eliminate the need to transfer smaller units of data across the processor's data bus to perform one or more arithmetic operations, one data element at a time.

[0130] In at least one embodiment, the execution unit 1208 may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuits. In at least one embodiment, the computer system 1200 may include, but is not limited to, memory 1220. In at least one embodiment, memory 1220 may be implemented as a dynamic random access memory ("DRAM") device, a static random access memory ("SRAM") device, a flash memory device, or other memory device. In at least one embodiment, memory 1220 may store (one or more) instructions 1219 and / or data 1221, which are represented by data signals that can be executed by the processor 1202.

[0131] In at least one embodiment, a system logic chip may be coupled to the processor bus 1210 and memory 1220. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub ("MCH") 1216, and the processor 1202 may communicate with the MCH 1216 via the processor bus 1210. In at least one embodiment, the MCH 1216 may provide a high-bandwidth memory path 1218 to memory 1220 for instruction and data storage, as well as for the storage of graphics commands, data, and textures. In at least one embodiment, the MCH 1216 may direct data signals between the processor 1202, memory 1220, and other components in the computer system 1200, and bridge data signals between the processor bus 1210, memory 1220, and system I / O 1222. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, the MCH1216 may be coupled to memory 1220 through a high-bandwidth memory path 1218, and the graphics / video card 1212 may be coupled to the MCH1216 via an Accelerated Graphics Port ("AGP") interconnect 1214.

[0132] In at least one embodiment, the computer system 1200 may use a system I / O 1222, which is a proprietary hub interface bus for coupling the MCH 1216 to the I / O controller hub ("ICH") 1230. In at least one embodiment, the ICH 1230 may provide direct connectivity to several I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripherals to memory 1220, the chipset, and the processor 1202. Examples may include, but are not limited to, an audio controller 1229, a firmware hub ("Flash BIOS") 1228, a wireless transceiver 1226, data storage 1224, a legacy I / O controller 1223 including a user input and keyboard interface 1225, a serial expansion port 1227 such as a Universal Serial Bus ("USB"), and a network controller 1234 which in some embodiments may include a data processing unit. The data storage 1224 may comprise a hard disk drive, a floppy disk drive, a CD-ROM device, a flash memory device, or other mass storage device.

[0133] In at least one embodiment, Figure 12 shows a system including interconnected hardware devices or “chips,” while in other embodiments, Figure 12 may show an exemplary system-on-a-chip (“SoC”). In at least one embodiment, devices may be interconnected by proprietary interconnects, standard interconnects (e.g., PCIe), or any combination thereof. In at least one embodiment, one or more components of computer system 1200 are interconnected using compute express link (CXL) interconnects.

[0134] The inference and / or training logic 1015 is used to perform inference and / or training operations associated with one or more embodiments. Details relating to the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B. In at least one embodiment, the inference and / or training logic 1015 may be used in the system of Figure 12 for inference or prediction operations, at least in part, based on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0135] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0136] Figure 13 is a block diagram showing an electronic device 1300 for utilizing the processor 1310 according to at least one embodiment. In at least one embodiment, the electronic device 1300 may be, for example, a notebook, a tower server, a rack server, a blade server, a laptop, a desktop, a tablet, a mobile device, a telephone, an embedded computer, an edge device, an IoT device, or any other suitable electronic device.

[0137] In at least one embodiment, the system 1300 may include a processor 1310 communicably coupled to any number or type of preferred components, peripherals, modules, or devices, but not limited to these. In at least one embodiment, the processor 1310 is coupled using a bus or interface such as a 1°C bus, a System Management Bus ("SMBus"), a Low Pin Count (LPC) bus, a Serial Peripheral Interface ("SPI"), a High Definition Audio ("HDA") bus, a Serial Advance Technology Attachment ("SATA") bus, a Universal Serial Bus ("USB") (versions 1, 2, or 3), or a Universal Asynchronous Receiver / Transmitter ("UART") bus. In at least one embodiment, Figure 13 shows a system including interconnected hardware devices or “chips,” while in other embodiments, Figure 13 may show an exemplary system-on-a-chip (“SoC”). In at least one embodiment, the devices shown in Figure 13 may be interconnected by proprietary interconnects, standard interconnects (e.g., PCIe), or any combination thereof. In at least one embodiment, one or more components of Figure 13 are interconnected using a Compute Express Link (CXL) interconnect.

[0138] In at least one embodiment, Figure 13 includes a display 1324, a touchscreen 1325, a touchpad 1330, a Near Field Communication ("NFC") unit 1345, a sensor hub 1340, a thermal sensor 1346, an Express Chipset ("EC") 1335, a Trusted Platform Module ("TPM") 1338, a BIOS / firmware / flash memory ("BIOS,FW flash") 1322, a DSP 1360, a drive 1320 such as a Solid State Disk ("SSD") or Hard Disk Drive ("HDD"), a Wireless Local Area Network ("WLAN") unit 1350, a Bluetooth unit 1352, and a Wireless Wide Area Network ("WWAN") unit. This may include a network (1356), a Global Positioning System (GPS) (1355), a camera such as a USB 3.0 camera ("USB 3.0 camera") (1354), and / or a Low Power Double Data Rate ("LPDDR") memory unit ("LPDDR3") (1315) implemented, for example, in the LPDDR3 standard. Each of these components may be implemented in any preferred manner.

[0139] In at least one embodiment, other components may be communicatively coupled to the processor 1310 through the components described above. In at least one embodiment, the accelerometer 1341, the ambient light sensor ("ALS") 1342, the compass 1343, and the gyroscope 1344 may be communicatively coupled to the sensor hub 1340. In at least one embodiment, the thermal sensor 1339, the fan 1337, the keyboard 1336, and the touchpad 1330 may be communicatively coupled to the EC 1335. In at least one embodiment, the speaker 1363, the headphones 1364, and the microphone ("mic") 1365 may be communicatively coupled to the audio unit ("audio codec and class d amplifier") 1362, and the audio unit 1362 may be communicatively coupled to the DSP 1360. In at least one embodiment, the audio unit 1364 may include, for example, an audio coder / decoder ("codec") and a Class D amplifier. In at least one embodiment, a SIM card ("SIM") 1357 may be communicatively coupled to the WWAN unit 1356. In at least one embodiment, components such as the WLAN unit 1350 and the Bluetooth unit 1352, as well as the WWAN unit 1356, may be implemented in a Next Generation Form Factor ("NGFF").

[0140] The inference and / or training logic 1015 is used to perform inference and / or training operations associated with one or more embodiments. Details relating to the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B. In at least one embodiment, the inference and / or training logic 1015 may be used in the system of Figure 13 for inference or prediction operations, at least in part, based on weight parameters calculated using the neural network training operations, neural network functions and / or architectures, or neural network use cases described herein.

[0141] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0142] Figure 14 is a block diagram of a processing system according to at least one embodiment. In at least one embodiment, system 1400 includes one or more processors 1402 and one or more graphics processors 1408, and may be a single-processor desktop system, a multi-processor workstation system, or a server system having a large number of processors 1402 or processor cores 1407. In at least one embodiment, system 1400 is a processing platform embedded in a system-on-a-chip (SoC) integrated circuit for use in mobile devices, handheld devices, edge devices, or embedded devices.

[0143] In at least one embodiment, system 1400 may include, or be incorporated within, a server-based gaming platform, a game console including a game and media console, a mobile gaming console, a handheld game console, or an online game console. In at least one embodiment, system 1400 is a mobile phone, a smartphone, a tablet computing device, or a mobile internet device. In at least one embodiment, processing system 1400 may also include, be coupled with, or be incorporated within, wearable devices such as a smartwatch wearable device, a smart eyewear device, an augmented reality device, or a virtual reality device. In at least one embodiment, processing system 1400 is a television or set-top box device having one or more processors 1402 and a graphical interface produced by one or more graphics processors 1408.

[0144] In at least one embodiment, one or more processors 1402 each include one or more processor cores 1407 for processing instructions that perform actions for system and user software when executed. In at least one embodiment, each of the one or more processor cores 1407 is configured to process a particular instruction set 1409. In at least one embodiment, the instruction set 1409 may facilitate computing via composite instruction set computing (CISC), reduced instruction set computing (RISC), or very long instruction words (VLIW). In at least one embodiment, each processor core 1407 may process a different instruction set 1409, and the instruction set 1409 may include instructions to facilitate emulation of other instruction sets. In at least one embodiment, the processor core 1407 may also include other processing devices, such as a digital signal processor (DSP).

[0145] In at least one embodiment, the processor 1402 includes cache memory 1404. In at least one embodiment, the processor 1402 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory is shared among the various components of the processor 1402. In at least one embodiment, the processor 1402 also uses an external cache (e.g., a Level 3 (L3) cache or a Last Level Cache (LLC)) (not shown), and the external cache may be shared among the processor cores 1407 using known cache coherency techniques. In at least one embodiment, additionally, a register file 1406 is included in the processor 1402, and the register file 1406 may include different types of registers for storing different types of data (e.g., integer registers, floating-point registers, status registers, and instruction pointer registers). In at least one embodiment, the register file 1406 may include general-purpose registers or other registers.

[0146] In at least one embodiment, one or more processors 1402 are coupled with one or more interface buses 1410 to transmit communication signals, such as addresses, data, or control signals, between the processors 1402 and other components in the system 1400. In at least one embodiment, the interface bus 1410 in one embodiment may be a processor bus, such as a version of the Direct Media Interface (DMI) bus. In at least one embodiment, the interface 1410 is not limited to the DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), a memory bus, or other types of interface buses. In at least one embodiment, the (one or more) processors 1402 include an integrated memory controller 1416 and a platform controller hub 1430. In at least one embodiment, the memory controller 1416 facilitates communication between the memory device and other components of the system 1400, and the platform controller hub (PCH) 1430 provides connectivity to I / O devices via the local I / O bus.

[0147] In at least one embodiment, the memory device 1420 may be a dynamic random-access memory (DRAM) device, a static random-access memory (SRAM) device, a flash memory device, a phase-change memory device, or any other memory device having performance suitable for acting as process memory. In at least one embodiment, the memory device 1420 may act as system memory for the system 1400 to store data 1422 and instructions 1421 for use when one or more processors 1402 execute an application or process. In at least one embodiment, the memory controller 1416 may also be coupled to an optional external graphics processor 1412, which may communicate with one or more graphics processors 1408 in the processor 1402 to perform graphics and media operations. In at least one embodiment, the display device 1411 may be connected to one or more processors 1402. In at least one embodiment, the display device 1411 may include one or more internal display devices, such as in the case of a mobile electronic device or laptop device, or external display devices attached via a display interface (e.g., DisplayPort). In at least one embodiment, the display device 1411 may include a head-mounted display (HMD), such as a stereoscopic display device for use in a virtual reality (VR) application or an augmented reality (AR) application.

[0148] In at least one embodiment, the platform controller hub 1430 enables peripherals to connect to the memory device 1420 and processor 1402 via a high-speed I / O bus. In at least one embodiment, the I / O peripherals include, but are not limited to, an audio controller 1446, a network controller 1434, a firmware interface 1428, a wireless transceiver 1426, a touch sensor 1425, and a data storage device 1424 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 1424 may be connected via a storage interface (e.g., SATA) or via a peripheral bus such as a peripheral component interconnect bus (e.g., PCI, PCI Express). In at least one embodiment, the touch sensor 1425 may include a touch screen sensor, a pressure sensor, or a fingerprint sensor. In at least one embodiment, the wireless transceiver 1426 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, or Long Term Evolution (LTE) transceiver. In at least one embodiment, the firmware interface 1428 enables communication with system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, the network controller 1434 may enable network connectivity to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to the interface bus 1410. In at least one embodiment, the audio controller 1446 is a multi-channel high-definition audio controller.In at least one embodiment, system 1400 includes an optional legacy I / O controller 1440 for connecting legacy devices (e.g., Personal System 2 (PS / 2)) to the system. In at least one embodiment, platform controller hub 1430 may also connect to one or more Universal Serial Bus (USB) controller 1442-connected input devices, such as a keyboard and mouse combination 1443, a camera 1444, or other USB input devices.

[0149] In at least one embodiment, instances of the memory controller 1416 and the platform controller hub 1430 may be incorporated into a discreet external graphics processor, such as an external graphics processor 1411. In at least one embodiment, the platform controller hub 1430 and / or the memory controller 1416 may be external to one or more processors 1402. For example, in at least one embodiment, the system 1400 may include an external memory controller 1416 and a platform controller hub 1430, which may be configured as a memory controller hub and peripheral controller hub within a system chipset communicating with (one or more) processors 1402.

[0150] The inference and / or training logic 1015 is used to perform the inference and / or training operations associated with one or more embodiments. Details relating to the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B. In at least one embodiment, part or all of the inference and / or training logic 1015 may be incorporated into the graphics processor 1408. For example, in at least one embodiment, the training and / or inference techniques described herein may use one or more ALUs embodied in the graphics processor. Furthermore, in at least one embodiment, the inference and / or training operations described herein may be performed using logic other than the logic shown in Figure 10A or Figure 10B. In at least one embodiment, weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown) that constitute the ALUs of the graphics processor for performing one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0151] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0152] Figure 15 is a block diagram of a processor 1500 having one or more processor cores 1502A to 1502N, an integrated memory controller 1513, and an integrated graphics processor 1508, according to at least one embodiment. In at least one embodiment, the processor 1500 may include additional cores up to additional core 1502N, represented by dashed boxes. In at least one embodiment, each of the processor cores 1502A to 1502N includes one or more internal cache units 1504A to 1504N. In at least one embodiment, each processor core also has access to one or more shared cache units 1506.

[0153] In at least one embodiment, the internal cache units 1504A-1504N and the shared cache unit 1506 represent a cache memory hierarchy within the processor 1500. In at least one embodiment, the cache memory units 1504A-1504N may include at least one level of instruction and data cache within each processor core, and one or more levels of shared intermediate level caches such as Level 2 (L2), Level 3 (L3), Level 4 (L4), or other levels of cache, where the highest level cache before external memory is classified as LLC. In at least one embodiment, cache coherency logic maintains coherency among the various cache units 1506 and 1504A-1504N.

[0154] In at least one embodiment, the processor 1500 may also include a set of one or more bus controller units 1516 and a system agent core 1510. In at least one embodiment, one or more bus controller units 1516 manage a set of peripheral buses, such as one or more PCI or PCI Express buses. In at least one embodiment, the system agent core 1510 provides management functionality for various processor components. In at least one embodiment, the system agent core 1510 includes one or more integrated memory controllers 1513 for managing access to various external memory devices (not shown).

[0155] In at least one embodiment, one or more of the processor cores 1502A to 1502N include support for simultaneous multithreading. In at least one embodiment, the system agent core 1510 includes components for coordinating and operating the cores 1502A to 1502N during multithreaded processing. In at least one embodiment, the system agent core 1510 may additionally include a power control unit (PCU), the PCU including logic and components for regulating the power state of one or more of the processor cores 1502A to 1502N and the graphics processor 1508.

[0156] In at least one embodiment, the processor 1500 additionally includes a graphics processor 1508 for performing graphics processing operations. In at least one embodiment, the graphics processor 1508 is coupled with a system agent core 1510 which includes a shared cache unit 1506 and one or more integrated memory controllers 1513. In at least one embodiment, the system agent core 1510 also includes a display controller 1511 for driving graphics processor outputs to one or more coupled displays. In at least one embodiment, the display controller 1511 may also be a separate module coupled with the graphics processor 1508 via at least one interconnection, or it may be incorporated within the graphics processor 1508.

[0157] In at least one embodiment, a ring-based interconnect unit 1512 is used to connect the internal components of the processor 1500. In at least one embodiment, alternative interconnect units such as point-to-point interconnects, switching interconnects, or other techniques may be used. In at least one embodiment, the graphics processor 1508 is connected to the ring interconnect 1512 via an I / O link 1513.

[0158] In at least one embodiment, I / O link 1513 represents at least one of several types of I / O interconnects, including on-package I / O interconnects that facilitate communication between various processor components and high-performance embedded memory modules 1518, such as eDRAM modules. In at least one embodiment, each of the processor cores 1502A to 1502N and the graphics processor 1508 use the embedded memory module 1518 as a shared last-level cache.

[0159] In at least one embodiment, the processor cores 1502A to 1502N are homogeneous cores that execute a common instruction set architecture. In at least one embodiment, the processor cores 1502A to 1502N are heterogeneous in terms of instruction set architecture (ISA), where one or more of the processor cores 1502A to 1502N execute a common instruction set, and the other cores of one or more of the processor cores 1502A to 1502N execute a subset of the common instruction set or a different instruction set. In at least one embodiment, the processor cores 1502A to 1502N are heterogeneous in terms of microarchitecture, where one or more cores with relatively high power consumption are coupled with one or more power cores with lower power consumption. In at least one embodiment, the processor 1500 may be implemented on one or more chips or as an SoC integrated circuit.

[0160] The inference and / or training logic 1015 is used to perform the inference and / or training operations associated with one or more embodiments. Details relating to the inference and / or training logic 1015 are provided below in conjunction with Figures 10A and / or 10B. In at least one embodiment, part or all of the inference and / or training logic 1015 may be incorporated into the processor 1500. For example, in at least one embodiment, the training and / or inference techniques described herein may use one or more of the graphics processor 1508, (one or more) graphics cores 1502A to 1502N, or other components embodied in Figure 15. Furthermore, in at least one embodiment, the inference and / or training operations described herein may be performed using logic other than the logic shown in Figure 10A or Figure 10B. In at least one embodiment, the weight parameters may be stored in on-chip or off-chip memory and / or registers (shown or not shown) that constitute the ALU of the graphics processor 1500 for implementing one or more machine learning algorithms, neural network architectures, use cases, or training techniques described herein.

[0161] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0162] Figure 16 is an exemplary data flow diagram for a process 1600 that generates and deploys an image processing and inference pipeline, according to at least one embodiment. In at least one embodiment, process 1600 may be deployed in one or more facilities 1602 for use with imaging devices, processing devices, and / or other device types. Process 1600 may run within a training system 1604 and / or deployment system 1606. In at least one embodiment, training system 1604 may be used to carry out the training, deployment, and implementation of machine learning models (e.g., neural networks, object detection algorithms, computer vision algorithms, etc.) for use in deployment system 1606. In at least one embodiment, deployment system 1606 may be configured to offload processing and compute resources between distributed computing environments to reduce infrastructure requirements at facility 1602. In at least one embodiment, one or more applications in the pipeline may use or call services of the deployment system 1606 (e.g., inference, visualization, compute, AI, etc.) while the application is running.

[0163] In at least one embodiment, some of the applications used in the advanced processing and inference pipeline may use machine learning models or other AI to perform one or more processing steps. In at least one embodiment, the machine learning model may be trained at facility 1602 using data 1608 (such as imaging data) generated at facility 1602 (and stored at facility 1602 in one or more picture archiving and communication system (PACS) servers), or it may be trained using imaging or sequencing data 1608 from one or more other facilities, or a combination thereof. In at least one embodiment, the training system 1604 may be used to provide applications, services, and / or other resources for generating a working and deployable machine learning model for the deployment system 1606.

[0164] In at least one embodiment, the model registry 1624 may be backed up by object storage capable of supporting version control and object metadata. In at least one embodiment, the object storage may be accessible, for example, from within a cloud platform, through a cloud storage (e.g., cloud 1726 in Figure 17) compatible application programming interface (API). In at least one embodiment, machine learning models in the model registry 1624 may be uploaded, listed, modified, or deleted by a system developer or partner interacting with the API. In at least one embodiment, the API may provide access to a way that enables appropriately qualified users to associate models with applications, thereby allowing the models to run as part of a containerized instantiation of the application.

[0165] In at least one embodiment, the training pipeline 1704 (Figure 17) may include a scenario in which the facility 1602 is training its own machine learning model or has an existing machine learning model that needs to be optimized or updated. In at least one embodiment, imaging data 1608 generated by (one or more) imaging devices, sequencing devices, and / or other device types may be received. In at least one embodiment, once the imaging data 1608 is received, AI-assisted annotation 1610 may be used to help generate annotations corresponding to the imaging data 1608 to be used as ground truth data for the machine learning model. In at least one embodiment, AI-assisted annotation 1610 may include one or more machine learning models (e.g., convolutional neural networks (CNNs)), and one or more machine learning models may be trained to generate annotations corresponding to several types of imaging data 1608 (e.g., from several devices). In at least one embodiment, the AI-assisted annotation 1610 may then be used directly to generate ground truth data, or it may be modified or fine-tuned using an annotation tool. In at least one embodiment, the AI-assisted annotation 1610, labeled clinic data 1612, or a combination thereof may be used as ground truth data for training a machine learning model. In at least one embodiment, the trained machine learning model may be referred to as output model 1616 and may be used by the deployment system 1606 as described herein.

[0166] In at least one embodiment, the training pipeline 1704 (Figure 17) may include a scenario in which facility 1602 requires a machine learning model to use when performing one or more processing tasks for one or more applications in the deployment system 1606, but facility 1602 may not currently have such a machine learning model (or may not have a model optimized, efficient, or effective for such purposes). In at least one embodiment, an existing machine learning model may be selected from the model registry 1624. In at least one embodiment, the model registry 1624 may include machine learning models trained to perform a variety of different inference tasks on imaging data. In at least one embodiment, the machine learning models in the model registry 1624 may be trained on imaging data from a different facility than facility 1602 (e.g., a facility in a remote location). In at least one embodiment, the machine learning model may be trained on imaging data from one location, two locations, or any number of locations. In at least one embodiment, when trained with respect to imaging data from a specific location, training may be performed at that location, or at least in a manner that protects the confidentiality of the imaging data or restricts the transfer of the imaging data outside the premises. In at least one embodiment, once a model is trained, or partially trained, at one location, the machine learning model may be added to the model registry 1624. In at least one embodiment, the machine learning model may then be retrained or updated at any number of other facilities, and the retrained or updated model may be made available in the model registry 1624. In at least one embodiment, the machine learning model may then be selected from the model registry 1624, sometimes referred to as the output model 1616, and may be used in the deployment system 1606 to perform one or more processing tasks for one or more applications of the deployment system.

[0167] In at least one embodiment, the training pipeline 1704 (Figure 17), scenario may include a facility 1602 requiring a machine learning model for use in performing one or more processing tasks for one or more applications in the deployment system 1606, but facility 1602 may not currently have such a machine learning model (or may not have an efficient or effective model optimized for such purposes). In at least one embodiment, a machine learning model selected from the model registry 1624 may not be fine-tuned or optimized for imaging data 1608 generated at facility 1602 due to differences in the population, the robustness of the training data used to train the machine learning model, the diversity of anomalies in the training data, and / or other issues associated with the training data. In at least one embodiment, AI-assisted annotation 1610 may be used to help generate annotations corresponding to the imaging data 1608 to be used as ground truth data for retraining or updating the machine learning model. In at least one embodiment, labeled data 1612 may be used as ground truth data for training the machine learning model. In at least one embodiment, retraining or updating a machine learning model may be referred to as model training 1614. In at least one embodiment, model training 1614, for example, AI-assisted annotation 1610, labeled clinic data 1612, or a combination thereof, may be used as ground truth data for retraining or updating a machine learning model. In at least one embodiment, the trained machine learning model may be referred to as output model 1616, which may be used by the deployment system 1606 as described herein.

[0168] In at least one embodiment, the deployment system 1606 may include software 1618, services 1620, hardware 1622, and / or other components, features, and functionalities. In at least one embodiment, the deployment system 1606 may include a software "stack" so that software 1618 may be built on top of services 1620 and use services 1620 to perform some or all of the processing tasks, and services 1620 and software 1618 may be built on top of hardware 1622 and use hardware 1622 to perform the processing, storage, and / or other compute tasks of the deployment system 1606. In at least one embodiment, software 1618 may include any number of different containers, each of which may perform the instantiation of an application. In at least one embodiment, each application may perform one or more processing tasks in the advanced processing and inference pipeline (e.g., inference, object detection, feature detection, segmentation, image enhancement, calibration, etc.). In at least one embodiment, the advanced processing and inference pipeline may be defined based on the selection of different containers desired or required to process the imaging data 1608, in addition to a container that receives and configures the imaging data for use by each container and / or by facility 1602 after processing it through the pipeline (for example, to convert the output to an available data type). In at least one embodiment, the combination of containers in software 1618 (for example, building up the pipeline) may be referred to as a virtual device (as described in more detail herein), and the virtual device may leverage services 1620 and hardware 1622 to perform some or all of the processing tasks of the applications instantiated in the containers.

[0169] In at least one embodiment, the data processing pipeline may receive input data (e.g., imaging data 1608) in a specific format in response to an inference request (e.g., a request from a user of the deployment system 1606). In at least one embodiment, the input data may represent one or more images, videos, and / or other data representations generated by one or more imaging devices. In at least one embodiment, the data may undergo preprocessing as part of the data processing pipeline to prepare the data for processing by one or more applications. In at least one embodiment, postprocessing may be performed on the output of one or more inference tasks or other processing tasks in the pipeline to prepare output data for subsequent applications and / or output data for user transmission and / or use (e.g., as a response to an inference request). In at least one embodiment, the inference task may be performed by one or more machine learning models, such as a trained or deployed neural network, which may include the output model 1616 of the training system 1604.

[0170] In at least one embodiment, tasks in a data processing pipeline may be encapsulated in one or more containers, each representing a separate, fully functional instantiation of an application and a virtualized computing environment capable of referencing machine learning models. In at least one embodiment, a container or application may be exposed to a private (e.g., restricted access) area of ​​a container registry (described in more detail herein), and trained or deployed models may be stored in a model registry 1624 and associated with one or more applications. In at least one embodiment, an image of an application (e.g., a container image) may be available in the container registry, and once selected by a user from the container registry for deployment in a pipeline, the image may be used to generate a container for instantiation of the application for use by the user's system.

[0171] In at least one embodiment, a developer (e.g., a software developer, clinician, physician, etc.) may develop, publish, and store an application (e.g., as a container) for performing image processing and / or inference on supplied data. In at least one embodiment, development, publication, and / or storage may be performed using a software development kit (SDK) associated with the system (e.g., to ensure that the developed application and / or container conforms to or is compatible with the system). In at least one embodiment, the developed application may be tested locally (e.g., at a first facility, against data from a first facility) using an SDK that can support at least some of the services 1620 as a system (e.g., system 1700 in Figure 17). In at least one embodiment, because a DICOM object may contain anywhere one to hundreds of images or other data types, and due to data variability, the developer may be responsible for managing the extraction and preparation of incoming data (e.g., setting up constructs for the application, incorporating preprocessing into the application, etc.). In at least one embodiment, once validated by System 1700 (for example, with respect to accuracy), the application may become available in a container registry for user selection and / or implementation to perform one or more processing tasks on data at the user's facility (for example, a second facility).

[0172] In at least one embodiment, the developer may then share the application or container over a network for access and use by users of the system (e.g., system 1700 in Figure 17). In at least one embodiment, the completed and validated application or container may be stored in a container registry, and associated machine learning models may be stored in a model registry 1624. In at least one embodiment, a requesting entity providing an inference or image processing request may browse the container registry and / or model registry 1624 for applications, containers, datasets, machine learning models, etc., select a desired combination of elements to include in the data processing pipeline, and submit an imaging processing request. In at least one embodiment, the request may include input data (and, in some examples, associated patient data) necessary to perform the request, and / or include the selection of (one or more) applications and / or machine learning models to be executed when processing the request. In at least one embodiment, the request may then be passed to one or more components of the deployment system 1606 (e.g., the cloud) to perform the processing in the data processing pipeline. In at least one embodiment, processing by the deployment system 1606 may include referencing selected elements (e.g., applications, containers, models, etc.) from the container registry and / or model registry 1624. In at least one embodiment, once the pipeline has generated results, the results may be returned to the user for reference (e.g., for viewing in a viewing application suite running on a local, on-premises workstation or terminal).

[0173] In at least one embodiment, service 1620 may be utilized to assist in the processing or execution of an application or container in a pipeline. In at least one embodiment, service 1620 may include compute services, artificial intelligence (AI) services, visualization services, and / or other service types. In at least one embodiment, service 1620 may provide functionality common to one or more applications in software 1618, and thus the functionality may be abstracted to a service that can be called or utilized by the applications. In at least one embodiment, the functionality provided by service 1620 may operate dynamically and more efficiently, and may scale well by enabling applications to process data in parallel (for example, using a parallel computing platform 1730 (Figure 17)). In at least one embodiment, service 1620 may be shared among and between various applications, rather than each application sharing the same functionality provided by service 1620 being required to have its own instance of service 1620. In at least one embodiment, the service may include, in non-limiting examples, an inference server or engine that can be used to perform detection or segmentation tasks. In at least one embodiment, a model training service may be included that can provide machine learning model training and / or retraining capability. In at least one embodiment, a data augmentation service may further be included that can provide GPU-accelerated data (e.g., DICOM, RIS, CIS, REST-compliant, RPC, raw, etc.), resizing, scaling, and / or other augmentation. In at least one embodiment, a visualization service may be used that can add image rendering effects such as ray tracing, rasterization, denoising, and sharpening to add realism to two-dimensional (2D) and / or three-dimensional (3D) models.In at least one embodiment, a virtual instrument service may be included that provides beamforming, segmentation, inference, imaging, and / or support for other applications within the virtual instrument pipeline.

[0174] In at least one embodiment, if service 1620 includes an AI service (e.g., an inference service), one or more machine learning models may be executed by calling the inference service (e.g., an inference server) (e.g., as an API call) to execute the machine learning models, or their processing, as part of the application execution. In at least one embodiment, if another application includes one or more machine learning models for a segmentation task, the application may call the inference service to execute the machine learning models to perform one or more processing operations associated with the segmentation task. In at least one embodiment, software 1618 implementing an advanced processing and inference pipeline including a segmentation application and an anomaly detection application may be streamlined because each application may call the same inference service to perform one or more inference tasks.

[0175] In at least one embodiment, the hardware 1622 may include a GPU, CPU, DPU, graphics card, AI / deep learning system (e.g., an AI supercomputer such as NVIDIA's DGX), cloud platform, or a combination thereof. In at least one embodiment, different types of hardware 1622 may be used to provide efficient and dedicated support for the software 1618 and service 1620 in deployment system 1606. In at least one embodiment, the use of GPU processing for processing locally (e.g., at facility 1602) may be implemented within the AI / deep learning system, in the cloud system, and / or in other processing components of deployment system 1606 to improve the efficiency, accuracy, and effectiveness of image processing and generation. In at least one embodiment, the software 1618 and / or service 1620 may, as a non-limiting example, be optimized for GPU processing relating to deep learning, machine learning, and / or high-performance computing. In at least one embodiment, at least a portion of the computing environment of the deployment system 1606 and / or the training system 1604 may be run on one or more supercomputers or high-performance computing systems in a data center using GPU-optimized software (e.g., a combination of hardware and software from NVIDIA's DGX system). In at least one embodiment, the hardware 1622 may include any number of GPUs, which may be called to perform parallel processing of data as described herein. In at least one embodiment, the cloud platform may further include GPU processing for GPU-optimized execution of deep learning tasks, machine learning tasks, or other computing tasks. In at least one embodiment, the cloud platform may further include DPU processing for sending data received over the network and / or through a network controller or other network interface directly to one or more GPUs (e.g., their memory).In at least one embodiment, the cloud platform (e.g., NVIDIA's NGC) may run using (one or more) AI / deep learning supercomputers and / or GPU-optimized software (e.g., provided on NVIDIA's DGX systems) as a hardware abstraction and scaling platform. In at least one embodiment, the cloud platform may integrate an application container clustering system or orchestration system (e.g., Kubernetes) for multiple GPUs to enable seamless scaling and load balancing.

[0176] Figure 17 is a system diagram for an exemplary system 1700 for generating and implementing an imaging introduction pipeline, according to at least one embodiment. In at least one embodiment, system 1700 may be used to implement process 1600 of Figure 16, and / or other processes including advanced processing and inference pipelines. In at least one embodiment, system 1700 may include a training system 1604 and an introduction system 1606. In at least one embodiment, the training system 1604 and the introduction system 1606 may be implemented using software 1618, services 1620, and / or hardware 1622 as described herein.

[0177] In at least one embodiment, System 1700 (e.g., training system 1604 and / or deployment system 1606) may be implemented in a cloud computing environment (e.g., using cloud 1726). In at least one embodiment, System 1700 may be implemented locally with respect to a healthcare service facility, or as a combination of both cloud computing resources and local computing resources. In at least one embodiment, access to APIs in cloud 1726 may be restricted to authorized users through established security measures or protocols. In at least one embodiment, the security protocol may include web tokens, which may be signed by an authentication service (e.g., AuthN, AuthZ, Gluecon, etc.) and may have appropriate authorizations. In at least one embodiment, APIs of virtual devices (as described herein), or other instantiations of System 1700, may be restricted to a set of public IPs that are inspected or authorized for interaction.

[0178] In at least one embodiment, various components of System 1700 may communicate with one another using any of various different network types, including, but not limited to, local area networks (LANs) and / or wide area networks (WANs), via wired and / or wireless communication protocols. In at least one embodiment, communication between a facility and components of System 1700 (e.g., for sending inference requests, for receiving the results of inference requests) may be conducted via (one or more) data buses, wireless data protocols (Wi-Fi), wired data protocols (e.g., Ethernet), etc.

[0179] In at least one embodiment, the training system 1604 may execute a training pipeline 1704 similar to that described herein with respect to Figure 16. In at least one embodiment, if one or more machine learning models are to be used in the introduction pipeline 1710 by the introduction system 1606, the training pipeline 1704 may be used to train or retrain one or more (e.g., pre-trained) models, and / or to implement one or more of the pre-trained models 1706 (e.g., without the need for retraining or updating). In at least one embodiment, one or more output models 1616 may be produced as a result of the training pipeline 1704. In at least one embodiment, the training pipeline 1704 may include any number of processing steps, including, but not limited to, image data (or other input data) conversion or adaptation. In at least one embodiment, different training pipelines 1704 may be used for different machine learning models used by the introduction system 1606. In at least one embodiment, a training pipeline 1704 similar to the first example described with respect to Figure 16 may be used for a first machine learning model, a training pipeline 1704 similar to the second example described with respect to Figure 16 may be used for a second machine learning model, and a training pipeline 1704 similar to the third example described with respect to Figure 16 may be used for a third machine learning model. In at least one embodiment, any combination of tasks within the training system 1604 may be used depending on what is required for each respective machine learning model. In at least one embodiment, one or more of the machine learning models may already be trained and ready for deployment, and therefore may not undergo any processing by the training system 1604, but may be implemented by the deployment system 1606.

[0180] In at least one embodiment, the (one or more) output models 1616 and / or the (one or more) pre-trained models 1706 may include any type of machine learning model depending on the implementation or embodiment. In at least one embodiment, and not limited to, the machine learning models used by System 1700 may include (one or more) machine learning models using linear regression, logistic regression, decision trees, support vector machines (SVMs), naive Bayes, k-nearest neighbors (Knn), k-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., autoencoders, convolutions, recurrents, perceptrons, long / short-term memory (LSTMs), Hopfield, Boltzmann, deep belief, deconvolution, adversarial generative, liquid state machines, etc.), and / or other types of machine learning models.

[0181] In at least one embodiment, the training pipeline 1704 may include AI-assisted annotation, as will be described in more detail herein with respect to at least Figure 16B. In at least one embodiment, labeled data 1612 (e.g., conventional annotation) may be generated by any number of techniques. In at least one embodiment, labels or other annotations may be generated in a drawing program (e.g., an annotation program), a computer-aided design (CAD) program, a labeling program, another type of program suitable for generating annotations or labels for ground truth, and / or, in some examples, by handwriting. In at least one embodiment, the ground truth data may be synthetically generated (e.g., generated from a computer model or rendering), realistically generated (e.g., designed and generated from real-world data), machine-automated (e.g., using feature analysis and learning to extract features from data and then generate labels), human-annotated (e.g., a labeller or annotation expert defines the location of the labels), and / or a combination thereof. In at least one embodiment, for each instance of the imaging data 1608 (or other data types used by the machine learning model), there may be corresponding ground truth data generated by the training system 1604. In at least one embodiment, AI-assisted annotation may be performed as part of the introduction pipeline 1710, either in addition to or instead of AI-assisted annotation included in the training pipeline 1704. In at least one embodiment, the system 1700 may include a multilayer platform, which may include a software layer (e.g., software 1618) of a diagnostic application (or other application type) capable of performing one or more medical imaging and diagnostic functions. In at least one embodiment, the system 1700 may be communicably coupled (e.g., via an encrypted link) to a PACS server network of one or more facilities.In at least one embodiment, the system 1700 may be configured to access and reference data from a PACS server in order to perform operations such as training a machine learning model, deploying a machine learning model, image processing, inference, and / or other operations.

[0182] In at least one embodiment, the software layer may be implemented as a secure, encrypted, and / or authenticated API through which an application or container may be invoked (e.g., called) from (one or more) external environments (e.g., facility 1602). In at least one embodiment, the application may then call or execute one or more services 1620 to perform compute, AI, or visualization tasks associated with each application, and the software 1618 and / or services 1620 may leverage the hardware 1622 to perform the processing tasks in an effective and efficient manner.

[0183] In at least one embodiment, the introduction system 1606 may execute an introduction pipeline 1710. In at least one embodiment, the introduction pipeline 1710 may include any number of applications that may be applied sequentially, discontinuously, or otherwise to imaging data (and / or other data types) generated by imaging devices, sequencing devices, genomics devices, etc., including AI-assisted annotation as described above. In at least one embodiment, as described herein, the introduction pipeline 1710 for individual devices may be referred to as a virtual instrument for the device (e.g., a virtual ultrasound instrument, a virtual CT scanning instrument, a virtual sequencing instrument, etc.). In at least one embodiment, there may be two or more introduction pipelines 1710 for a single device, depending on the information desired from the data generated by the device. In at least one embodiment, there may be a first introduction pipeline 1710 if anomaly detection is desired from the MRI machine, and there may be a second introduction pipeline 1710 if image enhancement is desired from the output of the MRI machine.

[0184] In at least one embodiment, the image generation application may include processing tasks that involve the use of a machine learning model. In at least one embodiment, the user may wish to use their own machine learning model or select a machine learning model from the model registry 1624. In at least one embodiment, the user may implement their own machine learning model or select a machine learning model to include in the application in order to perform the processing tasks. In at least one embodiment, the application may be selectable and customizable, and by defining the construction of the application, the deployment and implementation of the application for a particular user is presented as a more seamless user experience. In at least one embodiment, by leveraging other features of the system 1700, such as services 1620 and hardware 1622, the deployment pipeline 1710 may become even more user-friendly, provide easier integration, and produce more accurate, efficient, and timely results.

[0185] In at least one embodiment, the deployment system 1606 may include a user interface 1714 (e.g., a graphical user interface, a web interface, etc.) which may be used to select applications for inclusion in one or more deployment pipelines 1710, to deploy applications, to modify or change applications or their parameters or construction, to use and interact with one or more deployment pipelines 1710 during setup and / or deployment, and / or to interact with the deployment system 1606 in other ways. In at least one embodiment, although not shown with respect to the training system 1604, the user interface 1714 (or a different user interface) may be used to select models for use in the deployment system 1606, to select models for training or retraining in the training system 1604, and / or to interact with the training system 1604 in other ways.

[0186] In at least one embodiment, a pipeline manager 1712 may be used in addition to the application orchestration system 1728 to manage interactions between applications or containers in (one or more) deployment pipelines 1710 and services 1620 and / or hardware 1622. In at least one embodiment, the pipeline manager 1712 may be configured to facilitate application-to-application interactions, applications-to-service interactions, and / or applications or services-to-hardware interactions. In at least one embodiment, it is shown to be included in software 1618, but this is not intended to limit it, and in some examples (for example, shown in Figure 15), the pipeline manager 1712 may be included in services 1620. In at least one embodiment, the application orchestration system 1728 (e.g., Kubernetes, DOCKER, etc.) may include a container orchestration system, which may group applications into containers as logical units for coordination, management, scaling, and deployment. In at least one embodiment, by associating applications (e.g., reconstruction applications, segmentation applications, etc.) from one or more deployment pipelines 1710 with individual containers, each application can run in a self-contained environment (e.g., at the kernel level) to improve speed and efficiency.

[0187] In at least one embodiment, each application and / or container (or its image) may be developed, modified, and deployed individually (for example, a first user or developer may develop, modify, and deploy a first application, and a second user or developer may develop, modify, and deploy a second application independently of the first user or developer), which allows for focus and attention on the tasks of a single application and / or container without being interrupted by the tasks of other applications or containers. In at least one embodiment, communication and coordination between different containers or applications may be assisted by the pipeline manager 1712 and the application orchestration system 1728. In at least one embodiment, the application orchestration system 1728 and / or the pipeline manager 1712 may facilitate communication between each of the applications or containers and the sharing of resources between them, as long as the expected inputs and / or outputs of each container or application are known by the system (for example, based on the construction of the application or container). In at least one embodiment, one or more applications or containers in the deployment pipeline 1710 may share the same services and resources, so that the application orchestration system 1728 can orchestrate, load balance, and decide on sharing services or resources among the various applications or containers. In at least one embodiment, a scheduler may be used to track the resource requirements of the applications or containers, the current or planned usage of these resources, and the availability of the resources. In at least one embodiment, the scheduler may therefore allocate resources to different applications and distribute resources among applications, taking into account the system requirements and availability.In some examples, the scheduler (and / or other components of the application orchestration system 1728) may determine the availability and distribution of resources based on constraints imposed on the system (e.g., user constraints), such as quality of service (QoS) and the urgency of the need for data output (e.g., to determine whether real-time processing or delayed processing should be performed).

[0188] In at least one embodiment, the services 1620 utilized and shared by applications or containers in the deployment system 1606 may include compute services 1716, AI services 1718, visualization services 1720, and / or other service types. In at least one embodiment, an application may call (e.g., execute) one or more of the services 1620 to perform processing operations for the application. In at least one embodiment, compute service 1716 may be utilized by an application to perform supercomputing or other high-performance computing (HPC) tasks. In at least one embodiment, one or more compute services 1716 may be utilized to process data through one or more applications and / or to perform parallel processing (e.g., using a parallel computing platform 1730) for substantially simultaneous processing of one or more tasks of a single application. In at least one embodiment, the parallel computing platform 1730 (e.g., NVIDIA's CUDA) may enable general-purpose computing (GPGPU) on a GPU (e.g., GPU 1722). In at least one embodiment, the software layer of the parallel computing platform 1730 may provide access to a virtual instruction set and parallel computing elements of the GPU for the execution of a compute kernel. In at least one embodiment, the parallel computing platform 1730 may include memory, which in some embodiments may be shared among multiple containers and / or among different processing tasks within a single container.In at least one embodiment, inter-process communication (IPC) calls may be generated for multiple containers and / or multiple processes within containers to use the same data from a shared segment of memory on the parallel computing platform 1730 (for example, when multiple different stages of an application or multiple applications are processing the same information). In at least one embodiment, the same data in the same location in memory may be used for any number of processing tasks (for example, at the same time, at different times, etc.) rather than making copies of the data and moving the data to different locations in memory (e.g., read / write operations). In at least one embodiment, when data is used and new data is generated as a result of processing, this information about the new location of the data may be stored and shared among various applications. In at least one embodiment, the location of the data and the location of the updated or modified data may be part of the definition of how the payload is understood within the container.

[0189] In at least one embodiment, the AI ​​service 1718 may be leveraged to perform inference services for running one or more machine learning models associated with an application (e.g., assigned the role of performing one or more processing tasks of the application). In at least one embodiment, the AI ​​service 1718 may leverage the AI ​​system 1724 to run one or more machine learning models (e.g., neural networks such as CNNs) for segmentation, reconstruction, object detection, feature detection, classification, and / or other inference tasks. In at least one embodiment, an application in one or more introductory pipelines 1710 may perform inference on imaging data using one or more output models 1616 from the training system 1604 and / or other models of the application. In at least one embodiment, two or more examples of inference using the application orchestration system 1728 (e.g., a scheduler) may be available. In at least one embodiment, the first category may include high-priority / low-latency routes that can achieve a higher service level agreement, such as for performing reasoning regarding urgent requests in emergencies or for radiologists during diagnosis. In at least one embodiment, the second category may include standard-priority routes that may be used for requests that may not be urgent or where analysis may be performed later. In at least one embodiment, the application orchestration system 1728 may distribute resources (e.g., services 1620 and / or hardware 1622) based on priority routes for different reasoning tasks of the AI ​​service 1718.

[0190] In at least one embodiment, shared storage may be attached to the AI ​​service 1718 within system 1700. In at least one embodiment, shared storage may act as a cache (or other storage device type) and may be used to process inference requests from an application. In at least one embodiment, when an inference request is submitted, the request may be received by a set of API instances of deployment system 1606, and one or more instances may be selected to process the request (e.g., for best fit, for load balancing, etc.). In at least one embodiment, to process the request, the request may be placed in a database, a machine learning model may be located from the model registry 1624 if it is not already in the cache, and a verification step may ensure that a suitable machine learning model is loaded into the cache (e.g., shared storage) and / or a copy of the model can be stored in the cache. In at least one embodiment, if the application is not yet running or there are not enough instances of the application, a scheduler (e.g., of pipeline manager 1712) may be used to start the application referenced in the request. In at least one embodiment, an inference server may be started if one has not yet been started to run the model. Any number of inference servers may be started for each model. In at least one embodiment, in a clustered pull model, the model may be cached whenever load balancing is advantageous. In at least one embodiment, the inference server may be statically loaded onto the corresponding distributed server.

[0191] In at least one embodiment, inference may be performed using an inference server running in a container. In at least one embodiment, an instance of the inference server may be associated with a model (optionally, multiple versions of the model). In at least one embodiment, when a request to perform inference against a model is received and no instance of the inference server exists, a new instance may be loaded. In at least one embodiment, when the inference server is started, a model may be passed to the inference server, thereby allowing the same container to be used to serve different models, as long as the inference server is running as a different instance.

[0192] In at least one embodiment, while the application is running, an inference request for a given application may be received, a container (e.g., hosting an instance of the inference server) may be loaded (if not already loaded), and a start procedure may be called. In at least one embodiment, preprocessing logic in the container may load, decode, and / or perform any additional preprocessing on incoming data (e.g., using one or more CPUs and / or one or more GPUs and / or one or more DPUs). In at least one embodiment, once the data is prepared for inference, the container may perform inference on the data as needed. In at least one embodiment, this may involve a single inference call for a single image (e.g., an X-ray of a hand) or may require inference on hundreds of images (e.g., a CT scan of a chest). In at least one embodiment, the application may summarize results before completion, which may include, but are not limited to, generating a single confidence score, pixel-level segmentation, voxel-level segmentation, visualizations, or text to summarize the findings. In at least one embodiment, different models or applications may be assigned different priorities. For example, some models may have real-time priority (TAT < 1 minute), while others may have lower priority (e.g., TAT < 12 minutes). In at least one embodiment, model execution time may be measured from the requesting institution or entity and may include partner network traversal time and execution for inference services.

[0193] In at least one embodiment, the transfer of requests between service 1620 and the inference application may be hidden behind a software development kit (SDK), and robust transport may be provided through a queue. In at least one embodiment, for each application / tenant ID combination, requests are queued via an API, and the SDK retrieves the requests from the queue and delivers them to the application. In at least one embodiment, a name for the queue may be provided in the environment in which the SDK picks up requests. In at least one embodiment, asynchronous communication through a queue may be useful because the communication may allow any instance of the application to pick up the work when the work becomes available. The results may be returned through the queue to ensure that no data is lost. In at least one embodiment, the queue may also provide the ability to segment work, so that the highest priority work may proceed to a queue to which most instances of the application are connected, while the lowest priority work may proceed to a queue to which a single instance is connected, processing tasks in the order they are received. In at least one embodiment, the application may run on a GPU-accelerated instance generated in Cloud 1726, and the inference service may perform inference on the GPU.

[0194] In at least one embodiment, the visualization service 1720 may be utilized to generate visualizations for viewing the output of an application and / or (one or more) introductory pipelines 1710. In at least one embodiment, the visualization service 1720 may utilize the GPU 1722 to generate visualizations. In at least one embodiment, rendering effects such as ray tracing may be implemented by the visualization service 1720 to generate higher quality visualizations. In at least one embodiment, visualizations may include, but are not limited to, 2D image rendering, 3D volume rendering, 3D volume reconstruction, 2D tomographic slicing, virtual reality display, augmented reality display, etc. In at least one embodiment, a virtualized environment may be used to generate a virtual interactive display or environment (e.g., a virtual environment) for interaction by a user of the system (e.g., a doctor, nurse, radiologist, etc.). In at least one embodiment, the visualization service 1720 may include an internal visualizer, cinematics, and / or other rendering or image processing capabilities or functionalities (e.g., ray tracing, rasterization, internal optics, etc.).

[0195] In at least one embodiment, hardware 1622 may include GPU 1722, AI system 1724, cloud 1726, and / or any other hardware used to run training system 1604 and / or deployment system 1606. In at least one embodiment, GPU 1722 (e.g., NVIDIA's TESLA and / or QUADRO GPUs) may include any number of GPUs, any number of GPUs may be used to perform processing tasks for any of the compute service 1716, AI service 1718, visualization service 1720, other services, and / or features or functionalities of software 1618. For example, with respect to AI service 1718, GPU 1722 may be used to perform preprocessing on imaging data (or other data types used by the machine learning model), postprocessing on the output of the machine learning model, and / or to perform inference (e.g., to run the machine learning model). In at least one embodiment, the cloud 1726, the AI ​​system 1724, and / or other components of system 1700 may use the GPU 1722. In at least one embodiment, the cloud 1726 may include a GPU-optimized platform for deep learning tasks. In at least one embodiment, the AI ​​system 1724 may use a GPU, and the cloud 1726, or at least a portion assigned the role of deep learning or inference, may run using one or more AI systems 1724. Thus, although hardware 1622 is shown as a separate component, this is not intended to be limiting, and any component of hardware 1622 may be combined with any other component of hardware 1622 and utilized by them.

[0196] In at least one embodiment, the AI ​​system 1724 may include a dedicated computing system (e.g., a supercomputer or HPC) configured for inference, deep learning, machine learning, and / or other artificial intelligence tasks. In at least one embodiment, the AI ​​system 1724 (e.g., NVIDIA's DGX) may include GPU-optimized software (e.g., a software stack), which may run using multiple GPUs 1722 in addition to the DPU, CPU, RAM, storage, and / or other components, features, or functionalities. In at least one embodiment, one or more AI systems 1724 may be implemented in a cloud 1726 (e.g., in a data center) to perform some or all of the AI-based processing tasks of system 1700.

[0197] In at least one embodiment, the cloud 1726 may include a GPU-accelerated infrastructure (e.g., NVIDIA's NGC) which may provide a GPU-optimized platform for performing processing tasks of system 1700. In at least one embodiment, the cloud 1726 may include (one or more) AI systems 1724 (e.g., as a hardware abstraction and scaling platform) for performing one or more of the AI-based tasks of system 1700. In at least one embodiment, the cloud 1726 may integrate with an application orchestration system 1728, leveraging multiple GPUs to enable seamless scaling and load balancing between applications and services 1620. In at least one embodiment, the cloud 1726 may be tasked with performing at least some of the services 1620 of system 1700, including compute service 1716, AI service 1718, and / or visualization service 1720, as described herein. In at least one embodiment, Cloud 1726 may perform large and small batch inference (e.g., running NVIDIA's TENSOR RT), provide accelerated parallel computing APIs and platforms 1730 (e.g., NVIDIA's CUDA), run application orchestration systems 1728 (e.g., KUBERNETES), provide graphics rendering APIs and platforms (e.g., for ray tracing, 2D graphics, 3D graphics, and / or other rendering techniques to produce higher quality cinematics), and / or provide other functionality for system 1700.

[0198] Figure 18A shows a data flow diagram of process 1800 for training, retraining, or updating a machine learning model, according to at least one embodiment. In at least one embodiment, process 1800 may be performed using system 1700 in Figure 17 as a non-limiting example. In at least one embodiment, process 1800 may leverage the services 1620 and / or hardware 1622 of system 1700 as described herein. In at least one embodiment, the improved model 1812 produced by process 1800 may be executed by deployment system 1606 for one or more containerized applications in deployment pipeline 1710.

[0199] In at least one embodiment, model training 1614 may include retraining or updating the initial model 1804 (e.g., a pre-trained model) using new training data (e.g., customer dataset 1806, and / or new input data such as new ground truth data associated with the input data). In at least one embodiment, in order to retrain or update the initial model 1804, one or more output or loss layers of the initial model 1804 may be reset or deleted and / or replaced with one or more updated or new output or loss layers. In at least one embodiment, the initial model 1804 may have parameters (e.g., weights and / or biases) that have been previously fine-tuned and remain from the previous training, and therefore training or retraining 1614 may not take as long or require as much processing as training the model from scratch. In at least one embodiment, during model training 1614, by having one or more reset or replaced output or loss layers of the initial model 1804, the parameters may be updated or readjusted for the new dataset based on the loss calculation associated with the accuracy of one or more output or loss layers when generating predictions with respect to the new customer dataset 1806 (e.g., image data 1608 in Figure 16).

[0200] In at least one embodiment, the pre-trained model 1706 may be stored in a data store or registry (for example, the model registry 1624 in Figure 16). In at least one embodiment, the pre-trained model 1706 may be trained at least partially at one or more facilities other than the facility where process 1800 is performed. In at least one embodiment, in order to protect the privacy and rights of patients, subjects, and customers at different facilities, the pre-trained model 1706 may be trained on-site using on-site generated customer or patient data. In at least one embodiment, the pre-trained model 1706 may be trained using the cloud 1726 and / or other hardware 1622, but privacy-protected sensitive patient data may not be transferred to any component of the cloud 1726 (or other off-site hardware), may not be used by such components, or may be inaccessible to such components. In at least one embodiment, if the pre-trained model 1706 is trained using patient data from two or more facilities, the pre-trained model 1706 may be trained individually for each facility and then trained with respect to patient or customer data from another facility. In at least one embodiment, customer or patient data from any number of facilities may be used to train the pre-trained model 1706 on-premises and / or off-premises, such as in a data center or other cloud computing infrastructure, if the customer or patient data is released from privacy concerns (e.g., by waiver, for experimental use, etc.) or if the customer or patient data is included in a public data set.

[0201] In at least one embodiment, when selecting an application for use in the deployment pipeline 1710, the user may also select a machine learning model to be used for that particular application. In at least one embodiment, the user may not have a model for use and therefore may select a pre-trained model 1706 for use with the application. In at least one embodiment, the pre-trained model 1706 may not be optimized to produce accurate results with respect to the user's facility's customer dataset 1806 (based on, for example, patient diversity, demographics, and the type of medical imaging device used). In at least one embodiment, before deploying the pre-trained model 1706 into the deployment pipeline 1710 for use with one or more applications, the pre-trained model 1706 may be updated, retrained, and / or fine-tuned for use at each respective facility.

[0202] In at least one embodiment, the user may select a pre-trained model 1706 to be updated, retrained, and / or fine-tuned, which may be referred to as the initial model 1804 for the training system 1604 within process 1800. In at least one embodiment, a customer dataset 1806 (e.g., imaging data, genomics data, sequencing data, or other data types generated by devices in a facility) may be used to perform model training 1614 (which may include, but is not limited to, transfer learning) with respect to the initial model 1804 to generate an improved model 1812. In at least one embodiment, ground truth data corresponding to the customer dataset 1806 may be generated by the training system 1604. In at least one embodiment, ground truth data may be generated at least partially by clinicians, scientists, physicians, or practitioners in a facility (e.g., as labeled clinic data 1612 in Figure 16).

[0203] In at least one embodiment, AI-assisted annotation 1610 may be used in several examples to generate ground truth data. In at least one embodiment, AI-assisted annotation 1610 (implemented, for example, using the AI-assisted annotation SDK) may leverage a machine learning model (e.g., a neural network) to generate suggestive or predictive ground truth data about a customer dataset. In at least one embodiment, user 1810 may use the annotation tool within a user interface (graphical user interface (GUI)) on computing device 1808.

[0204] In at least one embodiment, user 1810 may interact with a GUI via computing device 1808 to edit or fine-tune (automatic) annotations. In at least one embodiment, polygon editing features may be used to move polygon vertices to more precise or fine-tuned locations.

[0205] In at least one embodiment, if the customer dataset 1806 has relevant ground truth data, the ground truth data (e.g., from AI-assisted annotation, manual labeling, etc.) may be used during model training 1614 to generate an improved model 1812. In at least one embodiment, the customer dataset 1806 may be applied to the initial model 1804 any number of times, and the ground truth data may be used to update the parameters of the initial model 1804 for the improved model 1812 until an acceptable level of accuracy is achieved. In at least one embodiment, once the improved model 1812 is generated, the improved model 1812 may be deployed in one or more deployment pipelines 1710 at the facility to perform one or more processing tasks on medical imaging data.

[0206] In at least one embodiment, the improved model 1812 may be uploaded to a pre-trained model 1706 in a model registry 1624 to be selected by another facility. In at least one embodiment, the process may be completed in any number of facilities, thereby allowing the improved model 1812 to be further improved any number of times with respect to new datasets to generate a more universal model.

[0207] Figure 18B is an exemplary diagram of a client-server architecture 1832 for extending an annotation tool with a pre-trained annotation model, according to at least one embodiment. In at least one embodiment, an AI-assisted annotation tool 1836 may be instantiated based on the client-server architecture 1832. In at least one embodiment, the annotation tool 1836 in an imaging application may assist a radiologist in identifying, for example, organs and anomalies. In at least one embodiment, the imaging application may include, as a non-limiting example, a software tool that helps a user 1810 identify several extreme points on a particular organ of interest in a raw image 1834 (e.g., in a 3D MRI or CT scan) and receives automatically annotated results for all 2D slices of the particular organ. In at least one embodiment, the results may be stored in a data store as training data 1838 and used as ground truth data for training (e.g., but not limited to). In at least one embodiment, when computing device 1808 sends out extreme points for AI-assisted annotation 1610, a deep learning model, for example, may receive this data as input and return inference results for segmented organs or anomalies. In at least one embodiment, a pre-instantiated annotation tool, such as AI-assisted annotation tool 1836B in Figure 18B, may be extended by making an API call (e.g., API call 1844) to a server, such as annotation support server 1840, which may contain a set of pre-trained models 1842 stored, for example, in an annotation model registry. In at least one embodiment, the annotation model registry may store pre-trained models 1842 (e.g., machine learning models such as deep learning models) that have been pre-trained to perform AI-assisted annotation for specific organs or anomalies. These models may be further updated by using the training pipeline 1704.In at least one embodiment, the pre-installed annotation tool may improve over time as new labeled clinic data 1612 is added.

[0208] Such components can be used to generate synthetic data that mimics failure scenarios in the network training process, which can help improve network performance while limiting the amount of synthetic data to avoid overfitting.

[0209] Other variations are within the scope of this disclosure. Thus, the disclosed techniques can be modified and constructed in various ways, some exemplary embodiments of which are shown in the drawings and described in detail above. However, this disclosure is not intended to limit itself to any particular one or more disclosed forms, but rather to encompass all modifications, alternative constructions, and equivalents that fall within the spirit and scope of the disclosure, as defined in the appended claims.

[0210] In the context describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar demonstrative pronouns, should be interpreted as encompassing both singular and plural, and not as definitions of terms, unless otherwise stated herein or clearly refuted by the context. The terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (meaning “including, but not limited to,”) unless otherwise stated. When “connected” is unmodified and refers to a physical connection, it should be interpreted as being partially or completely contained, attached, or joined to one another, even if there is something intervening. The detailing of ranges of values ​​herein is merely intended to serve as a concise way of individually referring to each distinct value that falls within a range, unless otherwise stated herein and unless each distinct value is incorporated into the specification as if it were individually detailed herein. In at least one embodiment, the use of the terms “set” (e.g., “set of items”) or “subset” should be interpreted as a non-empty set comprising one or more members, unless otherwise stated or denied by the context. Furthermore, unless otherwise stated or denied by the context, the term “subset” of a corresponding set does not necessarily refer to a strict subset of the corresponding set, and a subset and a corresponding set can be equivalent.

[0211] Combinations such as “at least one of A, B, and C” or “at least one of A, B, and C” are understood in contexts generally used to indicate that an item, term, etc., can be either A, B, or C, or any non-empty subset of the set of A, B, and C, unless otherwise specifically stated or explicitly denied by the context. For example, in a descriptive example of a set having three members, the combinations “at least one of A, B, and C” and “at least one of A, B, and C” refer to any of the following sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus, such combinations do not imply as a whole that some embodiments require the presence of each of A, B, and C. Furthermore, unless otherwise stated or negated by the context, the term "plurality" refers to a state of being multiple (for example, "a plurality of items" refers to multiple items). In at least one embodiment, the number of items in plurality is at least two, but can be more when explicitly or by contextual indication. Furthermore, unless otherwise stated or clarified by the context, the phrase "based on" means "at least partially based on," and does not mean "based solely on."

[0212] The operation of the processes described herein may be carried out in any preferred order unless otherwise stated herein or explicitly refuted by the context. In at least one embodiment, a process such as the processes described herein (or variations and / or combinations thereof) is carried out under the control of one or more computer systems consisting of executable instructions and is implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or by a combination thereof. In at least one embodiment, the code is stored in a computer-readable storage medium, for example, in the form of a computer program comprising multiple instructions executable by one or more processors. In at least one embodiment, the computer-readable storage medium is a non-temporary computer-readable storage medium that excludes temporary signals (e.g., transient electrical or electromagnetic transmissions that propagate) but includes non-temporary data storage circuit elements (e.g., buffers, caches, and queues) in transceivers for temporary signals. In at least one embodiment, code (e.g., executable code or source code) is stored in one or more sets of non-temporary computer-readable storage media, which store executable instructions (or have other memory for storing executable instructions) that cause the computer system to perform the operations described herein when executed by one or more processors of the computer system (i.e., as a result of execution). In at least one embodiment, the set of non-temporary computer-readable storage media comprises a plurality of non-temporary computer-readable storage media, where one or more of the individual non-temporary storage media of the plurality of non-temporary computer-readable storage media do not contain all of the code, but the plurality of non-temporary computer-readable storage media collectively contain all of the code. In at least one embodiment, the executable instructions are executed such that different instructions are executed by different processors.

[0213] Accordingly, in at least one embodiment, the computer system is configured to implement one or more services that individually or collectively perform the operations of the processes described herein, and such a computer system consists of applicable hardware and / or software that enables the performance of the operations. Furthermore, a computer system implementing at least one embodiment of the present disclosure is a single device, and in another embodiment, a distributed computer system comprising multiple devices operating in different ways so that the distributed computer system performs the operations described herein and the single device does not perform all of the operations.

[0214] Any use of any examples or illustrative language provided herein (e.g., "such as") is intended solely to further illustrate the embodiments of this disclosure and, unless otherwise asserted, does not limit the scope of this disclosure. Nothing in this specification should be construed as indicating any unclaimed element as essential to the practice of this disclosure.

[0215] All references cited herein, including publications, patent applications, and patents, are incorporated herein by reference to the same extent as if they were included herein in their entirety, provided that each reference is clearly indicated by individual reference.

[0216] In the specification and claims, the terms “joined” and “connected” may be used together with their derivatives. It should be understood that these terms may not be intended to be synonymous with one another. Rather, in certain instances, “connected” or “joined” may be used to indicate that two or more elements are in direct or indirect physical or electrical contact with one another. “Joined” may also mean that two or more elements are not in direct contact with one another, but still interact or communicate with one another.

[0217] Unless otherwise specified, throughout this specification, terms such as “processing,” “computing,” “calculating,” or “determining” refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulate and / or transform data, represented as electronic or other physical quantities, in the registers and / or memory of a computing system into other data, similarly represented as physical quantities in the memory, registers, or other such information storage, transmission, or display device of a computing system.

[0218] Similarly, the term “processor” may refer to any device or part of a device that processes electronic data from registers and / or memory and converts that electronic data into other electronic data that can be stored in registers and / or memory. A “computing platform” may comprise one or more processors. As used herein, “software” processes may include software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Each process may also refer to multiple processes for executing instructions serially or in parallel, continuously or intermittently. In at least one embodiment, the terms “system” and “method” are used interchangeably herein only if one or more methods can embody a system and a method can be considered a system.

[0219] This specification may refer to acquiring, obtaining, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of acquiring, obtaining, receiving, or inputting analog or digital data may be implemented in various ways, such as by receiving the data as a parameter to a function call or a call to an application programming interface. In at least one embodiment, the process of acquiring, obtaining, receiving, or inputting analog or digital data may be implemented by transferring the data via a serial or parallel interface. In at least one embodiment, the process of acquiring, obtaining, receiving, or inputting analog or digital data may be implemented by transferring the data via a computer network from a providing entity to an acquiring entity. In at least one embodiment, it may also refer to providing, outputting, transmitting, sending out, or presenting analog or digital data. In various applications, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be achieved by transferring data as input or output parameters of a function call, an application programming interface, or a parameter of an inter-process communication mechanism.

[0220] While the description in this specification describes exemplary embodiments of the techniques described, other architectures may be used to implement the described functionality and are intended to be within the scope of this disclosure. Furthermore, specific distributions of responsibility may be defined for illustrative purposes, but various functions and responsibilities may be distributed and divided in different ways depending on the context.

[0221] Furthermore, while the subject matter is described in language specific to structural features and / or methodological actions, it should be understood that the subject matter claimed in the attached claims is not necessarily limited to the described specific features or actions. Rather, the specific features and actions are disclosed as exemplary forms that implement the claims.

Claims

1. A step of receiving a first design state of the data path circuit using a processing device, The steps include providing the first design state of the data path circuit as input to a machine learning model, The steps include: using the machine learning model to implement one or more reinforcement learning techniques in order to cause the machine learning model to output the final design state of the data path circuit; Includes, The aforementioned one or more reinforcement learning techniques are A step of processing the first design state of the data path circuit using the machine learning model, wherein the machine learning model outputs a second design state of the data path circuit which is a modification of the first design state. A step of determining a first parameter value for the first design state and a second parameter value for the second design state, A step of updating the machine learning model based on a comparison of the first parameter value and the second parameter value, Includes, A method wherein the final design state of the data path circuit is associated with a final parameter value that is closer to the target than the first parameter value associated with the first design state.

2. The step of implementing one or more reinforcement learning techniques is: A step of processing the first design state or the second design state of the data path circuit using the updated machine learning model, wherein the updated machine learning model outputs a third design state of the data path circuit which is a modification of the first design state or the second design state. The method according to claim 1, including the method described in claim 1.

3. A step of processing the first design state using a second model, wherein the second model outputs the first parameter values ​​associated with the first design state. A step of processing the second design state using the second model, wherein the second model outputs the second parameter value associated with the second design state, The steps include: performing the comparison between the second parameter value and the first parameter value to determine which of the first parameter value and the second parameter value is closer to the target; The method according to claim 2, further comprising:

4. The machine learning model receives a grid representation of the first graph of the first design state as input and outputs a grid representation of the second graph of the second design state. The second model is a metric predictor model that receives the grid representation of the first graph in the first design state as a first input and the grid representation of the second graph in the second design state as a second input. The method according to claim 3.

5. The machine learning model receives a grid representation of the first graph of the first design state as input, outputs an action for generating the grid representation of the second graph of the second design state, and the method Steps to convert the grid representation of the second graph of the second design state to the circuit implementation form of the second design state. It further includes, The method according to claim 3, wherein the second model receives the circuit implementation configuration of the first design state as a first input and the circuit implementation configuration of the second design state as a second input.

6. A step of determining the difference between the second parameter value and the first parameter value, wherein the machine learning model is updated based on the difference. The method according to claim 2, further comprising:

7. A step of processing the first design state using a circuit synthesis tool, wherein the circuit synthesis tool outputs the first parameter values ​​associated with the first design state, A step of processing the second design state using the circuit synthesis tool, wherein the circuit synthesis tool outputs the second parameter value associated with the second design state, The steps include: performing the comparison between the second parameter value and the first parameter value to determine which of the first parameter value and the second parameter value is closer to the target; The method according to claim 2, further comprising:

8. The method according to claim 7, wherein the circuit synthesis tool processes the first design state at least partially simultaneously with the machine learning model outputting an action for generating the second design state of the data path circuit.

9. Before determining the second parameter value, the step is to determine that the second design state is an invalid state, A step of modifying the second design state of the data path circuit to create an updated second design state which is valid, wherein the second parameter value is determined from the updated second design state. The method according to claim 2, further comprising:

10. The steps include generating a first parallel prefix graph representing the first design state, A step of generating a grid representation of the first parallel prefix graph, wherein the grid representation of the first parallel prefix graph is provided as input to the machine learning model, and the machine learning model outputs one or more actions for constructing a grid representation of the final parallel prefix graph representing the final design state. The method according to claim 2, further comprising:

11. The method according to claim 1, wherein the data path circuit comprises a parallel prefix circuit, or a parallel prefix circuit to which additional circuit logic is connected at its input or output.

12. The method according to claim 1, wherein the first parameter value and the final parameter value each represent at least one of the area associated with the data path circuit, the delay associated with the data path circuit, the predicted power consumption associated with the data path circuit, or any combination thereof.

13. The method according to claim 12, wherein the first parameter value and the final parameter value each represent at least one of the following predictions: the area associated with the data path circuit, the delay associated with the data path circuit, or the weighted value of the power consumption associated with the data path circuit.

14. The method according to claim 1, wherein the machine learning model iteratively modifies the design state of the data path circuit from a first design state to a final design state, each iteration comprising at least one of removing or adding nodes to a graph of the data path circuit, each node of the graph being associated with one or more components of the data path circuit.

15. Receiving the first design state of the data path circuit, Processing the first design state of the data path circuit using a machine learning model, wherein the machine learning model generates an output that includes a second design state of the data path circuit, which is a modification of the first design state. A first processing device that will perform this task, Determining a first parameter value for the first design state and a second parameter value for the second design state, The reward associated with the output is determined based on a comparison between the first parameter value and the second parameter value. A second processing device configured to perform the following: A system equipped with, The first processing device further updates the machine learning model based on the reward. A system in which, after one or more updates to the machine learning model, the machine learning model outputs the final design state of the data path circuit, and the final design state of the data path circuit is associated with a final parameter value that is closer to the target than the first parameter value associated with the first design state.

16. The first processing device further, Processing the first or second design state of the data path circuit using the updated machine learning model, Outputting a third design state of the data path circuit, which is a modification of the first or second design state. The system according to claim 15, which will perform the following actions.

17. The second processing device described above further, Processing the first design state using a second model, wherein the second model outputs the first parameter values ​​associated with the first design state. Processing the second design state using the second model, wherein the second model outputs the second parameter value associated with the first design state. To determine which of the first parameter value and the second parameter value is closer to the target, the second parameter value and the first parameter value are compared. The system according to claim 15, which will perform the following actions.

18. At least one of the second processing device or the third processing device further, Processing the first design state using a circuit synthesis tool, wherein the circuit synthesis tool outputs the first parameter values ​​associated with the first design state. Processing the second design state using the circuit synthesis tool, wherein the circuit synthesis tool outputs the second parameter value associated with the first design state. To determine which of the first parameter value and the second parameter value is closer to the target, the second parameter value and the first parameter value are compared. The system according to claim 15, configured to perform the following:

19. The system according to claim 15, wherein the first processing device comprises a graphical processing unit (GPU) and the second processing device comprises a central processing unit (CPU).

20. A non-temporary computer-readable medium on which instructions are stored, wherein when an instruction is executed by a processing device, the processing device... Receiving the first design state of the data path circuit, The first design state of the data path circuit is provided as input to a machine learning model, To cause the machine learning model to output the final design state of the data path circuit, one or more reinforcement learning techniques are applied using the machine learning model, The aforementioned one or more reinforcement learning techniques are Processing the first design state of the data path circuit using the machine learning model, wherein the machine learning model outputs a second design state of the data path circuit which is a modification of the first design state. Determining a first parameter value for the first design state and a second parameter value for the second design state, Updating the machine learning model based on a comparison between the first parameter value and the second parameter value, Includes, A non-temporary computer-readable medium in which the final design state of the data path circuit is associated with a final parameter value that is closer to the target than the first parameter value associated with the first design state.

Images