Efficient machine learning acceleration in general-purpose computing SoCs

Abstract

Disclosed herein is a method for optimized data computation for machine learning (ML) operations using a plurality of small ML accelerators integrated into a general-purpose computing system on a chip (SoC). One or more processors of the general-purpose computing SoC can be configured to receive a workload and divide the workload into a plurality of sub-workloads. The plurality of sub-workloads can be distributed among the plurality of small ML accelerators. Each ML accelerator of the plurality of ML accelerators can be configured to output a partial result. The output of each ML accelerator can be combined using an atomic store operation with a dedicated opcode and a specified memory address.

Description

Background Technology

[0001] As machine learning workloads and their associated computational complexity increase, integrating machine learning (ML) accelerators into general-purpose computing systems-on-chips (SoCs) can be advantageous. Most dedicated ML accelerators and application-specific integrated circuits (ASICs) such as tensor processing units (TPUs) typically use expensive and limited-capacity high-bandwidth memory (HBM). Integrating ML accelerators into general-purpose computing SoCs allows some computational tasks from the TPU to be offloaded to the general-purpose SoC, freeing up some of the TPU's HBM for higher-priority operations and eliminating bottlenecks.

[0002] Integrating a single large ML accelerator into a general-purpose computing SoC can be challenging and inefficient, as it can lead to increased die size and manufacturing costs. Furthermore, general-purpose computing SoCs have multiple dedicated crossover points, which are interfaces used for SoC interconnects, and the bandwidth of a single crossover point is limited. Therefore, to accommodate a single, larger ML accelerator, multiple crossover points need to be distributed across the general-purpose computing SoC with multiple lines, resulting in wiring congestion. Additionally, a single large ML accelerator can have very high power density, which can lead to thermal hotspots in the general-purpose computing SoC. Summary of the Invention

[0003] This technique typically involves using multiple small ML accelerators integrated into a general-purpose computing system-on-a-chip (SoC) for efficient and optimized data computation for machine learning (ML) operations. Multiple ML accelerators can be configured to perform distributed operations for specific ML applications, such as complex matrix multiplication. The outputs of each ML accelerator can be combined using atomic storage operations that guarantee complete and indivisible writes to memory to ensure data integrity.

[0004] One aspect of this disclosure provides a system-on-a-chip (SoC) including a memory, a plurality of machine learning (ML) accelerators, and one or more processors communicating with the one or more ML accelerators, the one or more processors being configured to: receive a workload; divide the workload into one or more subtasks; assign the one or more subtasks to the one or more ML accelerators, wherein each ML accelerator is configured to output a corresponding partial result based on the received assigned subtask, wherein the corresponding partial result includes a special opcode and a specified memory address; store the corresponding partial result of each ML accelerator in the memory; and combine the stored partial results using atomic storage operations at the specified memory address to obtain a total result.

[0005] In some examples, each of the multiple ML accelerators can be placed in a different location within the SoC.

[0006] In some examples, the memory may include at least one of system-level cache (SLC) or double data rate (DDR) memory.

[0007] In some examples, each ML accelerator can be configured to transfer the dedicated opcode, the specified memory address, and the corresponding partial result to the memory via the same interconnect protocol.

[0008] In some examples, the memory can be configured to perform parallel arithmetic or logical operations based on different data formats of the corresponding partial results received from each ML accelerator.

[0009] In some examples, the special opcode can be modified based on the different data format of the corresponding partial result received from each ML accelerator and the type of the arithmetic or logical operation.

[0010] In some examples, the special opcode can be modified to include parallel floating-point arithmetic operations, including at least one summation, multiplication, subtraction, or division.

[0011] In some examples, this special opcode can be modified to include parallel logical operations, including Boolean operations, finding the maximum or minimum value, or the sum of squares.

[0012] In some examples, the one or more processors can be configured to use memory-mapped input / output (MMIO) to write to each ML accelerator the time when the corresponding partial result will be stored in the memory.

[0013] In some examples, the multiple ML accelerators can be configured to use memory-mapped input / output (MMIO) writes to each ML accelerator to notify the time when the corresponding partial result will be stored in the SLC or DDR memory.

[0014] Another aspect of this disclosure provides a method for optimizing data computation in machine learning (ML) operations. The method may include: receiving a workload by one or more processors; dividing the workload into one or more subtasks by the one or more processors; assigning the one or more subtasks to one or more ML accelerators by the one or more processors; outputting multiple partial results by the one or more ML accelerators, wherein the multiple partial results include special opcodes and specified memory addresses; storing the multiple partial results in the specified memory by the one or more ML accelerators; and combining the stored multiple partial results at the specified memory addresses using atomic storage operations to obtain a total result.

[0015] In some examples, each of the multiple ML accelerators can be placed in a different location within the system-on-a-chip (SoC).

[0016] In some examples, the memory may include at least one of system-level cache (SLC) or double data rate (DDR) memory.

[0017] In some examples, the method may further include the transfer of the dedicated opcode, the specified memory address, and the corresponding partial result from the one or more ML accelerators to the memory via the same interconnect protocol.

[0018] In some examples, the method may further include the memory performing parallel arithmetic or logical operations based on different data formats of the corresponding partial results received from each ML accelerator.

[0019] In some examples, the special opcode can be modified based on the different data format of the corresponding partial result received from each ML accelerator and the type of the arithmetic or logical operation.

[0020] In some examples, the special opcode can be modified to include parallel floating-point arithmetic operations, including at least one summation, multiplication, subtraction, or division.

[0021] In some examples, this special opcode can be modified to include parallel logical operations, including Boolean operations, finding the maximum or minimum value, or the sum of squares.

[0022] In some examples, the one or more processors can use memory-mapped input / output (MMIO) to notify each of the one or more ML accelerators when the corresponding partial result will be stored in the memory.

[0023] In some examples, each of the one or more ML accelerators can use memory-mapped input / output (MMIO) writes to notify each other when the corresponding partial result is to be stored in the memory. Attached Figure Description

[0024] Figure 1 An example general-purpose computing SoC integrating multiple ML accelerators is depicted according to various aspects of this disclosure.

[0025] Figure 2 An example ML accelerator control system according to various aspects of this disclosure is described.

[0026] Figure 3 A functional block diagram illustrating example atomic storage operations according to various aspects of this disclosure is provided.

[0027] Figure 4 A block diagram depicting example matrix multiplication according to various aspects of this disclosure is provided.

[0028] Figure 5 A flowchart illustrating example methods according to various aspects of this disclosure is provided. Detailed Implementation

[0029] This paper discloses a method for optimized data computation of machine learning (ML) operations using multiple small ML accelerators integrated into a general-purpose computing SoC. One or more processors of the general-purpose computing SoC can be configured to receive workloads and divide the workloads into multiple sub-workloads. These sub-workloads can be distributed among multiple small ML accelerators. Each of the multiple ML accelerators can be configured to output a portion of the workload. The outputs of each ML accelerator can be combined using atomic memory operations that utilize dedicated opcodes and specified memory addresses.

[0030] For example, multiple small ML accelerators can be placed in different locations within a general-purpose computing SoC. Instead of adding dedicated new interfaces between multiple ML accelerators, they can be configured to communicate via interconnects already in use by the general-purpose computing SoC. ML accelerators are configured to perform specific ML applications, such as matrix multiplication. Such operations can be efficiently distributed across multiple small ML accelerators. Each accelerator can be configured to process a portion of the input data, thus producing a partial output. The final result can be obtained by efficiently combining the partial outputs of all ML accelerators using existing interconnects.

[0031] In some examples, the end result is combined using atomic memory operations to allow a single memory write per operation. Therefore, each ML accelerator does not need to perform a "read, modify, write" operation on memory. ML accelerators are configured to transmit a dedicated opcode and memory address when transmitting partial output. In some examples, the dedicated opcode can be modified to include various operations, not just summation. Partial results from all ML accelerators are transferred to memory to perform atomic memory operations. A portion of memory can be assigned to a specific memory address, and the output received from each ML accelerator can be combined at that designated address within memory, reducing additional memory accesses via the SoC interconnect.

[0032] Figure 1 An example general-purpose computing SoC 100 with multiple ML accelerators is depicted. The general-purpose computing SoC 100 may include multiple arrays of CPUs and SLCs, such as CPUs 10A-H, 12A-H, 14A-H, 16A-H and SLCs 10A-H, 12A-H, 14A-H, and 16A-H. Each array of CPUs and SLCs is interconnected via a corresponding bus and cross-point (XP), such as XP 110 and bus 112. The general-purpose computing SoC 100 may also include multiple Double Data Rate (DDR) Synchronous Dynamic Random Access Memory (SDRAM), such as DDR 104A-B, 106A-B, 108A-B, and 110A-B. The general-purpose computing SoC 100 may also include machine learning accelerators, such as ML accelerators 102A-D. It should be understood that a general-purpose computing SoC may include N arrays of DDR, ML accelerators, or CPUs and SLCs, where N is an integer. Each ML accelerator 102A-D is likely small and configured to be housed within a general-purpose computing SoC 100. Each ML accelerator can be configured to execute a specific ML application and process a portion of the input data, thereby producing a partial output. Each partial output can be transmitted via, for example, Figure 1 The existing bus and XP shown are transferred to memory, thus eliminating the need for additional space for additional interconnects. In one example, each ML accelerator 102A-D can be assigned to compute a portion of a matrix multiplication. Each ML accelerator 102A-D can use atomic store operations to send the corresponding partial output to memory via a crosspoint. When each accelerator 102A-D sends its corresponding partial output, each ML accelerator 102A-D can send a special opcode along with a specified memory address. The partial outputs can then be combined at the specified memory address.

[0033] Figure 2This is a block diagram illustrating an example ML accelerator control system. The computing device 200 can take various configurations, such as, for example, a controller or microcontroller, or a processor such as a CPU, GPU, or custom logic including a Tensor Processing Unit (TPU). The computing device may further include an ML accelerator controller 202. The ML accelerator controller 202 can be configured to distribute workloads among multiple ML accelerators disposed within a general-purpose computing SoC and receive a portion of the output from each ML accelerator.

[0034] The ML accelerator controller 202 may include a processor 210, a memory 204 including data 206 and instructions 208, an ML accelerator allocation module 212 and an ML accelerator combination module 214, and other components typically found in server computing devices. In other examples, such operations may be performed by one or more computing devices in a data center or elsewhere.

[0035] Memory 204 may store information accessible by processor 210, including instructions 208 executable by processor 210. Memory 204 may also include data 206 that can be retrieved, manipulated, or stored by processor 210. Memory 204 may be a non-transitory computer-readable medium capable of storing information accessible by processor 210. Processor 210 may also be configured to communicate with external memory, such as hard disk drives, solid-state drives, tape drives, optical storage devices, memory cards, ROM, RAM, DVDs, CD-ROMs, writable memory, and read-only memory. Processor 210 may be a well-known processor or other lesser-known types of processors. Alternatively, processor 210 may be a dedicated controller, such as custom logic.

[0036] Instructions 208 may be a set of instructions (such as machine code) that are directly executed by processor 210, or a set of instructions (such as a script) that are indirectly executed. In this regard, the terms "instruction," "step," and "program" are used interchangeably herein. Instructions 208 may be stored in object code format for direct processing by processor 210, or in other types of computer languages, including scripts or sets of standalone source code modules that are interpreted on demand or compiled in advance. For example, instructions 208 may include instructions for performing complex number machine learning computations using atomic storage operations.

[0037] Data 206 can be retrieved, stored, or modified by processor 210 according to instructions 208. For example, although the system and method are not limited to a specific data structure, data 206 can be stored in a computer register, stored as a table with multiple different fields and records in a relational database, or stored in an XML document. Data 206 can also be formatted in a computer-readable format, such as, but not limited to, binary values, ASCII, or Unicode. Furthermore, data 206 may include information sufficient to identify relevant information such as numbers, descriptive text, proprietary codes, pointers, or references to data stored in other memory locations, including other network locations. Data 206 may include input data for ML calculations such as complex matrix multiplication.

[0038] The ML accelerator allocation module 212 can divide specific workloads and assign the divided workloads to each ML accelerator. For example, an ML accelerator can be assigned to a computational task involving the multiplication of two large matrices. Each matrix can be divided into multiple sub-matrices, each with multiple rows and columns. Each ML accelerator can be assigned to a sub-task involving the multiplication of sub-matrices. Each ML accelerator can be assigned to sub-matrices of different sizes based on its bandwidth. The ML accelerator allocation module 212 can be configured to command each ML accelerator to perform similar computations using the assigned input data.

[0039] The ML accelerator combining module 214 can be configured to command each ML accelerator to transmit a corresponding portion of its output to the ML accelerator controller 202 via a corresponding bus and crosspoint. The ML accelerator combining module 214 can be configured to combine the collected portion outputs to obtain the final output.

[0040] Figure 2 While processor 210 and memory 204 are functionally shown as residing in the same enclosure, processor 210 and memory 204 may alternatively include multiple processors and memories that may or may not be stored in the same physical housing. For example, some of instructions 208 and data 206 may be stored on a removable CD-ROM, while others may be within a read-only computer chip. Some or all of the instructions and data may be stored in a location physically remote from processor 210 but still accessible from it. Similarly, processor 210 may include a series of processors that may or may not operate in parallel.

[0041] It should be understood that, in this example, the ML accelerator allocation module 212 and the ML accelerator combination module 214 are shown as part of the ML accelerator controller 202. In other examples, the ML accelerator allocation module 212 and the ML accelerator combination module 214 may be implemented in software in one or more other systems or computing devices.

[0042] Figure 3 A functional block diagram of an example atomic memory operation is depicted. An atomic memory operation can refer to a read-modify-write sequence performed without interference. For example, when requester 302 AD attempts a read-modify-write, other requesters cannot interfere with the read-modify-write sequence of requester 302 AD. In one example, requester 302 AD can transfer transaction data to memory 306 containing an arithmetic logic unit (ALU). The ALU can perform arithmetic and logical operations on operands in computer instructions. Once the ALU receives the transaction data from requester 302 AD, the ALU can execute sub-operation 304 AD, which may be indicated in the sub-opcode of the transaction data. In some examples, the SLC can be configured to read the address data indicated in the sub-opcode of the transaction data, and the SLC can be configured to use the transaction data and address data in the transaction as operands. In some examples, the SLC may already have a local, up-to-date copy of the address. When the ALU completes the operation, the ALU can store the result of the operation in a specific address of memory 306 based on the address data. Each requester 302A-D can be a different ML accelerator, and each ML accelerator can use its own sub-operations 304A-D to transfer transaction data using a specified address in memory 306. Although each ML accelerator transfers different transaction data and results, each ML accelerator can use the same address in memory 306 to transfer this data, allowing each transferred data to be combined at a specified address within memory 306. Memory 306 can be configured to perform all combinations with minimal additional access to general-purpose computing SoC memory.

[0043] In some examples, such as Figure 1 As shown, each ML accelerator 102A-D is configured to transmit a dedicated opcode and memory address when transferring a portion of its output to the SLC. All partial outputs from the ML accelerators 102A-D are transferred to memory 306. Memory 306 may include the SLC. For a specific computation being performed by the ML accelerator 102A-D, a specific address of the SLC can be assigned to memory 306. Once all partial outputs are received from the ML accelerators 102A-D, a summation can be performed at a specific address in memory 306.

[0044] According to one aspect of this disclosure, an enhanced ALU may include circuitry such as arithmetic adders to enhance atomic memory operations, enabling the enhanced ALU to perform computations on an entire cache line of data (e.g., 64 bytes). In many existing SoCs, conventional atomic memory operations can only perform computations using a 64-bit (8-byte) cache line because conventional atomic memory operations are typically designed for low-bandwidth control flow and therefore do not require handling operands larger than 64 bits. However, ML operations may require higher bandwidth. By adding the additional hardware described above, the enhanced ALU can have up to eight (8) adders that can perform computations in parallel using 64 bytes of data. If the cache line width is 64 bytes and the data format is 4 bytes, the enhanced ALU may contain up to 16 adders or other arithmetic or logic units, depending on the cache line and operand size. It should be understood that the ALU may include a greater number of adders, or the number of adders required based on the cache line size.

[0045] Enhanced ALUs can include adders to support floating-point addition, which is not supported by traditional atomic memory operations. Enhanced ALUs can also add new specified opcodes or parameters to existing opcodes to inform memory that the current operation will be performed using enhanced atomic memory operations, and to indicate the exact data format and requested operation.

[0046] For example, the memory including the enhanced ALU can be configured to perform parallel arithmetic or logical operations based on different data formats received from each ML accelerator. For instance, if the 64 bytes of output data consist of 16 32-bit floating-point values, the SLC can be configured to perform 16 sums simultaneously. If the output consists of 32 16-bit values, the SLC can be configured to perform 32 sums simultaneously. If the output data consists of 64 8-bit values, the SLC can be configured to perform 64 sums simultaneously.

[0047] In some examples, if the order of arithmetic or logical operations is critical, each ML accelerator can be notified via memory-mapped input / output (MMIO) controlled by a central processing unit (CPU) of the time when each can perform its summation. In other examples, ML accelerators can be configured to notify each other of the order of arithmetic or logical operations.

[0048] Figure 4A block diagram illustrating an example matrix multiplication is provided. In one example, multiple ML accelerators can be assigned to the multiplication of matrices 404 and 406. Matrix 404 and 406 can each comprise multiple rows and columns. Matrix 404 can consist of M rows × K columns, and matrix 406 can consist of K rows and N columns. Matrix 404 can be divided into multiple submatrices 405A-D, and matrix 406 can be divided into multiple submatrices 407A-D. Each submatrix can have one or more rows and one or more columns. Each matrix can be equally partitioned based on the number of ML accelerators. In other examples, each matrix can be unequally partitioned based on the bandwidth of each ML accelerator. Each ML accelerator can be assigned to a specific pair of submatrices. For example, the first ML accelerator can be assigned to submatrices 405A and 407A. The size of submatrices 405A and 407A can vary depending on the bandwidth of the ML accelerators. Each ML accelerator can be assigned to multiply submatrices, such as the multiplication of submatrices 405A and 407A, 405B and 407B, 405C and 407C, and 405D and 407D. Each ML accelerator can output a corresponding partial output. These partial outputs can be combined to produce the final output matrix 402.

[0049] Each ML accelerator can be configured to transmit a corresponding partial output via dedicated interconnects and cross-points embedded on a general-purpose computing SoC. Each partial output can be indicated with a special opcode and a specified memory address for the corresponding atomic memory operation. Once the memory has received all the partial outputs, these partial outputs can be combined to output the final output matrix 402.

[0050] Figure 5 A flowchart illustrating an example method is depicted. According to box 502, the system can be configured to receive workloads. Workloads can include specific ML applications, such as complex matrix multiplication. In some examples, workloads can include other types of arithmetic operations, including summation, subtraction, or division. Workloads can also include Boolean and logical operations, such as finding the maximum, minimum, or sum of squares.

[0051] According to box 504, the system can be configured to divide the workload into one or more subtasks. For example, each matrix can be divided into multiple submatrices consisting of a smaller number of rows and columns. In some examples, each matrix can be equally divided based on the number of ML accelerators. In other examples, each matrix can be unequally divided based on the bandwidth of each ML accelerator.

[0052] According to box 506, the system can be configured to assign one or more subtasks to one or more ML accelerators. Such workloads can be efficiently distributed across multiple small ML accelerators embedded in a general-purpose SoC. Each ML accelerator can be configured to process a portion of the input data, thereby producing a portion of the output.

[0053] According to box 508, the system can be configured to instruct ML accelerators to output corresponding partial outputs based on the assigned subtasks they receive, wherein the corresponding partial outputs include a dedicated opcode and a specified memory address. Each ML accelerator can be configured to transmit the dedicated opcode and memory address when transmitting the corresponding partial output. In some examples, the dedicated opcode can be modified to include various operations, such as floating-point arithmetic operations.

[0054] According to box 510, the system can be configured to store a corresponding portion of the output received from each ML accelerator in one or more memory addresses. A portion of the memory can be assigned to a specified address based on specified address data contained in a dedicated opcode. The portion of the output received from the ML accelerator can be combined at the specified addresses within memory. This combination within memory can reduce the overall number of memory accesses. In some examples, the memory may include SLC, DDR memory, a memory controller, or a combination thereof.

[0055] According to box 512, the system can be configured to combine stored portions of output using atomic store operations at specified memory addresses to obtain a total result. Atomic store operations can be used to combine output received from each ML accelerator, guaranteeing a complete and indivisible write to memory to ensure data integrity. To enhance atomic store operations, the existing ALU in memory can be enhanced with an additional adder circuitry system to perform computations on an entire cache line of data (e.g., 64 bytes). In some examples, the system can execute boxes 510 and 512 simultaneously or nearly simultaneously.

[0056] While this document has described the technology with reference to specific examples, it should be understood that these examples are merely illustrative of the principles and applications of the technology. Therefore, it should be understood that many modifications can be made, and other arrangements can be designed without departing from the spirit and scope of the technology as defined by the appended claims.

[0057] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but can be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the preceding description should be presented in an illustrative rather than restrictive manner. Furthermore, the examples described herein and the provision of terms such as “such as,” “comprising,” etc., should not be construed as limiting the subject matter of the claims to specific examples; rather, these examples are intended to illustrate only one of many possible implementations. Moreover, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A system on chip (SoC), comprising: include: Memory; Multiple machine learning (ML) accelerators; as well as One or more processors, which communicate with the one or more ML accelerators, are configured to: Receive workload; Divide the workload into one or more subtasks; The one or more subtasks are assigned to the one or more ML accelerators, wherein each ML accelerator is configured to output a corresponding partial result based on the assigned subtask it receives, wherein the corresponding partial result includes a special opcode and a specified memory address; The corresponding partial results of each ML accelerator are stored in the memory; as well as Atomic storage operations are used at the specified memory address to combine the stored partial results to obtain the total result.

2. The SoC of claim 1, wherein, Each of the multiple ML accelerators is placed in a different location within the SoC.

3. The SoC as described in claim 1, characterized in that, The memory mentioned therein includes at least one of system-level cache (SLC) or double data rate (DDR) memory.

4. The SoC as described in claim 3, characterized in that, Each ML accelerator is configured to transfer the dedicated opcode, the specified memory address, and the corresponding partial result to the memory via the same interconnect protocol.

5. The SoC as described in claim 4, characterized in that, The memory is configured to perform parallel arithmetic or logical operations based on different data formats of the corresponding partial results received from each ML accelerator.

6. The SoC as described in claim 5, characterized in that, The special opcode is modified based on the different data formats of the corresponding partial results received from each ML accelerator and the type of the arithmetic or logical operation.

7. The SoC as described in claim 6, characterized in that, The special opcode is modified to include parallel floating-point arithmetic operations, which include at least one summation, multiplication, subtraction, or division.

8. The SoC as described in claim 6, characterized in that, The special opcode is modified to include parallel logic operations, including Boolean operations, finding the maximum or minimum value, or the sum of squares.

9. The SoC as described in claim 5, characterized in that, The one or more processors are configured to use memory-mapped input / output (MMIO) writes to each ML accelerator to notify the corresponding partial result of the time to be stored in the memory.

10. The SoC as claimed in claim 5, characterized in that, The plurality of ML accelerators are configured to use memory-mapped input / output (MMIO) writes to notify each ML accelerator of the time when the corresponding partial results will be stored in the memory.

11. A method for optimizing data computation in machine learning (ML) operations, characterized in that, The method includes: The workload is received by one or more processors; The workload is divided into one or more subtasks by the one or more processors; The one or more processors assign the one or more subtasks to one or more ML accelerators; The one or more ML accelerators output multiple partial results, wherein the multiple partial results include special opcodes and specified memory addresses; The plurality of partial results are stored in memory by the one or more ML accelerators; and The one or more ML accelerators use atomic storage operations at the specified memory address to combine multiple stored partial results to obtain a total result.

12. The method as described in claim 11, characterized in that, Each of the multiple ML accelerators is placed in a different location within the system-on-chip (SoC).

13. The method as described in claim 11, characterized in that, The memory mentioned therein includes at least one of system-level cache (SLC) or double data rate (DDR) memory.

14. The method as described in claim 13, characterized in that, This further includes the transfer of the dedicated opcode, the specified memory address, and the corresponding partial result to the memory by the one or more ML accelerators via the same interconnect protocol.

15. The method as described in claim 14, characterized in that, It further includes the memory performing parallel arithmetic or logical operations based on different data formats of the corresponding partial results received from each ML accelerator.

16. The method as described in claim 15, characterized in that, The special opcode is modified based on the different data formats of the corresponding partial results received from each ML accelerator and the type of the arithmetic or logical operation.

17. The method as described in claim 16, characterized in that, The special opcode is modified to include parallel floating-point arithmetic operations, which include at least one summation, multiplication, subtraction, or division.

18. The method as described in claim 16, characterized in that, The special opcode is modified to include parallel logic operations, including Boolean operations, finding the maximum or minimum value, or the sum of squares.

19. The method as described in claim 15, characterized in that, The one or more processors use Memory-Mapped Input / Output (MMIO) to write to each of the one or more ML accelerators, notifying them of the time when the corresponding partial result will be stored in the memory.

20. The method as described in claim 15, characterized in that, Each of the one or more ML accelerators uses memory-mapped input / output (MMIO) writes to notify each other of the time when the corresponding partial results will be stored in the memory.

Images