Awareness of data layout in memory architecture for executing neural network models

By processing neural network models in memory and using PIM technology to optimize data transmission and calculation, the problem of large data transmission in the von Neumann architecture is solved, the computing efficiency and memory utilization are improved, and the computing efficiency and memory utilization are improved. Overall performance.

CN112906877BActive Publication Date: 2024-12-03ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011305314.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-11-19
Filing Date
2020-11-19
Publication Date
2024-12-03
Estimated Expiration
2040-11-19

AI Technical Summary

Technical Problem

When processing deep neural network models on a conventional von Neumann architecture, the data transmission volume between the processing unit and the storage unit is large, resulting in high computing volume and large storage occupancy, and it is difficult to effectively reduce data movement inside the memory, affecting the impact of performance.

Method used

Using the technology of supporting in-memory processing (PIM) to perform calculations in memory, use memory block components and data links to perform data transmission, optimize data alignment and transmission paths, reduce unnecessary data movement, and improve memory utilization and overall performance. .

Benefits of technology

By processing the neural network model in memory, data transmission between the processing unit and the storage unit is reduced, computing efficiency and memory utilization are improved, energy consumption and occupation are reduced, and execution performance of the neural network model is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112906877B_ABST
    Figure CN112906877B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a device supporting processing in memory (PIM) for executing a neural network model. The PIM-supporting device includes a memory block component, which includes a first memory block array; a second memory block array adjacent to the first memory block array; a plurality of first data links associated with the first memory block array and the second memory block array, wherein each of the plurality of first data links communicatively connects two corresponding memory blocks from the first memory block array and the second memory block array, respectively; and, a second data link communicatively connected to the plurality of first data links, data from a first memory block of the first memory block array can be transmitted to a second memory block of the second memory block array via the plurality of first data links and the second data link.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Machine learning has been widely used in various fields such as natural language processing, speech recognition, image classification, etc. In machine learning, neural network models have been increasing and becoming more and more complex. As neural network models, especially deep neural network (DNN) models, become large and complex, with hundreds of layers and millions of weights, executing neural network models is not only computationally intensive, but also occupies a large amount of memory. Because the conventional von Neumann architecture has separate processing units and storage units, when processing neural network models on the conventional von Neumann architecture, a large amount of data transfer occurs between the processing units and the storage units, which becomes a bottleneck for processing neural network models.

[0002] Processing In Memory (PIM) technology that enables computation to be performed in memory units has emerged to solve such problems by reducing data movement between processing units and memory units. However, data or processor alignment suitable for executing neural network models on PIM architecture is very important to reduce unnecessary data movement within memory units, improve memory utilization, and improve overall performance. Summary of the invention

[0003] An embodiment of the present disclosure provides a device supporting processing in memory (PIM) for executing a neural network model. The PIM-supporting device includes a storage block component, which includes a first storage block array; a second storage block array adjacent to the first storage block array; a plurality of first data links associated with the first storage block array and the second storage block array, wherein each of the plurality of first data links communicatively connects two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and, a second data link communicatively connected to the plurality of first data links, data from a first storage block of the first storage block array can be transmitted to a second storage block of the second storage block array via the plurality of first data links and the second data link.

[0004] An embodiment of the present disclosure also provides a method for executing a neural network model on a PIM-supported device including a memory block component, the method comprising: processing a first operation of the neural network model on a first memory block of a first memory block array in the memory block component; transmitting output data from the first memory block in the memory block component to the second memory block of a second memory block array via a data link that communicatively connects the first memory block and the second memory block in the memory block component; and processing a second operation of the neural network model on the second memory block based on the output data.

[0005] An embodiment of the present disclosure also provides a non-transitory computer-readable storage medium storing an instruction set that can be executed by at least one processor in a terminal supporting processing in memory (PIM) to enable the terminal to perform the following method: processing a first operation of a neural network model on a first memory block of a first memory block array in a memory block component; transmitting output data from the first memory block in the memory block component to the second memory block of a second memory block array via a data link that communicatively connects the first memory block and the second memory block in the memory block component; and processing a second operation of the neural network model on the second memory block based on the output data.

[0006] An embodiment of the present disclosure also provides a terminal, comprising: a host unit; and a device supporting processing in memory (PIM), the device supporting processing in memory (PIM) being communicatively connected to the host unit, the device supporting processing in memory (PIM) comprising: a storage block component, the storage block component comprising: a first storage block array; a second storage block array adjacent to the first storage block array; a plurality of first data links associated with the first storage block array and the second storage block array, wherein each data link in the plurality of first data links can communicatively connect two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and a second data link communicatively connected to the plurality of first data links, wherein data from a first storage block of the first storage block array can be transmitted to a second storage block of the second storage block array via the plurality of first data links and the second data links.

[0007] Additional features and advantages of the disclosed embodiments will be partially set forth in the following description, and in part will become apparent from the description, or may be learned through the implementation of the embodiments. The features and advantages of the disclosed embodiments may be realized and obtained through the elements and combinations set forth in the claims.

[0008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 An exemplary in-memory processing block configuration consistent with embodiments of the present disclosure is shown.

[0010] Figure 2A An exemplary neural network accelerator architecture consistent with embodiments of the present disclosure is shown.

[0011] Figure 2B A schematic diagram of an exemplary cloud system incorporating a neural network accelerator architecture consistent with an embodiment of the present disclosure is shown.

[0012] Figure 3 An exemplary memory tile configuration consistent with embodiments of the present disclosure is shown.

[0013] Figure 4 Exemplary memory block components in a memory bank consistent with embodiments of the present disclosure are shown.

[0014] Figure 5A An exemplary data transfer table for recording data transfers consistent with an embodiment of the present disclosure is shown.

[0015] Figure 5B An exemplary block table for recording storage block status consistent with an embodiment of the present disclosure is shown.

[0016] Figure 5C An exemplary state transition cycle for a memory block consistent with embodiments of the present disclosure is shown.

[0017] Fig. 6A An example of a neural network model consistent with an embodiment of the present disclosure is shown.

[0018] Figure 6B A method for executing the method consistent with the embodiment of the present disclosure is shown. Fig. 6A An exemplary node mapping in a memory block component of a neural network model.

[0019] Figure 6C The embodiment consistent with the present disclosure is shown in the implementation Fig. 6A The state transitions of the memory blocks in the memory block component during the neural network model.

[0020] Figure 7 An embodiment consistent with the present disclosure is shown. Figure 2A An exemplary flowchart for training a neural network model on a neural network accelerator architecture. DETAILED DESCRIPTION

[0021] Reference will now be made in detail to exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, wherein, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments set forth in the following description of the exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with aspects related to the present invention as described in the appended claims.

[0022] Figure 1An exemplary process-in-memory (PIM) block configuration consistent with an embodiment of the present disclosure is shown. The PIM block 100 includes a memory cell array 110, a block controller 120, a block row driver 131, and a block column driver 132. Although some embodiments will be illustrated using ReRAM (resistive random access memory) as an example, it should be understood that the PIM block 100 according to an embodiment of the present disclosure can be implemented based on various memory technologies including DRAM (dynamic random access memory), SRAM (static random access memory), ReRAM, etc.

[0023] The memory cell array 110 may include m rows r1 to r2. m , and n columns c1 to c n .like Figure 1 The cross memory 111 can be connected in m rows r1 to r m Each of the n columns c1 to c n In some embodiments, data may be stored in the crossbar memory 111 as a multi-bit memristor.

[0024] The block row driver 131 and the block column driver 132 may provide power to m rows r1 to r2. m and to n columns c1 to c n Signals such as voltage signals are provided to process corresponding operations. In some embodiments, the block row driver 131 and the block column driver 132 may be configured to pass analog signals through the crossbar memory 111. In some embodiments, the analog signals may be converted from digital input data.

[0025] The block controller 120 may include an instruction register for storing instructions. In some embodiments, the instructions may include instructions on when the block row driver 131 or the block column driver 132 provides signals to the corresponding columns or rows, which signals are to be provided, etc. The block controller 120 may decode the instructions stored in the register into signals used by the block row driver 131 or the block column driver 132.

[0026] The PIM block 100 may further include a row sense amplifier 141 or a column sense amplifier 142 for reading data from a memory cell or storing data in a memory cell. In some embodiments, the row sense amplifier 141 and the column sense amplifier 142 may store data for caching. In some embodiments, the PIM block 100 may further include a DAC 151 (digital-to-analog converter) or an ADC 152 (analog-to-digital converter) to convert input signals or output data between an analog domain and a digital domain. In some embodiments of the present disclosure, the row sense amplifier 141 or the column sense amplifier 142 may be omitted because the calculations in the PIM block 100 may be performed directly on the values ​​stored in the memory cell without reading out the values ​​or using any sense amplifiers.

[0027] According to an embodiment of the present disclosure, the PIM block 100 implements parallel computing by using the memory as multiple SIMD (single instruction, multiple data) processing units. The PIM block 100 can support computing operations including bitwise operations, addition, subtraction, multiplication, and division of integer and floating point values. For example, in Figure 1 In the memory cell array 110, the first column c1 and the second column c2 can store the first vector A and the second vector B, respectively. By applying the formatted signals to the first to third columns (c1 to c3) and the corresponding rows for the lengths of the vectors A, B and C, the vector operation result C obtained by adding the vectors A and B can be stored in the third column c3. Similarly, Figure 1 The memory cell array 110 can also support vector multiplication and addition operations. For example, a calculation C=aA+bB can be performed by applying a voltage signal corresponding to multiplier a to the first column c1 and a voltage signal corresponding to multiplier b to the second column c2, and adding by applying formatted signals to the corresponding columns and rows, and the result C can be stored in the third column c3.

[0028] In some embodiments, a vector may be stored in multiple columns to represent the n-bit value of an element. For example, a vector whose elements have 2-bit values ​​may be stored in two columns of storage cells. In some embodiments, when the length of a vector exceeds the number of rows of the storage cell array 110 constituting the storage block, the vector may be stored in multiple storage blocks. Multiple storage blocks may be configured to calculate different vector segments in parallel. Although in an embodiment the PIM architecture performs computational operations without using arithmetic logic other than storage cells, the present disclosure may also be applied to a PIM architecture including arithmetic logic for performing arithmetic operations in the PIM architecture.

[0029] The PIM architecture can be used to process neural network models to address the overhead of data movement between processing units and storage units. As previously shown, computational operations (such as addition, multiplication, etc.) can also be performed as column-wise vector calculations in the PIM architecture. It is important to properly align the data before processing such operations in the memory to reduce unnecessary data movement in the memory and maximize the computing power of the PIM architecture. The disclosed embodiments provide a PIM accelerator architecture that is capable of effective data alignment when processing neural network models. The disclosed embodiments provide a data layout-aware PIM accelerator architecture for improving the execution efficiency of neural network models.

[0030] Figure 2A An exemplary neural network accelerator architecture 200 consistent with an embodiment of the present disclosure is shown. In the context of the present disclosure, a neural network accelerator may also be referred to as a machine learning accelerator or a deep learning accelerator. In some embodiments, the accelerator architecture 200 may be referred to as a neural network processing unit (NPU) architecture 200. Figure 2A As shown, the accelerator architecture 200 may include a PIM accelerator 210 , an interface 212 , etc. It should be understood that the PIM accelerator 210 may perform algorithmic operations based on the transmitted data.

[0031] The PIM accelerator 210 may include one or more memory blocks 2024. In some embodiments, the memory blocks 2024 may include multiple memory blocks for data storage and computation. The memory blocks may be configured to perform one or more operations (e.g., multiplication, addition, multiply-accumulate, etc.) on the communicated data. In some embodiments, each memory block included in the memory blocks 2024 may have a memory block having a plurality of memory blocks. Figure 1 The PIM block 100 shown has the same configuration. Due to the hierarchical design of the PIM accelerator 210, the PIM accelerator 210 can provide versatility and scalability. The PIM accelerator 210 can include any number of memory blocks 2024, and each memory block 2024 can have any number of memory blocks.

[0032] The interface 221 (e.g., a PCIe interface) serves as (usually) an inter-chip bus to provide communication between the PIM accelerator 210 and the host unit 220. The inter-chip bus connects the PIM accelerator 210 to other devices (e.g., off-chip memory or peripheral devices). In some embodiments, the accelerator architecture 200 may also include a DMA unit (not shown) that assists in transferring data between the host memory 221 and the PIM accelerator 210. In addition, the DMA unit may assist in transferring data between multiple accelerators. The DMA unit may allow off-chip devices to access on-chip and off-chip memory without causing a host CPU interrupt. Thus, the DMA unit may also generate memory addresses and initiate memory read or write cycles. The DMA unit may also include several hardware registers that one or more processors can read and write, including a memory address register, a byte count register, one or more control registers, and other types of registers. These registers may specify some combination of source, destination, transfer direction (read from or write to an input / output (I / O) device), the size of the transfer unit, and / or the number of bytes transferred in a burst. It should be understood that the accelerator architecture 200 may include a second DMA unit that may be used to transfer data between other accelerator architectures to allow multiple accelerator architectures to communicate directly without involving the host CPU.

[0033] although Figure 2A The accelerator architecture 200 is interpreted as including a memory block (e.g., Figure 1 The disclosed embodiments include a PIM accelerator 210 of a PIM memory block 100 of a PIM memory block 100, but it should be understood that the disclosed embodiments can be applied to any type of memory block that supports arithmetic operations for accelerating certain applications (such as deep learning).

[0034] The accelerator architecture 200 may also communicate with a host unit 220. The host unit 220 may be one or more processing units (eg, an X86 central processing unit). In some embodiments, the PIM accelerator 210 may be considered a co-processor of the host unit 220.

[0035] like Figure 2AAs shown, the host unit 220 can be associated with a host memory 221. In some embodiments, the host memory 221 can be an integrated memory or an external memory associated with the host unit 220. The host memory 221 can be a local or global memory. In some embodiments, the host memory 221 can include a host disk, which is an external memory configured to provide additional memory for the host unit 220. The host memory 221 can be a double data rate synchronous dynamic random access memory (e.g., DDR SDRAM) or the like. Compared with the on-chip memory of the PIM accelerator 210 used as a high-level cache, the host memory 221 can be configured to store a large amount of data at a lower access speed. The data stored in the host memory 221 can be transmitted to the PIM accelerator 210 for executing the neural network model.

[0036] In some embodiments, the host system having the host unit 220 and the host memory 221 may include a compiler (not shown). A compiler is a program or computer software that converts computer code written in a programming language into instructions to create an executable program. In machine learning applications, the compiler may perform various operations, such as preprocessing, lexical analysis, parsing, semantic analysis, conversion of input programs to intermediate representations, code optimization, and code generation, or a combination thereof.

[0037] In some embodiments, the compiler may push one or more commands to the host unit 220. Based on these commands, the host unit 220 may assign any number of tasks to one or more memory banks (e.g., memory bank 2024) or processing elements. Some commands may instruct the DMA unit to transfer instructions and data from the host memory (e.g., Figure 2A of the host memory 221) to the accelerator (e.g., Figure 2A Instructions may be loaded into each memory block (eg, Figure 2A 's storage block 2024), and one or more storage blocks can process these instructions.

[0038] It should be understood that the first few instructions may indicate loading / storing data from the host memory 221 into one or more local memories of the memory bank. Each memory bank may then initiate an instruction pipeline that includes fetching instructions from local memory (e.g., via a fetch unit), decoding instructions (e.g., via an instruction decoder), generating local memory addresses (e.g., corresponding to operands), reading source data, executing or loading / storing operations, and then writing back results.

[0039] Figure 2BA schematic diagram of an exemplary cloud system incorporating a neural network accelerator architecture consistent with an embodiment of the present disclosure is shown. Figure 2B As shown, cloud system 230 may have a cloud service that provides artificial intelligence (AI) functions and may include multiple computing servers (e.g., 232 and 234). In some embodiments, computing server 232 may be incorporated into, for example, Figure 2A The neural network accelerator architecture 200. For simplicity and clarity, the Figure 2B A neural network accelerator architecture 200 is shown.

[0040] With the help of the neural network accelerator architecture 200, the cloud system 230 can provide extended AI functions such as image recognition, facial recognition, translation, 3D modeling, etc. It should be understood that the neural network accelerator architecture 200 can be deployed to computing devices in other forms. For example, the neural network accelerator architecture 200 can also be integrated into computing devices such as smart phones, tablets, and wearable devices.

[0041] Figure 3 An exemplary memory block configuration consistent with embodiments of the present disclosure is shown. Memory block 300 may include memory block components 310 , controller 320 , row driver 331 , column driver 332 , and global buffer 340 .

[0042] The memory block assembly 310 may include a plurality of memory blocks arranged in a two-dimensional grid consistent with an embodiment of the present disclosure, which will be referred to as Figure 4 Provide detailed explanation.

[0043] The controller 320 may provide commands to each memory block in the memory block assembly 310 via a row driver 331, a column driver 332, and a global buffer 340. The row driver 331 is connected to each row of memory blocks in the memory block assembly 310, and the column driver 332 is connected to each column of memory blocks in the memory block assembly 310. In some embodiments, a block controller (e.g., Figure 1 The block controller 120 in the embodiment may be configured to receive commands via the row driver 331 or the column driver 332 and send commands to the block row driver (eg, Figure 1 131) and block column drivers (e.g., Figure 1 The block column driver 132 in the memory sends a signal to perform a corresponding operation in the memory. According to an embodiment of the present disclosure, the memory blocks can independently perform different operations by using the block controller in the memory block of the memory block assembly 310, so block-level parallel processing can be performed. The block controller can effectively transfer data between the memory cells arranged in rows and columns in the corresponding memory blocks.

[0044] In some embodiments, the global buffer 340 may be used to transfer data between storage blocks in the storage block assembly 310. For example, when transferring data from one storage block to another storage block in the storage block assembly 310, the controller 320 may use the global buffer 340. According to some embodiments of the present disclosure, the global buffer 340 may be shared by all storage blocks in the storage block assembly 310. The global buffer 340 may be configured to store commands for each storage block to handle the assigned tasks in processing the neural network model. In some embodiments, the controller 320 is configured to send the commands stored in the global buffer 340 to the corresponding storage blocks via the row driver 331 and the column driver 332. In some embodiments, the global buffer 340 may be sent from a host unit (e.g., Figure 2A The global buffer 340 may be configured to store and send data for processing tasks assigned to the memory block. In some embodiments, the global buffer 340 may be configured to store and send data for processing tasks assigned to the memory block. Figure 2A The controller 320 may transmit the data stored in and sent from the global buffer 340 to the host unit 220 of the memory block assembly 310 or other memory blocks in the memory block assembly 310. In some embodiments, the controller 320 is configured to store the data in the memory blocks in the memory block assembly 310 into the global buffer 340. In some embodiments, the controller 320 may receive and store the data of an entire row of one memory block in the memory block assembly 310 into the global buffer 340 in one cycle. Similarly, the controller 320 may send the data of an entire row from the global buffer 340 to another memory block in one cycle.

[0045] Reference now Figure 4 , Figure 4 An exemplary memory block assembly 310 in a memory bank 300 consistent with an embodiment of the present disclosure is shown. The memory block assembly 310 may include a plurality of memory blocks 410, a plurality of switches 420, and a plurality of selectors 430.

[0046] According to an embodiment of the present disclosure, operations for processing a neural network model in a PIM architecture can be converted into column-based vector calculations in a memory. Column-based vector calculations in a memory, the output data of a certain neural network layer or neural network operation can be stored in a storage block 410 as a set of column-based vectors. The storage block component 310 can be configured to support or facilitate column-based vector calculations so as to minimize unnecessary data movement within the storage block component 310 and maximize storage utilization when executing a neural network model. In some embodiments, multiple storage blocks 410 in the storage block component 310 can be interconnected to facilitate data movement between storage blocks, which makes the storage block component 310 suitable and efficient for processing vector calculations. Multiple storage blocks 410 can be arranged in a 2D grid. In some embodiments, multiple storage blocks 410 can be arranged in M ​​rows and N columns in the storage block component 310, where M and N are natural numbers greater than 1. Each storage block 410 can be located at the intersection of each row and column. Although Figure 4 As an example, 12 memory blocks 410_11 to 410_34 are shown arranged in 3 rows and 4 columns, but it should be understood that the memory block assembly 310 can have any number of memory blocks and can be arranged in any manner. In some embodiments, each memory block 410 can be a PIM block (e.g., Figure 1 In some embodiments, the memory block 410 is a basic computing unit for performing various in-memory operations.

[0047] According to an embodiment of the present disclosure, the row data link 451 interconnects two adjacent storage blocks in the same row of the plurality of storage blocks 410. Figure 4 As shown, the first storage block 410_11 and the second storage block 410_12 placed adjacently in the first row are connected via a row data link 451_1, and the fifth storage block 410_21 and the sixth storage block 410_22 placed adjacently in the second row are connected via a row data link 451_2. Similarly, two adjacent storage blocks in the same row can be interconnected via their own row data links.

[0048] According to an embodiment of the present disclosure, the storage block assembly 310 may further include a column data link that connects a plurality of data links between two adjacent storage block columns in the plurality of storage blocks 410. Figure 4As shown, two column data links 452_1 and 452_2 connect three row data links 451_1 to 451_3 between the first column of storage blocks 410_11 to 410_31 and the second column of storage blocks 410_12 to 410_32. In some embodiments, via the column data link 452 and the row data link 451 between two adjacent columns of storage blocks, one storage block column in two adjacent columns of storage blocks can share data with another storage block column. For example, column data link 452_1 or 452_2 enables the first column of storage blocks 410_11 to 410_31 to share data with the entire second column of storage blocks 410_12 to 410_31 by cooperating with row data links 451_1 to 451_3.

[0049] According to an embodiment of the present disclosure, a switch 420 may be provided for each row data link 451. For example, a first switch 420_1 and a second switch 420_2 may be provided for the row data link 451_1, and a third switch 420_3 and a fourth switch 420_4 may be provided for the row data link 451_2. In some embodiments, each switch 420 is configured to open or close the data path to / from the switch 420 in the same row to / from the corresponding data link 451 to / from the adjacent storage block. For example, when the first switch 420_1 is turned on. The data path from the first storage block 410_11 to the row data link 451_1 is open. In this example, data from the first storage block 410_11 may be transmitted through row data links 451_1, 451_2, and 451_3 and column data links 452_1 and 452_2. In the same example, when the second switch 420_2 is turned on, the data path from the row data link 451_1 to the second memory block 410_12 is opened. In this example, data shared through the row data links 451_1, 451_2, and 451_3 and the column data links 452_1 and 452_2 may be transmitted to the second memory block 410_12.

[0050] According to some embodiments of the present disclosure, a selector 430 is provided for multiple switches in the same column. For example, a selector 430_1 is provided for switches 420_1, 420_3, and 420_5 of the first column, and a selector 430_2 is provided for switches 420_2, 420_4, and 420_6 of the second column. In some embodiments, the selector 430 may be configured to select a switch to be turned on among the multiple switches in the corresponding column. For example, the selector 430_1 may be configured to select a switch to be turned on among the multiple switches 420_1, 420_3, and 420_5 in the first column, and the selector 430_2 may be configured to select a switch to be turned on among the multiple switches 420_2, 420_4, and 420_6 in the second column.

[0051] When it is necessary to transfer data from the first storage block 410_11 to the tenth storage block 410_32, the selector 430_1 can select the first switch 420_1 to be turned on, and the selector 430_2 can select the sixth switch 420_6 to be turned on. In this example, the data path from the first storage block 410 to the row data link 451_1 is opened by turning on the first switch 420_1, and the data from the first storage block 410_11 can be accessed via the storage blocks 410_12, 410_22 and 410_32 of the second column through two column data links 452_1 and 452_2 and row data links 451_1, 451_2 and 451_3. Because only the sixth switch 420_6 is selected to be turned on, and therefore the data path from the row data link 451_3 to the tenth storage block 410_32 is disconnected, the data from the first storage block 410 can be transmitted to the tenth storage block 410_32.

[0052] According to some embodiments of the present disclosure, data transmission from one storage block to multiple storage blocks can be supported. The selector 430 that controls the data path from the row data link 451 to the adjacent storage block 410 can select more than one switch at a time (e.g., in one cycle) to transfer data to multiple storage blocks 410. For example, in the above example, the selector 430_2 can select the second switch 420_2 and the sixth switch 420_6 to transfer data from the first storage block 410_11 to two storage blocks 410_12 and 410_32 in the same cycle. In the present disclosure, an embodiment of selecting a switch in one cycle will be described. In these embodiments, data transmission from one storage block to many storage blocks (also referred to as "one-to-many" data transmission) can be performed in multiple cycles. For example, in the above example, the selector 430_2 may select the second switch 420_2 in the first cycle and the sixth switch 420_6 in the second cycle, while the selector 430_1 selects the first switch 420_1 in the first cycle and the second cycle to transfer data from the first memory block 410_11 to the two memory blocks 410_12 and 410_32.

[0053] According to some embodiments, when data is transferred from one storage block to another storage block, the values ​​stored in a column of the storage cells of the transmitting storage block may be stored in a row of the storage cells of the receiving storage block. For example, when data is transferred from the first storage block 410_11 to the tenth storage block 410_32, the values ​​stored in a column of the storage cells of the first storage block 410_11 (e.g., the column of storage cells storing the output vector) may be stored in a row of the storage cells in the tenth storage block 410_32. In some embodiments, data of an entire column of storage cells from a storage block (e.g., 410_11) may be transferred to one or more adjacent storage blocks (e.g., 410_12 and 410_32) in one cycle. In some embodiments, the transmission latency may depend on the size of each element in the vector.

[0054] As described above, operations for processing a neural network model in a PIM architecture may be converted to column-wise vector calculations in memory. Column-wise vector calculations in memory, output data of a certain neural network layer or neural network operation may be stored in a memory block as a set of column-wise vectors. According to some embodiments of the present disclosure, the global buffer 340 may be suitable for transmitting data stored in a row of a storage unit, and may show optimal performance when the entire data stored in a row of a storage unit is moved in one cycle. Transmitting column-wise vectors through the global buffer 340 may require sequential transmission for each element of the vector. Moving data through the global buffer 340 shared with all storage blocks 410 may result in severe conflicts, which may reduce system throughput. According to an embodiment of the present disclosure, data transmission between adjacent storage blocks may be performed without using the global buffer 340 via data links 451 and 452, which improves the data transmission performance between adjacent storage blocks and the overall throughput when executing a neural network model on the PIM accelerator 210.

[0055] According to an embodiment of the present disclosure, the storage block component 310 may further include a transmission buffer 440 for transmitting data between non-adjacent storage blocks 410. In some embodiments, the transmission buffer 440 may be provided to the columns of the storage blocks. For example, the first transmission buffer 440_1 may be provided to the first column of the storage blocks 410_11, 410_21, and 410_31, the second transmission buffer 440_2 may be provided to the second column of the storage blocks 410_12, 410_22, and 410_32, the third buffer 440_3 may be provided to the third column of the storage blocks 410_12, 410_23, and 410_33, and the fourth transmission buffer 440_4 may be provided to the fourth column of the storage blocks 410_14, 410_24, and 410_34. Similar to the storage block, the row data link 451_4 extends between two adjacent transmission buffers 440_1 and 440_2, and the two column data links 452_1 and 452_2 extend to the row data link 451_4 between the first column of the storage block and the second column of the storage block. In some embodiments, the transmission buffer 440 may have only a row buffer to store data in a row of memory cells. When data needs to be transferred from the first storage block 410_11 in the first column to the third storage block 410_13 in the third column, the selector 430_1 can select the first switch 420_1 to be turned on in the first cycle, and the selector 430_2 can select the eighth switch 420_8 to be turned on. In this first cycle, data from the first storage block 410_11 can be stored in the row buffer of the second transmission buffer 440_2. In the second cycle, the selector 430_3 may select the 15th switch 420_15 to be turned on, the selector 430_4 may select the 10th switch 420_10 to be turned on, and then the data stored in the second transmission buffer 440_2 may be transmitted to the third storage block 410_13.

[0056] Although Figure 4The first transmission buffer 440_1 and the fourth transmission buffer 440_4 and the seventh switch 420_7 and the 24th switch 420_24 are shown, but those elements may be omitted in some embodiments because the first transmission buffer 440_1 does not provide data to be transmitted and the fourth transmission buffer 440_4 does not become the final destination of the data transmission. According to an embodiment of the present disclosure, via the transmission buffer 440, data transmission between non-adjacent memory blocks may be performed without using the global buffer 340, which also improves the data transmission performance between non-adjacent memory blocks and the overall throughput when executing a neural network on the PIM accelerator 210. According to some embodiments of the present disclosure, data transmission between non-adjacent memory blocks may take H cycles, where H is the column difference between two non-adjacent memory blocks. For example, when data is transferred from the first storage block 410_11 in the first column to the third storage block 410_13 in the third column, the data transfer may require 2 cycles (e.g., the first cycle is from the first storage block 410_11 to the second transmission buffer 440_2 and the second cycle is from the second transmission buffer 440_2 to the third storage block 410_13). When each transmission from the first storage block 410_11 to the second transmission buffer 440_2 and from the second transmission buffer 440_2 to the third storage block 410_13 requires C cycles due to the size of the data, the data transfer may take a total of H*C cycles.

[0057] In some embodiments, data may be transferred in one direction in data block component 310 via row and column data links 451 and 452 or transfer buffer 440. For example, in Figure 4 In the memory block component 310, the Figure 4 In the data block component 310, data is transmitted from the left side to the right side (i.e., from the first column to the second column) through the row and column data links 451 and 452 or the transmission buffer 440. When data needs to be transmitted in other directions, such as from the right side to the left side or in a direction parallel to the column direction, such data transmission can be performed via the global buffer 340. For example, when moving data from one storage block (e.g., 410_11) to another storage block (410_21) located in the same column as the one storage block, the controller 320 can be configured to perform data transmission via the global buffer 340. When moving data from one storage block (e.g., 410_12) to another storage block (e.g., 410_11), the controller 320 can be configured to perform such data transmission via the global buffer 340. Although it has been explained in the embodiments that Figure 4 Data is transmitted from the left side to the right side of the storage block assembly 310, but it should be understood that embodiments of the present disclosure can be implemented so that data can be transmitted in any other direction.

[0058] According to an embodiment of the present disclosure, a neural network model may be mapped to a plurality of storage blocks 410, and the storage blocks may process corresponding tasks, such as training or reasoning, in a pipeline manner to maximize system throughput. Figure 3 The memory block 300 may include an instruction memory 350 configured to store instructions for executing the neural network model in the memory block component 310 in a pipelined manner. The instruction memory 350 may store instructions for calculation or data movement between memory blocks in the memory block component 310. The controller 320 may be configured to access the instruction memory 350 to retrieve instructions stored in the instruction memory 350.

[0059] The instruction memory 350 may be configured to have a separate instruction segment assigned to each storage block. In some embodiments, the instruction segment assigned to the corresponding storage block 410 may be divided into a plurality of parts, for example, three parts including a RECEIVE part, a COMPUTE part, and a SEND part. The RECEIVE part may store instructions for loading data, which are used for processing tasks assigned to the corresponding storage block 410. Data may be transferred from other storage blocks 410, a transmission buffer 440, a global buffer 340, or a host unit 220. For example, instructions on which source location the data is transferred from may be stored in the RECEIVE part. The COMPUTE part may store instructions for performing computation-related operations of a neural network model. For example, instructions on which computational operations are to be performed may be stored in the COMPUTE part. The SEND part may store instructions for transferring output data to other storage blocks 410, a transmission buffer 440, a global buffer 340, or a host unit 220. For example, instructions on which destination location the output data is transferred to may be stored in the SEND part. In some embodiments, the compiler may determine the source address and the destination address of the operand of the corresponding instruction based on the memory layout and the data layout mechanism in the memory block component 310 .

[0060] According to an embodiment of the present disclosure, the storage block 300 may further include one or more control tables. In some embodiments, the storage block 300 may include a data transfer table for recording data transfer in the storage block 300 . Figure 5A An exemplary data transfer table 360 ​​for recording data transfers in a storage block 300 consistent with an embodiment of the present disclosure is shown. The data transfer table 360 ​​may be configured to record data transfers between storage blocks. In some embodiments, the data transfer table 360 ​​may be configured to record pending data transfers. Figure 5AAs shown, the data transfer table 360 ​​may have a plurality of fields, for example, including a source field (Src), a destination field (Dst), and a buffer field (Bfr). The source field may include information indicating a storage block that sends data, and the destination field may include information indicating a storage block that receives data. The buffer field may include information indicating in which buffer (e.g., the transmission buffer 440 or the global buffer 340) the data is currently placed. The controller 320 may schedule instructions at runtime to execute a neural network model with reference to the information stored in the data transfer table 360. According to some embodiments of the present disclosure, the storage block 410 may perform one of calculation or data transfer at one time, and the size of data that can be transferred within one cycle is limited, and one data transfer may require multiple cycles to complete.

[0061] In some embodiments, the memory block 300 may include a block table for indicating which state a corresponding memory block is in. Figure 5B An exemplary block table 370 for recording storage block status consistent with an embodiment of the present disclosure is shown. Figure 5B As shown, the block table 370 may have a state field (State) storing the current state of the corresponding storage block. According to some embodiments of the present disclosure, during the execution of the neural network model, the storage block 410 may have one of the states, for example, including an idle state (I), a computing state (C), and a ready state (R). Referring to FIG. Figure 5C , will describe the status of the storage block. Figure 5C As shown, once the storage block 410 receives all the data for performing a particular computing operation, the state of the storage block 410 changes from the idle state (I) to the computing state (C). Once the storage block 410 completes the computing operation, the state of the storage block 410 changes from the computing state (C) to the ready state (R). Once the storage block 410 completes sending the output data to the destination location, the state of the storage block 410 changes from the ready state (R) to the idle state (I).

[0062] According to some embodiments of the present disclosure, the state transition cycle may be repeated during the execution of the neural network model. In some embodiments, when the target memory is full, the data transmission may be suspended. For example, when data is transmitted to the transmission buffer 440 and the transmission buffer 440 is full or has no space for additional data transmission, the scheduled data transmission may be suspended until the transmission buffer 440 is emptied. In some embodiments, Figure 5BThe block table 370 may include a field for recording the size of received data or the size of transmitted data. In some embodiments, the size of received data may be recorded by counting the number of iterations of transmitting data. For example, when the size of data that can be transmitted within one cycle is fixed or limited and the data to be transmitted is larger than the size of data that can be transmitted within one cycle, the data to be transmitted may be divided into smaller parts, each part having a size of data that is equal to or smaller than the size of data that can be transmitted within one cycle. The divided smaller parts may then be transferred from one location to another in multiple cycles. Therefore, in some embodiments, counting the number of iterations of transmitting data may be used to determine the size of the data transmitted. Figure 5B As shown, the block table 370 may have a field (#Rec) for recording the iteration number of receiving data and a field (#Send) for recording the iteration number of sending data.

[0063] Back to reference Figure 3 , the controller 320 may be configured to schedule instructions (e.g., stored in the memory 350) by coordinating the hardware components for the neural network model pipeline execution in the storage block component 310. According to some embodiments of the present disclosure, the controller 320 may be configured to decode instructions, schedule instructions, and update control tables (e.g., data transfer tables 360 and block tables 370). During runtime, the controller 320 may periodically check the control table (e.g., data transfer table 360 ​​or block table 370) to determine the next step of each storage block 410. For example, the controller 320 may determine which state (e.g., idle state (I), calculation state (C), or ready state (R)) the storage block 410 will enter in the next cycle by referring to the data transfer table 360 ​​or the block table 370. After checking the next step, the controller 320 may schedule the instruction corresponding to the next step of the storage block 410 by referring to the instruction memory 350. The controller 320 may be configured to update the data transfer table 360 ​​or the block table 370 according to the schedule. According to some embodiments of the present disclosure, the controller 320 may be configured to manage data communications using the global buffer 340. In some embodiments, the controller 320 may be configured to manage data communications to / from the host unit 220 or the host memory 221.

[0064] As described above, the controller 320 may schedule the calculation of the storage blocks 410 in the storage block assembly 310 and the data transfer in the storage block assembly 310, for example, based on the information stored in the instruction memory 350, the data transfer table 360, or the block table 370. According to an embodiment of the present disclosure, the controller 320 may schedule the calculation and data transfer of the storage blocks in the storage block assembly 310 so as to execute the neural network model in a pipeline manner. FIG. 6A to FIG. 6C Description for Figure 3An example of executing a neural network model in a pipelined manner on storage block 300.

[0065] Fig. 6A An example of a neural network model consistent with an embodiment of the present disclosure is shown. In machine learning or deep learning, a neural network model can be represented by a computational graph or data structure graph, which includes nodes and edges organized as a directed acyclic graph (DAG). Nodes represent variables or computational operations, while edges represent data or tensor flows from one node to another. Nodes representing computational operations can consume input data flowing into the node along input edges, and output data produced by the computational operation can flow out of the node along output edges. A typical machine learning or deep learning model may have tens of thousands or even millions of variables and computational operations.

[0066] As shown in the figure, Fig. 6A As shown, the computation graph includes a plurality of nodes n1 to n5 and an edge connecting two nodes among the plurality of nodes n1 to n5. Fig. 6A The computational graph shown can be part of a neural network model. It should be noted that Fig. 6A The computational graph is only illustrative and not restrictive, and therefore, the embodiments of the present disclosure can be applied to computational graphs with any number of nodes and edges. The second node n2 and the third node n3 receive the output data of the first node n1. The output data of the second node n2 is provided to the fourth node n4 as input data. The output data of the fourth node n4 is provided to the fifth node n5 as input data. The output data of the third node n3 is provided to the fifth node n5 as input data. In some embodiments, nodes n1 to n5 can represent corresponding computing operations. The edge can represent the correlation between two nodes connected by the corresponding edge. That is, the node at the edge end point can be processed only after the processing of the node at the edge starting point is completed. For example, the fifth node n5 can be processed only after the third node n3 and the fourth node n4 are processed and the output data of the third and fourth nodes n3 and n4 are provided to the fifth node n5.

[0067] According to the embodiment of the present disclosure, based on the data correlation between nodes n1 to n5, it can be considered that Fig. 6A The computation graph has four computation layers. For example, the first node n1 may constitute the first layer, and because the second and third nodes n2 and n3 receive the output data of the first node n1, the second and third nodes n2 and n3 may constitute the second layer. Similarly, the fourth node n4 may constitute the third layer, and the fifth node n5 may constitute the fourth layer. In some embodiments, the second node n2 may constitute the second layer, and the third node n3 and the fourth node n4 may constitute the third layer.

[0068] Figure 6B1 shows a method for executing in the storage block component 310 consistent with an embodiment of the present disclosure. Fig. 6A An example of a node map for a neural network model. Figure 6B Shows Figure 4 A portion of a storage block assembly 310 is shown, Figure 4 The illustrated storage component 310 includes eight storage blocks 410_11 to 410_24 arranged in two rows and four columns, and four transmission buffers 440_1 to 440_4. Figure 6B For simplicity, the remaining storage blocks and data links are omitted. Figure 4 As explained, data may be transferred in one direction within the memory block assembly 310, such as from left to right.

[0069] In this example, by considering data movement and node dependencies between storage blocks, Fig. 6A The computational graph of Figure 6B On the memory block in the memory block component 310. Figure 6B As shown, the first node n1 can be assigned to the first storage block 410_11 in the first column, which is the leftmost column in the storage block assembly 310, because the first node n1 does not receive data from other storage blocks assigned to nodes n2 to n5. The second node n2 and the third node n3 are assigned to the second storage block 410_12 and the sixth storage block 410_22 in the second column, because the second node n2 and the third node n3 receive output data from the first node n1, which is assigned to the first storage block 410_11 in the first column. Similarly, the fourth node n4 and the fifth node n5 are assigned to the third storage block 410_13 and the fourth storage block 410_14 in the third column and the fourth column, respectively.

[0070] Figure 6C The following is shown in accordance with an embodiment of the present disclosure, which is performed in the storage block component 310. Fig. 6A The state transitions of the memory blocks during the processing of the neural network model. During the runtime of processing the neural network model, the state of the block memory is periodically updated to the block table 370. According to some embodiments of the present disclosure, the controller 320 checks the block table 370 and adds information from the memory block 410 with the ready state R to the data transfer table 360. The information added to the data transfer table 360 ​​may include the source location and destination location of the data to be transferred and the buffer information that can be used to transfer the data. According to an embodiment of the present disclosure, the controller 320 can schedule the pipeline processing of the neural network model in the memory block component 310 by using the data transfer table 360 ​​and the block table 370, which can maximize memory utilization and improve the overall throughput of the system.

[0071] Figure 6Cshows the pipelined execution during runtime. Fig. 6A The state changes of the storage blocks and transmission buffers of the computation graph are taken as examples. Figure 6C As shown, during the first time period t1, the first storage block 410_11 performs its assigned calculation (assuming that all data for processing the assigned calculation are retrieved), for example, the calculation of the first node n1 and the state of the first storage block 410_11 in the first time period t1 can be represented as a calculation state C. Since the calculations of all other nodes n2 to n5 can be processed only after the output data of the first node n1 is provided to the corresponding storage blocks (e.g., 410_12 and 410_22), the states of the other storage blocks 410_12, 410_22, 410_13, and 410_14 are represented as an idle state I during the first time period t1.

[0072] In such Fig. 6A As shown in the second time period t2, after the first storage block 410_11 completes the calculation of the first node n1, the first storage block 410_11 completes the calculation of the first node n1, and the state of the first storage block 410_11 can be changed to the ready state R, which indicates that the data is ready for transmission. In the second time period t2, the other storage blocks 410_12, 410_22, 410_13 and 410_14 remain in the idle state I. In the second time period t2, the output data generated from the first storage block 410_11 can be transmitted to the second storage block 410_12 and the sixth storage block 410_22 assigned to the second node n2 and the third node n3, respectively. In some embodiments, the controller 320 can update the data transmission table 360 ​​for recording the pending data transmission of the second time period t2. Fig. 6A In FIG. 1 , an arrow is used to indicate data transmission from the starting point of the arrow to the end point of the arrow. Because the output data is transmitted to two memory blocks 410_12 and 410_22 in a column (e.g., the second column) adjacent to the first memory block 410_11, data links (e.g., row data links 451_1 and 451_2 and column data link 452_1) can be used and switches (e.g., Figure 4 The first switch 420_1, the second switch 420_2 and the fourth switch 420_4 are used to transmit the output data without using the transmission buffer 440 or the global buffer 340. In some embodiments, after the data transmission between the first storage block 410_11 and the second storage block 410_12 is completed, the data transmission can be performed between the first storage block 410_11 and the sixth storage block 410_22. In some other embodiments, the data transmission between the first storage block 410_11 and the second storage block 410_12 and the data transmission between the first storage block 410_11 and the sixth storage block 410_22 can be performed simultaneously.

[0073] After the data transmission from the first storage block 410_11 to the second storage block 410_12 and the sixth storage block 410_22 is completed, the second storage block 410_12 and the sixth storage block 410_22 can start to use the data received from the first storage block 410_11 to process the allocation calculation of the second node n2 and the third node n3. In some embodiments, once the corresponding transmission is completed, the controller 320 can also update the data transmission table 360 ​​to remove the entry for data transmission from the first storage block 410_11 to the second storage block 410_12 and the sixth storage block 410_22. In the third time period t3, the second storage block 410_12 and the sixth storage block 410_22 may have a calculation state C, and the remaining storage blocks have an idle state I. In some embodiments, the first storage block 410_11 may receive input data to process the allocated calculation in subsequent processing, and during the third time period t3, the first storage block 410_11 is in an idle state I.

[0074] like Fig. 6A As shown in the fourth time period t4 in FIG, after the second storage block 410_12 and the sixth storage block 410_22 complete their allocation calculations for the second node n2 and the third node n3, the states of the second storage block 410_12 and the sixth storage block 410_22 can be changed to the ready state R. In the fourth time period t4, the third storage block and the fourth storage blocks 410_13 and 410_14 maintain the idle state I. During the fourth time period t4, as indicated by the arrow, the output data generated from the second storage block 410_12 can be transmitted to the third storage block 410_13. According to an embodiment of the present disclosure, when the output data generated from the second storage block 410_12 is transmitted to the third storage block 410_13, the output data generated from the sixth storage block 410_22 will not be transmitted at the same time, because the two data transmissions use the same data link between the second column storage block and the third column storage block, and the output data generated by the sixth storage block 410_22 is different from the output data generated by the second storage block 410_12. In this example, after the data transmission between the second storage block 410_12 and the third storage block 410_13 is completed, the data transmission between the sixth storage block 410_22 and the fourth storage block 410_14 can be started (in the fifth time period t5). In some other embodiments, after the data transmission between the sixth storage block 410_22 and the fourth storage block 410_14 is completed, the data transmission between the second storage block 410_12 and the third storage block 410_13 can be performed. At the same time, the first storage block 410_11 can perform its allocated calculation during the fourth time period t4.

[0075] In the fifth time period t5, the third storage block 410_13 can use the data received from the second storage block 410_12 to process its allocation calculation for the fourth node n4. The data transmission between the sixth storage block 410_22 and the fourth storage block 410_14 can start in the fifth time period t5. Because the sixth storage block 410_22 and the fourth storage block 410_14 are not placed adjacent to each other, the data transmission between them can be performed by using the transmission buffer 440. In the fifth time period t5, the data link (e.g., Figure 4 The row data link 451 and the column data link 452 between the second column and the third column of the storage block and turn on the switch (for example, Figure 4 The 11th switch 420_11 and the 16th switch 420_16) transmit the output data generated from the sixth storage block 410_22 to the third transmission buffer 440_3. Figure 6C The last column shows that the third transmission buffer 440_3 is being used in the fifth time period t5. At the same time, the second storage block and the fourth storage blocks 410_12 and 410_14 may be in the idle state I, and the first storage block 410_11 may be in the ready state R. In the fifth time period t5, the output data generated in the first storage block 410_11 during the fourth time period t4 may be transmitted to the second storage block 410_12, which is shown in FIG. Figure 6C Not shown.

[0076] After the third storage block 410_13 completes its calculation, the third storage block 410_13 may enter the ready state R in the sixth time period t6. In the sixth time period t6, the second storage block 410_12, the sixth storage block 410_22, and the fourth storage block 410_14 may be in the idle state I. In the sixth time period t6, the data link (e.g., Figure 4 The output data from the third storage block 410_13 is transmitted to the fourth storage block 410_14 without using the transmission buffer 440, which is between the third column and the fourth column of the storage block. Figure 6C In the sixth time period t6, the output data generated by the first storage block 410_11 in the fourth time period t4 can also be transmitted to the sixth storage block 410_22, because the data transmission between the first storage block 410-11 and the sixth storage block 410_22 can use a data link different from the data transmission between the third storage block 410_13 and the fourth storage block 410_14 (for example, in Figure 4According to an embodiment of the present disclosure, the output data generated by the first storage block 410_11 in the fourth time period t4 cannot be transmitted to the sixth storage block 410_22 in the fifth time period t5 because the sixth storage block 410_22 is sending its output data to other destinations, and the storage blocks 410 are either performing calculations or performing data transmission at a time.

[0077] like Figure 6C As shown in the seventh time period t7, after the data transmission is completed, the state of the third storage block 410_13 becomes the idle state I. In the seventh time period t7, the fourth storage block 410_14 remains in the idle state I, and the fourth storage block 410_14 cannot start processing its assigned calculation because the output data generated by the sixth storage block 410_22 has not been received. In the seventh time period t7, the data stored in the third transmission buffer 440_3 that has been transmitted from the sixth storage block 410_22 in the fifth time period t5 is transmitted to the fourth storage block 410_14. At the same time, the second storage block 410_12 and the sixth storage block 410_22 can perform their assigned calculations because the second storage block 410_12 and the sixth storage block 410_22 receive data from the first storage block 410_11 in the fifth and sixth time periods t5 and t6, respectively. In some embodiments, the first storage block 410_11 can be in the idle state I and receive input data to be used in the next cycle.

[0078] In the eighth time period t8, the fourth storage block 410_14 can perform its allocation calculation to the fifth node n5, because the fourth storage block 410_14 has received the input from the sixth storage block 410_22 in the sixth time period t6 and has received the input of the third storage block 410_13 in the fifth time period t5 and the seventh time period t7. Similar to the fourth time period t4, in the eighth time period t8, the second storage block 410_12 and the sixth storage block 410_22 can be in the ready state R, and the third storage block 410_13 can be in the idle state I. In the eighth time period t8, the output data generated by the second storage block 410_12 can be transferred to the third storage block 410_13. The first storage block 410_11 can perform its allocated calculation in the eighth time period t8.

[0079] It should be understood that Figure 6CThe time periods t1 to t8 may have different time lengths, depending on the processing time of the calculations, the size of the data to be transmitted, the size of the data that can be transmitted in one cycle, etc. For example, if the data to be transmitted is larger than the size of the data that can be transmitted in one cycle, the second time period t2 may correspond to multiple transmission operation cycles. Similarly, the processing time for different calculations may be different, so the first time period t1 for processing the calculation of the first node n1 may be different from the third time period t3 for processing the calculation of the second node n2 and the calculation of the third node n3. Figure 1 FIG. 6A to FIG. 6C As shown, the neural network model can be efficiently executed in a pipeline manner in the storage block 300 based on the layout of the storage block component 310 and the control table (e.g., the data transfer table 360 ​​and the block table 370). According to an embodiment of the present disclosure, the storage utilization of the storage block 300 can be maximized, and the overall system efficiency can also be improved.

[0080] Figure 7 An embodiment consistent with the present disclosure is shown. Figure 2A An exemplary flowchart of training a neural network model on a neural network accelerator architecture of a neural network accelerator. A neural network accelerator architecture (e.g., Figure 2A The neural network accelerator architecture 200 in can be used to train neural network models according to some embodiments of the present disclosure.

[0081] In step S710, input data for executing the neural network model may be prepared. In some embodiments, the host unit (e.g., Figure 2A The host unit 220 of the embodiment can offload the input data to the storage block (e.g., Figure 3 In some embodiments, the host unit can access the storage block 300 via an interface (e.g., Figure 2A The host unit may prepare different weight values ​​for each of the one or more storage blocks.

[0082] In step S720, consistent with embodiments of the present disclosure, one or more storage blocks may execute a neural network model based on the received input data. In some embodiments, one or more storage blocks (e.g., Figure 3 The storage block 300) can execute the neural network model simultaneously based on a set of different weight values. In some embodiments, each storage block can execute the neural network model in a feedforward process.

[0083] At step S730, after the execution of the neural network model is completed, the execution results from one or more storage blocks may be collected. In some embodiments, the execution results may be collected through an interface (e.g., Figure 2AThe execution results are sent to the host unit through an interface 212 of the processor, and the host unit can determine the output error of each execution result. In some embodiments, the host unit can have information about the correct execution results for the corresponding neural network model. The host unit can determine the average error of the execution results from one or more storage blocks.

[0084] In step S740, the weight value of the neural network model may be updated based on the error determined in step S730. According to an embodiment of the present disclosure, the determined error may be an average error of execution results from one or more memory blocks. In some embodiments, the host unit may send the average error to the one or more memory blocks. The one or more memory blocks may execute the neural network model in a back-propagation manner to determine the weight value based on the average error.

[0085] In some embodiments, the neural network model is used to train Figure 7 Can be repeated until a certain condition is met. For example, Figure 7 The condition line of the training process may be that the average error determined in step S730 is less than a threshold. Figure 7 The training conditions can be Figure 7 The number of iterations of the step is greater than a threshold. According to an embodiment of the present disclosure, multiple storage blocks can be used to train a neural network model, and the training accuracy can be improved.

[0086] According to an embodiment of the present disclosure, a neural network accelerator architecture can be used for an inference process. In some embodiments, a host unit can send the same set of weights to multiple blocks, and multiple blocks can perform inference processes in parallel based on the same set of weights with different input values. By using multiple blocks for the same neural network model, according to an embodiment of the present disclosure, the throughput of inference can be improved.

[0087] Embodiments of the present disclosure may provide a data layout aware PIM architecture that implements flexible data layout transformations and supports efficient data movement patterns for executing neural network models. Embodiments of the present disclosure enable efficient data transfer between storage blocks via a storage block design having interconnected storage blocks arranged in a 2D mesh topology as well as data links and switches. Embodiments of the present disclosure also enable computation and data transfer in a pipelined manner via transfer buffers and control tables. According to embodiments of the present disclosure, data layout across storage blocks may be obtained, and thus a neural network model may be flexibly mapped to a PIM accelerator architecture, which improves memory utilization and parallel computing.

[0088] Embodiments may be further described using the following claims:

[0089] 1. A processing-in-memory (PIM)-enabled device for executing a neural network model, comprising:

[0090] Storage block components, including:

[0091] a first storage block array;

[0092] a second storage block array adjacent to the first storage block array;

[0093] a plurality of first data links associated with the first storage block array and the second storage block array, wherein each data link of the plurality of first data links communicatively connects two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and

[0094] a second data link communicatively connected to the plurality of first data links,

[0095] The data from the first storage block of the first storage block array can be transmitted to the second storage block of the second storage block array via the multiple first data links and the second data links.

[0096] 2. The PIM-enabled device of clause 1, wherein the storage block component further comprises:

[0097] a first switch configured to control turning on or off a data path from the first storage block to a first data link among a plurality of first data links communicatively connected to the first storage block; and

[0098] A second switch is configured to control turning on or off a data path from a first data link among a plurality of first data links communicatively connected to the second storage block to the second storage block.

[0099] 3. The PIM-enabled device of clause 2, wherein the storage block component further comprises:

[0100] a first selector configured to select the first switch to switch on a data path from the first memory block to the first data link communicatively connected to the first memory block; and

[0101] A second selector is configured to select the second switch to connect a data path from a first data link communicatively connected to the second storage block to the second storage block.

[0102] 4. A PIM enabled device as claimed in any one of clauses 1 to 3, wherein each memory block comprises an array of memory cells arranged in a plurality of rows and a plurality of columns.

[0103] 5. A PIM-enabled device according to any one of clauses 1 to 4, wherein the plurality of first data links and the second data link are configured to transfer data stored in a column of memory cells in the first memory block to a row of memory cells in the second memory block in one cycle.

[0104] 6. The PIM-enabled device of clause 1, wherein the storage block component further comprises:

[0105] a third storage block array adjacent to the second storage block array;

[0106] a plurality of third data links associated with the second storage block array and the third storage block array, wherein each data link of the plurality of third data links is communicatively connected to two corresponding storage blocks from the second storage block array and the third storage block array, respectively;

[0107] a fourth data link communicatively connected to the plurality of third data links; and

[0108] a transmission buffer communicatively coupled to the second data link and the fourth data link,

[0109] The data from the first storage block of the first storage block array can be transmitted to the third storage block of the third storage block array through a transmission buffer.

[0110] 7. The PIM-enabled device of clause 6, wherein the storage block component further comprises:

[0111] a first buffer data link configured to communicatively connect the second data link and the transmission buffer; and

[0112] A second buffer data link is configured to communicatively connect the fourth data link and the transmission buffer.

[0113] 8. The PIM-enabled device of clause 7, wherein the storage block component further comprises:

[0114] a first switch configured to control turning on or off a data path from the first storage block to a first data link among a plurality of first data links communicatively connected to the first storage block;

[0115] a second switch configured to control turning on or off a data path from the first buffer data link to the transmission buffer;

[0116] a third switch configured to control turning on or off a data path from the transmission buffer to the second buffer data link; and

[0117] A fourth switch is configured to control turning on or off a data path from a third data link among a plurality of third data links communicatively connected to the third storage block to the third storage block.

[0118] 9. The PIM-enabled device of clause 8, wherein the storage block component further comprises:

[0119] a first selector configured to select the first switch to switch on a data path from the first memory block to the first data link communicatively connected to the first memory block;

[0120] a second selector configured to select the second switch to switch on a data path from the first buffer data link to the transmission buffer;

[0121] a third selector configured to select the third switch to switch on a data path from the transmission buffer to a second buffer data link, and

[0122] A fourth selector is configured to select the fourth switch to connect a data path from a third data link communicatively connected to the third memory block to the third memory block.

[0123] 10. A PIM enabled device as recited in any of clauses 6-9, wherein each memory block comprises an array of memory cells arranged in a plurality of rows and a plurality of columns.

[0124] 11. A PIM-enabled device according to any one of clauses 6-10, wherein the plurality of first data links, the second data link, and the first buffer data link are configured to transfer data stored in columns of storage cells in the first storage block to rows of storage cells in the transmission buffer within one cycle.

[0125] 12. The PIM-enabled device of clause 11, wherein the second buffer data link, the fourth data link, and the plurality of third data links are configured to transfer data stored in the row of memory cells in the transmission buffer to the row of memory cells in the third memory block in another cycle.

[0126] 13. The PIM-enabled device according to any one of clauses 1 to 12, further comprising: a block table for recording a status of each storage block in the storage block assembly,

[0127] The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

[0128] 14. A PIM-enabled device according to any one of clauses 1 to 13, further comprising a data transfer table for recording pending data transfers in the storage block component,

[0129] The data transmission table includes a source field indicating a storage block for sending data and a destination field indicating a storage block for receiving data.

[0130] 15. The PIM enabled device of clause 14, wherein the data transfer table further comprises a buffer field indicating a buffer in which the data resides.

[0131] 16. A PIM-enabled device as described in any one of clauses 1 to 15, further comprising a controller configured to provide commands to the memory block component to perform a first operation of the neural network model on the first memory block and a second operation of the neural network model on the second memory block, the second operation using output data from performing the first operation as input.

[0132] 17. A method for executing a neural network model on a PIM-enabled device including a memory block component, the method comprising:

[0133] processing a first operation of the neural network model on a first memory block of a first memory block array in the memory block assembly;

[0134] transmitting output data from the first memory block in the memory block assembly to the second memory block of a second memory block array via a data link communicatively connecting the first memory block and the second memory block in the memory block assembly; and

[0135] A second operation of the neural network model is processed on the second storage block based on the output data.

[0136] 18. The method according to clause 17, further comprising:

[0137] transferring the output data from the first memory block to a third memory block of the second memory block array; and

[0138] processing a third operation of the neural network model on the third storage block based on the output data,

[0139] The transmitting of the output data from the first storage block to the third storage block is performed during or after the transmitting of the output data from the first storage block to the second storage block.

[0140] 19. The method of clause 17, wherein transmitting the output data comprises:

[0141] transmitting the output data from the first memory block to a transmission buffer in the memory block assembly; and

[0142] transmitting the output data from the transmission buffer to the second storage block,

[0143] The transmission buffer is communicatively connected with the first storage block and the second storage block.

[0144] 20. A method according to any of clauses 17-19, wherein transmitting the output data comprises:

[0145] The output data stored in the columns of the memory cells of the first memory block are transferred to the rows of the memory cells of the second memory block.

[0146] 21. A method according to any of clauses 17-20, wherein transmitting the output data is performed in one or more cycles depending on the size of the output data.

[0147] 22. The method according to any one of clauses 17-21, further comprising:

[0148] Record the status of each storage block in the storage block component in a block table,

[0149] The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

[0150] 23. The method of any of clauses 17-22, further comprising:

[0151] recording the data transfer to be processed in the storage block component in a data transfer table,

[0152] The data transmission table includes a source field indicating a storage block that sends data and a destination field indicating a storage block that receives the data.

[0153] 24. A non-transitory computer-readable storage medium storing a set of instructions executable by at least one processor in a terminal supporting processing in memory (PIM) to cause the terminal to perform the following method:

[0154] processing a first operation of the neural network model on a first memory block of a first memory block array in the memory block assembly;

[0155] transmitting output data from the first memory block in the memory block assembly to the second memory block of a second memory block array via a data link communicatively connecting the first memory block and the second memory block in the memory block assembly; and

[0156] A second operation of the neural network model is processed on the second storage block based on the output data.

[0157] 25. The computer-readable medium of clause 24, wherein the set of instructions is executable by at least one processor of the terminal to cause the terminal to further perform:

[0158] transferring the output data from the first memory block to a third memory block of the second memory block array; and

[0159] processing a third operation of the neural network model on the third storage block based on the output data,

[0160] The transmitting of the output data from the first storage block to the third storage block is performed during or after the transmitting of the output data from the first storage block to the second storage block.

[0161] 26. The computer-readable medium of clause 24, wherein the set of instructions is executable by at least one processor of the terminal to cause the terminal to further perform:

[0162] transmitting the output data from the first memory block to a transmission buffer in the memory block assembly; and

[0163] transferring the output data from the transmission buffer to the second storage block,

[0164] The transmission buffer is communicatively connected with the first storage block and the second storage block.

[0165] 27. The computer-readable medium of any of clauses 24-26, wherein transmitting the output data comprises:

[0166] The output data stored in the columns of the memory cells of the first memory block are transferred to the rows of the memory cells of the second memory block.

[0167] 28. The computer-readable medium according to any of clauses 24-27, wherein transmitting the output data is performed in one or more cycles depending on the size of the output data.

[0168] 29. A computer-readable medium according to any of clauses 24 to 28, wherein the set of instructions is executable by at least one processor of the terminal to cause the terminal to further perform:

[0169] Record the status of each storage block in the storage block combination in the block table,

[0170] The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

[0171] 30. A computer-readable medium according to any of clauses 24 to 29, wherein the set of instructions is executable by at least one processor of the terminal to cause the terminal to further perform:

[0172] recording the data transfer to be processed in the storage block component in a data transfer table,

[0173] The data transmission table includes a source field indicating a storage block for sending data and a destination field indicating a storage block for receiving data.

[0174] 31. A terminal, comprising:

[0175] host unit; and

[0176] A device supporting processing in memory (PIM), the device supporting processing in memory being communicatively connected to the host unit, the device supporting processing in memory (PIM) comprising:

[0177] Storage block components, including:

[0178] a first storage block array;

[0179] a second storage block array adjacent to the first storage block array;

[0180] a plurality of first data links associated with the first storage block array and the second storage block array, wherein each data link of the plurality of first data links communicatively connects two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and

[0181] a second data link communicatively connected to the plurality of first data links,

[0182] The data from the first storage block of the first storage block array can be transmitted to the second storage block of the second storage block array via the multiple first data links and second data links.

[0183] Embodiments herein include database systems, methods, and tangible non-transitory computer-readable media. The method may be performed, for example, by at least one processor receiving instructions from a tangible non-transitory computer-readable storage medium. Similarly, a system consistent with the present disclosure may include at least one processor and a memory, and the memory may be a tangible non-transitory computer-readable storage medium. As used herein, a tangible non-transitory computer-readable storage medium refers to any type of physical memory on which information or data readable by at least one processor can be stored. Examples include random access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drive, CD ROM, DVD, flash drive, disk, register, cache, and any other known physical storage medium. Singular terms such as "memory" and "computer-readable storage medium" may also refer to multiple structures, such as multiple memories or computer-readable storage media. As mentioned herein, unless otherwise specified, "memory" may include any type of computer-readable storage medium. A computer-readable storage medium may store instructions for execution by at least one processor, including instructions for causing a processor to execute steps or stages consistent with the embodiments herein. Additionally, a computer-implemented method may be implemented using one or more computer-readable storage media.The term "non-transitory computer-readable storage medium" should be understood to include tangible items and exclude carrier waves and transient signals.

[0184] As used herein, unless expressly stated otherwise, the term "or" encompasses all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless expressly stated otherwise or not feasible, the database may include A, or B, or A and B. As a second example, if it is stated that a database may include A, B, or C, then unless expressly stated otherwise or not feasible, the database may include A, or B, or C, or A and B, or A and C, or B and C, or A and B and C.

[0185] In the foregoing description, embodiments have been described with reference to many specific details that may vary from embodiment to embodiment. Certain modifications and changes may be made to the described embodiments. Other embodiments will be apparent to those skilled in the art in view of the description and practice of the invention disclosed herein. The description and embodiments are to be considered exemplary only, with the true scope and spirit of the invention being indicated by the appended claims. The order of steps shown in the accompanying drawings is intended to be for illustrative purposes only and is not intended to be limited to any particular order of steps. Thus, it will be appreciated by those skilled in the art that these steps may be performed in different orders while implementing the same method.

Claims

1. A device for executing a neural network model supporting in-memory processing of PIM, include: Storage block components, including: a first storage block array; a second storage block array adjacent to the first storage block array; a plurality of first data links associated with the first storage block array and the second storage block array, wherein each data link of the plurality of first data links communicatively connects two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and a second data link communicatively connected to the plurality of first data links, wherein data from a first storage block of the first storage block array can be transmitted to a second storage block of the second storage block array via the plurality of first data links and the second data link; a third storage block array adjacent to the second storage block array; a plurality of third data links associated with the second storage block array and the third storage block array, wherein each data link of the plurality of third data links is communicatively connected to two corresponding storage blocks from the second storage block array and the third storage block array, respectively; a fourth data link communicatively connected to the plurality of third data links; and a transmission buffer communicatively coupled to the second data link and the fourth data link, The data from the first storage block of the first storage block array can be transmitted to the third storage block of the third storage block array through the transmission buffer.

2. The PIM-enabled device according to claim 1, in, The storage block assembly also includes: a first switch configured to control turning on or off a data path from the first storage block to a first data link among a plurality of first data links communicatively connected to the first storage block; and A second switch is configured to control turning on or off a data path from a first data link among a plurality of first data links communicatively connected to the second storage block to the second storage block.

3. The PIM-enabled device of claim 2, in, The storage block assembly also includes: a first selector configured to select the first switch to switch on a data path from the first memory block to the first data link communicatively connected to the first memory block; and A second selector is configured to select a second switch to connect a data path from a first data link communicatively connected to the second storage block to the second storage block.

4. The PIM-enabled device of claim 1, in, Each memory block includes an array of memory cells arranged in a plurality of rows and columns.

5. The PIM-enabled device of claim 1, in, The plurality of first data links and the second data link are configured to transfer data stored in a column of memory cells in the first memory block to a row of memory cells in the second memory block in one cycle.

6. The PIM-enabled device of claim 1, in, The storage block assembly also includes: a first buffer data link configured to communicatively connect the second data link and the transmission buffer; and A second buffer data link is configured to communicatively connect the fourth data link and the transmission buffer.

7. The PIM-enabled device of claim 6, in, The storage block assembly also includes: a first switch configured to control turning on or off a data path from the first storage block to a first data link among a plurality of first data links communicatively connected to the first storage block; a second switch configured to control turning on or off a data path from the first buffer data link to the transmission buffer; a third switch configured to control turning on or off a data path from the transmission buffer to the second buffer data link; and A fourth switch is configured to control turning on or off a data path from a third data link among a plurality of third data links communicatively connected to the third storage block to the third storage block.

8. The PIM-enabled device of claim 7, in, The storage block assembly also includes: a first selector configured to select the first switch to switch on a data path from the first memory block to the first data link communicatively connected to the first memory block; a second selector configured to select the second switch to switch on a data path from the first buffer data link to the transmission buffer; a third selector configured to select the third switch to switch on a data path from the transmission buffer to a second buffer data link, and A fourth selector is configured to select the fourth switch to connect a data path from a third data link communicatively connected to the third memory block to the third memory block.

9. The PIM-enabled device of claim 1, in, Each memory block includes an array of memory cells arranged in a plurality of rows and columns.

10. The PIM-enabled device of claim 6, in, The plurality of first data links, the second data link, and the first buffer data link are configured to transfer data stored in columns of storage cells of the first storage block to rows of storage cells in the transmission buffer in one cycle.

11. The PIM-enabled device of claim 10, wherein the second buffer data link, the fourth data link, and the plurality of third data links are configured to transfer data stored in the row of storage cells in the transmission buffer to the row of storage cells in the third storage block in another cycle.

12. The PIM-enabled device of claim 1, further comprising: include: a block table, the block table being used to record the status of each storage block in the storage block component, The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

13. The PIM-enabled device of claim 1 , further comprising a data transfer table for recording pending data transfers in the storage block component, in, The data transfer table includes a source field indicating a memory block that transmits data and a destination field indicating a memory block that receives data.

14. The PIM-enabled device of claim 13, in, The data transfer table also includes a buffer field indicating the buffer in which the data resides.

15. The PIM-enabled device of claim 1, further comprising a controller configured to provide commands to the memory block component to perform a first operation of the neural network model on the first memory block and a second operation of the neural network model on a second memory block, the second operation using output data from performing the first operation as input.

16. A method for executing a neural network model on a PIM-enabled device including a memory block component, the PIM-enabled device comprising the device of any one of claims 1 to 15, the method include: processing a first operation of the neural network model on a first memory block of a first memory block array in the memory block assembly; transmitting output data from the first memory block in the memory block assembly to the second memory block of a second memory block array via a data link communicatively connecting the first and second memory blocks in the memory block assembly; and A second operation of the neural network model is processed on the second storage block based on the output data.

17. The method according to claim 16, in, Transmission output data includes: The output data stored in the columns of the memory cells of the first memory block are transferred to the rows of the memory cells of the second memory block.

18. The method according to claim 16, in, Transmitting the output data is performed in one or more cycles depending on the size of the output data.

19. The method according to claim 16, further comprising: include: Record the status of each storage block in the storage block component in a block table, The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

20. The method according to claim 16, further comprising: include: recording the data transfer to be processed in the storage block component in a data transfer table, The data transmission table includes a source field indicating a storage block that sends data and a destination field indicating a storage block that receives the data.

21. A non-transitory computer-readable storage medium storing a set of instructions executable by at least one processor in a terminal supporting processing of a PIM in a memory to cause the terminal to perform the following method: processing a first operation of the neural network model on a first memory block of a first memory block array in the memory block assembly; transmitting output data from the first memory block in the memory block assembly to the second memory block of a second memory block array via a data link communicatively connecting the first and second memory blocks in the memory block assembly; and processing a second operation of the neural network model on the second storage block based on the output data; The terminal comprises the device according to any one of claims 1 to 15.

22. The computer readable medium of claim 21, in, Transmission output data includes: The output data stored in the columns of the memory cells of the first memory block are transferred to the rows of the memory cells of the second memory block.

23. The computer readable medium of claim 21, wherein transmitting the output data is performed in one or more cycles depending on the size of the output data.

24. The computer readable medium of claim 21, in, The set of instructions may be executed by at least one processor of the terminal to cause the terminal to further perform: Record the status of each storage block in the storage block combination in the block table, The states include: a computing state, which indicates that the corresponding storage block is performing an assigned operation; a ready state, which indicates that the corresponding storage block has data ready for transmission; and an idle state, which indicates that the corresponding storage block is ready to receive data.

25. The computer readable medium of claim 21, in, The set of instructions may be executed by at least one processor of the terminal to cause the terminal to further perform: recording the data transfer to be processed in the storage block component in a data transfer table, The data transmission table includes a source field indicating a storage block for sending data and a destination field indicating a storage block for receiving data.

26. A terminal, include: Host unit; and A device supporting in-memory processing of PIM, the device supporting in-memory processing being communicatively connected to the host unit, the device supporting in-memory processing of PIM comprising: Storage block components, including: a first storage block array; a second storage block array adjacent to the first storage block array; a plurality of first data links associated with the first storage block array and the second storage block array, wherein each data link of the plurality of first data links communicatively connects two corresponding storage blocks from the first storage block array and the second storage block array, respectively; and a second data link communicatively connected to the plurality of first data links, wherein data from a first storage block of the first storage block array can be transmitted to a second storage block of the second storage block array via the plurality of first data links and second data links; a third storage block array adjacent to the second storage block array; a plurality of third data links associated with the second storage block array and the third storage block array, wherein each data link of the plurality of third data links is communicatively connected to two corresponding storage blocks from the second storage block array and the third storage block array, respectively; a fourth data link communicatively connected to the plurality of third data links; and a transmission buffer communicatively coupled to the second data link and the fourth data link, The data from the first storage block of the first storage block array can be transmitted to the third storage block of the third storage block array through the transmission buffer.