Storage access device, board card, storage access method and apparatus thereof

Abstract

The invention relates to an apparatus and method for reading and writing multidimensional data, wherein the computing apparatus of the invention is comprised in an integrated circuit apparatus, which comprises a general purpose interconnect interface and other processing apparatus. The computing apparatus interacts with the other processing apparatus to jointly perform a user-specified computing operation. The integrated circuit apparatus can further comprise a storage apparatus, which is connected to the computing apparatus and the other processing apparatus, respectively, for data storage of the computing apparatus and the other processing apparatus.

Description

Technical Field

[0001] This invention generally relates to the field of computers. More specifically, this invention relates to memory access devices, integrated circuit devices, circuit boards, methods, computer-readable storage media, computer program products, and computer devices for reading and writing multidimensional data. Background Technology

[0002] With the ever-increasing volume of information in data today, it is often necessary to present data in a multi-dimensional manner. However, data is stored in memory in a single-dimensional, rule-based manner. Taking image recognition or computer vision in the field of artificial intelligence as an example, image data is defined in a multi-dimensional way. If each image is three-dimensional data with H (height), W (width), C (channels), etc., since C is the lowest dimension, it is arranged consecutively in the order of C when stored on off-chip memory.

[0003] When the on-chip system wants to process images, it first moves the image data from off-chip memory to on-chip. While temporarily stored on-chip, multiple image data items are still arranged consecutively in C-dimensional order. However, when the on-chip system processes the images, it does so on a per-image basis. That is, it processes image HW1 first, then image HW2, then image HW3, and so on, in that order. The location where each image needs to be sent may be different. Since multiple image data items are consecutive in C-dimensional terms, but not for the entire image, discrete writing is required.

[0004] When moving image data from off-chip memory to on-chip storage, traditional techniques typically involve multiple reads per continuous dimension of the image data to store it on-chip. For example, for each image, the entire image might be moved to the chip in H×W reads, with C dimensions as the unit. However, since the C-dimensional data size of each image is often small, the bus bandwidth cannot be efficiently utilized for a single read operation. For instance, if an image is 64 bytes in size and the bus bandwidth is also 64 bytes, theoretically one image can be written in one cycle. However, as mentioned earlier, image data is not stored per image in an on-chip system. Therefore, each write operation may only require data in the C-dimensional dimension (e.g., 8 bytes). In this case, it would take 8 cycles to move a complete image, resulting in very low bus efficiency.

[0005] Therefore, an efficient solution for transporting multidimensional data is urgently needed. Summary of the Invention

[0006] In order to at least partially solve the technical problems mentioned in the background art, the present invention provides a memory access device for reading and writing multidimensional data, an integrated circuit device, a board, a method, a computer-readable storage medium, a computer program product, and a computer device.

[0007] In one aspect, the present invention discloses a memory access device for reading and writing multidimensional data, comprising: a source-end access unit and a destination-end access unit. The source-end access unit is used to continuously read multidimensional sub-data from the source-end memory when the multidimensional data is stored continuously along a first source dimension and adjacent dimensions. The destination-end access unit is used to discretely write the multidimensional sub-data into the destination-end memory when the multidimensional data is stored discretely along a second destination dimension.

[0008] In another aspect, the present invention discloses an integrated circuit device including the aforementioned memory access device, and also discloses a board including the aforementioned integrated circuit device.

[0009] In another aspect, the present invention discloses a method for reading and writing multidimensional data, comprising: when the multidimensional data is stored continuously in the source memory along the first dimension and adjacent dimensions, continuously reading multidimensional sub-data from the source memory; and when the multidimensional data is stored discretely in the destination memory along the second dimension, discretely writing the multidimensional sub-data into the destination memory.

[0010] In another aspect, the present invention discloses a computer-readable storage medium storing computer program code for reading and writing multidimensional data, wherein the computer program code, when run by a processing device, performs the aforementioned method; the present invention further discloses a computer program product comprising a computer program for reading and writing multidimensional data, characterized in that the computer program, when executed by a processor, implements the steps of the aforementioned method; the present invention further discloses a computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the aforementioned method.

[0011] Under specific conditions, this invention directly divides multidimensional data into multidimensional sub-data, and can move multidimensional sub-data at once. Compared with the one-dimensional moving method of the prior art, this invention can move more data in the same time period, improving read and write efficiency and bus bandwidth utilization. Attached Figure Description

[0012] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts. Wherein:

[0013] Figure 1 This is a structural diagram of a board card according to an embodiment of the present invention;

[0014] Figure 2 This is a structural diagram illustrating an integrated circuit device according to an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram illustrating the internal structure of a computing device according to an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram illustrating the internal structure of the processor core according to an embodiment of the present invention;

[0017] Figure 5 This is a schematic diagram illustrating a memory access device according to an embodiment of the present invention;

[0018] Figure 6 This is a flowchart illustrating the framework for reading and writing multidimensional data according to another embodiment of the present invention; and

[0019] Figure 7 This is a flowchart illustrating one embodiment of another embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be understood that the terms "first," "second," "third," and "fourth," etc., in the claims, specification, and drawings of this invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" used in the specification and claims of this invention indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0022] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0023] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."

[0024] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] This invention directly decodes multidimensional data into microinstructions for reading and writing multidimensional data, rather than microinstructions for reading and writing one-dimensional data. Therefore, a single microinstruction can perform the reading and writing of multidimensional data. Specifically, existing technologies break down a read / write instruction into multiple one-dimensional microinstructions, each of which carries one-dimensional sub-data from the multidimensional data. In contrast, this invention breaks down a read / write instruction into multiple multidimensional microinstructions, each of which carries multidimensional sub-data from the multidimensional data. For example, a two-dimensional microinstruction can carry two-dimensional sub-data, and a three-dimensional microinstruction can carry three-dimensional sub-data, thereby improving read / write efficiency and bus bandwidth utilization.

[0026] Figure 1 A schematic diagram of the structure of a board 10 according to an embodiment of the present invention is shown. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC) integrating one or more combined processing units. These combined processing units are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0027] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0028] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0029] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and off-chip memory 204.

[0030] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0031] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0032] The processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping the computing device 201. Depending on the implementation, the processing device 203 may be one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or other general-purpose and / or special-purpose processors. These processors include, but are not limited to, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, the computing device 201 of this invention can be considered as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0033] The off-chip memory 204 is used to store data to be processed. It is DDR memory, typically 16G or larger, and is used to store data of computing device 201 and / or processing device 203.

[0034] Figure 3 A schematic diagram of the internal structure of computing device 201 is shown. Computing device 201 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 201 in the figure adopts a multi-core hierarchical structure design. As a system-on-a-chip, computing device 201 includes multiple clusters, and each cluster includes multiple processor cores. In other words, computing device 201 is constructed in a hierarchical structure of system-on-a-chip, cluster, and processor core.

[0035] From the perspective of system-on-a-chip hierarchy, such as Figure 3 As shown, the computing device 201 includes an external storage controller 301, a peripheral communication module 302, an on-chip interconnect module 303, a synchronization module 304, and multiple clusters 305.

[0036] There can be multiple external storage controllers 301; two are shown as an example in the figure. These controllers are used to respond to access requests from the processor core to access external storage devices, such as… Figure 2The external memory 204 allows for reading or writing data from external memory. The peripheral communication module 302 receives control signals from the processing device 203 via the interface device 202, initiating the computing device 201 to execute tasks. The on-chip interconnect module 303 connects the external storage controller 301, the peripheral communication module 302, and multiple clusters 305, enabling data and control signal transmission between modules. The synchronization module 304 is a global barrier controller (GBC) used to coordinate the work progress of each cluster, ensuring information synchronization. The multiple clusters 305 are the computing core of the computing device 201; four are exemplarily shown in the figure. With hardware development, the computing device 201 of this invention can also include eight, sixteen, sixty-four, or even more clusters 305. The clusters 305 are used to efficiently execute deep learning algorithms.

[0037] From the perspective of cluster hierarchy, such as Figure 3 As shown, each cluster 305 includes multiple processor cores (IPU cores) 306 and one memory core (MEM core) 307.

[0038] Four processor cores 306 are shown exemplarily in the figure, but the present invention does not limit the number of processor cores 306. Its internal architecture is as follows: Figure 4 As shown. Each processor core 306 includes three main modules: a control module 41, an arithmetic module 42, and a storage module 43.

[0039] The control module 41 coordinates and controls the operation of the computation module 42 and the storage module 43 to complete the deep learning task. It includes an instruction fetch unit (IFU) 411 and an instruction decode unit (IDU) 412. The instruction fetch unit 411 fetches instructions from the processing device 203, and the instruction decode unit 412 decodes the fetched instructions and sends the decoding result as control information to the computation module 42 and the storage module 43.

[0040] The computation module 42 includes a vector operation unit 421 and a matrix operation unit 422. The vector operation unit 421 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 422 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0041] Storage module 43 is used to store or move related data, including neuron RAM (NRAM) 431, weight RAM (WRAM) 432, input / output direct memory access (IODMA) 433, and move direct memory access (MVDMA) 434. NRAM 431 is used to store feature maps for computation by processor core 306 and intermediate results after computation; WRAM 432 is used to store the weights of the deep learning network; IODMA 433 controls the access of NRAM 431 / WRAM 432 to external memory 204 through broadcast bus 309; MVDMA 434 controls the access of NRAM 431 / WRAM 432 to SRAM 308.

[0042] Back Figure 3 The storage core 307 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 306, and for communication between the execution cluster 305 and off-chip memory 204, communication between clusters 305, and communication between processor cores 306. In other embodiments, the storage core 307 has scalar operation capabilities and is used to perform scalar operations.

[0043] Storage core 307 includes a shared memory unit (SRAM) 308, a broadcast bus 309, a cluster direct memory access (CDMA) module 310, and a global direct memory access (GDMA) module 311. SRAM 308 acts as a high-performance data relay station. Data multiplexed between different processor cores 306 within the same cluster 305 does not need to be obtained from the off-chip memory 204 by each processor core 306 individually. Instead, it is relayed between processor cores 306 via SRAM 308. Storage core 307 only needs to quickly distribute the multiplexed data from SRAM 308 to multiple processor cores 306, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access.

[0044] Broadcast bus 309, CDMA 310, and GDMA 311 are used to perform communication between processor cores 306, communication between clusters 305, and data transfer between cluster 305 and off-chip memory 204, respectively. These will be explained below.

[0045] The broadcast bus 309 is used to complete high-speed communication between the processor cores 306 within the cluster 305. In this embodiment, the broadcast bus 309 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point (i.e., data transmission from one processor core to another) data transmission. Multicast is a communication method that transmits a piece of data from SRAM 308 to several specific processor cores 306. Broadcast is a communication method that transmits a piece of data from SRAM 308 to all processor cores 306, and is a special case of multicast.

[0046] CDMA 310 is used to control SRAM 308 access between different clusters 305 within the same computing device 201.

[0047] GDMA 311 works in conjunction with external memory controller 301 to control access to external memory 204 from SRAM 308 of cluster 305, or to read data from external memory 204 into SRAM 308. As described above, communication between external memory 204 and NRAM 431 or WRAM 432 can be achieved through two channels. The first channel is direct communication between external memory 204 and NRAM 431 or WRAM 432 via IODAM 433; the second channel involves first transferring data between external memory 204 and SRAM 308 via GDMA 311, and then transferring data between SRAM 308 and NRAM 431 or WRAM 432 via MVDMA 434. Although the second channel appears to require more components and has a longer data flow, in some embodiments, the bandwidth of the second channel is actually much greater than that of the first channel. Therefore, communication between off-chip memory 204 and NRAM 431 or WRAM 432 may be more efficient through the second channel. Embodiments of the present invention can select the data transmission channel based on their hardware capabilities.

[0048] In other embodiments, the functions of GDMA 311 and IODMA 433 can be combined in the same component. For ease of description, GDMA 311 and IODMA 433 are considered as different components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those of this invention, they fall within the scope of protection of this invention. Furthermore, the functions of GDMA 311, IODMA 433, CDMA 310, and MVDMA 434 can also be implemented by the same component.

[0049] The on-chip storage (or intra-chip storage) of the system-on-chip may include the aforementioned NRAM 431, WRAM 432, and SRAM 308, which can interact with the off-chip memory 204. In this embodiment, when writing neuron data from the off-chip source memory to the on-chip destination memory (e.g., NRAM 431), the data is mainly written from the off-chip memory 204 to the on-chip storage (e.g., NRAM 431) through a memory access device (i.e., IODMA 433).

[0050] Figures 1 to 4 This is merely a specific structure of one embodiment of the present invention. The present invention does not limit the source memory to be off-chip memory 204 or the destination memory to NRAM 431. Any form of off-chip memory is considered source memory, and any form of on-chip memory is considered destination memory. The structure of this embodiment is merely an example.

[0051] The memory access device of the present invention can fold adjacent dimensions of multidimensional data when they are consecutive, so as to obtain multidimensional sub-data of multiple consecutive dimensions of multidimensional data in a single data read operation. Specifically, the memory access device may include a source-end access unit, which is used to continuously read multidimensional sub-data from the source-end memory when the multidimensional data is stored continuously along the source-end first dimension and adjacent dimensions, thereby realizing multidimensional sub-data of multiple consecutive dimensions of multidimensional data. The source-end first dimension may refer to the lowest dimension when the multidimensional data is stored in the source-end processor, and the multidimensional data is stored continuously along the lowest dimension. Further, the memory access device may also include a destination-end access unit, used to discretely write the multidimensional sub-data into the destination-end memory when the multidimensional data is stored discretely along the destination-end second dimension, wherein the destination-end first dimension may be the lowest dimension when the data is stored in the destination-end memory, and the multidimensional data is also stored continuously along the lowest dimension in the destination-end memory, and the destination-end second dimension may be an adjacent dimension to the first dimension. In this embodiment of the disclosure, the source access unit and the destination access unit described above can be used to continuously read multi-dimensional sub-data of multiple dimensions from the source memory and discretely write the read data into the destination memory, thereby making full use of the bandwidth of the data bus and reducing the memory access overhead during the data read and write process.

[0052] In one embodiment, when the source-end access unit continuously reads multidimensional data from the source-end memory, it can read the multidimensional data according to a portion of the continuously stored dimensions, or it can read the multidimensional data according to all the continuously stored dimensions. When the source-end access unit reads multidimensional sub-data according to a portion of the continuously stored dimensions, the number of dimensions of the multidimensional sub-data is less than the number of dimensions of the multidimensional data. For example, for three-dimensional multidimensional data, if the step size of the second dimension of the source-end data is equal to the length of the first dimension of the source-end data, it indicates that the first and second dimensions of the source-end data are continuous. In this case, two-dimensional sub-data can be read according to the continuous first and second dimensions of the source-end data. Of course, for four-dimensional multidimensional data, if the step size of the second dimension of the source-end data is equal to the length of the first dimension of the source-end data, and the step size of the third dimension of the source-end data is equal to the length of the first dimension multiplied by the length of the second dimension of the source-end data, it indicates that the multidimensional data is continuously stored in the source-end memory along the first, second, and third dimensions of the source-end data. In this case, three-dimensional sub-data can be read from the multidimensional data. This is just an example and is not a specific limitation.

[0053] When the source access unit reads the multidimensional sub-data according to all dimensions of the continuously stored multidimensional data, the number of dimensions of the multidimensional sub-data is equal to the number of dimensions of the multidimensional data. Then, the destination access unit can discretely write the multidimensional sub-data into the destination memory according to the number of dimensions of the multidimensional sub-data. The multidimensional data can be two-dimensional, three-dimensional, or four-dimensional or higher; this disclosure mainly describes two-dimensional and three-dimensional data.

[0054] If the multidimensional data is two-dimensional, then when certain conditions are met, multidimensional sub-data (i.e., two-dimensional sub-data) can be read according to all dimensions of the multidimensional data (such as the first and second dimensions at the source). These conditions include, but are not limited to, the step size of the second dimension at the source equals the length of the first dimension at the source. When the multidimensional data meets these conditions, it indicates that all dimensions of the multidimensional data are continuous, and in this case, two-dimensional sub-data can be read according to all dimensions of the two-dimensional data. If the multidimensional data is three-dimensional, then when certain conditions are met, multidimensional sub-data can be read according to some dimensions of the multidimensional data (such as the first and second dimensions at the source), for example, reading two-dimensional sub-data according to two dimensions of three-dimensional data. If the multidimensional data is three-dimensional, then when certain conditions are met, multidimensional sub-data (i.e., three-dimensional sub-data) can also be read according to all dimensions of the multidimensional data (such as the first, second, and third dimensions at the source). These conditions include: the step size of the third dimension at the source end is equal to the length of the first dimension multiplied by the length of the second dimension at the source end, and the step size of the second dimension at the source end is equal to the length of the first dimension at the source end. When the multidimensional data meets the above conditions, it means that the multidimensional data is stored contiguously along the first, second, and third dimensions at the source end in the source end memory. In this case, multidimensional sub-data can be read according to all dimensions of the multidimensional data. Specific conditions are described below.

[0055] Furthermore, the memory access device can implement the reading and writing of the aforementioned multidimensional data through instructions. The memory access device may also include an instruction receiving unit and an input / output decoding unit. The memory access device can receive read / write instructions for multidimensional data through the instruction receiving unit, and generate at least one multidimensional micro-instruction from the read / write instructions through the input / output decoding unit. This enables the source-end access unit to continuously read multidimensional sub-data from the source-end memory according to the multidimensional micro-instruction, and the destination-end access unit to discretely write the multidimensional sub-data into the destination-end memory according to the multidimensional micro-instruction.

[0056] The following section primarily illustrates the implementation of this disclosure from the perspective of instructions; please refer to the description below for details. Those skilled in the art will understand that the specific conditions for continuous reading and discrete writing of multidimensional data described below are not limited to the application scenario of generating corresponding instructions. The conditions for continuous reading and discrete writing of multidimensional data described below can be applied to various possible implementation scenarios of multidimensional data reading and writing.

[0057] Figure 5The diagram shows a memory access device in an embodiment of the present disclosure. The memory access device (e.g., IODMA433) may include an instruction receiving unit 511, an input / output decoding unit 512, a routing unit 513, a source access unit 514, and a destination access unit 515.

[0058] When the system needs a memory access device (e.g., IODMA 433) to read neuron data (i.e., images) from off-chip memory 204 and write it to on-chip storage (e.g., NRAM 431), the system sends a read / write instruction to the processor core 306. The instruction decoding unit 412 of the control module 41 determines that the read / write instruction requires data to be read from off-chip memory 204 and sends the read / write instruction to the corresponding access device. After receiving the read / write instruction, the instruction receiving unit 511 caches the instruction in its issue queue. Due to its dependencies, the instruction needs to wait in the issue queue until all dependencies are resolved before it can enter the execution state. After all dependencies are resolved, the instruction receiving unit 511 issues the read / write instruction to the input / output decoding unit 512.

[0059] The input / output decoding unit 512 decomposes the read / write instruction into multiple micro-instructions in a certain order. A micro-instruction is a combination of micro-commands that can perform micro-operations. A micro-operation refers to the basic operation that a component can perform, and is the smallest independent operation, relative to the function performed by the micro-instruction. Each micro-instruction carries a source address (i.e., read address), a destination address (i.e., write address), and the data size. In this embodiment, the input / output decoding unit 512 can decompose the read / write instruction into multiple two-dimensional micro-instructions, three-dimensional micro-instructions, or other multi-dimensional micro-instructions; no specific limitation is made here. Regarding the reading and writing of multi-dimensional data, two-dimensional micro-instructions can read and write on two consecutive dimensions of the multi-dimensional data, and three-dimensional micro-instructions can read and write on three consecutive dimensions of the multi-dimensional data.

[0060] The routing unit 513 sends each microinstruction to the corresponding memory access unit based on the source address on the microinstruction. When the source address on the microinstruction points to the off-chip memory 204, the routing unit 513 will send the microinstruction to the source access unit 514.

[0061] Data access between the system and the off-chip memory 204 can be implemented based on a bus protocol (such as the AXI bus protocol). Specifically, data access between the source access unit 514 and the off-chip memory 204 is based on a bus protocol, and data is read from the off-chip memory 204 in bursts according to the source address and data size in the microinstruction. After obtaining the data, the source access unit 514 sends the destination address along with the data to the routing unit 513, and sends a read completion acknowledgment to the routing unit 513 to notify the routing unit 513 that the read micro-operation has been completed. The routing unit 513 can determine the destination memory based on the destination address, and then sends the destination address along with the read data to the destination access unit 515. At the same time, after receiving the read completion acknowledgment from the source access unit 514, it forwards the read completion acknowledgment to the input / output decoding unit 512.

[0062] The destination access unit 515 controls the writing of data to the destination memory. Based on the destination address, it writes the received data to a specific address in the destination memory. The destination memory then sends a write end acknowledgment to the destination access unit 515 to indicate that the write operation is complete. The destination access unit 515 sends the write end acknowledgment back to the routing unit 513, which then sends it back to the input / output decoding unit 512. When the input / output decoding unit 512 receives all the end acknowledgments (including read end acknowledgment and write end acknowledgment) of a micro-instruction corresponding to a read / write instruction, it indicates that all the micro-operations included in the read / write instruction have been executed. After collecting all the end acknowledgments of the micro-instructions, the input / output decoding unit 512 sends a storage resource release request and an execution end acknowledgment to the instruction receiving unit 511. The instruction receiving unit 511 responds to the storage resource release request and execution end acknowledgment and issues another instruction to the input / output decoding unit 512. Each unit repeats the aforementioned operation.

[0063] In this embodiment, the input / output decoding unit 512 attempts to directly decode N-dimensional data into two-dimensional, three-dimensional, or M-dimensional micro-instructions (M≤N), rather than one-dimensional micro-instructions. Therefore, a single micro-instruction performs the reading and writing of multi-dimensional data. Specifically, the input / output decoding unit 512 in this embodiment breaks down the multi-dimensional data transport corresponding to a read / write instruction into multiple two-dimensional micro-instructions, each of which transports a portion of the two-dimensional sub-data from the multi-dimensional data. The source access unit 514 and the destination access unit 515 perform read and write tasks based on the two-dimensional micro-instructions, enabling a single micro-instruction to control the two-dimensional data transport, thereby improving read / write efficiency and bus bandwidth utilization.

[0064] When the data in this embodiment is exemplarily images, and these images are exemplarily defined by two dimensions WC or HC, where the first dimension (lowest dimension) is channel C, and the second dimension (highest dimension) is height H or width W. Under certain conditions, as described above, two-dimensional microinstructions are generated to continuously read two-dimensional data from off-chip memory 204 at once, and the two-dimensional data is discretely written to the destination memory according to the operation requirements of the operation module 42. When the images are completely transferred to the destination memory, their arrangement in the destination memory is the order required by the operation module 42.

[0065] More specifically, when the multidimensional data is stored contiguously in the off-chip memory 204 along the first source dimension and its adjacent dimension (i.e., the second source dimension W), the source access unit 514 continuously reads the two-dimensional sub-data from the two-dimensional data based on the first and second source dimensions, that is, reads the two-dimensional data all at once. When the multidimensional data is stored discretely in the destination memory along the second destination dimension, that is, when the step size of the second destination dimension is greater than the length of the first source dimension, the destination access unit 515 discretly writes the two-dimensional sub-data into the destination memory. Specifically, when the step size of the second source dimension is equal to the length of the first source dimension, the two-dimensional data is stored contiguously in the off-chip memory 204 along the first and second source dimensions; when the step size of the second destination dimension is greater than the length of the first source dimension, the two-dimensional data is stored discretely in the destination memory NRAM 431 along the second destination dimension.

[0066] When the data in this embodiment is exemplarily images, and these images are exemplarily defined by three dimensions HWC, where the first dimension (lowest dimension) is, for example, channel C, followed by, for example, the second dimension width W, and the third dimension (highest dimension) is height H. Under certain conditions, as described above, two-dimensional microinstructions are generated to continuously read two-dimensional data from off-chip memory 204 at once, and the two-dimensional data is discretely written to the destination memory according to the computation requirements of the operation module 42. At this time, even if the multidimensional data is three-dimensional data, two-dimensional microinstructions are only generated according to a portion of the dimensions in the three-dimensional data, so that one two-dimensional sub-data is read each time according to the two-dimensional microinstructions. When the images are completely transferred to the destination memory, their arrangement in the destination memory is the order required by the operation module 42.

[0067] More specifically, the images are stored in the off-chip memory 204 in the order of the lowest dimension (i.e., channel C). In this embodiment, several consecutive addresses of two-dimensional data are read at once within the acceptable range of bus bandwidth, and then these two-dimensional data are written discretely to the appropriate address of the destination memory in units of images, so that the images are stored in the destination memory in units of single images, which makes it convenient for the computing module 42 to perform operations in units of single images.

[0068] For ease of explanation, this paper defines that images are stored in off-chip memory 204 based on the first, second, and third dimensions of the source end, and in the destination memory based on the first, second, and third dimensions of the destination end. Regardless of the source or destination end, the lowest dimension is the first dimension, followed by the second dimension, and the highest dimension is the third dimension. The first, second, and third dimensions correspond to channels, width, and height, respectively. This invention does not limit the number of dimensions of the image; it can be higher than three dimensions, such as four-dimensional data of N (batch), H (height), W (width), and C (channels). For example, dimension C can be the lowest dimension, and the adjacent second dimension can be either dimension H or dimension W. Since image data is stored sequentially off-chip in the order of the lowest dimension (first dimension), it is definitely stored continuously in the first dimension. The memory access device in this embodiment can generate two-dimensional microinstructions based on the lowest dimension and its adjacent continuous dimensions. In other embodiments of this invention, the memory access device can generate two-dimensional microinstructions based on any two consecutive dimensions.

[0069] The input / output decoding unit 512 breaks down a read / write instruction into multiple two-dimensional micro-instructions. Each two-dimensional micro-instruction performs a micro-operation to move a portion of the two-dimensional data to be processed. When the input / output decoding unit 512 receives a read / write instruction from the instruction receiving unit 511, it determines whether the conditions for generating the two-dimensional micro-instruction are met. Multiple conditions may exist; if any condition is not met, it means that a two-dimensional micro-instruction cannot be generated. In this case, the memory access device generates a one-dimensional micro-instruction according to existing technology to perform one-dimensional data read / write micro-operations. If all conditions are met, the input / output decoding unit 512 generates a two-dimensional micro-instruction based on the first and second dimensions of the source end. The memory access device executes micro-operations according to the two-dimensional micro-instructions, with each two-dimensional micro-instruction moving a two-dimensional sub-data of size W×C to improve efficiency.

[0070] These conditions include: the image is stored contiguously along the first and second dimensions of the source end in off-chip memory 204; and the image is stored discretely along the second dimension of the destination end in the destination end memory. These conditions may also include combinations of one or more of the following: the length of the first dimension of the source end is not greater than a threshold; the length of the second dimension of the source end is the same as the length of the second dimension of the destination end; the source end memory is off-chip memory; and the destination end memory is on-chip memory. These will be explained one by one below.

[0071] When considering that the image is stored contiguously along the first and second source dimensions in off-chip memory 204, since the image is already stored contiguously along the first source dimension in off-chip memory 204, no special judgment is needed. Specifically, the input / output decoding unit 512 determines whether the step size of the second source dimension is equal to the length of the first source dimension, that is, whether the step size of the image in the W dimension in off-chip memory 204 is equal to the length of the C dimension. If so, it means that the image is stored contiguously along the C and W dimensions in off-chip memory 204.

[0072] When considering that the image is discretely stored along the second dimension of the destination memory, specifically, the input / output decoding unit 512 determines whether the step size of the second dimension of the destination is greater than the length of the first dimension of the source, that is, whether the step size of the W dimension of the image in the destination memory is greater than the length of the C dimension of the image in the off-chip memory 204. If so, it means that the image is discretely stored along the W dimension of the destination memory.

[0073] When considering that the length of the first dimension at the source end is not greater than a threshold, specifically, the input / output decoding unit 512 determines whether the length of the first dimension at the source end is not greater than the threshold, that is, whether the length of the C dimension of the image in the off-chip memory 204 is not greater than the threshold. This threshold is related to the bus bandwidth, and is usually set to an integer multiple of the threshold. Taking a bus bandwidth of 512 bytes as an example, the threshold can be 64 bytes, and the bus bandwidth is 8 times the threshold. This condition can be further set to a minimum value for the length of the C dimension, for example, setting it to be between 1 byte and 64 bytes.

[0074] When considering that the length of the second dimension at the source end is the same as the length of the second dimension at the destination end, specifically, the input / output decoding unit 512 determines whether the length of the W dimension in the off-chip memory 204 is the same as the W dimension in the destination memory. If so, it means that the length of the second dimension at the source end is the same as the length of the second dimension at the destination end. Specifically, when the data is stored in three dimensions in both the source and destination memories, it is necessary to consider whether the second dimension at the source end and the second dimension at the destination end are the same to ensure that the shape of the data remains unchanged when stored in both memories. Of course, when the data is stored in two dimensions in either the source or destination memory, the second dimension at the source end or the second dimension at the destination end is equal to 0, and this determination process can be omitted.

[0075] When considering that the source memory is off-chip memory, the input / output decoding unit 512 determines whether the source memory pointed to by the read / write instruction is off-chip memory based on the source address. For example, when the source address points to off-chip memory 204, this condition is met since off-chip memory 204 is off-chip memory. This invention does not limit the type of off-chip memory; in other cases, off-chip memory can also be scratch pad memory (SPM).

[0076] When considering that the destination memory is on-chip memory, the input / output decoding unit 512 determines whether the destination memory is on-chip memory based on the destination address. For example, when the destination address points to NRAM 431, this condition is satisfied because NRAM 431 is on-chip memory storing neuron data. In other cases, when the destination address points to WRAM 432, this condition is still satisfied because WRAM 432 is also on-chip memory.

[0077] The aforementioned conditions are merely examples, and those skilled in the art can add, delete, or modify them according to actual circumstances. For instance, one or more of the above conditions can be deleted: the length of the first dimension at the source end is not greater than a threshold, the source end memory is off-chip memory, and the destination end memory is on-chip memory. In this case, the input / output decoding unit can generate corresponding two-dimensional microinstructions based on the remaining conditions.

[0078] When the above-mentioned conditions are met, it means that a two-dimensional microinstruction can be generated. The input / output decoding unit 512 sends the two-dimensional microinstruction to the routing unit 513. The two-dimensional microinstruction carries information such as the source address, destination address, and data size.

[0079] The routing unit 513 sends the two-dimensional micro-instruction to the corresponding access unit according to the source address in the two-dimensional micro-instruction, which is the source access unit 514 in this embodiment.

[0080] The source access unit 514 reads W×C two-dimensional sub-data of the image from the off-chip memory 204 based on two-dimensional micro-instructions. Therefore, a read / write instruction that moves an H×W×C image will be split into H two-dimensional micro-instructions that move W×C data, and the image will be moved to the destination memory in H steps.

[0081] In order to achieve the purpose of continuous read and discrete write, the source access unit 514 decodes the two-dimensional micro-instruction that identifies continuous read and discrete write. The read request sent to the off-chip memory 204 is still further divided into two-dimensional micro-operations according to the 64-byte burst of the bus bandwidth. However, for the data and read control information returned by the off-chip memory 204, the source access unit 514 needs to further parse the two-dimensional information on the destination side in this micro-operation to perform discrete write on the read data.

[0082] The source-side access unit 514 includes a segmentation module and a merging module. The segmentation module, based on the bandwidth and the length of the first dimension of the source end, segments the sub-data corresponding to the two-dimensional microinstruction into multiple data segments. For example, if the bandwidth is 64 bytes and the length of the C dimension of the off-chip memory 204 is 8 bytes, the segmentation module will divide the corresponding two-dimensional sub-data into 8 data segments. The merging module, based on the step size of the second dimension of the destination end (i.e., the W dimension of the data in the destination memory), arranges these data segments so that the destination-side access unit can discretely write these data segments into the destination memory.

[0083] The decoding method of the source access unit 514 is divided into aligned mode and unaligned mode, which adopt different segmentation and merging strategies. In aligned mode, the C dimension of the multidimensional data stored in the off-chip memory 204 and the destination memory must meet the following two conditions.

[0084] The first condition is that the length of the C dimension of the multidimensional data stored in the off-chip memory 204 is a power of 2. The source access unit 514 will determine whether the length of the C dimension of the data stored in the off-chip memory 204 involved in the two-dimensional microinstruction that identifies continuous read and discrete write is a power of 2, such as the length of the C dimension of the data stored in the off-chip memory 204 being 1 byte, 2 bytes, 4 bytes, 8 bytes, 16 bytes, etc.

[0085] The second condition is whether the step size of the second dimension of the data in the destination memory (e.g., the step size of the W dimension) is an integer multiple of the length of the C dimension of the data in the external memory 204. Specifically, when the length of the C dimension of the data in the external memory 204 is 1 byte, the source access unit 514 will determine whether the step size of the W dimension of the data in the destination memory is any integer byte from 2 bytes to 63 bytes; when the length of the C dimension of the data in the external memory 204 is 2 bytes, the source access unit 514 will determine whether the step size of the W dimension of the data in the destination memory is 2N bytes, where N is an integer from 2 to 31, that is, the step size of the W dimension of the data in the destination memory can be any 4 bytes, 6 bytes...62 bytes; when the length of the C dimension of the data in the external memory 204 is 4 bytes, the source access unit 514 will determine whether the step size of the W dimension of the data in the destination memory is 4N bytes, where N is an integer from 2 to 1... The integer 5 indicates that the step size of the W dimension of the data in the destination memory can be any of 8 bytes, 12 bytes...60 bytes. When the length of the C dimension of the data in the off-chip memory 204 is 8 bytes, the source access unit 514 will determine whether the step size of the W dimension of the data in the destination memory is 8N bytes, where N is an integer from 2 to 7, that is, the step size of the W dimension of the data in the destination memory can be any of 16 bytes, 24 bytes...56 bytes. When the length of the C dimension of the data in the off-chip memory 204 is 16 bytes, the source access unit 514 will determine whether the step size of the W dimension of the data in the destination memory is 16N bytes, where N is 2 or 3, that is, the step size of the W dimension of the data in the destination memory can be 32 bytes or 48 bytes.

[0086] If both of the aforementioned conditions are met, then it is in alignment mode. The source access unit 514 aligns and splits the second dimension step (e.g., the W-dimensional step) of the data in the destination memory and the address of the destination memory according to the C-dimensional length of the off-chip memory 204, in order to fully utilize the bandwidth. In alignment mode, peak bandwidth efficiency can be improved by the following factors:

[0087]

[0088] In alignment mode, the source access unit 514 may include a front-end register and a back-end register. In alignment mode, the front-end register caches multiple data segments corresponding to two-dimensional microinstructions from off-chip memory 204. These data segments are arranged in the front-end register according to the data arrangement of off-chip memory 204, i.e., stored contiguously in dimension C. The source access unit 514 reorganizes these data segments according to the step size of the second dimension (e.g., dimension W) of the data in the destination memory, arranging them in the same way as the data arrangement in the destination memory. That is, these data segments are discretely stored in the back-end register with intervals of the second dimension (e.g., dimension W) of the destination memory. The source access unit 514 sends the multidimensional sub-data stored in the back-end register to the routing unit 513, so that the multidimensional sub-data can be written to the destination memory via the destination access unit 515. Further, the source access unit 514 also includes a selector located between the front-end register and the back-end register, used to select the corresponding data segment from the front-end register according to a control signal and store the selected data segment in the back-end register. Specifically, the data segment stored in the front-stage register has multiple bytes, each byte corresponding to an index value. The selector can select at least one byte of valid data from the front-stage register based on the control signal and the index value in the front-stage register, and write the at least one byte of valid data into the back-stage register.

[0089] In alignment mode, the control signal can be a one-dimensional control signal generated by the source access unit 514. This control signal may include preset bits, which are proportional to the bus bandwidth. The preset bits may include multiple sets of sub-bits, each set controlling the allowable byte of bus bandwidth. Normally, valid data at both the source and destination ends are arranged from the least significant bit. Therefore, each set of sub-bits in the control signal can indicate the (i+1)th byte of the destination data, counting from the least significant bit. Each set of sub-bits includes a flag bit indicating whether the corresponding byte is valid. The other bits in this set of sub-bits can indicate which valid byte among multiple bytes of data from the preceding register, for example, 0 represents the first byte and 1 represents the second byte.

[0090] Furthermore, the source access unit 514 accumulates an index (i.e., the valid byte in the front-end register has an index value) for each valid byte of data control information in the one-dimensional control signal to record the starting position of the next part of data in the front-end register. Then, the source access unit 514 shifts the intermediate information to the left by several bits. These bits become the starting index of the first valid data in the back-end register, thereby generating control information for controlling the placement of data. The control information can indicate the (i+1)th byte starting from the least significant bit of the destination data. The control information contains a flag bit to identify whether this byte is valid. The other bits in the control information can be used to indicate which valid byte the data comes from among the multiple bytes of data in the front-end register.

[0091] If the aforementioned two conditions are not simultaneously met, it belongs to the unaligned mode. In the unaligned mode, the segmentation module of the source access unit 514 splits the multidimensional microinstruction, whose length is the length of the C dimension of the data in the off-chip memory 204 multiplied by the number of original microinstructions, into one-dimensional microinstructions whose size is the number of original microinstructions and whose size is the length of the C dimension of the data in the off-chip memory 204. The source access unit 514 reads the multidimensional sub-data based on the one-dimensional microinstructions. Here, the original microinstructions refer to the one-dimensional microinstructions generated by the prior art based on the C dimension of the data in the off-chip memory 204 to control the reading and writing of multidimensional data. The information of the one-dimensional microinstructions in the prior art is carried in the two-dimensional microinstructions of this embodiment. For example, if the length of the C dimension of the data in the off-chip memory 204 is 8 bytes and the number of original microinstructions is 4 (meaning that the micro-operation of the two-dimensional microinstruction is equivalent to the micro-operation of 4 original microinstructions), then the segmentation module splits a two-dimensional microinstruction with a length of 8×4=32 bytes into 4 one-dimensional microinstructions, each of which is 8 bytes.

[0092] The source access unit 514 also includes a write decoder, which decodes the aforementioned one-dimensional microinstructions. These one-dimensional microinstructions may include at least one operation field: source address offset, destination address, one-dimensional data size, etc. The write decoder decodes the one-dimensional microinstructions to generate control information for data shifting, thereby controlling the switching and placement of data. Specifically, the write decoder uses the source address in the one-dimensional microinstruction as the source address offset, the destination base address in the one-dimensional microinstruction as the destination address, the C-dimensional length of the data in the off-chip memory 204 as the one-dimensional data size, and the number of one-dimensional microinstructions as the count value of the remaining data entries. Each time the source access unit 514 sends out a sub-data item, it decrements the number of one-dimensional microinstructions by 1, increments the destination base address by a C-dimensional step size of the destination memory, and increments the source address by a C-dimensional length of the data in the off-chip memory 204. After each update, the difference between the source and destination addresses is used to control the shift of the data's destination address.

[0093] In unaligned mode, each read data received by the source access unit 514 from external memory 204 requires multiple cycles to be discretely written out. Therefore, each read data is buffered. When the length of the C dimension of the data in external memory 204 plus a specific bit in the source address is greater than or equal to 64 bytes, it indicates that the current read data has been completely discretely written out. In unaligned mode, the peak bandwidth is as follows:

[0094]

[0095] Regardless of whether it is in aligned or unaligned mode, the source access unit 514 transmits the two-dimensional sub-data and the destination address in the two-dimensional micro-instruction to the routing unit 513. The routing unit 513 sends the two-dimensional sub-data to the destination access unit 515 according to the destination address. The destination access unit 515 sends the two-dimensional sub-data to the destination memory according to the destination address.

[0096] In this embodiment, when both the first and second dimensions of the source end are continuous, the two-dimensional microinstructions can be generated in units of reading and writing W×C data. That is, the first two-dimensional microinstruction controls the movement of 1×W×C of two-dimensional data, the second controls 2×W×C, the third controls 3×W×C, and so on. Since the third dimension of the source end is H, the read / write instruction in this embodiment will be split into H two-dimensional microinstructions.

[0097] In summary, under specific conditions, this embodiment directly decodes the transfer of multidimensional data into two-dimensional microinstructions. Compared with the one-dimensional microinstructions of the prior art, this embodiment can transfer more data in the same time period, improving read and write efficiency and bus bandwidth utilization.

[0098] Another embodiment of the present invention also has the following characteristics: Figures 1 to 5 The structure allows a read / write instruction to be broken down into multiple multi-dimensional micro-instructions, each of which performs a micro-operation to move a portion of the multi-dimensional data to be processed. Specifically, when the input / output decoding unit 512 receives a read / write instruction from the instruction receiving unit 511, it determines whether the conditions for generating multi-dimensional micro-instructions are simultaneously met. There may be multiple conditions; if any condition is not met, it means that multi-dimensional micro-instructions cannot be generated, and the memory access device performs one-dimensional data read / write micro-operations according to the existing one-dimensional micro-instructions.

[0099] If all conditions are met, the input / output decoding unit 512 generates multidimensional microinstructions based on several consecutive dimensions of the source end. For example, if N consecutive dimensions of the source end are continuous, the input / output decoding unit 512 will generate N-dimensional microinstructions to move N-dimensional data at once. Taking three dimensions as an example, the input / output decoding unit 512 will generate multidimensional microinstructions based on at least two adjacent dimensions of the first, second, and third dimensions of the source end. Therefore, the multidimensional microinstructions in this embodiment may be two-dimensional or three-dimensional microinstructions. In other words, the source end access unit 514 can continuously read multidimensional sub-data based on the first dimension and all adjacent dimensions of the source end, and the dimension of this multidimensional sub-data is the same as the dimension of the multidimensional data. The memory access device performs micro-operations based on two-dimensional or three-dimensional microinstructions to improve efficiency.

[0100] Among these conditions, the input / output decoding unit 512 first determines whether the length of the second dimension at the source end and the length of the second dimension at the destination end are greater than or equal to zero. A greater than zero length indicates the existence of the second dimension at the source end, while a length equal to zero indicates its non-existence. Based on whether the length of the second dimension at the source end is greater than or equal to zero, it can be determined whether the multidimensional data is two-dimensional. Based on whether the length of the second dimension at the destination end is greater than or equal to zero, it can be determined whether the multidimensional data is discretized in the destination memory according to three dimensions. Based on the dimension information of the multidimensional data and the storage dimension information in the destination memory, the following four cases can be distinguished.

[0101] The first case is that the lengths of the second dimension at both the source and destination ends are greater than zero, meaning the multidimensional data is three-dimensional data. The input / output decoding unit 512 further determines whether the following conditions are met.

[0102] These conditions include that the image is stored contiguously along the first and second source dimensions in off-chip memory 204. As mentioned earlier, the image is indeed stored contiguously along the first source dimension in off-chip memory 204, and no special judgment is needed. Specifically, the input / output decoding unit 512 determines whether the step size of the second source dimension is equal to the length of the first source dimension, that is, whether the step size of the image in the W dimension of off-chip memory 204 is equal to the length of the C dimension. If so, it means that the image is stored contiguously along the W dimension in off-chip memory 204.

[0103] These conditions also include that the image is stored contiguously along the third dimension of the source end in off-chip memory 204. Specifically, the input / output decoding unit 512 determines that the step size of the third dimension of the source end is equal to the length of the first dimension of the source end multiplied by the length of the second dimension of the source end, which means determining whether the step size of the H dimension of the image in off-chip memory 204 is equal to the length of the C dimension. If so, combined with the previous condition, it indicates that the image is stored contiguously along the C, W, and H dimensions in off-chip memory 204.

[0104] These conditions also include that the image is discretely stored in the destination memory along the second dimension. Specifically, the input / output decoding unit 512 determines whether the step size of the second dimension of the destination is greater than the length of the first dimension of the source, that is, whether the step size of the W dimension of the image in the destination memory is greater than the length of the C dimension of the image in the off-chip memory 204. If so, it means that the image is discretely stored in the destination memory along the W dimension.

[0105] These conditions also include that the image is stored in the destination memory in two dimensions. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the destination is equal to the step size of the second dimension of the destination multiplied by the length of the second dimension of the destination, that is, whether the step size of the H dimension of the image in the destination memory is equal to the step size of the W dimension of the data in the destination memory multiplied by the length of the W dimension of the data in the destination memory. If so, it means that the image is stored in the destination memory in two dimensions, C and W.

[0106] These conditions also include the length of the first dimension at the source not exceeding a threshold. Specifically, the input / output decoding unit 512 determines whether the length of the first dimension at the source is not greater than the threshold, that is, whether the length of the C dimension of the image in the off-chip memory 204 is not greater than the threshold. This threshold is related to the bus bandwidth, and is generally set to an integer multiple of the threshold. Taking a bus bandwidth of 512 bytes as an example, the threshold can be 64 bytes, and the bus bandwidth is 8 times the threshold. This condition can be further set to a minimum value for the length of the C dimension, for example, setting it to be between 1 byte and 64 bytes.

[0107] These conditions also include the source memory being off-chip memory, that is, the input / output decoding unit 512 determines whether the source memory pointed to by the read / write instruction is off-chip memory based on the source address.

[0108] These conditions also include that the destination memory is on-chip memory.

[0109] When all the above conditions are met, it means that the first dimension, the second dimension, and the third dimension of the source are all continuous. The input-output decoding unit 512 can generate three-dimensional micro-instructions based on the first dimension, the second dimension, and the third dimension of the source at the same time. The reading and writing operations of the entire image can be controlled by a single three-dimensional micro-instruction, that is, a single micro-instruction can move the entire image.

[0110] The second case is when the length of the second dimension at the source end is zero and the length of the second dimension at the destination end is greater than zero. This indicates that the image is stored in off-chip memory 204 based only on the first and third dimensions at the source end, meaning the image is two-dimensional data. The input / output decoding unit 512 further determines whether the following conditions are met.

[0111] These conditions also include that the image is stored contiguously along the first and third dimensions of the source end in off-chip memory 204. Similarly, whether it is contiguous along the first dimension of the source end does not require special judgment. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the source end is equal to the length of the first dimension of the source end, that is, whether the step size of the H dimension of the image in off-chip memory 204 is equal to the length of the C dimension. If so, it means that the image is stored contiguously along the H dimension of the off-chip memory 204, that is, the image is stored contiguously along two adjacent dimensions in off-chip memory 204. At this time, since the multidimensional data only has two dimensions in the source end memory, the third dimension of the source end can also be regarded as the second dimension of the source end.

[0112] These conditions also include that the image is discretely stored in the destination memory along the second dimension. Specifically, the input / output decoding unit 512 determines whether the step size of the second dimension of the destination is greater than the length of the first dimension of the source, that is, whether the step size of the W dimension of the image in the destination memory is greater than the length of the C dimension of the image in the off-chip memory 204. If so, it means that the image is discretely stored in the destination memory along the W dimension.

[0113] These conditions also include storing the image in the destination memory along two dimensions. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the destination is equal to the step size of the second dimension of the destination multiplied by the length of the second dimension of the destination, that is, whether the step size of the H dimension of the image in the destination memory is equal to the step size of the W dimension of the data in the destination memory multiplied by the length of the W dimension of the data in the destination memory. If so, it means that the image is stored in the destination memory along two dimensions, C and W.

[0114] These conditions include that the length of the first dimension at the source end is not greater than a threshold. Specifically, the input / output decoding unit 512 determines whether the length of the first dimension at the source end is not greater than the threshold, that is, whether the length of the C dimension of the image in the off-chip memory 204 is not greater than the threshold. This condition can be further set to a minimum value for the length of the C dimension, for example, setting it to be between 1 byte and 64 bytes.

[0115] These conditions also include the source memory being off-chip memory, that is, the input / output decoding unit 512 determines whether the source memory pointed to by the read / write instruction is off-chip memory based on the source address.

[0116] These conditions also include that the destination memory is on-chip memory.

[0117] When all the above conditions are met, it means that the first dimension and the third dimension (adjacent dimensions) of the source end are continuous. The input-output decoding unit 512 can generate two-dimensional micro-instructions based on the first dimension and the third dimension of the source end at the same time. The read and write operations of the entire two-dimensional image with only the first and third dimensions can be controlled by a single two-dimensional micro-instruction. That is, a single micro-instruction can also move the entire image.

[0118] The third case is when the length of the second dimension at the source end is greater than zero and the length of the second dimension at the destination end is equal to zero, indicating that the image is three-dimensional data and is stored in the destination memory according to two dimensions: the first dimension and the third dimension at the destination end. The input / output decoding unit 512 further determines whether the following conditions are met.

[0119] These conditions also include that the image is stored contiguously along the first and second source dimensions in off-chip memory 204. The first source dimension does not require special consideration. Specifically, the input / output decoding unit 512 determines whether the step size of the second source dimension is equal to the length of the first source dimension, which is equivalent to determining whether the step size of the image along the W dimension in off-chip memory 204 is equal to the length of the C dimension. If so, it indicates that the image is stored contiguously along the W dimension in off-chip memory 204.

[0120] These conditions also include that the image is stored contiguously along the third dimension of the source end in off-chip memory 204. Specifically, the input / output decoding unit 512 determines that the step size of the third dimension of the source end is equal to the length of the first dimension of the source end multiplied by the length of the second dimension of the source end, which means determining whether the step size of the H dimension of the image in off-chip memory 204 is equal to the length of the C dimension. If so, combined with the previous condition, it indicates that the image is stored contiguously along the C, W, and H dimensions in off-chip memory 204.

[0121] These conditions also include that the image is discretely stored along the third dimension of the destination memory. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the destination is greater than the length of the first dimension of the source, that is, whether the step size of the H dimension of the image in the destination memory is greater than the length of the C dimension of the image in the off-chip memory 204. If so, it means that the image is discretely stored along the H dimension of the destination memory. At this time, since the multidimensional data only has two dimensions in the destination memory, the third dimension of the destination can also be regarded as the second dimension of the destination.

[0122] These conditions include that the length of the first dimension at the source end is not greater than a threshold. Specifically, the input / output decoding unit 512 determines whether the length of the first dimension at the source end is not greater than the threshold, that is, whether the length of the C dimension of the image in the off-chip memory 204 is not greater than the threshold. This condition can be further set to a minimum value for the length of the C dimension, for example, setting it to be between 1 byte and 64 bytes.

[0123] These conditions also include the source memory being off-chip memory, that is, the input / output decoding unit 512 determines whether the source memory pointed to by the read / write instruction is off-chip memory based on the source address.

[0124] These conditions also include that the destination memory is on-chip memory.

[0125] When all the above conditions are met, it means that the first dimension, the second dimension, and the third dimension of the source end are all continuous, but the second dimension of the destination end is zero. Therefore, the input-output decoding unit 512 generates multiple two-dimensional micro-instructions based on the first dimension and the third dimension of the source end at the same time, and moves the images in batches.

[0126] The fourth case is when the lengths of the second dimension at both the source and destination ends are zero, indicating that neither the second dimension exists. In reality, the image is two-dimensional data at both the source and destination ends. The input / output decoding unit 512 further determines whether the following conditions are met.

[0127] These conditions also include that the image is stored contiguously along the first and third dimensions of the source end in the off-chip memory 204. The first dimension of the source end does not require special judgment. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the source end is equal to the length of the first dimension of the source end, that is, whether the step size of the H dimension of the image in the off-chip memory 204 is equal to the length of the C dimension. If so, it indicates that the image is stored contiguously along the H dimension in the off-chip memory 204. In this case, since the multidimensional data only has two dimensions in the source-end memory, the third dimension of the source end can also be considered as the second dimension of the source end.

[0128] These conditions also include that the image is discretely stored along the third dimension of the destination memory. Specifically, the input / output decoding unit 512 determines whether the step size of the third dimension of the destination is greater than the length of the first dimension of the source, that is, whether the step size of the H dimension of the image in the destination memory is greater than the length of the C dimension of the image in the off-chip memory 204. If so, it means that the image is discretely stored along the H dimension of the destination memory. At this time, since the multidimensional data only has two dimensions in the destination memory, the third dimension of the destination can also be regarded as the second dimension of the destination.

[0129] These conditions include that the length of the first dimension at the source end is not greater than a threshold. Specifically, the input / output decoding unit 512 determines whether the length of the first dimension at the source end is not greater than the threshold, that is, whether the length of the C dimension of the image in the off-chip memory 204 is not greater than the threshold. This condition can be further set to a minimum value for the length of the C dimension, for example, setting it to be between 1 byte and 64 bytes.

[0130] These conditions also include the source memory being off-chip memory, that is, the input / output decoding unit 512 determines whether the source memory pointed to by the read / write instruction is off-chip memory based on the source address.

[0131] These conditions also include that the destination memory is on-chip memory.

[0132] When all the above conditions are met, it means that the first dimension and the third dimension of the source are continuous. The input-output decoding unit 512 generates a two-dimensional micro-instruction based on the first dimension and the third dimension of the source. The read and write operations of the entire two-dimensional image with only the first and third dimensions can be controlled by a single two-dimensional micro-instruction. That is, a single micro-instruction can also move the entire image.

[0133] The aforementioned conditions are merely examples, and those skilled in the art can add, delete, or modify conditions according to actual circumstances. For instance, regarding the four scenarios described above, one or more of the following conditions can be deleted: the length of the first dimension at the source end is not greater than a threshold, the source end memory is off-chip memory, and the destination end memory is on-chip memory. In this case, the input / output decoding unit can generate corresponding microinstructions based on the remaining conditions.

[0134] After the aforementioned two-dimensional or three-dimensional microinstructions are generated, the input / output decoding unit 512 sends the two-dimensional or three-dimensional microinstructions to the routing unit 513. The routing unit 513, based on the source address in the two-dimensional or three-dimensional microinstructions, sends the two-dimensional or three-dimensional microinstructions to the source access unit 514. The source access unit 514 reads the complete image or a portion thereof from the off-chip memory 204 based on the two-dimensional or three-dimensional microinstructions.

[0135] To support the continuous read discrete write function, the source access unit 514 decodes the two-dimensional or three-dimensional microinstructions that identify continuous read discrete write. Read requests sent to the off-chip memory 204 are still segmented into two-dimensional or three-dimensional micro-operations or micro-instructions according to a 64-byte bus bandwidth burst. However, for the data and read control information returned by the off-chip memory 204, the source access unit 514 needs to further parse the destination-side two-dimensional or three-dimensional information in this micro-operation to perform discrete write on the corresponding data. The source access unit 514 in this embodiment also includes a segmentation module and a merging module, and the decoding method is divided into aligned mode and unaligned mode based on the same or similar conditions as in the aforementioned embodiments. Those skilled in the art can understand the operation methods of the aligned mode and unaligned mode of this embodiment without creative effort based on the description of the foregoing embodiments, so they will not be elaborated further.

[0136] Regardless of whether it is in aligned or unaligned mode, the source access unit 514 transmits the two-dimensional or three-dimensional sub-data and the destination address in the two-dimensional or three-dimensional micro-instruction to the routing unit 513. The routing unit 513 sends the two-dimensional or three-dimensional sub-data to the destination access unit 515 according to the destination address. The destination access unit 515 sends the two-dimensional or three-dimensional sub-data to the destination memory according to the destination address.

[0137] Under certain conditions, this embodiment can directly decode multidimensional data into two-dimensional or even three-dimensional microinstructions. Compared with the one-dimensional microinstructions of the prior art, this embodiment can transfer more data in the same time period, improving read and write efficiency and bus bandwidth utilization.

[0138] Another embodiment of the present invention is a method for reading and writing multidimensional data. Figure 6 This embodiment shows a flowchart illustrating the framework for reading and writing multidimensional data.

[0139] This embodiment can also fold adjacent dimensions when they are continuous in multidimensional data, so as to obtain multidimensional sub-data of multiple continuous dimensions of multidimensional data in a single data read operation. Specifically, in step 601, when multidimensional data is stored continuously in the source memory along the first source dimension and adjacent dimensions, multidimensional sub-data in the multidimensional data is read continuously from the source memory, thereby realizing multidimensional sub-data of multiple continuous dimensions of multidimensional data. Here, the first source dimension can refer to the lowest dimension when the multidimensional data is stored in the source processor, and the multidimensional data is stored continuously along the lowest dimension. In step 602, when multidimensional data is stored discretely along the second destination dimension in the destination memory, the multidimensional sub-data is written discretely into the destination memory, where the first destination dimension can be the lowest dimension when the data is stored in the destination memory, and the multidimensional data is also stored continuously along the lowest dimension in the destination memory. The second destination dimension can be an adjacent dimension to the first dimension.

[0140] This embodiment can continuously read multi-dimensional sub-data of multiple dimensions from the source memory and discretely write the read data to the destination memory, thereby making full use of the data bus bandwidth and reducing memory access overhead during data read and write.

[0141] Figure 7 A flowchart illustrating one implementation of this embodiment is shown.

[0142] In step 701, read / write instructions are received.

[0143] In step 702, the read / write instruction is broken down into at least one multi-dimensional micro-instruction, which carries information such as the source address, destination address, and data size. This embodiment can directly decode the read / write instruction into a two-dimensional micro-instruction based on only the lowest two dimensions, or simultaneously evaluate the multi-dimensional information of the source and destination ends to decode the read / write instruction into two-dimensional or more-dimensional micro-instructions.

[0144] In the case where read / write instructions are directly decoded into two-dimensional microinstructions based solely on the lowest two-dimensional dimension, this step determines whether the conditions for generating two-dimensional microinstructions are met. Multiple conditions may exist; if any condition is not met, it means that two-dimensional microinstructions cannot be generated. If all conditions are met, this embodiment generates multiple two-dimensional microinstructions based on the source's first and second dimensions, with each two-dimensional microinstruction carrying a two-dimensional sub-data of size W×C.

[0145] These conditions include: the image is stored contiguously along the first and second dimensions of the source memory; and the image is stored discretely along the second dimension of the destination memory. These conditions may also include one or more of the following: the length of the first dimension of the source is not greater than a threshold; the length of the second dimension of the source is the same as the length of the second dimension of the destination; the source memory is off-chip memory; and the destination memory is on-chip memory. These conditions have been clearly explained in the foregoing embodiments and will not be repeated here.

[0146] When simultaneously evaluating the multidimensional information from both the source and destination ends to decode read / write instructions into two-dimensional or more-dimensional micro-instructions, this embodiment first determines whether the length of the second dimension at the source end and the length of the second dimension at the destination end are greater than or equal to zero. A length greater than zero indicates the existence of the second dimension, while a length equal to zero indicates its non-existence. Based on whether the lengths of the second dimension at the source and destination ends are greater than or equal to zero, the following four cases can be distinguished.

[0147] The first scenario involves both the length of the second dimension at the source end and the length of the second dimension at the destination end being greater than zero. Further checks are made to determine if the following conditions are simultaneously met: the length of the first dimension at the source end is not greater than a threshold; the image is stored contiguously along the first and second dimensions at the source end in the source end memory; the image is stored contiguously along the third dimension at the source end in the source end memory; the image is stored discretely along the second dimension at the destination end in the destination end memory; the image is stored along two dimensions at the destination end memory; the source end memory is off-chip memory; the destination end memory is on-chip memory, etc. When all the above conditions are met, this embodiment can simultaneously generate three-dimensional micro-instructions based on the first, second, and third dimensions at the source end, allowing the entire image's read and write operations to be controlled by a single three-dimensional micro-instruction—that is, a single micro-instruction can move the entire image.

[0148] The second scenario involves a source-side second dimension with a length of zero and a destination-side second dimension with a length greater than zero. Further checks are made to determine if the following conditions are met: the source-side first dimension's length is not greater than a threshold; the image is stored contiguously along both the source-side first and third dimensions in the source-side memory; the image is stored discretely along the destination-side second dimension in the destination-side memory; the image is stored along both dimensions in the destination-side memory; the source-side memory is off-chip memory; or the destination-side memory is on-chip memory. When all the above conditions are met, this embodiment simultaneously generates a two-dimensional micro-instruction based on both the source-side first and third dimensions. This allows for the control of reading and writing the entire two-dimensional image (containing only the first and third dimensions) with a single two-dimensional micro-instruction, meaning that a single micro-instruction can move the entire image.

[0149] The third scenario involves a source-side second dimension with a length greater than zero and a destination-side second dimension with a length equal to zero. Further checks are made to determine if the following conditions are met: the source-side first dimension's length is not greater than a threshold; the image is stored contiguously along the source-side first and second dimensions in the source-side memory; the image is stored contiguously along the source-side third dimension in the source-side memory; the image is stored discretely along the destination-side third dimension in the destination-side memory; the source-side memory is off-chip memory; or the destination-side memory is on-chip memory, etc. When all the above conditions are met, this embodiment generates multiple two-dimensional micro-instructions based on the source-side first and third dimensions to transport the images in batches.

[0150] The fourth scenario involves both the length of the second dimension at the source end and the length of the second dimension at the destination end being zero. Further checks are made to determine if the following conditions are met: the length of the first dimension at the source end is not greater than a threshold; the image is stored contiguously along the first and third dimensions at the source end in the source end memory; the image is stored discretely along the third dimension at the destination end memory; the source end memory is off-chip memory; or the destination end memory is on-chip memory, etc. When all the above conditions are met, this embodiment generates a two-dimensional microinstruction based on the first and third dimensions at the source end. The read and write operations of the entire two-dimensional image, which only has the first and third dimensions, can be controlled by a single two-dimensional microinstruction. In other words, a single microinstruction can move the entire image.

[0151] In step 703, each multidimensional microinstruction is sent to the corresponding memory access unit based on the source address on the multidimensional microinstruction.

[0152] In step 704, the corresponding data is read from off-chip memory according to the source address and data size in the multidimensional microinstructions. In this step, two-dimensional or more-dimensional sub-data of the image is read from the source memory based on two-dimensional or more-dimensional microinstructions.

[0153] To achieve continuous read and discrete write, this embodiment further segments and merges two-dimensional or more-dimensional sub-data. More specifically, based on the bandwidth and the length of the first dimension at the source end, the sub-data corresponding to the two-dimensional micro-instruction is segmented into multiple data segments, and then these data segments are arranged based on the step size of the second dimension at the destination end.

[0154] The decoding method in this embodiment is also divided into aligned mode and unaligned mode. The operation method has been clearly explained in the previous embodiment, so it will not be repeated here.

[0155] In step 705, the destination address, along with the read data, is sent to the destination access unit according to the destination address.

[0156] In step 706, data is written to on-chip memory based on the destination address.

[0157] Under certain conditions, this embodiment can directly decode multidimensional data into two-dimensional microinstructions, three-dimensional microinstructions, or even higher-dimensional microinstructions. Compared with the one-dimensional microinstructions of the prior art, this embodiment can transfer more data in the same time period, improving read and write efficiency and bus bandwidth utilization.

[0158] Another embodiment of the present invention is a computer-readable storage medium storing computer program code for reading and writing multidimensional data. When the computer program code is run by a processor, it executes the methods described in the preceding embodiments. In some implementation scenarios, the integrated units described above can be implemented as software program modules. If implemented as software program modules and sold or used as independent products, the integrated units can be stored in a computer-readable memory. Based on this, when the solution of this disclosure is embodied in the form of a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, a server, or a network device) to execute some or all of the steps of the methods described in the embodiments of this disclosure. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0159] Another embodiment of the present invention is a computer program product, including a computer program for reading and writing multidimensional data, characterized in that, when the computer program is executed by a processor, it implements the steps of the methods described in the preceding embodiments.

[0160] Another embodiment of the present invention is a computer device including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the methods described in the preceding embodiments.

[0161] This invention directly decodes multidimensional data into two-dimensional microinstructions or even higher-dimensional microinstructions. Compared with the one-dimensional microinstructions of the prior art, this embodiment can move more data in the same time period, perform continuous read and discrete write, and improve read and write efficiency and bus bandwidth utilization.

[0162] Depending on the application scenario, the electronic devices or apparatus of the present invention may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus of the present invention can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus of the present invention can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal computing. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present invention can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0163] It should be noted that, for the sake of brevity, this invention describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solution of this invention is not limited to the order of the described actions. Therefore, based on the disclosure or teachings of this invention, those skilled in the art will understand that some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this invention can be considered as optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions of this invention. In addition, depending on the solution, the description of some embodiments of this invention also has different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this invention can also refer to the relevant descriptions of other embodiments.

[0164] In terms of specific implementation, based on the disclosure and teachings of this invention, those skilled in the art will understand that the several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or device embodiments described above, this document has divided them based on logical functions, but in actual implementation, there may be other ways of division. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection utilizing an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0165] In this invention, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same position or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this invention. Additionally, in some scenarios, multiple units in the embodiments of this invention may be integrated into one unit or each unit may exist physically independently.

[0166] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (Dynamic Random Access Memory, off-chip memory 204), static random access memory (SRAM), enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, off-chip memory 204), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM, etc.

[0167] The foregoing can be better understood in accordance with the following terms:

[0168] Clause A1. A memory access device for reading and writing multidimensional data, the memory access device comprising: a source-end access unit, configured to continuously read multidimensional sub-data from the source-end memory when the multidimensional data is stored continuously along a first source-end dimension and adjacent dimensions in the source-end memory; and a destination-end access unit, configured to discretely write the multidimensional sub-data into the destination-end memory when the multidimensional data is stored discretely along a second destination-end dimension in the destination-end memory.

[0169] Clause A2, the memory access apparatus according to Clause A1, wherein the adjacent dimension includes a source-end second dimension, and the source-end access unit is configured to continuously read the multidimensional sub-data based on the source-end first dimension and the source-end second dimension, wherein the multidimensional sub-data is two-dimensional sub-data.

[0170] Clause A3, the memory access apparatus according to Clause A2, wherein the multidimensional data is stored contiguously in the source memory along the source first dimension and the source second dimension when the step size of the source second dimension is equal to the length of the source first dimension.

[0171] Clause A4. A memory access device according to any one of Clauses A1 to 3, wherein when the step size of the second dimension of the destination is greater than the length of the first dimension of the source, the multidimensional data is discretely stored in the memory of the destination along the second dimension of the destination.

[0172] Clause A5, the memory access device according to Clause A4, wherein the multidimensional data is two-dimensional data or data of three or more dimensions.

[0173] Clause A6. The memory access device according to Clause A4, wherein when the multidimensional data is three-dimensional data, when the length of the source end second dimension of the multidimensional data is the same as the length of the destination end second dimension, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretely writes the multidimensional sub-data into the destination end memory.

[0174] Clause A7. The memory access device according to Clause A5, wherein when the multidimensional data is two-dimensional data, when the step size of the third dimension of the destination end is equal to the step size of the second dimension of the destination end multiplied by the length of the second dimension of the destination end, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretely writes the multidimensional sub-data into the destination end memory.

[0175] Clause A8. The memory access apparatus according to Clause A4, wherein the source-end access unit is further configured to continuously read the multidimensional sub-data based on the source-end first dimension and all the adjacent dimensions, wherein the dimension of the multidimensional sub-data is the same as the dimension of the multidimensional data.

[0176] Clause A9. The memory access device according to Clause A8, wherein the adjacent dimensions include a source-end second dimension and a source-end third dimension, and when the multidimensional data is stored continuously in the source-end memory along the source-end first dimension, the source-end second dimension and the source-end third dimension, the source-end access unit continuously reads the multidimensional sub-data based on the source-end first dimension, the source-end second dimension and the source-end third dimension.

[0177] Clause A10, the memory access apparatus according to Clause A9, wherein when the step size of the source-end third dimension is equal to the length of the source-end first dimension multiplied by the length of the source-end second dimension, and the step size of the source-end second dimension is equal to the length of the source-end first dimension, the multidimensional data is stored contiguously in the source-end memory along the source-end first dimension, the source-end second dimension, and the source-end third dimension.

[0178] Clause A11, the memory access device according to Clause A10, wherein when the step size of the third dimension of the destination is equal to the step size of the second dimension of the destination multiplied by the length of the second dimension of the destination, the source access unit continuously reads the multidimensional sub-data, and the destination access unit discretly writes the multidimensional sub-data into the destination memory.

[0179] Clause A12. The memory access apparatus according to Clause A1 further includes an input / output decoding unit for determining whether the multidimensional data is stored continuously in the source memory along the first source dimension and adjacent dimensions; wherein the adjacent dimensions of the first source dimension are the second source dimension and / or the third source dimension; and whether the multidimensional data is stored discretely in the destination memory along the second destination dimension.

[0180] Clause A13. The memory access device according to Clause A12, wherein when the input / output decoding unit determines that the length of the first dimension of the source end is not greater than a threshold, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretely writes the multidimensional sub-data into the destination end memory.

[0181] Clause A14, the memory access apparatus according to Clause A12, wherein the source memory is off-chip memory and the destination memory is on-chip memory.

[0182] Clause A15. The memory access apparatus according to Clause A1, wherein the source-end access unit divides the multidimensional sub-data into multiple data segments according to the length and bandwidth of the first dimension of the source end, and sequentially reads the multiple data segments from the source-end memory.

[0183] Clause A16. The memory access apparatus according to Clause A15, wherein the source access unit includes a front-end register and a back-end register, wherein when the step size of the second dimension of the destination and the address of the destination memory are both aligned with the length of the first dimension of the source, the front-end register caches the plurality of data segments according to the data arrangement of the source memory, the source access unit concatenates the plurality of data segments cached in the front-end register, and the plurality of data segments in the back-end register are arranged according to the data arrangement of the destination memory, with a step size of the second dimension of the destination.

[0184] Clause A17, the memory access apparatus according to Clause A16, wherein when the length of the first dimension of the source end is a power of 2, the step size of the second dimension of the destination end and the address of the memory of the destination end are aligned.

[0185] Clause A18, the memory access apparatus according to Clause A17, wherein the source access unit arranges the plurality of data segments according to a step size of the second dimension of the destination end and caches them in the subsequent register.

[0186] Clause A19. The memory access apparatus according to Clause A15, wherein when the step size of the second dimension of the destination end and the address of the destination end memory are not aligned with the length of the first dimension of the source end, the source end access unit splits the multidimensional microinstruction into one-dimensional microinstructions of the number of original microinstructions and a size equal to the length of the first dimension of the source end, and the source end access unit reads the multidimensional sub-data based on the one-dimensional microinstructions, wherein the original microinstructions control the micro-operations of reading and writing one-dimensional sub-data in the multidimensional data based on the first dimension of the source end.

[0187] Clause A20. The memory access apparatus according to Clause A1, wherein the input / output decoding unit is configured to generate at least one multidimensional microinstruction according to the read / write instructions of the received multidimensional data, the source access unit is configured to continuously read multidimensional sub-data from the source memory according to the multidimensional microinstruction, and the destination access unit is configured to discretely write the multidimensional sub-data into the destination memory according to the multidimensional microinstruction.

[0188] Clause A21. The memory access apparatus according to Clause A20 further includes an instruction receiving unit for receiving read / write instructions.

[0189] Clause A22. The memory access device according to Clause A20 further includes a routing unit for sending the multidimensional microinstruction to the source access unit according to the source address in the multidimensional microinstruction, wherein the source access unit transmits the multidimensional sub-data and the destination address in the multidimensional microinstruction back to the routing unit.

[0190] Clause A23. The memory access device according to Clause A22 further includes a destination access unit, wherein the routing unit sends the multidimensional sub-data to the destination access unit according to the destination address, and the destination access unit sends the multidimensional sub-data to the destination memory for discrete storage according to the destination address.

[0191] Clause A24, an integrated circuit device including a memory access device according to any one of Clauses A1 to A23.

[0192] Clause A25, a board including an integrated circuit device as described in Clause A24.

[0193] Clause A26. A method for reading and writing multidimensional data, comprising: continuously reading multidimensional sub-data from the source memory when the multidimensional data is stored continuously along a first source dimension and adjacent dimensions in a source memory; and discretely writing the multidimensional sub-data into the destination memory when the multidimensional data is stored discretely along a second destination dimension in a destination memory.

[0194] Clause A27. A computer-readable storage medium having stored thereon computer program code for reading and writing multidimensional data, wherein when the computer program code is run by a processing device, it performs the method described in Clause A26.

[0195] Clause A28. A computer program product comprising a computer program for reading and writing multidimensional data, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described in Clause A26.

[0196] Clause A29. A computer apparatus comprising a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the method described in Clause A26.

[0197] The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A memory access device for reading and writing multidimensional data, the memory access device comprising: The source-end access unit is used to continuously read multidimensional sub-data from the source-end memory when the multidimensional data is stored continuously along the first source-end dimension and adjacent dimensions in the source-end memory. as well as The destination access unit is used to write the multidimensional sub-data discretely into the destination memory when the multidimensional data is stored discretely along the second dimension of the destination. When the step size of the second dimension of the source end is equal to the length of the first dimension of the source end, the multidimensional data is stored continuously along the first and second dimensions of the source end in the source end memory. Wherein, when the step size of the second dimension of the destination is greater than the length of the first dimension of the source, the multidimensional data is stored discretely along the second dimension of the destination in the destination memory.

2. The memory access device according to claim 1, wherein the adjacent dimension includes a source-end second dimension, and the source-end access unit is configured to continuously read the multidimensional sub-data based on the source-end first dimension and the source-end second dimension, wherein the multidimensional sub-data is two-dimensional sub-data.

3. The memory access device according to claim 1, wherein, The multidimensional data refers to two-dimensional data or data with three or more dimensions.

4. The memory access device according to claim 1, wherein when the multidimensional data is three-dimensional data, when the length of the second dimension of the source end of the multidimensional data is the same as the length of the second dimension of the destination end, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretely writes the multidimensional sub-data into the destination end memory.

5. The memory access device according to claim 3, wherein, When the multidimensional data is two-dimensional data, when the step size of the third dimension of the destination end is equal to the step size of the second dimension of the destination end multiplied by the length of the second dimension of the destination end, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretly writes the multidimensional sub-data into the destination end memory.

6. The memory access device according to claim 1, wherein, The source access unit is further configured to continuously read the multidimensional sub-data based on the first dimension of the source and all the adjacent dimensions, wherein the dimension of the multidimensional sub-data is the same as the dimension of the multidimensional data.

7. The memory access device according to claim 6, wherein, The adjacent dimensions include the source-end second dimension and the source-end third dimension. When the multidimensional data is stored continuously along the source-end first dimension, the source-end second dimension and the source-end third dimension in the source-end memory, the source-end access unit continuously reads the multidimensional sub-data based on the source-end first dimension, the source-end second dimension and the source-end third dimension.

8. The memory access apparatus of claim 7, wherein when the step size of the third dimension of the source end is equal to the length of the first dimension of the source end multiplied by the length of the second dimension of the source end, and the step size of the second dimension of the source end is equal to the length of the first dimension of the source end, the multidimensional data is stored contiguously in the source end memory along the first dimension of the source end, the second dimension of the source end, and the third dimension of the source end.

9. The memory access device according to claim 8, wherein, When the step size of the third dimension of the destination is equal to the step size of the second dimension of the destination multiplied by the length of the second dimension of the destination, the source access unit continuously reads the multidimensional sub-data, and the destination access unit discretly writes the multidimensional sub-data into the destination memory.

10. The memory access device according to claim 1, further comprising an input / output decoding unit for determining: Whether the multidimensional data is stored contiguously along the first source dimension and adjacent dimensions in the source-end memory; wherein... The adjacent dimensions of the first dimension of the source end are the second dimension and / or the third dimension of the source end; and Whether the multidimensional data is stored discretely along the second dimension of the destination in the destination memory.

11. The memory access device according to claim 10, wherein when the input / output decoding unit determines that the length of the first dimension of the source end is not greater than a threshold, the source end access unit continuously reads the multidimensional sub-data, and the destination end access unit discretely writes the multidimensional sub-data into the destination end memory.

12. The memory access device according to claim 10, wherein the source memory is an off-chip memory and the destination memory is an on-chip memory.

13. The memory access device according to claim 1, wherein the source access unit divides the multidimensional sub-data into multiple data segments according to the length and bandwidth of the first dimension of the source end, and sequentially reads the multiple data segments from the source end memory.

14. The memory access apparatus according to claim 13, wherein the source access unit includes a front-end register and a back-end register, and when the step size of the second dimension of the destination end and the address of the destination end memory are both aligned with the length of the first dimension of the source end, the front-end register caches the plurality of data segments according to the data arrangement of the source end memory, the source access unit concatenates the plurality of data segments cached in the front-end register, and the plurality of data segments in the back-end register are arranged according to the data arrangement of the destination end memory, with a step size of the second dimension of the destination end.

15. The memory access apparatus of claim 14, wherein when the length of the first dimension of the source end is a power of 2, the step size of the second dimension of the destination end and the address of the memory at the destination end are aligned.

16. The memory access apparatus of claim 15, wherein the source access unit arranges the plurality of data segments according to a step size of the second dimension of the destination end and caches them in the subsequent register.

17. The memory access apparatus of claim 13, wherein when the step size of the second dimension of the destination end and the address of the destination end memory are not aligned with the length of the first dimension of the source end, the source end access unit splits the multidimensional microinstruction into one-dimensional microinstructions of the original number of microinstructions and a size equal to the length of the first dimension of the source end, and the source end access unit reads the multidimensional sub-data based on the one-dimensional microinstructions, wherein, The original micro-instructions control the micro-operations of reading and writing one-dimensional sub-data in the multi-dimensional data based on the first dimension of the source end.

18. The memory access device according to claim 1, wherein, The input / output decoding unit is used to generate at least one multidimensional micro-instruction according to the read / write instructions of the received multidimensional data. The source-end access unit is used to continuously read multidimensional sub-data from the source-end memory according to the multidimensional micro-instruction. The destination-end access unit is used to discretely write the multidimensional sub-data into the destination-end memory according to the multidimensional micro-instruction.

19. The memory access apparatus according to claim 18 further includes an instruction receiving unit for receiving read / write instructions.

20. The memory access apparatus of claim 18 further includes a routing unit for sending the multidimensional microinstruction to the source access unit according to the source address in the multidimensional microinstruction, wherein the source access unit transmits the multidimensional sub-data and the destination address in the multidimensional microinstruction back to the routing unit.

21. The memory access apparatus according to claim 20, further comprising a destination access unit, wherein the routing unit sends the multidimensional sub-data to the destination access unit according to the destination address, and the destination access unit sends the multidimensional sub-data to the destination memory for discrete storage according to the destination address.

22. An integrated circuit device comprising a memory access device according to any one of claims 1 to 21.

23. A board comprising the integrated circuit device according to claim 22.

24. A method for reading and writing multidimensional data, comprising: When the multidimensional data is stored continuously along the first dimension and adjacent dimensions in the source-end memory, the multidimensional sub-data in the multidimensional data is continuously read from the source-end memory. as well as When the multidimensional data is stored discretely along the second dimension of the destination in the destination memory, the multidimensional sub-data is discretely written into the destination memory. When the step size of the second dimension of the source end is equal to the length of the first dimension of the source end, the multidimensional data is stored continuously along the first and second dimensions of the source end in the source end memory. Wherein, when the step size of the second dimension of the destination is greater than the length of the first dimension of the source, the multidimensional data is stored discretely along the second dimension of the destination in the destination memory.

25. A computer-readable storage medium having stored thereon computer program code for reading and writing multidimensional data, wherein when the computer program code is run by a processing device, it performs the method of claim 24.

26. A computer program product comprising a computer program for reading and writing multidimensional data, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 24.

27. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 24.

Images