Data access method for chip and chip

Abstract

The embodiment of the present disclosure provides a data access method for a chip and the chip, wherein the method comprises the following steps: reading target data to be written into an on-chip memory from an off-chip memory in response to a data loading instruction; the target data comprises at least one group of data blocks, and each data block comprises at least one data unit; using a first address rearrangement module to rearrange the write address of each data unit to obtain the rearranged address of each data unit; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged address of each data unit in the data block is located in the memory bank corresponding to the data block; and writing the data unit into the on-chip memory according to the rearranged address of the data unit. In this way, the data reading efficiency can be improved while reducing the occurrence of bank conflicts, the complexity of the software processing logic in the chip application process can be reduced, the number of instructions can be reduced, and the universality of the chip application can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the technical field of computer, and particularly relates to a data access method for a chip and the chip. BACKGROUND

[0002] In a chip, in order to improve data processing efficiency, data to be processed is usually loaded from an off-chip memory to an on-chip memory, and the required data can be quickly read from the on-chip memory in the process of data processing. However, in the related art, the chip may read multiple rows of data in the same bank from the on-chip memory in the same clock cycle in the process of data processing, which will cause a bank access conflict (i.e., a bank conflict), thereby affecting the data reading efficiency and the subsequent data processing efficiency. SUMMARY

[0003] Therefore, the embodiments of the present disclosure at least provide a data access method for a chip and the chip.

[0004] The technical scheme of the embodiments of the present disclosure is implemented as follows:

[0005] The embodiments of the present disclosure provide a data access method for a chip, and the method comprises:

[0006] In response to a data loading instruction, reading target data to be written into an on-chip memory from an off-chip memory; the target data comprises at least one group of data blocks, and each data block comprises at least one data unit;

[0007] Using a first address rearrangement module to rearrange the write addresses of each data unit to obtain the rearranged addresses of each data unit; wherein each data block in the same group corresponds to a different bank in the on-chip memory after rearrangement, and the rearranged address of each data unit in the data block is located in the bank corresponding to the data block;

[0008] According to the rearranged address of each data unit, writing each data unit into the on-chip memory.

[0009] The embodiments of the present disclosure provide a chip, and the chip comprises:

[0010] A data storage engine and an on-chip memory, the data storage engine comprising a first read data module, a first address rearrangement module and a first write data module; wherein:

[0011] The first read data module is configured to read target data to be written into the on-chip memory from an off-chip memory in response to a data loading instruction; the target data comprises at least one group of data blocks, and each data block comprises at least one data unit;

[0012] The first address rearrangement module is configured to rearrange the write addresses of the data units to obtain rearranged addresses of the data units; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged addresses of the data units in the data block are located in the memory bank corresponding to the data block.

[0013] The first write data module is configured to write the data units into the on-chip memory according to the rearranged addresses of the data units.

[0014] In the embodiments of the present disclosure, in response to a data loading instruction, target data to be written into the on-chip memory is read from an off-chip memory; the target data includes at least one group of data blocks, and each data block includes at least one data unit; a first address rearrangement module is used to rearrange the write addresses of the data units to obtain rearranged addresses of the data units; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged addresses of the data units in the data block are located in the memory bank corresponding to the data block; and the data units are written into the on-chip memory according to the rearranged addresses of the data units. In this way, on the one hand, each data block in the same group can be rearranged to be stored in a different memory bank in the on-chip memory, so that in the same clock cycle, data units in multiple data blocks in the same group can be read from the on-chip memory, thereby increasing the data reading efficiency while reducing the occurrence of bank conflicts; on the other hand, the first address rearrangement module is used to rearrange the write addresses, which can hardwareize the address rearrangement logic into the chip, without the need to additionally increase the address rearrangement logic at the software level, thereby reducing the complexity of the software processing logic in the chip application process, reducing the number of instructions, and further reducing the development burden of software developers, which is conducive to improving the universality of the chip application.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0017] Figure 1A An implementation flowchart of a data access method for a chip provided in the embodiments of the present disclosure;

[0018] Figure 1B A constituent structure diagram of an on-chip memory provided in the embodiments of the present disclosure;

[0019] Figure 2 An implementation flowchart of a data access method for a chip is provided for an embodiment of the present disclosure;

[0020] Figure 3A A component structure diagram of a chip is provided for an embodiment of the present disclosure;

[0021] Figure 3B A component structure diagram of a chip is provided for an embodiment of the present disclosure Figure Two ;

[0022] Figure 4A A component structure diagram of a chip is provided for an embodiment of the present disclosure;

[0023] Figure 4B A storage diagram of an A matrix in a local memory in the related art is provided;

[0024] Figure 4C An arrangement diagram of each data block in a local memory after address rearrangement of a chip in a matrix operation scenario is provided for an embodiment of the present disclosure;

[0025] Figure 4D An arrangement diagram of each data block in a local memory after address rearrangement of a chip in a case where a data type of a matrix element is FP16 is provided for an embodiment of the present disclosure;

[0026] Figure 4E An arrangement diagram of each data block in a local memory after address rearrangement of a chip in a case where a data type of a matrix element is TF16 is provided for an embodiment of the present disclosure;

[0027] Figure 4F A current available row size diagram of a local memory in a chip is provided for an embodiment of the present disclosure;

[0028] Figure 4G A current available row size diagram of a local memory in a chip is provided for an embodiment of the present disclosure Figure Two ;

[0029] Figure 4H An arrangement diagram of each data block in a local memory after address rearrangement of a chip in a convolution operation scenario without introducing an address interval step is provided for an embodiment of the present disclosure;

[0030] Figure 4I An arrangement diagram of each data block in a local memory after address rearrangement of a chip in a convolution operation scenario with introducing an address interval step is provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure are further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limitations of the present disclosure. All other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present disclosure.

[0032] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0033] The terms "first / second / third" involved are only to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first / second / third" can interchange specific order or sequence as allowed, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the present disclosure and are not intended to limit the present disclosure.

[0035] In order to better understand the solutions of the embodiments of the present disclosure, the data access method of a chip in the related art will be described first.

[0036] In the related art, if multiple rows of data need to be read from the same memory bank in the on-chip memory within one clock cycle, a bank conflict will occur.

[0037] In some related art, in the process of loading data to be processed from an off-chip memory to an on-chip memory, only the minimum size of data supported by the chip for processing in a single clock cycle is read each time, so as to store the data to be read in a single clock cycle in the same row of storage units in the on-chip memory. In this way, only the same row of storage units in the on-chip memory needs to be accessed in a single clock cycle in the process of reading data from the on-chip memory for processing, so that no bank conflict occurs. However, this solution will result in a decrease in data multiplexing rate and a decrease in access bandwidth of reading data in the data processing process due to the small amount of data loaded from the off-chip memory each time, and will increase the number of accesses to the off-chip memory, affecting the overall efficiency of data processing.

[0038] In view of this, the embodiments of the present disclosure provide a data access method for a chip. The method can be executed by the chip. In some embodiments, the chip can be any suitable processor chip. For example, the processor chip can include, but is not limited to, at least one of a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a data processing unit (DPU), and the like. Figure 1A An implementation flowchart of a data access method for a chip provided by the embodiments of the present disclosure is shown in FIG. 1. The method can include the following steps S101-S103: Figure 1A

[0039] In step S101, in response to a data load instruction, target data to be written into an on-chip memory is read from an off-chip memory; the target data includes at least one group of data blocks, and each data block includes at least one data unit.

[0040] Here, the off-chip memory can include, but is not limited to, at least one of a main memory, a global memory, and the like. The on-chip memory can include, but is not limited to, at least one of a local memory in the chip, a register stack, and the like.

[0041] The data load instruction is used to instruct to read the target data from the off-chip memory and write the read target data into the on-chip memory. The target data can be any suitable data to be processed by the chip, and the embodiments of the present disclosure are not limited thereto. In some embodiments, the data load instruction can be sent by a host processor to the chip. According to the data load instruction, the storage address of each data unit in the off-chip memory and the corresponding write address of each data unit in the on-chip memory can be determined.

[0042] It can be understood that the data amount of the target data, the number of groups of data blocks in the target data, the number of data blocks included in each group of data blocks, and the number of data units included in each data block can be determined according to actual data processing requirements.

[0043] In some embodiments, the data amount of the target data can be determined according to the available storage capacity of the on-chip memory. For example, the data amount of the target data can be equal to the available storage capacity of the on-chip memory, or can be less than the available storage capacity of the on-chip memory. ​

[0044] In some embodiments, a data chunk can be data that needs to be read from the on-chip memory in a single clock cycle of the chip for processing. For example, in a case where the chip is used to perform a matrix multiplication operation AxB, the target data can include matrix data of matrices A / B, a set of target data chunks can include a matrix slice in the matrices A / B, a data unit can include an element in the matrices A / B, and a matrix slice in the matrices A / B needs to be read from the on-chip memory in a single clock cycle of the chip for the matrix operation. For another example, in a case where the chip is used to perform a convolution operation, the target data can include image data, a set of data chunks can include an image slice in the image data, a data unit can include pixel data in the image slice, and an image slice in the image data needs to be read from the on-chip memory in a single clock cycle of the chip for the matrix operation.

[0045] In step S102, a first address rearrangement module is used to rearrange the write addresses of the data units to obtain rearranged addresses of the data units; wherein the data chunks in the same group correspond to different banks in the on-chip memory after rearrangement, and the rearranged addresses of the data units in the data chunks are located in the banks corresponding to the data chunks.

[0046] Here, the chip can include a first address rearrangement module. In implementation, the first address rearrangement module can use any suitable address rearrangement logic to rearrange the write addresses of the data units, so that the rearranged addresses of the data units in different data chunks in the same group fall into different banks in the on-chip memory, which is not limited in the embodiments of the present disclosure. For example, the first address rearrangement module can use at least one of a rearrangement logic based on a bit XOR operation, a rearrangement logic based on a sequential offset, etc. to rearrange the write addresses of the data units.

[0047] In some embodiments, the address offset corresponding to each write address can be determined in advance, and the first address rearrangement module can offset the write addresses of the data units according to the corresponding address offset to obtain the rearranged addresses of the data units.

[0048] In some embodiments, the correspondence between each write address and the corresponding rearranged address can be determined in advance, and the first address rearrangement module can query the correspondence according to the write addresses of the data units to obtain the rearranged addresses of the data units.

[0049] In step S103, the data units are written into the on-chip memory according to the rearranged addresses of the data units.

[0050] In the embodiments of the present disclosure, in response to a data loading instruction, target data to be written into the on-chip memory is read from the off-chip memory; the target data includes at least one group of data blocks, and each data block includes at least one data unit; a first address rearrangement module is used to rearrange the write addresses of the data units to obtain rearranged addresses of the data units; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged addresses of the data units in the data block are located in the memory bank corresponding to the data block; and the data units are written into the on-chip memory according to the rearranged addresses of the data units. In this way, on the one hand, each data block in the same group can be rearranged to be stored in a different memory bank in the on-chip memory, so that in the same clock cycle, data units in multiple data blocks in the same group can be read from the on-chip memory, thereby increasing data reading efficiency while reducing bank conflicts; on the other hand, the first address rearrangement module is used to rearrange the write addresses, which can hardwareize the address rearrangement logic into the chip, without the need to additionally increase the address rearrangement logic at the software level, thereby reducing the complexity of the software processing logic in the chip application process, reducing the number of instructions, and reducing the development burden of software developers, which is conducive to improving the universality of the chip application.

[0051] In some embodiments, the above step S102 can include the following step S111:

[0052] In step S111, a first address rearrangement module is used to convert the write addresses of the data units into rearranged addresses of the data units based on address conversion parameters.

[0053] Here, the address conversion parameters can include, but are not limited to, at least one of the following: a row size of the on-chip memory, a block size of the data block, an address interval step size between two adjacent data blocks in the same group of data blocks, and the like.

[0054] It can be understood that in different application scenarios, different address conversion parameters can be set, so that the first address rearrangement module can convert the write addresses of the data units according to the address conversion parameters using appropriate address conversion logic. In this way, the flexibility of address rearrangement can be improved to better adapt to different application scenarios and improve the versatility of the first address rearrangement module.

[0055] In some embodiments, as Figure 1BAs shown, the on-chip memory 100 includes a plurality of memory partitions 110 arranged in an array, each memory partition 110 includes at least one memory cell 111 which is address-continuous, at least one memory cell 111 in each memory partition 110 is used to store at least one data cell in a data partition, and each memory partition 110 in the same column belongs to the same memory bank (e.g., bank0, bank1, …). The address translation parameter includes a row size of a single row of memory partitions in the on-chip memory.

[0056] The address translation based on the address translation parameter in step S111 to convert the write address of the data cell to the rearranged address of the data cell can include the following steps S121 to S123:

[0057] In step S121, a first row identifier corresponding to the write address and a first intra-row address offset are determined based on the write address of the data cell and the row size.

[0058] Here, the row size of a single row of memory partitions in the on-chip memory is the current available row size of the on-chip memory. For example, the current available row size of the on-chip memory can include, but is not limited to, 128 bytes (Byte), or 256 Byte, etc.

[0059] In some embodiments, the current available row size of the on-chip memory can be determined according to the maximum access bandwidth provided by the on-chip memory. For example, in the case of the on-chip memory being a local memory, a computing engine in the chip can directly read data from the local memory for calculation, the maximum access bandwidth provided by the local memory for the computing engine is 256 Byte, and the row size of the local memory can be 256 Byte. For another example, in the case of the on-chip memory being a local memory, the chip can read data from the local memory to a register file through a local memory access pipeline, and the computing engine can read data from the register file for calculation, the maximum access bandwidth provided by the local memory for the local memory access pipeline is 128 Byte, and the current available row size of the on-chip memory can be 128 Byte.

[0060] In some embodiments, the write address of the data cell can be divided by the row size to obtain a first row identifier corresponding to the write address, and the write address can be taken modulo the row size to obtain a first intra-row address offset corresponding to the write address.

[0061] For example, if the write address of the data unit is lms_addr, and the row size of the single-row storage partition in the on-chip memory is line_size, then the first row identifier line_id corresponding to the write address is lms_addr / line_size, and the first intra-row address offset line_offset1 corresponding to the write address is lms_addr%line_size.

[0062] In some embodiments, the write address of the data unit can be divided by an integer multiple of the row size to obtain the first row identifier corresponding to the write address, and the write address can be divided by the integer multiple of the row size to obtain the first intra-row address offset corresponding to the write address.

[0063] In step S122, the second intra-row address offset is determined based on the first row identifier and the first intra-row address offset.

[0064] The second intra-row address offset refers to the intra-row address offset of the data unit after address rearrangement. In implementation, the second intra-row address offset corresponding to the data unit can be determined based on the first row identifier and the first intra-row address offset corresponding to the write address of the data unit according to the actually adopted address rearrangement logic, so that the second intra-row address offsets corresponding to the data units in different data partitions in the same group respectively fall into different memory banks in the on-chip memory.

[0065] In some embodiments, each row identifier can correspond to a different address offset step, and the first intra-row address offset can be offset by the address offset step corresponding to the first row identifier to obtain the second intra-row address offset, where the address offset step can be an integer multiple of the block size of the data partition.

[0066] In step S123, the rearranged address of the data unit is determined based on the first row identifier and the second intra-row address offset.

[0067] In some embodiments, the row start address of the row where the data unit is located can be determined based on the first row identifier corresponding to the data unit, and the rearranged address of the data unit can be obtained based on the row start address and the second intra-row address offset.

[0068] For example, the product line_id*line_size of the first row identifier line_id corresponding to the data unit and the row size line_size of the single-row storage partition in the on-chip memory can be determined as the row start address of the row where the data unit is located, and in the case where the second intra-row address offset corresponding to the data unit is line_offset2, the rearranged address of the data unit can be determined as line_id*line_size+line_offset2.

[0069] In some embodiments, based on the first row identifier corresponding to the data unit, a row start address of the row where the data unit is located can be determined, and based on the row start address, an intra-row reserved size, and the second intra-row address offset, the rearranged address of the data unit can be obtained. The intra-row reserved size can be a size of an address space reserved in the single-row storage block for storing a specific flag field or attribute field.

[0070] For example, the product line_id x line_size of the first row identifier line_id corresponding to the data unit and the row size line_size of the single-row storage block in the on-chip memory can be determined as the row start address of the row where the data unit is located; the intra-row reserved size line_offset0 of the row where the data unit is located; and in the case where the second intra-row address offset corresponding to the data unit is line_offset2, the rearranged address of the data unit can be determined as line_id x line_size + line_offset0 + line_offset2.

[0071] In the above embodiments, based on the write address of the data unit and the row size of the single-row storage block in the on-chip memory, the first row identifier and the first intra-row address offset corresponding to the write address are determined; based on the first row identifier and the first intra-row address offset, the second intra-row address offset is determined; and based on the first row identifier and the second intra-row address offset, the rearranged address of the data unit is determined. In this way, the write address of the data unit can be quickly and accurately converted into the rearranged address according to the row size of the single-row storage block in the on-chip memory, so that the first address rearrangement module can be adapted to on-chip memories with different row sizes.

[0072] In some embodiments, the address conversion parameter further includes a block size of the data block. The above step S122 can include the following steps S131 to S133:

[0073] In step S131, based on the first intra-row address offset and the block size, a first block identifier and a first intra-block address offset corresponding to the write address are determined.

[0074] Here, the block size of the data block refers to the amount of data contained in the data block, i.e., the sum of the amounts of data of each data unit in the data block.

[0075] In some embodiments, the data size of the data unit can be determined according to the data type of the data unit, and the block size of the data block can be determined according to the data size and the number of data units contained in the data block. For example, the data type of the data unit is FP16, and the number of data units contained in a single data block is 8, then the data size of a single data unit is 2 bytes, and the block size of the data block is 16 bytes.

[0076] In some embodiments, the first block identifier corresponding to the write address can be obtained by dividing the first intra-line address offset corresponding to the write address by the block size, and the first intra-block address offset corresponding to the write address can be obtained by taking the remainder of the first intra-line address offset corresponding to the write address divided by the block size.

[0077] For example, the first intra-line address offset corresponding to the write address lms_addr is line_offset1, and the block size of the data block is sector_sz, then the first block identifier sector1_in_line corresponding to the write address is line_offset1 / sector_sz, and the first intra-block address offset offset_in_sector corresponding to the write address is line_offset1%sector_sz.

[0078] In some embodiments, the first block identifier corresponding to the write address can be obtained by dividing the first intra-line address offset corresponding to the write address by an integer multiple of the block size, and the first intra-block address offset corresponding to the write address can be obtained by taking the remainder of the first intra-line address offset corresponding to the write address divided by the integer multiple of the block size.

[0079] In step S132, the second block identifier is determined based on the first row identifier and the first block identifier.

[0080] The second block identifier refers to the block identifier of the storage block corresponding to the data unit in the row after address rearrangement. In implementation, the second block identifier corresponding to the data unit can be determined based on the first row identifier and the first block identifier corresponding to the write address of the data unit according to the actually adopted address rearrangement logic, so that the second block identifiers corresponding to the data units in different data blocks in the same group fall into different storage banks in the on-chip memory respectively.

[0081] In some embodiments, each row identifier can correspond to a different block offset, and the first block identifier can be offset according to the block offset corresponding to the first row identifier to obtain the second block identifier.

[0082] In some embodiments, each combination of row identifier and block identifier can correspond to a different block offset, and the first block identifier can be offset according to the block offset corresponding to the first row identifier and the first block identifier to obtain the second block identifier.

[0083] In step S133, the second in-line address offset is determined based on the second block identifier and the first in-sector address offset.

[0084] In some embodiments, the starting address of the data chunk where the data unit is located can be determined based on the second block identifier corresponding to the data unit, and the second in-line address offset corresponding to the data unit can be obtained based on the starting address of the data chunk, the in-sector reserved size, and the first in-sector address offset. The in-sector reserved size can be the size of the address space reserved in the data chunk for storing a specific flag field or attribute field.

[0085] For example, the second in-line address offset line_offset2 corresponding to the data unit can be obtained by summing the product sector2_in_line x sector_sz of the second block identifier sector2_in_line corresponding to the data unit and the block size sector_sz of the data chunk, the line starting address line_id x line_size of the row where the data unit is located, and the first in-sector address offset offset_in_sector.

[0086] In some embodiments, the starting address of the data chunk where the data unit is located can be determined based on the second block identifier corresponding to the data unit, and the second in-line address offset corresponding to the data unit can be obtained based on the starting address of the data chunk, the in-sector reserved size, and the first in-sector address offset. The in-sector reserved size can be the size of the address space reserved in the data chunk for storing a specific flag field or attribute field.

[0087] In the above embodiments, the first block identifier and the first in-sector address offset corresponding to the write address are determined based on the first in-line address offset and the block size, the second block identifier is determined based on the first row identifier and the first block identifier, and the second in-line address offset is determined based on the second block identifier and the first in-sector address offset. In this way, the second in-line address offset corresponding to each data unit can be quickly and accurately determined according to the block size of the data chunk, so that the write address of the data unit is converted into the rearranged address, and thus the first address rearrangement module can adapt to address rearrangement of data units of different data types.

[0088] In some embodiments, the address conversion parameters further include an address interval step size between two adjacent data chunks in the same group of data chunks. The above step S132 can include the following steps S141 to S143:

[0089] In step S141, the row interval step size of the address rearrangement is determined based on the address interval step size and the block size.

[0090] Here, the address interval stride between two adjacent data chunks in the same group of data chunks can be determined according to the distance between two adjacent data chunks to be read in a single clock cycle in an actual data processing scenario. For example, the address interval stride can include 256 Bytes, 128 Bytes, 64 Bytes, or 32 Bytes, etc.

[0091] In some embodiments, the address interval stride between two adjacent data chunks in the same group of data chunks can be determined according to the current computing mode of the chip. For example, in the case that the current computing mode of the chip is a matrix multiplication operation, each data chunk in the same group corresponds to a plurality of storage chunks in the same storage bank, and thus the address interval stride between two adjacent data chunks in the same group of data chunks can be equal to the row size of a single row of storage chunks in the on-chip memory. For another example, in the case that the current computing mode of the chip is a convolution operation, each pixel data of an image is included in the target data, and the write addresses of the pixel data are continuous. Assuming that the data size of each pixel data in the image is 32 Bytes, each pixel data includes high 16 Bytes and low 16 Bytes of channel data, and the high 16 Bytes and the low 16 Bytes are respectively taken as a data chunk, in the case that the low 16 Bytes of data in a group of pixels need to be read in a single clock cycle, the address interval stride between two adjacent data chunks in the same group of data chunks can be equal to the data size of each pixel data, i.e., 32 Bytes. In this way, the first address rearrangement module can be adapted to different computing modes.

[0092] The row interval stride of address rearrangement refers to the number of rows included in the row range in which address rearrangement is performed, i.e., the row interval stride can be taken as an address rearrangement period, and address rearrangement is performed in a single address rearrangement period.

[0093] In some embodiments, the address interval stride can be divided by the block size to obtain the row interval stride of address rearrangement. For example, in the case that the address interval stride is stride and the block size is sector_sz, the row interval stride of address rearrangement can be stride / sector_sz.

[0094] In some embodiments, the address interval stride can be divided by an integer multiple of the block size to obtain the row interval stride of address rearrangement.

[0095] In step S142, a second row identifier is determined based on the first row identifier and the row interval stride.

[0096] Here, the second row identifier represents the row identifier of the address row corresponding to the write address of the data unit in the row interval stride.

[0097] In some embodiments, the first row identifier can be taken modulo the row interval step size to obtain the second row identifier. For example, in a case where the first row identifier is line_id and the row interval step size of the address rearrangement is swizzle_group, the second row identifier swizzle_line_id can be line_id % swizzle_group.

[0098] In some embodiments, the first row identifier can be taken modulo an integer multiple of the row interval step size to obtain the second row identifier.

[0099] In step S143, the second block identifier is determined based on the second row identifier and the first block identifier.

[0100] In some embodiments, each row identifier within the row interval step size can correspond to a different block offset, and the first block identifier can be offset according to the block offset corresponding to the second row identifier to obtain the second block identifier.

[0101] In some embodiments, the second block identifier can be obtained by performing a bitwise XOR operation on the second row identifier and the first block identifier.

[0102] In the above embodiments, the row interval step size of the address rearrangement is determined based on the address interval step size and the block size, the second row identifier is determined based on the first row identifier and the row interval step size, and the second block identifier is determined based on the second row identifier and the first block identifier. In this way, the second block identifier corresponding to each data unit can be quickly and accurately determined according to the address interval step size between two adjacent data blocks in the same group of data blocks, so as to convert the write address of the data unit into the rearranged address. In this way, the first address rearrangement module can be adapted to different data processing scenarios.

[0103] In some embodiments, the above data access method can further include the following step S151:

[0104] In step S151, the address conversion parameter is determined based on the data load instruction; the address conversion parameter includes at least one of the following: the row size of a single-row storage block in the on-chip memory, the block size of the data block, and the address interval step size between two adjacent data blocks in the same group of data blocks.

[0105] In some embodiments, the address conversion parameter can be directly carried in the data load instruction, and the chip can directly obtain the address conversion parameter from the data load instruction.

[0106] In some embodiments, the data load instruction can carry indication information for indicating the address conversion parameter, and the address conversion parameter can be obtained by analyzing the indication information in the data load instruction.

[0107] In the above embodiments, the address conversion parameters are configured through the data loading instruction, so that the flexibility of address rearrangement is further improved, the software logic in the data processing process is simplified, and the number of instructions is reduced.

[0108] The present disclosure provides a data access method for a chip. Figure 2 An implementation flowchart of a data access method for a chip provided by an embodiment of the present disclosure is shown in Figure 2 The method can include the following steps S201 to S207:

[0109] Step S201, in response to a data loading instruction, reading target data to be written into an on-chip memory from an off-chip memory; the target data includes at least one group of data blocks, and each data block includes at least one data unit.

[0110] Step S202, using a first address rearrangement module to rearrange the write addresses of each data unit to obtain the rearranged addresses of each data unit; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged addresses of each data unit in the data block are located in the memory bank corresponding to the data block.

[0111] Step S203, writing the data units into the on-chip memory according to the rearranged addresses of the data units.

[0112] Here, the above steps S201 to S203 correspond to steps S101 to S103 in the foregoing embodiments respectively, and in implementation, the implementation manners of the foregoing steps S101 to S103 can be referred to.

[0113] Step S204, obtaining a data reading instruction, the data reading instruction being used to indicate reading a group of target data blocks from the on-chip memory, and the target data blocks including at least one target data unit.

[0114] Step S205, determining the read addresses of each target data unit based on the data reading instruction.

[0115] Step S206, using a second address rearrangement module to rearrange the read addresses of each target data unit to obtain the rearranged addresses of each target data unit.

[0116] Step S207, reading each target data unit from the on-chip memory in parallel according to the rearranged addresses of each target data unit.

[0117] The data read instruction can be sent by the host processor to the chip, or generated by the computing engine in the chip. The target data chunk can be a data chunk currently to be processed by the chip.

[0118] The second address rearrangement module can be the same address rearrangement module as the first address rearrangement module, or another address rearrangement module with the same rearrangement logic as the first address rearrangement module.

[0119] The process of rearranging the read addresses of the target data units in step S206 corresponds to the process of rearranging the write addresses of the data units in step S102. In implementation, the read addresses of the target data units can be rearranged by referring to the implementation of step S102.

[0120] In the embodiments of the present disclosure, a data read instruction is obtained, the data read instruction being used to indicate reading a group of target data chunks from an on-chip memory, the target data chunks including at least one target data unit; based on the data read instruction, read addresses of the target data units are determined; a second address rearrangement module is used to rearrange the read addresses of the target data units to obtain rearranged addresses of the target data units; and the target data units are read from the on-chip memory in parallel according to the rearranged addresses of the target data units. In this way, since each target data chunk in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, the rearranged addresses of the target data units in the target data chunk are located in the memory bank corresponding to the data chunk, so that the target data units in multiple target data chunks in the same group can be read from the on-chip memory in the same clock cycle, thereby increasing the data read efficiency while reducing the occurrence of bank conflicts.

[0121] In some embodiments, the above method can further include the following step S211:

[0122] In step S211, address conversion parameters are determined based on the data read instruction; the address conversion parameters include at least one of the following: a row size of a single-row memory chunk in the on-chip memory, a block size of the data chunk, and an address interval step size between two adjacent data chunks in the same group of data chunks.

[0123] The above step S206 can include the following step S212:

[0124] In step S212, a second address rearrangement module is used to rearrange the read addresses of the target data units based on the address conversion parameters to obtain the rearranged addresses of the target data units.

[0125] In some embodiments, the address conversion parameter can be directly carried in the data read instruction, and the chip can directly obtain the address conversion parameter from the data read instruction.

[0126] In some embodiments, the data read instruction can carry indication information for indicating the address conversion parameter, and the address conversion parameter can be obtained by analyzing the indication information in the data read instruction.

[0127] In different application scenarios, the second address rearrangement module can convert the read address of the target data unit according to the corresponding address conversion parameter by using appropriate address conversion logic. In this way, the flexibility of address rearrangement can be improved to better adapt to different application scenarios and improve the versatility of the second address rearrangement module. In addition, by configuring the address conversion parameter through the data read instruction, the flexibility of address rearrangement can be further improved, and the software logic in the data processing process can be simplified, and the number of instructions can be reduced.

[0128] In some embodiments, the target data includes matrix data, a group of the target data blocks includes target matrix slices in the matrix data, and the target data unit includes matrix elements in the target matrix slices. The above method further includes the following step S221:

[0129] In step S221, each matrix element in the read target matrix slice is input into the computing unit array to perform a matrix product operation.

[0130] In some embodiments, the chip can include a computing unit array for performing data operations. The target matrix slice can be a matrix slice that needs to be read and subjected to a matrix product operation by the computing unit array in a single clock cycle.

[0131] It should be noted that in the process of performing the matrix product operation AxB of matrix A and matrix B, the target matrix slice can be matrix A or matrix B, and the embodiments of the present disclosure are not limited thereto. Matrix A and matrix B can be read from the off-chip memory and stored in the on-chip memory by using the data access method in the embodiments of the present disclosure, and can be read from the on-chip memory and output to the computing unit array to complete the matrix product operation.

[0132] In some embodiments, the target data includes image data, a group of the target data blocks includes target image slices in the image data, and the target data block includes channel data of a single pixel in the target image slice; the above method further includes the following step S231:

[0133] In step S231, the channel data of each pixel in the read target image slice is input into the computing unit array to perform a convolution operation.

[0134] In some embodiments, the channel data of each pixel in the target image slice can correspond to a data block respectively.

[0135] The chip provided by the embodiments of the present disclosure comprises a data storage engine and an on-chip memory. Figure 3A As shown in a schematic structural diagram of a chip provided by the embodiments of the present disclosure, Figure 3A The chip 300 comprises a data storage engine 310 and an on-chip memory 320, the data storage engine 310 comprises a first read data module 311, a first address rearrangement module 312 and a first write data module 313; wherein:

[0136] The first read data module 311 is configured to read target data to be written into the on-chip memory 320 from an off-chip memory in response to a data loading instruction; the target data comprises at least one group of data blocks, and each data block comprises at least one data unit;

[0137] The first address rearrangement module 312 is configured to rearrange the write addresses of each data unit to obtain rearranged addresses of each data unit; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory 320 after rearrangement, and the rearranged addresses of each data unit in the data block are located in the memory bank corresponding to the data block.

[0138] The first write data module 313 is configured to write the data unit into the on-chip memory 320 according to the rearranged address of the data unit.

[0139] In some embodiments, the data storage engine can comprise a tensor memory engine (TME) in a GPGPU chip.

[0140] In the embodiments of the present disclosure, by arranging the first address rearrangement module in the data storage engine of the chip, on the one hand, each data block in the same group can be rearranged to a different memory bank in the on-chip memory for storage, so that in the same clock cycle, the data units in multiple data blocks in the same group can be read from the on-chip memory, thus the data reading efficiency can be increased while reducing the occurrence of bank conflicts; on the other hand, the first address rearrangement module is used to rearrange the write addresses, which can hardwareize the address rearrangement logic to the chip, without the need to additionally increase the address rearrangement logic at the software level, thereby the complexity of the software processing logic in the chip application process can be reduced, the number of instructions can be reduced, and the development burden of the software developers can be reduced, which is conducive to improving the universality of the chip application.

[0141] In some embodiments, the first address rearrangement module is further configured to convert the write addresses of the data units into the rearranged addresses of the data units based on address conversion parameters.

[0142] In some embodiments, the data storage engine further includes:

[0143] a first parameter determination module configured to determine the address conversion parameter based on the data loading instruction, and send the address conversion parameter to the first address rearrangement module; the address conversion parameter includes at least one of the following: a row size of a single-row memory block in the on-chip memory, a block size of the data block, and an address interval step size between two adjacent data blocks in the same group of data blocks.

[0144] In some embodiments, as shown in Figure 3B Chip 300 further includes a computing engine 330, which includes an address determination module 331, a second address rearrangement module 332, a second data reading module 333, and a computing unit array 334 including a plurality of computing units; wherein:

[0145] The address determination module 331 is configured to obtain a data reading instruction, the data reading instruction being used to indicate reading a group of target data blocks from the on-chip memory 320, the target data blocks including at least one target data unit; and determine a reading address of each target data unit based on the data reading instruction.

[0146] The second address rearrangement module 332 is configured to rearrange the reading address of each target data unit to obtain a rearranged address of each target data unit.

[0147] The second data reading module 333 is configured to read each target data unit from the on-chip memory 320 in parallel according to the rearranged address of each target data unit, and input each target data unit in the group of target data blocks read to the computing unit array 334.

[0148] In some embodiments, the computing engine further includes:

[0149] A second parameter determination module is configured to determine an address conversion parameter based on the data reading instruction; the address conversion parameter includes at least one of the following: a row size of a single-row memory block in the on-chip memory, a block size of the data block, and an address interval step size between two adjacent data blocks in the same group of data blocks.

[0150] The second address rearrangement module is further configured to rearrange the reading address of each target data unit based on the address conversion parameter to obtain a rearranged address of each target data unit.

[0151] In some embodiments, in a case where the target data comprises matrix data, a group of the target data blocks comprises a target matrix tile in the matrix data, the target data cells comprise matrix elements in the target matrix tile, and the array of the calculation units is configured to perform a matrix multiplication operation based on each of the matrix elements in the read target matrix tile.

[0152] In a case where the target data comprises image data, a group of the target data blocks comprises a target image tile in the image data, and the target data blocks comprise channel data of a single pixel in the target image tile; and the array of the calculation units is configured to perform a convolution operation based on the channel data of each pixel in the read target image tile.

[0153] The following describes an application of the data access method for a chip provided by the embodiments of the present disclosure in an actual scenario.

[0154] In the related art, an artificial intelligence (AI) accelerator generally adopts an ATOMIC_CxATOMIC_K calculation unit configuration, and only one pixel data needs to be read each time an image is read, each pixel data contains ATOMIC_C channel data, and a storage unit is generally also designed specifically, and generally no bank conflict exists. However, a GPGPU needs to support general-purpose computing, and a Tensor Core is added to accelerate matrix multiplication and convolution operation. If the data source of the Tensor Core is a register, a mapping between the data and the register needs to be reasonably performed to avoid access conflict of the register. Although the Tensor Core can read data from the register, the data also needs to be moved from a local memory to the register, and the arrangement of matrix data in a global memory is linear and continuous. If data of multiple rows of storage units in the same bank is read at the same time, a bank conflict occurs. One solution is to read only data of the minimum size supported by the Tensor Core when the data is moved from the global memory to the local memory, so that the data is continuously stored in the same row of storage units in the local memory, and the bank conflict can be avoided, but data reuse is reduced, and access to the global memory is also increased.

[0155] The embodiments of the present disclosure provide a chip, such as Figure 4AAs shown, the chip 400 includes a tensor storage engine 410, a local memory 420, and a tensor computing engine 430, in which a hardware address rearrangement module (i.e., a first address rearrangement module) 411 is arranged in the tensor storage engine 410, which can realize address rearrangement of data from a global memory to the local memory 420, and a second address rearrangement module 431 same as the first address rearrangement module 411 is arranged in the tensor computing engine 430. The tensor storage engine 410 reads target data from the global memory, and before writing the target data into the local memory 420, the first address rearrangement module 411 is used to perform address rearrangement transformation on the original write addresses of each data unit in the target data. The transformation takes the row in the local memory 420 as a boundary, rearranges the write addresses of each data unit in the same row, so that the write addresses of the data units in the data blocks corresponding to different rows in the same bank are staggered in different banks after transformation. When the tensor computing engine 430 reads data from the local memory 420, multiple row data blocks in the same bank need to be read in the same clock cycle, at this time, the second address rearrangement module 431 same as the first address rearrangement module 411 can be used to perform address rearrangement to reduce bank conflicts. In this way, the chip provided by the embodiment of the present disclosure can reduce bank conflicts in the data access process, and can improve the data access bandwidth requirement in the process of matrix calculation and convolution calculation of the tensor computing engine 430, and reduce the burden of software personnel.

[0156] In some embodiments, the module adopts rearrangement logic based on XOR operation, and a user configures the current available row size line_size of the local memory, the block size sector_sz of the data block, and the address interval stride between adjacent two data blocks in the same group of data blocks in the data loading instruction and / or the data reading instruction according to the use scenario, where the row size line_size represents the amount of data that can be stored in a single row of the local memory, the block size sector_sz represents the amount of address-continuous data that needs to be read from a single row of the local memory in a single clock cycle, and the address interval stride represents the address interval between adjacent two data blocks in the same group of data blocks read in a single clock cycle. In this way, the first address rearrangement module and the second address rearrangement module can support the configurability of address transformation parameters, so as to meet various application scenarios and improve the flexibility of the chip.

[0157] In some embodiments, continuing to refer to Figure 4AIn the tensor computing engine 430, a dot product matrix 432 (corresponding to the array of computing units in the foregoing embodiments) of tile_Mx tile_N can be included, and each dot product unit DP in the dot product matrix 432 can perform tile_K multiply-add operations, for example, A0x B0+A1x B1+…, where tile_M, tile_N, and tile_K are positive integers. Using the dot product matrix 432, general matrix dot product operations, convolution operations, and the like can be performed on data read from the local memory 420.

[0158] In some embodiments, when the computing mode of the chip is a matrix dot product operation, the tensor storage engine can read matrix data from the global memory and write the matrix data into the local memory. For example, to calculate the product AxB between matrix A and matrix B, the tensor storage engine can read matrix data of matrix A / B from the global memory and write the matrix data into the local memory, and the memory access control unit in the tensor computing engine can read matrix data of matrix A / B to be operated on from the local memory. In each clock cycle, the memory access control unit in the tensor computing engine needs to read matrix data of tile_Mx tile_K elements of matrix A (corresponding to the target matrix tile in the foregoing embodiments) from the local memory. Figure 4B The storage diagram of the A matrix in the local memory in the related art is shown in FIG. 7A. As shown in FIG. 7A, the target matrix tile includes 8x8 elements A0_0 to A7_7 of matrix A, and the target matrix tile includes 8 rows of elements Line0 to Line7, and each row of elements (corresponding to the data chunk in the foregoing embodiments) is in the same bank bank0, so that bank conflicts occur during reading of the multiple rows of elements, which reduces the memory bandwidth of the local memory and affects the computing efficiency of the computing engine. Figure 4B

[0159] In the embodiments of the present disclosure, the amount of data of matrix A that needs to be read by each row in each clock cycle of the tensor computing engine can be taken as a data chunk, and the tile_K is taken as 8, and the data structure of the matrix elements is taken as FP16. For example, the size of the data chunk is 16 bytes, and the size of each row in the local memory is 256 bytes, so that there are 16 data chunks in each row of the local memory, and the addresses of the matrix elements in each data chunk are continuous. The matrix data of each row of the local memory can be rearranged in the granularity of the data chunk to obtain the rearranged addresses of the matrix elements in each data chunk. Figure 4C ​An arrangement diagram of each data block of a chip provided by an embodiment of the present disclosure after address rearrangement in a local memory in a matrix operation scenario is shown in FIG. 6. In the diagram, the block size of the data block is 16 bytes, the size of each row in the local memory is 256 bytes, and each row in the local memory includes 16 data blocks, i.e., data blocks 0, 1, 2, …, 15. In the original data row before address rearrangement, the data blocks 0, 1, 2, …, 15 in each row are arranged in sequence, so that the data blocks with the same serial number in each row (corresponding to a group of data blocks in the foregoing embodiment) are located in the same column in the local memory, i.e., fall in the same bank. Thus, when each data block in the same group (for example, each data block 0 or each data block 1) is read in a single clock cycle, there is a bank conflict problem. In each row s_line0 to s_line15 after address rearrangement, the data blocks with the same serial number in each row are located in different columns in the local memory, i.e., fall in different banks. Thus, when each data block in the same group (for example, each data block 0 or each data block 1) is read in a single clock cycle, there is no bank conflict problem.

[0160] For matrix point multiplication operation, the data arrangement of the input matrix can be in row-major order, i.e., the matrix is stored in memory row by row, so that each data unit in a single data block can correspond to a row of matrix elements in a target matrix tile; the data arrangement of the input matrix can also be in column-major order, i.e., the matrix is stored in memory column by column, so that each data unit in a single data block can correspond to a column of matrix elements in a target matrix tile. It can be understood that for matrix point multiplication operation of matrix A and matrix B, the matrix data of matrix A can be stored in row-major order, and the matrix data of matrix B can be stored in column-major order. The address rearrangement manner of each data block in the matrix data of matrix B can be similar to the address rearrangement manner of each data block in the matrix data of matrix A.

[0161] In some embodiments, the block size of the data block can correspond to the data type of each matrix element in the matrix data. For example, if the matrix element is of INT8 type, tile_M bytes of matrix data need to be read by the tensor computing engine per row per clock cycle, i.e., the block size of the data block is tile_M; if the matrix element is of FP16 type, tile_M*2 bytes of matrix data need to be read by the tensor computing engine per row per clock cycle, i.e., the block size of the data block is tile_M*2; if the matrix element is of TF16 type, tile_M*4 bytes of matrix data need to be read by the tensor computing engine per row per clock cycle, i.e., the block size of the data block is tile_M*4.

[0162] Figure 4D An arrangement diagram of each data block in the local memory after address rearrangement in the case that the data type of the matrix element in the chip is FP16 is shown in FIG. 6. Figure 4D As shown in FIG. 6, taking tile_M as 16 as an example, in the case that the data type of the matrix element is FP16, the block size of the data block is 32 Byte, the size of each row in the local memory is 256 Byte, and each row in the local memory includes 8 data blocks, i.e., data blocks 0, 1, 2, …, 7.

[0163] Figure 4E An arrangement diagram of each data block in the local memory after address rearrangement in the case that the data type of the matrix element in the chip is TF16 is shown in FIG. 7. Figure 4E As shown in FIG. 7, taking tile_M as 16 as an example, in the case that the data type of the matrix element is TF16, the block size of the data block is 64 Byte, the size of each row in the local memory is 256 Byte, and each row in the local memory includes 4 data blocks, i.e., data blocks 0, 1, 2, 3.

[0164] In some embodiments, in addition to reading data directly from the local memory for computation, the tensor computing engine can also read data from the register file for computation. In the case of reading data from the register file for computation, the input data of the matrix operation / convolution operation can be read from the local memory to the register file through a local memory access (LMA) pipeline. Since the data access bandwidth provided by the local memory to the tensor computing engine and the LMA pipeline can be different, the current available row size line_size of the local memory can be a first size in the case that the tensor computing engine reads data directly from the local memory for computation, and the current available row size line_size of the local memory can be a second size in the case that the tensor computing engine reads data from the register file for computation. Figure 4F An arrangement diagram of the current available row size of the local memory in the chip is shown in FIG. 8. Figure 4F As shown in FIG. 8, if the tensor computing engine reads data directly from the local memory for computation, the data access bandwidth provided by the local memory to the tensor computing engine can be 256 Byte, and the current available row size line_size of the local memory can be 256 Byte. Figure 4G An arrangement diagram of the current available row size of the local memory in the chip is shown in FIG. 9. Figure Two As shown in FIG. 9, if the tensor computing engine reads data from the register file for computation, the data access bandwidth provided by the local memory to the tensor computing engine can be 256 Byte, and the current available row size line_size of the local memory can be 256 Byte. Figure 4GAs shown, if the tensor computing engine reads data from the register file for computation, the data access bandwidth provided by the local memory for the LMA pipeline can be 128 Bytes, and the current available line size of the local memory can be 128 Bytes. In this way, by increasing the current available line size of the local memory as a configurable address translation parameter, a suitable available line size can be configured by software to adapt to different application scenarios.

[0165] In some embodiments, in a case where the computing mode of the chip is a convolution operation, the tensor storage engine can read image data from the global memory and write the image data into the local memory. For the convolution computation, the image data is stored in the local memory in a CxNDHW format, and is stored in a memory interleaved form in a channel dimension to ensure address continuity of pixel data in the channel dimension; where C represents the number of input channels, N represents the batch sample size of the convolution operation, D represents the number of output channels, and H and W represent the height and width of the convolution feature map, respectively. Figure 4H An arrangement diagram of each data block in the local memory after address rearrangement without introducing an address interval step in a convolution operation scenario is provided for the chip in the embodiments of the present disclosure. It is assumed that the data size of each pixel data in the image is 32 Bytes, each pixel data includes high 16 Byte and low 16 Byte channel data, the high 16 Byte data and the low 16 Byte data are respectively taken as a data block, the memory interleaving granularity (IG) adopted for storing the image data in the local memory is 32 Bytes, the width W of the convolution feature map corresponding to the image data is 32, and for a pixel Wi (for example, pixels W0, W1, W2, and the like), Wi lo is the low 16 Byte data of the pixel Wi, and Wi hi is the high 16 Byte data of the pixel Wi. If it is required to read the low 16 Byte data W0 lo to W15 lo of the pixels W0 to W15, since W0 lo to W15 lo are respectively located in different columns in the local memory, that is, in different banks, bank conflict will not occur in a case where the data blocks W0 lo to W15 lo are read in the same clock cycle.

[0166] Continuing to refer to Figure 4HIf low 16 Byte data W13 lo to W28 lo of pixels W13 to W28 need to be read, since W13 lo and W28 lo are located in the same column in the local memory, that is, in the same bank, a bank conflict occurs when the data blocks W13 lo to W28 lo are read in the same clock cycle. For this case, by introducing the address interval stride parameter, which is used to represent the address interval between two adjacent data blocks in the same group of data blocks read in a single clock cycle, and rearranging the addresses of the data blocks according to the address interval stride. Figure 4I The arrangement of the data blocks of the chip provided by the embodiment of the present disclosure after the address rearrangement in the convolution operation scenario is shown in the schematic diagram. For the case where the memory interleaving granularity is 32 Byte, the address interval stride between two adjacent data blocks in the same group of data blocks is set to 32 Byte, and the block size of the data block is 16 Byte, so only the address rearrangement in the range of 2 rows (32 Byte / 16 Byte) is needed, so that the bank conflict does not occur when the computing engine reads the single-channel data (high 16 Byte or low 16 Byte) of any 16 consecutive pixels in a single clock cycle.

[0167] In the embodiment of the present disclosure, by implementing the hardware address rearrangement module in the TCE and the TME, the bank conflict is reduced, the complexity of the software calculation address rearrangement logic is simplified, the number of instructions is reduced, and the flexibility of the address rearrangement is improved by increasing the three configurable parameters of the current available row size line_size of the local memory, the block size sector_sz of the data block, and the address interval stride stride between two adjacent data blocks in the same group of data blocks in the instructions, so that the first address rearrangement module and / or the second address rearrangement module are more general and can support multiple computing modes such as matrix multiplication and convolution operation.

[0168] It should be noted that the above description of the embodiments tends to emphasize the differences between the embodiments, and the same or similar parts can be referred to each other. The above description of the chip embodiments is similar to the above description of the method embodiments, and has similar beneficial effects as the method embodiments. In some embodiments, the chip provided by the embodiment of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure for understanding.

[0169] It should be understood that every feature, structure, or characteristic described in relation to one embodiment is applicable to at least one other embodiment, in combination or in isolation, unless specifically noted otherwise. It should be understood that the size and order of the steps / operations in the above described embodiments is not intended to be limiting, and the order of the steps / operations can be changed, and / or two or more steps / operations can be performed at the same time, unless specifically noted otherwise. The above described embodiments are merely exemplary and should not be taken as limiting.

[0170] It should be noted that, as used in this document, the terms "includes," "including," "has," "having," "contains," "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements, but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises a... " does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0171] In several embodiments provided in the present disclosure, it should be understood that the disclosed chip and method can be implemented in other manners. The embodiments described above are merely exemplary, and the division of the units is merely a logical function division, and there can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices, or units, and can be electrical, mechanical, or in other forms.

[0172] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units; they can be located in one place, or distributed on a plurality of network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments. In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in the form of hardware, or in the form of hardware plus software function units.

[0173] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the method embodiments when executed.

[0174] The above merely describes the embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure.

Claims

1. A method for data access of a chip, characterized in that, The method comprises: reading target data to be written into an on-chip memory from an off-chip memory in response to a data loading instruction; the target data comprises at least one group of data blocks, and each data block comprises at least one data unit; the group of data blocks comprises data that needs to be read from the on-chip memory for processing in a single clock cycle of a chip; rearranging, by using a first address rearrangement module, a write address of each data unit to obtain a rearranged address of each data unit; wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged address of each data unit in the data block is located in the memory bank corresponding to the data block; writing the data unit into the on-chip memory according to the rearranged address of the data unit; the rearranging, by using the first address rearrangement module, the write address of each data unit to obtain the rearranged address of each data unit comprises: converting, by using the first address rearrangement module, the write address of the data unit into the rearranged address of the data unit based on an address conversion parameter; wherein the address conversion parameter is determined according to a current application scenario of the chip, and the address conversion parameter comprises at least one of the following: a row size of a single-row memory block in the on-chip memory, a block size of the data block, and an address interval step between two adjacent data blocks in the same group of data blocks; a block offset of the rearranged address of the data unit relative to the write address is determined according to a row identifier and a block identifier corresponding to the write address under the address conversion parameter, different row identifiers and block identifiers correspond to different block offsets, and the block offset represents an offset direction and an offset amount of a memory block where the rearranged address of the data unit is located relative to a memory block where the write address of the data unit is located.

2. The data access method of claim 1, wherein, The on-chip memory comprises a plurality of memory blocks arranged in an array, each memory block comprises at least one memory unit with continuous addresses, the at least one memory unit is used to store at least one data unit in the data block, each memory block in the same column belongs to the same memory bank, and the address conversion parameter comprises a row size of a single-row memory block; the converting, based on the address conversion parameter, the write address of the data unit into the rearranged address of the data unit comprises: determining a first row identifier and a first intra-row address offset corresponding to the write address based on the write address of the data unit and the row size; determining a second intra-row address offset based on the first row identifier and the first intra-row address offset; determining the rearranged address of the data unit based on the first row identifier and the second intra-row address offset.

3. The data access method of claim 2, wherein, The address conversion parameter further comprises a block size of the data block; the determining a second intra-row address offset based on the first row identifier and the first intra-row address offset comprises: determining a first block identifier and a first intra-block address offset corresponding to the write address based on the first intra-row address offset and the block size; determining a second block identifier based on the first row identifier and the first block identifier; determining the second row address offset based on the second block identifier and the first block address offset.

4. The data access method of claim 3, wherein, the address conversion parameters further comprise an address interval stride between two adjacent data chunks in the same group of data chunks; determining the second block identifier based on the first row identifier and the first block identifier comprises: determining an inter-row stride of address rearrangement based on the address interval stride and the chunk size; determining a second row identifier based on the first row identifier and the inter-row stride of address rearrangement; determining the second block identifier based on the second row identifier and the first block identifier.

5. The data access method of claim 1, wherein, the method further comprises: determining the address conversion parameters based on the data load instruction.

6. The data access method according to any one of claims 1 to 5, characterized in that, the method further comprises: obtaining a data read instruction, the data read instruction being used to instruct reading a group of target data chunks from the on-chip memory, the target data chunks comprising at least one target data cell; determining read addresses of the target data cells based on the data read instruction; rearranging the read addresses of the target data cells by a second address rearrangement module to obtain rearranged addresses of the target data cells; reading the target data cells from the on-chip memory in parallel according to the rearranged addresses of the target data cells.

7. The data access method of claim 6, wherein, the method further comprises: determining address conversion parameters based on the data read instruction; the address conversion parameters comprising at least one of the following: a row size of a single-row storage chunk in the on-chip memory, a chunk size of the data chunk, an address interval stride between two adjacent data chunks in the same group of data chunks; rearranging the read addresses of the target data cells by the second address rearrangement module to obtain rearranged addresses of the target data cells comprises: rearranging the read addresses of the target data cells based on the address conversion parameters by the second address rearrangement module to obtain rearranged addresses of the target data cells.

8. The data access method of claim 6, wherein, the target data comprises matrix data, a group of the target data chunks comprises target matrix slices in the matrix data, and the target data cells comprise matrix elements in the target matrix slices; the method further comprises: inputting the read matrix elements in the target matrix slices into a computing cell array to perform a matrix product operation.

9. The data access method of claim 6, wherein, the target data comprises image data, a group of the target data chunks comprises target image slices in the image data, and the target data chunks comprise channel data of a single pixel in the target image slices; the method further comprises: inputting the read channel data of each pixel in the target image slices into a computing cell array to perform a convolution operation.

10. A chip, characterized by comprises: a data storage engine and an on-chip memory, the data storage engine comprising a first read data module, a first address rearrangement module, and a first write data module; wherein: The first reading data module is configured to read target data to be written into the on-chip memory from an off-chip memory in response to a data loading instruction, the target data comprising at least one group of data blocks, each data block comprising at least one data unit, and the group of data blocks comprising data that needs to be read from the on-chip memory for processing in a single clock cycle of the chip; The first address rearrangement module is configured to rearrange the write addresses of the data units to obtain rearranged addresses of the data units, wherein each data block in the same group corresponds to a different memory bank in the on-chip memory after rearrangement, and the rearranged addresses of the data units in the data block are located in the memory bank corresponding to the data block; The first writing data module is configured to write the data units into the on-chip memory according to the rearranged addresses of the data units; The first address rearrangement module is further configured to convert the write addresses of the data units into the rearranged addresses of the data units based on address conversion parameters; The address conversion parameters are determined according to a current application scenario of the chip, and the address conversion parameters comprise at least one of the following: a row size of a single-row memory block in the on-chip memory, a block size of the data block, and an address interval step between two adjacent data blocks in the same group of data blocks; a block offset of the rearranged address of the data unit relative to the write address is determined according to a row identifier and a block identifier corresponding to the write address under the address conversion parameters, different row identifiers and block identifiers correspond to different block offsets, and the block offset represents an offset direction and an offset amount of the memory block where the rearranged address of the data unit is located relative to the memory block where the write address of the data unit is located.

11. The chip of claim 10, wherein, The data storage engine further comprises: A first parameter determination module configured to determine the address conversion parameters based on the data loading instruction and send the address conversion parameters to the first address rearrangement module; the address conversion parameters comprise at least one of the following: a row size of a single-row memory block in the on-chip memory, a block size of the data block, and an address interval step between two adjacent data blocks in the same group of data blocks.

12. The chip according to claim 10 or 11, characterized in that The chip further comprises a computing engine, and the computing engine comprises an address determination module, a second address rearrangement module, a second reading data module, and a computing unit array comprising a plurality of computing units; wherein: The address determination module is configured to obtain a data reading instruction, the data reading instruction being used to indicate reading a group of target data blocks from the on-chip memory, the target data blocks comprising at least one target data unit; and determine read addresses of the target data units based on the data reading instruction; The second address rearrangement module is configured to rearrange the read addresses of the target data units to obtain rearranged addresses of the target data units; and The second reading data module is configured to read the target data blocks from the on-chip memory according to the rearranged addresses of the target data units. The second data reading module is configured to read each of the target data units from the on-chip memory in parallel according to the rearranged addresses of the target data units, and input each of the target data units in a group of the target data blocks read to the computing unit array.

13. The chip of claim 12, wherein, The computing engine further includes: A second parameter determining module configured to determine address conversion parameters based on the data reading instruction. The second address rearranging module is further configured to rearrange the reading addresses of each of the target data units based on the address conversion parameters to obtain the rearranged addresses of each of the target data units.

14. The chip of claim 13, wherein, In a case where the target data includes matrix data, a group of the target data blocks includes target matrix slices in the matrix data, the target data units include matrix elements in the target matrix slices, and the computing unit array is configured to perform a matrix product operation based on each of the matrix elements in the target matrix slices read. In a case where the target data includes image data, a group of the target data blocks includes target image slices in the image data, the target data blocks include channel data of a single pixel in the target image slices, and the computing unit array is configured to perform a convolution operation based on the channel data of each pixel in the target image slices read.

Images