Data access method and device applied to multi-NPU system, equipment and medium

Abstract

The invention provides a data access method and device applied to a multi-NPU system, equipment and a medium, and relates to the technical field of data processing. Any target NPU in the multi-NPU system applies the data access method, the target NPU is an NPU which needs to acquire a target matrix in the multi-NPU system, and the multi-NPU system comprises a plurality of target NPUs; the method comprises the steps of obtaining a storage area and a starting point address of a target matrix; the starting point addresses are located in the storage area, and at least part of the starting point addresses obtained in the multiple target NPUs are different; obtaining first data stored in the first target address interval; the first target address interval is an interval formed from the starting address to the termination address of the storage area; if the starting address is not the starting address of the storage area, acquiring second data stored in a second target address interval after acquiring the first data; the second target address interval is an interval formed from the starting address to the starting address of the storage area.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a data access method, apparatus, device and medium applied to a multi-NPU system. Background Art

[0002] In order to solve the load balancing problem of address access requests on NoC (Network On Chip), the data address space is usually interleaved, so that different data accesses can be stored in different data paths within the NoC as much as possible to achieve load balancing. However, when multiple NPU (Neural Network Processor) processors collaboratively process tensor data, data reuse usually occurs. For example, for matrix operations, if you choose to split the B matrix of the matrix operation in the column direction when executing on multiple NPU processors, each NPU processes different columns of the B matrix, so that multiple NPUs need to read the same A matrix. In this case, each processor accessing the A matrix will send the same address request to the NoC, and the existing balancing strategy of the NoC will distribute these multiple identical address requests to the same data path, resulting in the problem of unbalanced load on the data path. Summary of the invention

[0003] The present application provides a data access method, device, equipment and medium applied to a multi-NPU system to solve the problem in the prior art that multiple identical address requests are distributed to the same data path, resulting in an unbalanced load on the data path.

[0004] In a first aspect, the present application provides a data access method applied to a multi-NPU system, the data access method applied to a multi-NPU system, the multi-NPU system includes multiple NPUs, any target NPU in the multi-NPU system applies the data access method, the target NPU is the NPU in the multi-NPU system that needs to obtain a target matrix, and the multi-NPU system includes multiple target NPUs; the method includes: obtaining a storage area and a starting point address of the target matrix; the starting point address is located in the storage area, and at least some of the multiple target NPUs have different starting point addresses; obtaining first data stored in a first target address interval; the first target address interval is an interval formed from the starting point address to the end address of the storage area; if the starting point address is not the starting address of the storage area, then after obtaining the first data, obtaining the second data stored in a second target address interval; the second target address interval is an interval formed from the starting address of the storage area to the starting address.

[0005] In the embodiment of the present application, since at least some of the multiple target NPUs obtain different starting addresses, at least some of the multiple target NPUs will be assigned different data paths when obtaining the target matrix, so that compared with the existing method in which all NPUs obtain the target matrix through the same data path, this solution can better balance the load of different data paths. In addition, when the starting address is not the starting address of the storage area, after obtaining the first data, the second data stored in the second target address interval will also be obtained, thereby ensuring that each NPU can obtain the complete target matrix.

[0006] In combination with the technical solution provided in the first aspect above, in some possible implementations, obtaining the starting point address includes: obtaining a tensor offset; and obtaining the starting point address based on the starting address of the storage area and the tensor offset.

[0007] In the embodiment of the present application, the starting address is determined by the tensor offset and the starting address of the storage area. Compared with the method of directly obtaining the starting address, this method is simpler.

[0008] In combination with the technical solution provided in the first aspect above, in some possible implementations, the method further includes: obtaining a tensor shape; if the starting address is the starting address of the storage area, after obtaining the first data, the method further includes: determining whether the first data conforms to the tensor shape, and if not, re-obtaining the target matrix; if the starting address is not the starting address of the storage area, or after obtaining the first data and the second data, the method further includes: based on the tensor shape, determining whether the first data and the second data conform to the tensor shape, and if not, re-obtaining the target matrix.

[0009] In the embodiment of the present application, the obtained tensor shape is used to verify whether the obtained target matrix is ​​complete. If it is incomplete, the target matrix is ​​re-acquired to ensure the integrity of the data.

[0010] In combination with the technical solution provided in the first aspect above, in some possible implementations, when the target matrix is ​​used to perform matrix multiplication operations and the target matrix is ​​the multiplicand matrix of the multiplication operation, each row of the target matrix is ​​treated as an array and is stored in the corresponding storage area in sequence.

[0011] In the embodiment of the present application, since the rows of the multiplicand matrix are multiplied by the columns of the multiplier matrix in matrix multiplication, each row of the multiplicand matrix is ​​stored as an array, so that after a row of data of the multiplicand matrix is ​​subsequently obtained, calculations can be performed directly without waiting until the complete target matrix is ​​obtained before performing calculation operations.

[0012] In combination with the technical solution provided in the first aspect above, in some possible implementations, the method further includes: when each array of the target matrix is ​​obtained, performing a matrix multiplication operation based on the obtained array.

[0013] In the embodiment of the present application, after obtaining a row of data of the multiplicand matrix, the calculation is performed directly without waiting until the complete target matrix is ​​obtained before performing the calculation operation, thereby reducing the time required for the calculation.

[0014] In combination with the technical solution provided in the first aspect above, in some possible implementations, when the target matrix is ​​used to perform matrix multiplication operations and the target matrix is ​​a multiplier matrix of the multiplication operation, each column of the target matrix is ​​treated as an array and is stored in corresponding storage areas in sequence.

[0015] In the embodiment of the present application, since the matrix multiplication is performed by multiplying the rows of the multiplicand matrix by the columns of the multiplier matrix, each column of the multiplier matrix is ​​stored as an array, so that after a column of data of the multiplier matrix is ​​subsequently obtained, calculations can be performed directly without waiting until the complete target matrix is ​​obtained before performing calculation operations.

[0016] In combination with the technical solution provided in the first aspect above, in some possible implementations, the method further includes: when each array of the target matrix is ​​obtained, performing a matrix multiplication operation based on the obtained array.

[0017] In the embodiment of the present application, after obtaining a column of data of the multiplier matrix, calculation is performed directly without waiting until the complete target matrix is ​​obtained before performing calculation operations, thereby reducing the time required for calculation.

[0018] In combination with the technical solution provided in the first aspect above, in some possible implementations, the number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: N is greater than or equal to M, and the starting addresses obtained by each target NPU are different.

[0019] In combination with the technical solution provided in the first aspect above, in some possible implementations, the number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: the M is greater than the N and less than or equal to 2*N, the starting addresses obtained by the N target NPUs in each of the target NPUs are different from each other, and the starting addresses obtained by the remaining MN target NPUs are also different from each other.

[0020] In the second aspect, the present application provides a data access device applied to a multi-NPU system, which is deployed on any target NPU in the multi-NPU system, the target NPU is the NPU in the multi-NPU system that needs to obtain a target matrix, and the multi-NPU system includes multiple target NPUs; the device includes: a first acquisition module and a second acquisition module, the first acquisition module is used to obtain the storage area and starting address of the target matrix; the starting address is located in the storage area, and at least some of the multiple target NPUs have different starting addresses; the second acquisition module is used to obtain the first data stored in the first target address interval; the first target address interval is an interval formed from the starting address to the end address of the storage area; the second acquisition module is also used to obtain the second data stored in the second target address interval after obtaining the first data if the starting address is not the starting address of the storage area; the second target address interval is an interval formed from the starting address of the storage area to the starting address.

[0021] In a third aspect, the present application provides an electronic device, comprising: a memory and a multi-NPU system, wherein the memory and the multi-NPU system are connected; the memory is used to store programs; and each NPU in the multi-NPU system is used to call the program stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, the computer program executes the method described in the first aspect and / or in combination with any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flowchart of a data access method applied to a multi-NPU system shown in an embodiment of the present application;

[0025] Figure 2 A corresponding relationship diagram between a data channel and an NPU shown in an embodiment of the present application;

[0026] Figure 3A structural block diagram of a data access device applied to a multi-NPU system shown in an embodiment of the present application;

[0027] Figure 4 A structural block diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0029] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it is not necessary to further define and explain it in the subsequent drawings. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0030] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0031] In order to solve the problem that multiple identical address requests are distributed to the same data path in the prior art, resulting in unbalanced load on the data path, the present application provides a data access method applied to a multi-NPU system, any target NPU in the multi-NPU system applies the data access method, the target NPU is the NPU in the multi-NPU system that needs to obtain a target matrix, and the multi-NPU system includes multiple target NPUs.

[0032] See also Figure 1 , Figure 1 This is a flow chart of a data access method applied to a multi-NPU system shown in an embodiment of the present application. Figure 1 Describe the steps it involves.

[0033] S100: Obtain the storage area and starting point address of the target matrix.

[0034] The starting point address is located in the storage area, and the starting point addresses acquired by at least some of the multiple target NPUs are different.

[0035] Optionally, the storage area and the starting point address of the target matrix may be obtained from a received data acquisition command, and the data acquisition command carries the storage area and the starting point address of the target matrix.

[0036] Alternatively, the storage area and the starting point address of the target matrix may be acquired in advance and stored in a local storage medium, and may be directly called when needed.

[0037] In one implementation manner, obtaining the storage area of ​​the target matrix may be: obtaining the start address and the end address of the target matrix, and the storage area surrounded by the start address and the end address is the storage area of ​​the target matrix.

[0038] Alternatively, obtaining the storage area of ​​the target matrix may also be: obtaining the starting address and storage area size of the target matrix, and determining the ending address of the storage area according to the storage area size and the starting address. The storage area surrounded by the starting address and the ending address is the storage area of ​​the target matrix.

[0039] In one implementation manner, obtaining the starting point address may be directly or indirectly obtained.

[0040] Get it directly, that is, get the starting point address.

[0041] Indirect acquisition, that is, first obtain the tensor offset. Then, based on the starting address of the storage area and the tensor offset, get the starting address. That is, add the storage area and the tensor offset to get the starting address.

[0042] Here, the tensor offset is smaller than the length of the storage region.

[0043] In one implementation, if the number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: N is greater than or equal to M, then the starting point addresses obtained by each target NPU are different from each other.

[0044] Since N is greater than or equal to M, it can be achieved that the starting point address corresponding to each NPU is different, thereby ensuring load balancing.

[0045] Optionally, if the number of addresses of the storage area is N, the number of target NPUs in the multi-NPU system is M; wherein: M is greater than N and less than or equal to 2*N, the starting addresses obtained by the N target NPUs in each target NPU are different from each other, and the starting addresses obtained by the remaining MN target NPUs are also different from each other.

[0046] Since M is greater than N, the addresses corresponding to at most N target NPUs can be different. For the remaining MN target NPUs, the starting addresses obtained by the remaining MN target NPUs are also different. In this way, at most two target NPUs correspond to the same starting address, and load balancing can be guaranteed as much as possible.

[0047] Optionally, if the number of addresses of the storage area is N, the number of target NPUs in the multi-NPU system is M; where: M is greater than 2*N, then the maximum number of target NPUs obtaining the same starting point address is A, the minimum is B, and the maximum difference between A and B is 1.

[0048] S200: Acquire first data stored in a first target address interval.

[0049] The first target address interval is an interval formed from the starting address to the ending address of the storage area.

[0050] After obtaining the storage area and starting address of the target matrix, the target matrix can be obtained according to the starting address. That is, the starting address is used as the starting point for reading the target matrix, and the content of the target matrix is ​​obtained from the storage area in sequence.

[0051] S300: If the starting point address is not the start address of the storage area, after acquiring the first data, acquiring the second data stored in the second target address interval.

[0052] The second target address interval is an interval formed from the start address to the start address of the storage area.

[0053] Since the starting address is not necessarily the starting address of the storage area of ​​the target matrix, the first data stored in the first target address interval may not include all the contents of the target matrix. Therefore, when the starting address is not the starting address of the storage area, after obtaining the first data of the content of the first target address interval, it is also necessary to obtain the second data of the second target address interval. The first data and the second data together form the complete target matrix.

[0054] In one implementation, the hardware may provide an Infinite Memory (ring memory) mechanism to implement data jump acquisition. When the NPU accesses data, if the accessed physical address exceeds the Infinite Memory boundary (that is, the end address of the storage area of ​​the target matrix), the Infinite Memory will redirect the access address to the starting address of the Infinite Memory (that is, the starting address of the storage area of ​​the target matrix). Thus, when the starting address is not the starting address of the storage area, after acquiring the first data, the second data stored in the second target address interval is acquired.

[0055] In one implementation mode, in order to ensure the integrity of the acquired data, the shape of the tensor may be acquired first.

[0056] If the starting address is the starting address of the storage area, after obtaining the first data, it can also be determined whether the first data conforms to the tensor shape. If not, the target matrix is ​​re-acquired.

[0057] If the starting address is not the starting address of the storage area, or after obtaining the first data and the second data, it is also possible to determine whether the first data and the second data conform to the tensor shape based on the tensor shape. If not, the target matrix is ​​re-acquired.

[0058] Among them, the tensor shape is also called tensor shape. The tensor shape describes the size or length of the target matrix in each dimension. The specific meaning and implementation method of the tensor shape are well known to those skilled in the art. For the sake of brief description, they will not be repeated here.

[0059] Since the first data is all the data read from the storage area of ​​the target matrix when the starting address is the starting address of the storage area, it is directly determined whether the shape of the first data (the first data obtained when the starting address is the starting address of the storage area) is the same as the shape of the obtained tensor. If they are the same, it indicates that the first data obtained is the complete target matrix; if they are not the same, it indicates that the first data obtained is not the complete target matrix, so the target matrix needs to be obtained again.

[0060] For example, if the acquired tensor shape represents a matrix of size X0*Y0, and the acquired first data is a matrix of size X1*Y1. Then determine whether X1 is equal to X0, whether Y1 is equal to Y0, as long as any one of them is not equal, it is determined that the tensor shape of the first data is different from the acquired tensor shape. If both are equal, it is determined that the tensor shape of the first data is the same as the acquired tensor shape.

[0061] When the starting address is not the starting address of the storage area, in addition to the first data, the second data is also read. Therefore, it is necessary to combine the first data and the second data to determine whether the complete target matrix is ​​obtained. The first data and the second data can be spliced ​​together (the second data is in front and the first data is in the back). If the tensor shape of the spliced ​​matrix is ​​the same as the tensor shape obtained, it indicates that the first data and the second data obtained are the complete target matrix; if they are not the same, it indicates that the first data and the second data obtained are not the complete target matrix, so the target matrix needs to be re-acquired.

[0062] For example, if the shape of the acquired tensor represents a matrix of size X0*Y0, and the matrix after the acquired first data and second data are concatenated is a matrix of size X1*Y1, then determine whether X1 is equal to X0, and whether Y1 is equal to Y0. As long as any one of them is not equal, it is determined that the acquired first data and second data are not complete target matrices. If both are equal, it is determined that the acquired first data and second data are complete target matrices.

[0063] Alternatively, the tensor shape of the first data and the tensor shape of the second data may be added, and then it is determined whether the tensor shape after the addition is the same as the acquired tensor shape. If they are the same, it indicates that the acquired first data and second data are complete target matrices; if they are not the same, it indicates that the acquired first data and second data are not complete target matrices, and therefore the target matrix needs to be reacquired.

[0064] For example, if the shape of the acquired tensor represents a matrix of size X0*Y0, and the first data acquired is a matrix of size X2*Y2, and the second data is a matrix of size X3*Y3. Then add X2*Y2 and X3*Y3 to get (X2+X3)*(Y2+Y3). Determine whether (X2+X3) is equal to X0, and whether (Y2+Y3) is equal to Y0. As long as any one of them is not equal, it is determined that the first data and the second data acquired are not the complete target matrix. If both are equal, it is determined that the first data and the second data acquired are the complete target matrix.

[0065] In one implementation, after the first data and the second data are acquired, the first data and the second data may be concatenated to obtain a complete target matrix before performing subsequent matrix operations.

[0066] In one implementation, when the target matrix is ​​used to perform a matrix multiplication operation and the target matrix is ​​a multiplicand matrix of the multiplication operation, each row of the target matrix is ​​treated as an array and is sequentially stored in a corresponding storage area.

[0067] Correspondingly, in this case, when each array of the target matrix is ​​obtained, a matrix multiplication operation can be performed based on the obtained array.

[0068] Taking matrix multiplication of matrix A and matrix B as an example, if it is matrix A×matrix B, then matrix A is the multiplicand matrix and matrix B is the multiplier matrix.

[0069] Since matrix multiplication involves multiplying the rows of the multiplicand matrix by the columns of the multiplier matrix, each row of the multiplicand matrix is ​​stored as an array. After obtaining a row of data of the multiplicand matrix, the matrix multiplication calculation can be performed directly without waiting until the complete target matrix is ​​obtained, thereby reducing the time required for calculation.

[0070] In one implementation, when the target matrix is ​​used to perform a matrix multiplication operation and the target matrix is ​​a multiplier matrix of the multiplication operation, each column of the target matrix is ​​treated as an array and is sequentially stored in a corresponding storage area.

[0071] Correspondingly, in this case, when each array of the target matrix is ​​obtained, a matrix multiplication operation can be performed based on the obtained array.

[0072] Since matrix multiplication involves multiplying the rows of the multiplicand matrix by the columns of the multiplier matrix, each column of the multiplier matrix is ​​stored as an array. After obtaining a column of data of the multiplier matrix, calculations can be performed directly without waiting until the complete target matrix is ​​obtained, thereby reducing the time required for calculations.

[0073] In one implementation, the storage area, starting address, and tensor shape of the target matrix are obtained by obtaining a tensor description, that is, the tensor description includes the storage area, starting address (or tensor offset), and tensor shape of the target matrix.

[0074] The tensor description may be sent to the NPU synchronously when a computing task is sent to the NPU.

[0075] In order to facilitate understanding of the above-mentioned data access method applied to a multi-NPU system, the following is an example of explaining the storage area of ​​a target matrix, a tensor offset, and a tensor shape in a tensor description.

[0076] When different NPUs access the target matrix, they configure different tensor offset positions, such as Figure 2 As shown in the figure, the tensor offset of NPU:1 can be set to start from channel 1, and the rest of the tensor description of NPU:1 is the same as the tensor description of other NPUs. In this way, when the NPU accesses the target matrix and finds that the physical address exceeds the Infinite Memory boundary, it redirects the memory access address to the starting position of Infinite Memory. For NPU:2 and NPU:3, the offsets are set to channel 2 and channel 3 respectively. Different channels correspond to different starting addresses.

[0077] This solution also adjusts the calculation order of the target matrix, and accordingly we also need to adjust another calculation matrix in the matrix calculation. However, because the calculation matrix is ​​divided into each NPU and there is no data sharing between them, there is no need to worry about the load balancing problem of the same address access when adjusting the calculation matrix.

[0078] Based on the same technical concept, the present application also provides a data access device applied to a multi-NPU system. The data access device applied to a multi-NPU system is deployed on any target NPU in the multi-NPU system. The target NPU is the NPU in the multi-NPU system that needs to obtain the target matrix, and the multi-NPU system includes multiple target NPUs.

[0079] Among them, the multi-NPU system can be a system composed of multiple NPUs for accelerating the calculation of neural network models. The NPUs in the multi-NPU system can be the same type of NPUs or different types of NPUs. The NPU can be an embedded neural network processor, a neural network processor, a Google TPU, a Horizon BPU, a Cambrian NPU, etc. The types of NPUs are not limited to the aforementioned examples.

[0080] like Figure 3 As shown, the data access device 100 applied to a multi-NPU system includes a first acquisition module 110 and a second acquisition module 120 .

[0081] The first acquisition module 110 is used to acquire the storage area and the starting address of the target matrix; the starting address is located in the storage area, and the starting addresses acquired by at least part of the plurality of target NPUs are different.

[0082] The second acquisition module 120 is used to acquire the first data stored in the first target address interval; the first target address interval is an interval formed from the starting address to the ending address of the storage area.

[0083] The second acquisition module 120 is also used to acquire the second data stored in the second target address interval after acquiring the first data if the starting address is not the starting address of the storage area; the second target address interval is the interval formed from the starting address of the storage area to the starting address.

[0084] The first acquisition module 110 is specifically used to acquire a tensor offset; based on the starting address of the storage area and the tensor offset, the starting address is obtained.

[0085] The first acquisition module 110 is also used to obtain the tensor shape;

[0086] If the starting point address is the starting address of the storage area, after acquiring the first data, the second acquisition module 120 is further used to determine whether the first data conforms to the tensor shape, and if not, reacquire the target matrix;

[0087] If the starting address is not the starting address of the storage area, or after obtaining the first data and the second data, the second acquisition module 120 is also used to determine whether the first data and the second data conform to the tensor shape based on the tensor shape, and if not, re-acquire the target matrix.

[0088] In one implementation, when the target matrix is ​​used to perform a matrix multiplication operation and the target matrix is ​​a multiplicand matrix of the multiplication operation, each row of the target matrix is ​​taken as an array and is sequentially stored in a corresponding storage area. The second acquisition module 120 is further configured to perform a matrix multiplication operation based on each array of the target matrix when the array is acquired.

[0089] In one implementation, when the target matrix is ​​used to perform a matrix multiplication operation and the target matrix is ​​a multiplier matrix of the multiplication operation, each column of the target matrix is ​​taken as an array and is sequentially stored in a corresponding storage area. The second acquisition module 120 is further configured to perform a matrix multiplication operation based on the acquired array when each array of the target matrix is ​​acquired.

[0090] In one implementation, the number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: N is greater than or equal to M, and the starting point addresses obtained by the target NPUs are different from each other.

[0091] In one implementation, the number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: M is greater than N and less than or equal to 2*N, the starting addresses obtained by the N target NPUs in each of the target NPUs are different from each other, and the starting addresses obtained by the remaining MN target NPUs are also different from each other.

[0092] Optionally, if the number of addresses of the storage area is N, the number of target NPUs in the multi-NPU system is M; where: M is greater than 2*N, then the maximum number of target NPUs obtaining the same starting point address is A, the minimum is B, and the maximum difference between A and B is 1.

[0093] The data access device 100 applied to a multi-NPU system provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned data access method embodiment applied to a multi-NPU system. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned data access method embodiment applied to a multi-NPU system.

[0094] See also Figure 4 , which is an electronic device 200 provided in an embodiment of the present application. The electronic device 200 includes: a multi-NPU system 210 and a memory 220.

[0095] The memory 220 and the multi-NPU system 210 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 220 is used to store computer programs, such as Figure 3 The software function module shown in is the data access device 100 applied to the multi-NPU system. Among them, the data access device 100 applied to the multi-NPU system includes at least one software function module that can be stored in the memory 220 in the form of software or firmware or solidified in the operating system (OS) of the electronic device 200. The multi-NPU system 210 is used to execute the executable module stored in the memory 220, such as the software function module or computer program included in the data access device 100 applied to the multi-NPU system. At this time, each NPU in the multi-NPU system 210 is used to obtain the storage area and the starting address of the target matrix; the starting address is located in the storage area, and at least some of the multiple target NPUs obtain different starting addresses; obtain the first data stored in the first target address interval; the first target address interval is the interval formed from the starting address to the end address of the storage area; if the starting address is not the starting address of the storage area, then after obtaining the first data, obtain the second data stored in the second target address interval; the second target address interval is the interval formed from the starting address of the storage area to the starting address.

[0096] Among them, the memory 220 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0097] The multi-NPU system 210 may include a system for accelerating the calculation of a neural network model composed of multiple NPUs. The NPUs in the multi-NPU system may be of the same type or of different types. The NPU may be an embedded neural network processor, a neural network processor, a Google TPU, a Horizon BPU, a Cambrian NPU, etc. The types of NPUs are not limited to the aforementioned examples.

[0098] The electronic device 200 mentioned above includes but is not limited to a personal computer, a server, etc.

[0099] The embodiment of the present application also provides a computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored, and when the computer program is run by a computer such as the above-mentioned electronic device 200, the data access method applied to the multi-NPU system shown above is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0100] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A data access method applied to a multi-NPU system, characterized in that: The multi-NPU system includes a plurality of NPUs, any target NPU in the multi-NPU system applies the data access method, the target NPU is an NPU in the multi-NPU system that needs to obtain a target matrix, and the multi-NPU system includes a plurality of target NPUs; the method includes: Acquire a storage area and a starting point address of the target matrix; the starting point address is located in the storage area, and at least some of the target NPUs have different starting point addresses; Acquire the first data stored in the first target address interval; the first target address interval is the interval formed from the starting address to the ending address of the storage area; If the starting address is not the starting address of the storage area, after acquiring the first data, the second data stored in the second target address interval is acquired; the second target address interval is the interval from the starting address of the storage area to the starting address.

2. The method according to claim 1, characterized in that Obtaining the starting point address includes: Get tensor offset; The starting point address is obtained based on the starting address of the storage area and the tensor offset.

3. The method according to claim 1, characterized in that The method further comprises: Get the shape of a tensor; If the starting point address is the starting address of the storage area, after acquiring the first data, the method further includes: Determine whether the first data conforms to the tensor shape, and if not, reacquire the target matrix; If the starting point address is not the starting address of the storage area, or after the first data and the second data are obtained, the method further includes: Based on the tensor shape, determine whether the first data and the second data conform to the tensor shape; if not, reacquire the target matrix.

4. The method according to claim 1, characterized in that: When the target matrix is ​​used to perform a matrix multiplication operation and the target matrix is ​​a multiplicand matrix of the multiplication operation, each row of the target matrix is ​​taken as an array and is sequentially stored in a corresponding storage area.

5. The method according to claim 4, characterized in that The method further comprises: When each array of the target matrix is ​​obtained, a matrix multiplication operation is performed based on the obtained array.

6. The method according to claim 1, characterized in that When the target matrix is ​​used for matrix multiplication operation and the target matrix is ​​a multiplier matrix of the multiplication operation, each column of the target matrix is ​​regarded as an array and is sequentially stored in a corresponding storage area.

7. The method according to claim 6, characterized in that The method further comprises: When each array of the target matrix is ​​obtained, a matrix multiplication operation is performed based on the obtained array.

8. The method according to claim 1, characterized in that The number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: The N is greater than or equal to the M, and the starting point addresses acquired by the target NPUs are different from each other.

9. The method according to claim 1, characterized in that: The number of addresses of the storage area is N, and the number of target NPUs in the multi-NPU system is M; wherein: The M is greater than the N and less than or equal to 2*N, the starting point addresses acquired by the N target NPUs in each of the target NPUs are different from each other, and the starting point addresses acquired by the remaining MN target NPUs are also different from each other.

10. A data access device applied to a multi-NPU system, characterized in that: The device is deployed on any target NPU in a multi-NPU system, where the target NPU is an NPU in the multi-NPU system that needs to obtain a target matrix, and the multi-NPU system includes multiple target NPUs; the device includes: A first acquisition module is used to acquire a storage area and a starting point address of the target matrix; the starting point address is located in the storage area, and at least some of the target NPUs have different starting point addresses; A second acquisition module is used to acquire the first data stored in a first target address interval; the first target address interval is an interval formed from the starting address to the ending address of the storage area; The second acquisition module is also used to acquire the second data stored in the second target address interval after acquiring the first data if the starting address is not the starting address of the storage area; the second target address interval is the interval formed from the starting address of the storage area to the starting address.

11. An electronic device, characterized in that: include: A memory and a multi-NPU system, wherein the memory and the multi-NPU system are connected; The memory is used to store programs; Each NPU in the multi-NPU system is used to call the program stored in the memory to execute the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 9 is executed.

Images