Data synchronization method and chip in DLA architecture

Abstract

The application discloses a data synchronization method and chip in a DLA architecture. The method comprises the following steps: the DLA comprises a bottommost computing layer and at least one layer of carrying layers; the computing layer comprises at least one computing core; a first carrying layer adjacent to the computing layer comprises at least one first DMA corresponding to the computing core and a first cache area; the first DMA slices data in a previous storage area according to a preset order, a preset span and a preset size, and carries a first slice after slicing to the first cache area; after the first slice carrying is completed, the first DMA sends a first carrying completion signal to the computing core; after the computing core receives at least one first carrying completion signal, the computing core loads the first slice from the at least one first cache area; and the computing core calculates data in the loaded first slice. According to the data flow description of a computing operator, the application realizes data carrying and data calculation synchronization through the cooperation of the DMA and the computing core, and generates an efficient data flow.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning computing, and in particular to a data synchronization method in a DLA architecture and a chip. BACKGROUND

[0002] With the maturity and popularity of deep learning technology, the demand for high energy efficiency computing power is increasing. Deep learning computing, as an emerging technical field, often has strong regularity in its common computing mode and memory access form. Therefore, in recent years, the DLA (Deep Learning Accelerator) architecture for deep learning computing optimization has developed rapidly.

[0003] A popular DLA architecture uses multi-level buffer caching. Compared with the traditional CPU using multi-level cache caching architecture, this architecture is simpler to implement, has better energy efficiency, and can use more on-chip memory per unit area, and has strong competitiveness in the computing mode of deep learning. In this architecture design, in order to improve the utilization rate of the computing core and reduce the idle time of the computing core due to non-core computing logic, a dedicated DMA (Direct Memory Access) unit is often used to transfer data between buffer caches.

[0004] Deep learning computing is usually based on matrix and tensor for computing and memory access, but the on-chip cache cannot store the required matrix or tensor of an operator (such as matrix multiplication and convolution calculation). Therefore, how the DMA unit efficiently transfers data from the off-chip memory (such as DDR memory) with low bandwidth and far from the computing core to the on-chip memory (such as buffer cache) with high bandwidth and close to the computing core, while maintaining high utilization of on-chip resources and ensuring data integrity and correctness, is a technical problem that needs to be solved. SUMMARY

[0005] The present application provides a data synchronization method in a DLA architecture and a chip, which generates an efficient data stream according to a computing operator.

[0006] According to an aspect of the present application, a data synchronization method in a DLA architecture is provided, the DLA has at least two layers of structure, including a computing layer at the bottom and at least one transfer layer; the computing layer includes at least one computing core; the first transfer layer adjacent to the computing layer includes at least one first DMA corresponding to the computing core and at least one first-level cache area;

[0007] The first DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and carries the first slice after slicing to a first cache area;

[0008] After the first slice carrying is completed, the first DMA sends a first carrying completion signal to the computing core;

[0009] After the computing core receives the corresponding at least one first carrying completion signal, the computing core loads the first slice from the corresponding at least one first cache area;

[0010] The computing core calculates the data in the first slice after loading.

[0011] Optionally, when N layers of carrying layers are included, the Nth carrying layer includes an Nth DMA and an N-level cache area;

[0012] The Nth DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and carries the Nth slice after slicing to the N-level cache area, the upper-level storage area being an external storage area or an N+1-level cache area;

[0013] After the Nth DMA completes carrying the Nth slice, an Nth carrying completion signal is sent to the corresponding N-1th DMA;

[0014] After the N-1th DMA receives the corresponding Nth carrying completion signal, the Nth slice in the N-level cache area is sliced in a preset order with a preset span and a preset size, and the N-1th slice after slicing is carried to an N-1-level cache area;

[0015] After the Nth slice in the N-level cache area is completely carried, the N-1th DMA sends an N-1th carrying completion signal to the corresponding Nth DMA;

[0016] After the Nth DMA receives all the N-1th carrying completion signals sent by the N-1th DMA corresponding to the Nth DMA, the next Nth slice is carried from the upper-level storage area to the N-level cache area in the preset order.

[0017] Optionally, when N layers of carrying layers are included, the Nth carrying layer includes an Nth DMA and an N-level cache area;

[0018] The Nth DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and carries the Nth slice after slicing to the N-level cache area, the upper-level storage area being an external storage area or an N+1-level cache area;

[0019] After the Nth DMA finishes carrying the aM+bth Nth slice, an Nth carrying completion signal is sent to the corresponding bth N-1th DMA, and the next Nth slice is continuously carried, the Nth DMA corresponds to M N-1th DMAs, a is an integer greater than or equal to 0, b = 1, 2, … M;

[0020] After the bth N-1th DMA receives the Nth carrying completion signal, the corresponding Nth slice in the N-level cache area is sliced in a preset order with a preset span and a preset size, and the sliced N-1th slice is carried to an N-1-level cache area;

[0021] After the bth N-1th DMA finishes carrying the corresponding Nth slice, an N-1th carrying completion signal is fed back to the Nth DMA;

[0022] When the Nth DMA finishes carrying the aMth Nth slice and receives the N-1th carrying completion signal fed back by the first N-1th DMA, the Nth DMA continues to carry the aM+1th Nth slice;

[0023] After the Nth DMA finishes carrying the aM+1th Nth slice, an Nth carrying completion signal is sent to the corresponding first N-1th DMA, and the next Nth slice is continuously carried; after the Nth DMA receives the N-1th carrying completion signal fed back by the N-1th DMA corresponding to the last Nth slice in the N+1th slice, the Nth DMA carries the next N+1th slice from the upper-level storage area to the N-level cache area in the preset order.

[0024] Optionally, when N layers of carrying layers are included, the Nth carrying layer includes an Nth DMA and an N-level cache area;

[0025] When the Nth DMA is configured in parallel, the Nth DMA slices data in the upper-level storage area in a preset order with a preset span and a preset size, and carries the sliced Nth slice to the N-level cache area, the upper-level storage area being an external storage area or an N+1-level cache area;

[0026] After the Nth DMA finishes carrying the Nth slice, an Nth carrying completion signal is sent to the corresponding N-1th DMA, and the next Nth slice is continuously carried to the N-level cache area in the preset order;

[0027] After the N-1th DMA receives the Nth carrying completion signal, the Nth slice in the N-level cache area is sliced in a preset order with a preset span and a preset size, and the sliced N-1th slice is carried to an N-1-level cache area.

[0028] Optionally, after the computing core calculates the data in the first slice after loading, the method further comprises:

[0029] The computing core sends the data after calculation to the first cache area, so that the data is merged into a first merged slice in the first cache area.

[0030] After merging into the first merged slice, the computing core sends a first merging completion signal to the first DMA.

[0031] After the first DMA receives the first merging completion signal, the first DMA carries the first merged slice in the first cache area to a next storage area and merges the data into a second merged slice in the next storage area.

[0032] Optionally, when N layers of carrying layers are included, the Nth carrying layer includes an Nth DMA and an Nth cache area.

[0033] When the Nth DMA receives an Nth merging completion signal sent by the (N-1)th DMA, the Nth DMA carries an Nth merged slice in the Nth cache area to a next storage area and merges the data in the next storage area, the next storage area being an external storage area or an (N+1)th cache area.

[0034] Optionally, before the first DMA slices the data in the previous storage area according to the preset order, the preset span and the preset size, and carries the first slice to the first cache area, the method further comprises:

[0035] According to a data flow description of a preset operator, a processing node of a data flow in a DLA and a data flow direction in the processing node are obtained, the processing node including the Nth DMA and the computing core, and the data flow direction being a data carrying direction between the processing nodes.

[0036] A computation graph is determined according to the processing node and the data flow direction.

[0037] Optionally, after the computation graph is determined according to the processing node and the data flow direction, the method further comprises:

[0038] The Nth DMAs in the same carrying layer in the computation graph, which are configured identically and carry the same source data, are merged, the source data being tensor data in an operator.

[0039] Optionally, after the Nth DMAs in the same carrying layer in the computation graph, which are configured identically and carry the same source data, are merged, the method further comprises:

[0040] After traversing the computation graph, memory spaces required by the Nth DMA and the computation core are obtained;

[0041] According to the memory spaces required by the Nth DMA and the computation core, corresponding memory spaces are respectively allocated to the Nth DMA and the computation core.

[0042] Optionally, the method further comprises:

[0043] The Nth DMA carries data in a corresponding position from an address region closest to a physical position of the Nth DMA in an upper level storage area;

[0044] The Nth DMA carries data to an N-level cache area closest to a physical position of the Nth DMA.

[0045] According to another aspect of the present application, a chip is provided for performing a data synchronization method in a DLA architecture as described in the above aspect; wherein the chip comprises a DLA, the DLA is at least two layers of structure, the DLA comprises a computation layer at the bottom and at least one carrying layer; the computation layer comprises at least one computation core; a first carrying layer adjacent to the computation layer comprises at least one first DMA corresponding to the computation core and at least one first-level cache area.

[0046] The technical solution of the embodiment of the present application, after the DLA reads an operator, according to the data stream description of the operator, the data is carried by the DMA and the data is calculated by the computation core to realize the cooperative operation of the DMA and the computation core, realize the synchronization of data carrying and data calculation, and thus generate an efficient data stream. That is, in the embodiment of the present application, the DMA unit carries the data in the off-chip memory with low bandwidth and far away from the computation core to the on-chip memory with high bandwidth in a regular slice or block form, and repeatedly synchronizes with the computation core and other DMA units in the process, ensuring the integrity and correctness of the data, and improving the parallelism of the computation core and other DMA units, and improving the overall utilization of each unit.

[0047] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is a flow chart of a data synchronization method in a DLA architecture according to an embodiment of the present application;

[0050] Figure 2 is a method flow chart of a data synchronization method in a DLA architecture according to an embodiment of the present application;

[0051] Figure 3 is a flow chart of a data synchronization method in a DLA architecture according to an embodiment of the present application;

[0052] Figure 4 is a flow chart of a data synchronization method in a DLA architecture according to an embodiment of the present application;

[0053] Figure 5 is a flow chart of a data synchronization method in a DLA architecture according to an embodiment of the present application;

[0054] Figure 6 is a schematic diagram of parallel transfer between DMAs according to an embodiment of the present application;

[0055] Figure 7 is a flow chart of constructing a computation graph according to an embodiment of the present application;

[0056] Figure 8 is a schematic diagram of collaborative generation of efficient data flow between DMAs and between DMAs and computation cores according to an embodiment of the present application;

[0057] Figure 9 is a structural schematic diagram of a DLA in an embodiment of the present application;

[0058] Figure 10 is a schematic diagram of a computation graph according to an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person of ordinary skill in the art without creative labor should belong to the scope of protection of the present application.

[0060] It is to be understood that the terminology "first", "second" and the like used in the specification and the claims of the application as well as the foregoing drawings is merely intended to distinguish between similar objects and not necessarily to describe a particular sequential order. It is to be understood that the use of such terms can be interchanged in suitable instances to refer to a similar embodiment. Furthermore, the terms "comprise" and "include", and variations thereof, do not intend to exclude other technical features but to encompass the technical features listed in the claims and equivalents thereof. The technical features of the embodiments of the application described herein can be implemented in software and / or hardware. The embodiments of the application can be implemented in one or more computer systems or other processing systems.

[0061] Embodiment one

[0062] Figure 1 A flow chart of a data synchronization method in a DLA architecture provided for embodiment one of the application is shown in Figure 1 The DLA is a structure of at least two layers, including a computing layer at the bottom and at least one transport layer adjacent to the computing layer; the computing layer includes at least one computing core; the first transport layer adjacent to the computing layer includes at least one first DMA corresponding to the computing core and at least one first cache area, and the method includes:

[0063] 101. The first DMA slices the data in the upper storage area according to a preset order, a preset span and a preset size, and transports the first slice after slicing to the first cache area.

[0064] The first DMA and the first cache area can be in a one-to-one correspondence, and the first DMA can transport the first slice after slicing to the corresponding first cache area.

[0065] Specifically, the deep learning accelerator generally includes a multi-level cache structure, and each layer of cache is connected through a direct memory access unit DMA for data transport. The bottom layer of the deep learning accelerator is a computing core, which is used to read data in the adjacent cache and then perform calculation on the read data. In this embodiment, the computing core and the first DMA in the transport layer adjacent to the computing core can be used to form a calculation description of an operator. For example, for the calculation behavior C=A+A, the first DMA can transport the data A to the first cache area, and then the computing core can perform addition calculation on the data A in the first cache area, wherein the data A is generally tensor data.

[0066] In the actual data carrying process, there is a certain information interaction between the first DMA and the computing core, so that the first DMA and the computing core cooperate to produce efficient data flow. In this embodiment, the first DMA can slice the data in the upper storage area by a preset size, and carry the first slice after slicing to the first cache area. The upper cache area can be an external storage area or a second cache area in the carrying layer adjacent to the carrying layer where the first DMA is located; the first DMA can slice the tensor data in the second cache area or the external storage area, and the slicing method can be from top to bottom and from left to right in sequence, or from left to right and from top to bottom in sequence, and the embodiment does not make specific limitation on the slicing method. After slicing each piece of tensor data, the piece of data is carried from the second cache area or the external cache to the first cache area; the first DMA can slice according to a preset size, for example, data A is 64*64, and the first DMA can slice data A according to the size of 8*8.

[0067] 102、After the first slice carrying is completed, the first DMA sends a first carrying completion signal to the computing core.

[0068] 103、After the computing core receives the corresponding at least one first carrying completion signal, the computing core loads the first slice from the corresponding at least one first cache area.

[0069] 104、The computing core calculates the data in the loaded first slice.

[0070] Specifically, after the first DMA carries a piece of sliced data, the first DMA can send a first carrying completion signal to the computing core, notifying the computing core that the carrying of the first slice has been completed. After the computing core receives the first carrying completion signal sent by the first DMA, the computing core can load the first slice data from the first cache area for calculation. That is, in the actual carrying process, the first DMA will send a first carrying completion signal to the computing core after carrying the first slice each time, so that the computing core can continuously load the first slice from the first cache area and calculate the data of the first slice, thereby producing efficient data flow.

[0071] The embodiment of the present application can distribute the computing core and the plurality of first DMAs corresponding to the computing core according to the form of the operator. For example, when an operator such as C=A+B needs to be calculated, one computing core and two first DMAs corresponding to the computing core can be distributed to process the operator, that is, one first DMA is used to carry data A, and the other first DMA is used to carry data B; after one first DMA carries data A to the corresponding first cache area and the other first DMA carries data B to the corresponding first cache area, both the first DMAs send the first carrying completion signal to the computing core; after the computing core receives the first carrying completion signal sent by the corresponding two first DMAs, the computing core can load data A and data B from the two first cache areas respectively to perform calculation.

[0072] The technical scheme of the embodiment of the present application realizes the cooperative operation of the DMA and the computing core by using the DMA to carry data and the computing core to calculate data after the DLA reads the operator according to the data flow description of the operator, realizes the synchronization of the efficient carrying of data and the calculation of data, and thus generates an efficient data flow.

[0073] Embodiment two

[0074] Figure 2 is a method flow chart of a data synchronization method in a DLA architecture according to the embodiment two of the present application; applied to the case that the DLA only contains the bottommost computing layer and one carrying layer. The computing layer includes at least one computing core; the first carrying layer adjacent to the computing layer includes at least one first DMA corresponding to the computing core and at least one first cache area, and the method includes:

[0075] 201, the first DMA slices the data in the upper storage area according to the preset order, the preset span and the preset size, and carries the first slice after slicing to the first cache area.

[0076] 202, after the first slice is carried, the first DMA sends the first carrying completion signal to the computing core.

[0077] 203, after the computing core receives the at least one first carrying completion signal corresponding thereto, the computing core loads the first slice from the at least one first cache area corresponding thereto.

[0078] 204, the computing core calculates the data in the loaded first slice.

[0079] 205, the computing core sends the data after calculation to the first cache area, so that the data is merged into a first merged slice in the first cache area.

[0080] 206、After the first merged slice is merged, the computing core sends a first merged completion signal to the first DMA.

[0081] 207、After the first DMA receives the first merged completion signal, the first merged slice in the first cache area is moved to the next storage area, and the second merged slice is merged in the next storage area.

[0082] Specifically, after the computing core completes the calculation, it sends the data after the calculation to the first cache area, so that the data is merged into the first merged slice in the first cache area. After the first merged slice is merged, the computing core sends a first merged completion signal to the first DMA. After the first DMA receives the first merged completion signal, the first merged slice in the first cache area is moved to the next storage area. The first merged slice is merged in the next cache area in the preset size and preset order in the next storage area, and the second merged slice is merged. The second cache area in the next cache area is the second cache area in the second moving layer or the external cache. And the merging order in the next storage area can be determined according to the order in which the first DMA moves the data in the upper storage area to the first cache area. The size of the moved tensor data can also be determined according to the size of the data in the upper storage area moved by the first DMA to the first cache area.

[0083] On the one hand, the embodiment can realize the cooperative operation of the DMA and the computing core by using the DMA to move data and the computing core to calculate data, realize the synchronization of efficient data movement and data calculation, and thus generate efficient calculation data flow. On the other hand, the data is calculated by the computing core, the calculated data is merged and notified to the DMA, and then the data is moved by the DMA to realize efficient output of the data and generate efficient output data flow.

[0084] Embodiment three

[0085] Figure 3 A flowchart for data synchronization between the DMA and the DMA in the data synchronization method in the DLA architecture is provided for the embodiment three of the application. It is applied to the case where the DLA includes N moving layers and the bottommost computing layer, N is a positive integer, and the Nth moving layer includes the Nth DMA and the N-level cache area. As shown in Figure 3

[0086] 301、The Nth DMA slices the data in the upper storage area according to the preset order, the preset span and the preset size, and moves the Nth slice after slicing to the N-level cache area. The upper storage area is an external storage area or an N+1-level cache area.

[0087] ​Specifically, when N layers of carrying layers and the bottom layer of the calculation layer are included in the deep learning accelerator, the DMAs in adjacent carrying layers synchronize the carrying information. The Nth DMA slices the data in the upper storage area in a preset order with a preset span and a preset size, and carries the Nth slice after slicing to the N-level cache area. The upper storage area can be an external storage area or an N+1-level cache area. The Nth DMA can slice the data in the external storage area or the N+1-level cache area. The slicing method can be from top to bottom and from left to right in sequence, or from left to right and from top to bottom in sequence. After slicing each piece of tensor data, the piece of data is carried from the N+1-level cache area or the external cache to the N-level cache area. The Nth DMA can slice according to a preset size. For example, if the data A is 64*64, the Nth DMA can slice the data A according to the size of 8*8.

[0088] 302、After the Nth DMA finishes carrying the Nth slice, the Nth carrying completion signal is sent to the corresponding N-1th DMA.

[0089] Specifically, because the number of DMAs performing carrying operations in each layer of carrying layers is different, one DMA in the upper layer of carrying layers corresponds to multiple DMAs in the lower layer of carrying layers, or multiple DMAs in the upper layer of carrying layers correspond to one DMA in the lower layer of carrying layers. Therefore, after the Nth DMA finishes carrying the Nth slice, the Nth carrying completion signal is sent to the corresponding N-1th DMA. One Nth DMA can send the Nth carrying completion signal to one or more corresponding N-1th DMAs, or any one of multiple Nth DMAs can finish carrying the Nth slice, and then send the Nth carrying completion signal to the same N-1th DMA.

[0090] 303、After the N-1th DMA receives the corresponding Nth carrying completion signal, the Nth slice in the N-level cache area is sliced in a preset order with a preset span and a preset size, and the N-1th slice after slicing is carried to the N-1-level cache area.

[0091] Specifically, when the Nth DMA receives all the Nth transport completion signals sent by the corresponding N-1th DMAs, the Nth slice in the Nth cache area is sliced in the preset order with the preset span and the preset size, and the sliced N-1th slice is transported to the N-1th cache area. When one Nth DMA corresponds to one or more N-1th DMAs, the Nth DMA can send the Nth transport completion signal to the corresponding one or more N-1th DMAs at the same time, so that the N-1th DMAs synchronously perform the data transport of the Nth slice in the Nth cache area; when the N-1th DMA corresponds to multiple Nth DMAs, the multiple Nth DMAs all send the Nth transport completion signal to the N-1th DMA, and when the N-1th DMA receives all the Nth transport completion signals sent by the corresponding Nth DMAs, the Nth slice in the Nth cache area is sliced in the preset order with the preset span and the preset size, and the sliced N-1th slice is transported to the N-1th cache area.

[0092] 304、When all the Nth slices in the Nth cache area are transported, the N-1th DMA sends the N-1th transport completion signal to the corresponding Nth DMA.

[0093] 305、When the Nth DMA receives all the N-1th transport completion signals sent by the N-1th DMAs corresponding to the Nth DMA, the next Nth slice is transported from the upper storage area to the Nth cache area in the preset order.

[0094] Specifically, when one Nth DMA corresponds to one or more N-1th DMAs, if the multiple N-1th DMAs all transport the Nth slice in the Nth cache area, the multiple N-1th DMAs all send the N-1th transport completion signal to the corresponding Nth DMA, and when the Nth DMA receives all the N-1th transport completion signals sent by the corresponding N-1th DMAs, the next Nth slice is transported from the upper storage area to the Nth cache area in the preset order; when the multiple Nth DMAs correspond to one N-1th DMA, the N-1th DMA can send the N-1th transport completion signal to the corresponding multiple Nth DMAs, and when the Nth DMA receives the N-1th transport completion signal sent by the corresponding N-1th DMA, the next Nth slice is transported from the upper storage area to the Nth cache area in the preset order.

[0095] Figure 4 Another flowchart for data synchronization between DMAs in a data synchronization method in a DLA architecture is provided for the third embodiment of the application, which is applied to the case that the DLA includes N transport layers and a bottommost calculation layer, wherein the Nth transport layer includes an Nth DMA and an Nth cache area. As shown in Figure 4

[0096] ​401、the Nth DMA slices data in the upper storage area in a preset order with a preset span and a preset size, and carries the Nth slice after slicing to the N-level cache area, and the upper storage area is an external storage area or an N+1-level cache area.

[0097] Specifically, when the N-layer carrying layer is included in the deep learning accelerator and the calculation layer at the bottommost layer, the DMA in the adjacent carrying layer synchronizes the carrying information. The Nth DMA slices data in the upper storage area in a preset order with a preset span and a preset size, and carries the Nth slice after slicing to the N-level cache area, and the upper storage area can be an external storage area or an N+1-level cache area; the Nth DMA can slice data in the external storage area or the N+1-level cache area, and the slicing manner can be from top to bottom and from left to right in sequence, or from left to right and from top to bottom in sequence; in addition, the slicing manner can be continuous slicing, that is, the data flow between adjacent slices is continuous; the slicing manner can also be slicing with edge repetition, for example, the intersection of adjacent slices has overlapping data; the slicing manner can also be slicing with a gap, that is, after each slice is cut, the next slice is cut after skipping a certain amount of data, and the embodiment of the application does not limit the slicing manner. After each slice of tensor data is cut, the slice of data is carried from the N+1-level cache area or the external cache to the N-level cache area; the Nth DMA can slice according to a preset size, for example, if the data A is 64*64, the Nth DMA can slice the data A according to the size of 8*8.

[0098] 402、after the Nth DMA carries the aM+bth Nth slice, the Nth carrying completion signal is sent to the corresponding bth N-1 DMA, and the next Nth slice is continued to be carried, the Nth DMA corresponds to M N-1 DMAs, a is an integer greater than or equal to 0, b and M are positive integers, b=1, 2, …M.

[0099] Specifically, when one Nth DMA corresponds to multiple N-1 DMAs, after the Nth DMA carries the first Nth slice, the Nth carrying completion signal is sent to the corresponding first N-1 DMA; if there are M N-1 DMAs, when the Nth DMA carries the M+1th Nth slice, the Nth DMA will send the Nth carrying completion signal to the corresponding first N-1 DMA, that is, when the Nth DMA carries the Xth Nth slice, the Nth DMA will send the Nth carrying completion signal to the corresponding Xth %M N-1 DMA, % is the remainder symbol. After the Nth DMA carries the aM+bth Nth slice, the Nth carrying completion signal is sent to the corresponding bth N-1 DMA, and the aM+b+1th Nth slice is continued to be carried.

[0100] 403、the bth N-1 DMA receives the Nth carrying completion signal, slices the corresponding Nth slice in the Nth cache area according to the preset order, with a preset span and a preset size, and carries the sliced N-1th slice to the N-1 cache area.

[0101] Specifically, when the bth N-1 DMA receives the Nth carrying completion signal, the corresponding aM+bth Nth slice in the Nth cache area is sliced according to the preset order, with a preset span and a preset size, and the sliced N-1th slice is carried to the N-1 cache area; wherein the Nth DMA sends the Nth carrying completion signal to the corresponding bth N-1 DMA after carrying the aM+bth Nth slice, the bth N-1 DMA carries the aM+bth Nth slice, the N-1 DMA corresponds to the Nth slice, and the Nth DMA carries the next Nth slice according to the preset order after carrying one Nth slice each time, so that the Nth DMA corresponding to multiple N-1 DMAs synchronously carries, and each N-1 DMA carries different Nth slice. It should be noted that the Nth DMA can continue to carry the aM+1th Nth slice after receiving the N-1th carrying completion signal fed back by the first N-1 DMA, and the Nth DMA sends the Nth carrying completion signal to the first N-1 DMA corresponding to the Nth DMA after carrying the aM+1th Nth slice, at this time, the first N-1 DMA continues to carry the aM+1th Nth slice after carrying the 1th Nth slice.

[0102] 404、the bth N-1 DMA carries the corresponding Nth slice, and feeds back the N-1th carrying completion signal to the Nth DMA.

[0103] 405、when the Nth DMA carries the aMth Nth slice and receives the N-1th carrying completion signal fed back by the first N-1 DMA, the Nth DMA continues to carry the aM+1th Nth slice.

[0104] 406、after the Nth DMA carries the aM+1th Nth slice, the Nth carrying completion signal is sent to the first N-1 DMA corresponding to the Nth DMA, and the next Nth slice is continued to be carried.

[0105] Specifically, when the Nth DMA finishes carrying the aMth N-slice and receives the N-1th carrying completion signal of the first N-1th DMA feedback, the Nth DMA continues to carry the aM+1th N-slice; after the Nth DMA finishes carrying the aM+1th N-slice, the Nth carrying completion signal is sent to the corresponding 1st N-1th DMA, and the next N-slice is continued to be carried. At this time, the 1st N-1th DMA starts to carry the aM+1th N-slice after receiving the Nth carrying completion signal, so as to realize the data synchronization between the Nth DMA and the corresponding N-1th DMA.

[0106] 407、The Nth DMA receives the N-1th carrying completion signal of the last N-slice in the N+1th slice carried by the Nth DMA, and carries the next N+1th slice from the upper storage area to the N-level cache area in the preset order.

[0107] Specifically, when the Nth DMA finishes carrying the aMth N-slice and receives the N-1th carrying completion signal of the first N-1th DMA feedback, the Nth DMA continues to carry the aM+1th N-slice; after the Nth DMA finishes carrying the aM+1th N-slice, the Nth carrying completion signal is sent to the corresponding 1st N-1th DMA, and the next N-slice is continued to be carried. At this time, the 1st N-1th DMA starts to carry the aM+1th N-slice after receiving the Nth carrying completion signal, so as to realize the data synchronization between the Nth DMA and the corresponding N-1th DMA.

[0108] Figure 5 A third flowchart for data synchronization between DMA and DMA in a data synchronization method in a DLA architecture is provided for the third embodiment of the application, which is applied to the case that the DLA includes N layers of carrying layers and the bottommost calculation layer, wherein the Nth carrying layer includes the Nth DMA and the N-level cache area. As shown in Figure 5

[0109] 501、When the Nth DMA is configured in parallel, the Nth DMA slices the data in the upper storage area in the preset order, with a preset span and a preset size, and carries the sliced N-slice to the N-level cache area. The upper storage area is an external storage area or an N+1th cache area.

[0110] 502、After the Nth DMA finishes carrying the N-slice, the Nth carrying completion signal is sent to the corresponding N-1th DMA, and the next N-slice is continued to be carried to the N-level cache area in the preset order.

[0111] ​Specifically, when the Nth DMA contains a parallel configuration, the Nth DMA slices the data in the upper-level storage area in a preset order with a preset stride and a preset size, and carries the Nth slice after slicing to the N-level cache area, and the upper-level storage area is an external storage area or an N+1-level cache area; after the Nth DMA carries the Nth slice, the Nth DMA sends an Nth carrying completion signal to the corresponding N-1th DMA. At this time, the Nth DMA does not need to wait for the N-1th DMA to feed back an N-1th carrying completion signal, but continues to carry the next Nth slice to the N-level cache area in a preset order, and every time an Nth slice is carried, the Nth DMA sends an Nth carrying completion signal to the corresponding N-1th DMA, so that the Nth DMA and the N-1th DMA can carry in parallel. When the Nth DMA contains a parallel configuration, the parallel manner in which the Nth DMA carries data is exemplarily shown in Figure 6 Figure 6 is a schematic diagram of parallel carrying between DMAs according to Embodiment Three of the present application.

[0112] 503、After the N-1th DMA receives the Nth carrying completion signal, the N-1th DMA slices the Nth slice in the N-level cache area in a preset order with a preset stride and a preset size, and carries the N-1th slice after slicing to the N-1-level cache area.

[0113] Specifically, after the N-1th DMA receives the Nth carrying completion signal, the N-1th DMA slices the Nth slice in the N-level cache area in a preset order with a preset stride and a preset size, and carries the N-1th slice after slicing to the N-1-level cache area, and after the N-1th DMA carries the corresponding Nth slice, the N-1th DMA can continue to carry the next Nth slice in the next N-level cache area only after the N-1th DMA receives the Nth carrying completion signal corresponding to the next Nth slice sent by the Nth DMA. If the N-1th DMA has already received a new Nth carrying completion signal, the N-1th DMA can continue to carry the next Nth slice without waiting.

[0114] In a specific embodiment, when the DLA includes N layers of carrying layers and a bottommost calculation layer, wherein the Nth carrying layer includes an Nth DMA and an N-level cache area.

[0115] After the Nth DMA receives the Nth merging completion signal sent by the corresponding N-1th DMA, the Nth DMA carries the Nth merging slice in the N-level cache area to the next-level storage area, and merges data in the next-level storage area, and the next-level storage area is an external storage area or an N+1-level cache area.

[0116] ​Specifically, when the DLA includes N layers of carrying layers and the bottom layer of the computing layer, for data carrying between adjacent carrying layers, first, when the Nth DMA receives the Nth merge completion signal sent by the N-1th DMA, the Nth DMA carries the Nth merge slice in the N-level cache area to the next level storage area and merges the data in the next level storage area. The merging order of the Nth merge slice in the N-level cache area can be from top to bottom and from left to right, or from left to right and from top to bottom, and the merging order in the next level storage area can also be determined according to the order of carrying the data in the previous level storage area to the N-level cache area by the Nth DMA, and the size of the tensor data carried can also be determined according to the size of the data carried from the previous level storage area to the N-level cache area by the Nth DMA. The next level storage area is an external storage area or an N+1-level cache area.

[0117] In one specific embodiment, the exemplary, Figure 7 is a flow chart for constructing a graph according to the third embodiment of the present application, as Figure 7 shown, further comprising:

[0118] 701. According to the data flow description of the preset operator, the processing nodes of the data flow in the DLA and the data flow direction in the processing nodes are obtained, the processing nodes include the Nth DMA and the computing core, and the data flow direction is the data carrying direction between the processing nodes.

[0119] Specifically, the general operator includes matrix multiplication, convolution, element-by-element multiplication and other calculation modes, and such calculation modes have regularity, which includes: a calculation memory access mode based on structured loop nesting, and the range of the loop is fixed; the operation object, i.e., the input and output of the operator, is a multi-dimensional tensor; and the spatial and temporal locality of tensor access can be utilized by slicing.

[0120] Due to the following characteristics of the DLA architecture, for example, Figure 9 is a structural diagram of the DLA in the embodiment of the present application, as Figure 9 shown: multi-level buffer cache, different cache sizes, different bandwidths, and different shared access computing units; data carrying between multi-level buffer caches needs to pass through DMA units, DMA carrying can be performed in parallel with the computing unit; DMA units and DMA and computing units can be synchronized through hardware synchronization resources.

[0121] Therefore, according to the description of the data flow of the operator, the DLA can allocate the corresponding DMA and the computing core for the computing operator, and realize the hardware implementation of the operator. For example, when the operator is C=A+B, the corresponding DMA in the multi-layer adjacent carrying layer can be used to carry the data A, the corresponding DMA in another multi-layer adjacent carrying layer can be used to carry another data B, and the calculation of A+B can be completed in the computing core after the carrying is completed; and if the calculation includes A+B and A*C, the corresponding DMA in the multi-layer adjacent carrying layer can be used to carry the data A, the corresponding DMA in another multi-layer adjacent carrying layer can be used to carry another data B, and the corresponding DMA in the multi-layer adjacent carrying layer can be used to carry the data C, and the operator of A+B and A*C can be calculated in the computing core after the data A, B, and C are carried.

[0122] 702. Determine the computing graph according to the processing nodes and the data flow.

[0123] Specifically, after obtaining the processing nodes of the data flow, the computing graph can be constituted by the corresponding relationship of the DMA in different carrying layers and the computing core corresponding to the DMA.

[0124] 703. Merge the Nth DMA in the same carrying layer and configured with the same and carrying the same source data in the computing graph, and the source data is the tensor data in the operator.

[0125] Specifically, the DLA can merge the Nth DMA in the same carrying layer and configured with the same and carrying the same source data in the computing graph. For example, for the operator C=A+A, if the data carried by the DMA is all data A, the DMA carrying the data A can be merged, that is, after the corresponding DMA in the multi-layer adjacent carrying layer is used to carry the data A, the computing core performs the addition operation on the data A; or when the operator is A+B and A*C, the DMAs carrying A can be merged, that is, only the corresponding DMA in the multi-layer adjacent carrying layer is used to carry the data A, and the computing core calls the data A to perform addition and multiplication calculation, respectively. Since the configuration of the DMA also includes the transpose rearrangement, when the configurations of the DMAs are the same, the DMAs can be merged. After the Nth DMA in the computing graph is merged, the final computing graph structure can be determined. For example, the merged computing graph is as shown in Figure 10 Figure 10 The schematic diagram of the computing graph suitable for the embodiment of the present application.

[0126] 704. After traversing the computing graph, the memory space required by the Nth DMA and the computing core is obtained.

[0127] 705. According to the memory space required by the Nth DMA and the computing core, the corresponding memory space is allocated to the Nth DMA and the computing core, respectively. ​

[0128] Specifically, after the calculation graph is determined, the DLA can traverse the calculation graph, and the memory space required by each Nth DMA and the calculation core after the traversal of the calculation graph can be queried. The corresponding memory of the Nth DMA and the calculation core can be allocated adaptively according to the memory space after the traversal, so that the Nth DMA and the calculation core can run stably after the adaptive allocation of the memory.

[0129] It should be noted that the memory allocation mode of the DMA and the calculation core provided in the embodiments of the present application is only an example and is not a limitation of the present application, that is, the memory allocation of the DMA and the calculation core can be determined in advance according to actual needs, and then the method provided in the embodiments of the present application is used to realize data carrying and / or merging to generate an efficient data stream.

[0130] In a specific embodiment, the Nth DMA carries data in the corresponding position from the address region closest to the physical position of the Nth DMA in the upper level storage area.

[0131] The Nth DMA carries data to the N-level cache area closest to the physical position of the Nth DMA.

[0132] Specifically, the Nth DMA carries data in the corresponding position from the address region closest to the physical position of the Nth DMA in the upper level storage area, for example, Figure 9 The outer storage area DDR closest to the physical position of each DMA in the outermost layer is the affinity area of the DMA, and the area closest to the physical position of each DMA in the outermost layer on the L2Buf is the affinity area of the DMA. Therefore, each DMA in the outermost layer can carry the data stored in the affinity area on the outer storage area DDR to the affinity area on the L2Buf, and similarly, the Nth DMA can carry data from the affinity area in the upper level storage area to the affinity area in the N-level cache area.

[0133] Embodiment Four

[0134] Figure 8 The schematic diagram of the cooperation between the DMA and the DMA and between the DMA and the calculation core to generate an efficient data stream suitable for the fourth embodiment of the present application is applied to the case where the DLA includes two carrying layers and the calculation layer at the bottom.

[0135] As shown in Figure 8

[0136] The second DMA slices the data in the outer storage area DDR according to the preset order, the preset span and the preset size, and carries the second slice after the slicing to the second-level cache area L2Buffer;

[0137] After the second DMA carries the second slice, a second carrying completion signal is sent to the corresponding first DMA.​

[0138] After the first DMA receives the second carrying completion signal, the second slice in the second cache area L2Buffer is sliced in a preset order with a preset span and a preset size, and the first slice after slicing is carried to the first cache area L1Buffer;

[0139] After the second slice in the second cache area is carried, the first DMA sends a first carrying completion signal to the corresponding second DMA;

[0140] After the second DMA receives the first carrying completion signal sent by the first DMA corresponding to the second DMA, the next second slice is carried from the upper level storage area to the second cache area in a preset order;

[0141] The first DMA slices the data in the second cache area L2Buffer in a preset order with a preset span and a preset size, and carries the first slice after slicing to the first cache area L1Buffer;

[0142] After the first slice is carried, the first DMA sends a first carrying completion signal to the computing core;

[0143] After the computing core receives the first carrying completion signal, the first slice in the first cache area L1Buffer is loaded;

[0144] The computing core calculates the data in the loaded first slice;

[0145] The computing core sends the data after calculation to the first cache area L1Buffer, so that the data is merged into a first merged slice in the first cache area L1Buffer;

[0146] After the first merged slice is merged, the computing core sends a first merging completion signal to the first DMA;

[0147] After the first DMA receives the first merging completion signal, the first merged slice in the first cache area L1Buffer is carried to the second cache area L2Buffer, and is merged into a second merged slice in the second cache area L2Buffer;

[0148] When the second DMA receives the second merging completion signal sent by the corresponding first DMA, the second DMA carries the second merged slice in the second cache area to the external storage area DDR, and merges the data in the external storage area DDR.

[0149] The technical scheme of the embodiment of the present application improves the parallelism and utilization of the on-chip DMA and the computing unit, improves the multiplexing degree of the on-chip data, and reduces the bandwidth pressure of the off-chip memory with low bandwidth, by designing the data carrying rule, that is, the traversal direction, the traversal address rule, the slice size, and the synchronization relationship between the DMA units and between the DMA and the computing unit.

[0150] The embodiment of the present application also provides a chip, which is used for executing the data synchronization method in the DLA architecture provided by any of the above embodiments; wherein the chip comprises a DLA, the DLA is at least two layers of structure, the DLA comprises a computing layer at the bottom and at least one carrying layer; the computing layer comprises at least one computing core; the first carrying layer adjacent to the computing layer comprises at least one first DMA corresponding to the computing core and at least one first-level cache area.

[0151] The technical scheme of the embodiment of the present application organizes the working rhythm of the data carrying and the computing unit on the chip through the data synchronization method in the DLA architecture, so that they can be parallel, thereby improving the utilization of the chip. The chip can be a DSA (domain-specific accelerator), or more specifically, a DLA (deep-learning accelerator).

[0152] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical scheme of the present application can be achieved, which is not limited herein.

[0153] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for data synchronization in a DLA architecture, the method comprising: The DLA is at least a two-layer structure, including a bottommost computing layer and at least one transport layer; the computing layer includes at least one computing core; a first transport layer adjacent to the computing layer includes at least one first DMA corresponding to the computing core and at least one first-level cache area; The first DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and transports the first slice after slicing to the first-level cache area; After the first slice is transported, the first DMA sends a first transport completion signal to the computing core; After the computing core receives at least one corresponding first transport completion signal, the computing core loads the first slice from at least one corresponding first-level cache area; The computing core calculates the data in the loaded first slice; According to the data flow description of the operator, the first DMA transports the data in the first slice and the computing core calculates the data in the first slice, and the first DMA and the computing core operate cooperatively.

2. The method of claim 1, wherein, When the DLA includes N transport layers, the Nth transport layer includes an Nth DMA and an N-level cache area; The Nth DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and transports the Nth slice after slicing to the N-level cache area, the upper-level storage area being an external storage area or an N+1-level cache area; After the Nth DMA transports the Nth slice, the Nth DMA sends an Nth transport completion signal to the corresponding N-1th DMA; After the N-1th DMA receives the corresponding Nth transport completion signal, the N-1th DMA slices the Nth slice in the N-level cache area in a preset order with a preset span and a preset size, and transports the N-1th slice after slicing to an N-1-level cache area; After the Nth slice in the N-level cache area is all transported, the N-1th DMA sends an N-1th transport completion signal to the corresponding Nth DMA; After the Nth DMA receives all the N-1th transport completion signals sent by the N-1th DMA corresponding to the Nth DMA, the Nth DMA transports the next Nth slice from the upper-level storage area to the N-level cache area in the preset order.

3. The method of claim 1, wherein, When the DLA includes N transport layers, the Nth transport layer includes an Nth DMA and an N-level cache area; The Nth DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and transports the Nth slice after slicing to the N-level cache area, the upper-level storage area being an external storage area or an N+1-level cache area; After the Nth DMA transports the aM+bth Nth slice, the Nth DMA sends an Nth transport completion signal to the corresponding bth N-1th DMA, and continues to transport the next Nth slice, the Nth DMA corresponding to M N-1th DMAs, a being an integer greater than or equal to 0, and b=1, 2, …M; The bth N-1 DMA receives the Nth transport completion signal, slices the corresponding Nth slice in the N-level cache area according to the preset order, the preset span and the preset size, and transports the sliced N-1 slice to the N-1 level cache area; The bth N-1 DMA receives the Nth transport completion signal, slices the corresponding Nth slice in the N-level cache area according to the preset order, the preset span and the preset size, and transports the sliced N-1 slice to the N-1 level cache area; When the Nth DMA transports the aMth Nth slice and receives the N-1 transport completion signal fed back by the first N-1 DMA, the Nth DMA continues to transport the aM+1th Nth slice; After the Nth DMA transports the aM+1th Nth slice, the Nth transport completion signal is sent to the corresponding first N-1 DMA, and the next Nth slice is continued to be transported; After the Nth DMA receives the N-1 transport completion signal fed back by the N-1 DMA corresponding to the last Nth slice in the N+1 slice, the Nth DMA transports the next N+1 slice from the upper level storage area to the N-level cache area according to the preset order.

4. The method of claim 1, wherein, When N layers of transport layers are included, the Nth transport layer includes an Nth DMA and an N-level cache area; When the Nth DMA is configured in parallel, the Nth DMA slices data in the upper level storage area according to the preset order, the preset span and the preset size, and transports the sliced Nth slice to the N-level cache area, wherein the upper level storage area is an external storage area or an N+1 level cache area; After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; 5. The method of claim 1, wherein, After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; After the Nth DMA transports the Nth slice, the Nth transport completion signal is sent to the corresponding N-1 DMA, and the next Nth slice is continued to be transported to the N-level cache area according to the preset order; 6. The method of claim 5, wherein, When the Nth DMA receives the Nth merge completion signal sent by the corresponding N-1 DMA, the Nth DMA transports the Nth merged slice in the N-level cache area to the next level storage area, and merges data in the next level storage area, wherein the next level storage area is an external storage area or an N+1 level cache area. When N layers of transport layers are included, the Nth transport layer includes an Nth DMA and an N-level cache area; 7. The method of claim 1, wherein, Before the first DMA slices the data in the upper-level storage area in a preset order with a preset span and a preset size, and transports the first slice after slicing to a first cache area, further comprising: According to a preset operator data flow description, obtain processing nodes of a data flow in a DLA and data flow directions in the processing nodes, the processing nodes including the first NDMA and the computing core, the data flow directions being data transport directions between the processing nodes; Determine a computation graph from the processing nodes and the data flow directions.

8. The method of claim 7, wherein, After the determination of the computation graph from the processing nodes and the data flow directions, further comprising: Merge the first NDMA in the computation graph that are in the same transport layer, have the same configuration, and transport the same source data, the source data being tensor data in an operator.

9. The method of claim 8, wherein, After the merging of the first NDMA in the computation graph that are in the same transport layer, have the same configuration, and transport the same source data, further comprising: After traversing the computation graph, obtain memory spaces required by the first NDMA and the computing core; According to the memory spaces required by the first NDMA and the computing core, respectively allocate corresponding memory spaces for the first NDMA and the computing core.

10. The method of claim 1 or 9, wherein the DLA architecture is a GPU architecture. Further comprising: The first NDMA transports data in a corresponding position from an address area closest to the physical position of the first NDMA in the upper-level storage area; The first NDMA transports data to an N-level cache area closest to the physical position of the first NDMA.

11. A chip, characterized by A method for performing data synchronization in a DLA architecture as claimed in any one of claims 1-10; When the chip includes a DLA, the DLA is at least two layers of structure, the DLA includes a computing layer at the bottom and at least one transport layer; the computing layer includes at least one computing core; the first transport layer adjacent to the computing layer includes at least one first DMA corresponding to the computing core and at least one first cache area.

Images