Coarse-grained reconfigurable processor operator memory data multiplexing design method

Abstract

The invention relates to a coarse-grained reconfigurable processor operator memory data multiplexing design method, which comprises the following steps of: acquiring operator related information of all operators in a to-be-processed model, then determining an operator fusion strategy according to the operator related information of all the operators, and finally fusing all the operators according to the operator fusion strategy, so that the memory data multiplexing design of the coarse-grained reconfigurable processor operator is realized. And the fused operator is processed. According to the method, the fused operators are uniformly processed on the on-chip memory, or the on-chip memory is reused to more quickly process the intermediate data in the memory, so that the repeated carrying of the intermediate data can be reduced, the data carrying power consumption of the whole network operation can be reduced, the utilization rate of the computing unit can be improved, and the whole operation time can be shortened; and the data processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of coarse-grained reconfigurable processors, and in particular to a method for designing operator memory data reuse of a coarse-grained reconfigurable processor. Background Art

[0002] With the rapid development of artificial intelligence and big data technologies, neural network models are increasingly used in image recognition, natural language processing and other fields. However, these complex neural network models place extremely high demands on computing resources and memory bandwidth, resulting in the gradual emergence of memory bottlenecks. Traditional fixed-architecture processors are often limited by their fixed hardware structure when processing these memory-intensive tasks, making it difficult to efficiently cope with dynamically changing computing needs.

[0003] As a flexible and programmable processor architecture, coarse-grained reconfigurable processors (CGRP) can meet diverse application requirements to a certain extent by dynamically reconfiguring their hardware structure. However, current coarse-grained reconfigurable processors still have significant limitations in neural network computing. The existing coarse-grained reconfigurable processor architecture has the problem of low data processing efficiency when processing on-chip and off-chip memory interactions. Summary of the invention

[0004] Based on this, it is necessary to provide a coarse-grained reconfigurable processor operator memory data reuse design method that can improve data processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for designing data reuse of a coarse-grained reconfigurable processor operator memory, which is applied to a coarse-grained reconfigurable processor operator memory architecture, and the method includes:

[0006] Obtain operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency;

[0007] Determine the operator fusion strategy based on the operator-related information of all operators;

[0008] All operators are fused according to the operator fusion strategy, and the fused operators are processed.

[0009] In one embodiment, the operator fusion strategy includes a complete memory fusion strategy and a block memory fusion strategy. The operator fusion strategy is determined according to operator-related information of all operators, including:

[0010] Determine the amount of data interaction between all operators and the central processor based on the operator-related information of all operators;

[0011] If the data interaction amount is not greater than a preset interaction amount threshold, the operator fusion strategy is determined to be a complete memory fusion strategy; the preset interaction amount threshold is determined by the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture;

[0012] If the data interaction volume is greater than the preset interaction volume threshold, the operator fusion strategy is determined to be the block memory fusion strategy.

[0013] In one embodiment, determining the amount of data interaction between all operators and the central processor according to operator-related information of all operators includes:

[0014] Determine the amount of input data and output data between each operator and the central processor according to the operator-related information of each operator;

[0015] The amount of data interaction between all operators and the central processor is determined based on the amount of input data and output data between each operator and the central processor.

[0016] In one embodiment, the operator fusion strategy is a complete memory fusion strategy, the fused operators include multi-level operators, all operators are fused according to the operator fusion strategy, and the fused operators are processed, including:

[0017] Sort all operators, and generate the first fused operator based on the sorted operators;

[0018] The output data of the preceding operator in the first fused operator is stored in an on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and the input data of the succeeding operator in the fused operator is obtained from the on-chip memory.

[0019] In one embodiment, the operator fusion strategy is a block memory fusion strategy, all operators are fused according to the operator fusion strategy, and the fused operators are processed, including:

[0020] According to the preset interaction threshold, all operators are divided into multiple operator blocks;

[0021] Sorting all operators in any operator block, and generating a second fused operator based on the sorted operators;

[0022] The output data of the preceding operator in the second fused operator is stored in an on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and the input data of the succeeding operator in the fused operator is obtained from the on-chip memory.

[0023] In one embodiment, the method further comprises:

[0024] According to the network information of the network to be processed, all operators in the network to be processed are divided into multiple sub-networks, and any sub-network is used as a model to be processed.

[0025] In a second aspect, the present application also provides a coarse-grained reconfigurable processor operator memory architecture, the architecture comprising: a processor, on which a global memory and an on-chip memory are arranged, and the processor is used to execute the steps of the method of any embodiment in the first aspect.

[0026] In a third aspect, the present application also provides a coarse-grained reconfigurable processor operator memory data reuse design device, the device comprising:

[0027] An acquisition module, used to acquire operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle and input-output dependency;

[0028] A determination module is used to determine the operator fusion strategy based on the operator-related information of all operators;

[0029] The processing module is used to fuse all operators according to the operator fusion strategy and process the fused operators.

[0030] In a fourth aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0031] Obtain operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency;

[0032] Determine the operator fusion strategy based on the operator-related information of all operators;

[0033] All operators are fused according to the operator fusion strategy, and the fused operators are processed.

[0034] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0035] Obtain operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency;

[0036] Determine the operator fusion strategy based on the operator-related information of all operators;

[0037] All operators are fused according to the operator fusion strategy, and the fused operators are processed.

[0038] In a sixth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtain operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency;

[0040] Determine the operator fusion strategy based on the operator-related information of all operators;

[0041] All operators are fused according to the operator fusion strategy, and the fused operators are processed.

[0042] The above-mentioned coarse-grained reconfigurable processor operator memory data reuse design method obtains the operator-related information of all operators in the model to be processed, and then determines the operator fusion strategy based on the operator-related information of all operators, and finally fuses all operators according to the operator fusion strategy, and processes the fused operators. In the above-mentioned method, by uniformly processing the fused operators on the on-chip memory, or reusing the on-chip memory to more quickly process the intermediate data in the memory, the repeated transfer of the intermediate data can be reduced, the data transfer power consumption of the overall network operation can be reduced, the utilization rate of the computing unit can be improved, the overall running time can be reduced, and the data processing efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of a data processing method of a coarse-grained reconfigurable processor in the prior art;

[0044] Figure 2 A diagram of an application environment of the method described in one embodiment;

[0045] Figure 3 A schematic diagram of a process of the method described in one embodiment;

[0046] Figure 4 A schematic diagram of a process of the method described in another embodiment;

[0047] Figure 5 A schematic diagram of a process of the method described in another embodiment;

[0048] Figure 6 A schematic diagram of a process of the method described in another embodiment;

[0049] Figure 7 A schematic diagram of a process of the method described in another embodiment;

[0050] Figure 8 A schematic diagram of a process of the method described in another embodiment;

[0051] Fig. 9 A schematic diagram of a process of the method described in another embodiment;

[0052] Fig.10 A schematic diagram of a process of the method described in another embodiment;

[0053] Fig.11 A schematic diagram of a process of the method described in another embodiment;

[0054] Fig.12 A schematic diagram of a process of the method described in one embodiment;

[0055] Fig.13 A schematic diagram of a process of the method described in another embodiment;

[0056] Fig.14 A schematic diagram of a process of the method described in another embodiment;

[0057] Fig.15 A schematic diagram of a process of the method described in another embodiment;

[0058] Fig.16 A schematic diagram of a process of the method described in another embodiment;

[0059] Fig.17 A schematic diagram of a process of the method described in another embodiment;

[0060] Fig.18 A schematic diagram of a process of the method described in another embodiment;

[0061] Fig.19 A schematic diagram of a process of the method described in another embodiment;

[0062] Fig. 20 A schematic diagram of a process of the method described in another embodiment;

[0063] Fig.21 A schematic diagram of a process of the method described in another embodiment;

[0064] Fig. 22 A schematic diagram of a process of the method described in another embodiment;

[0065] Fig.23 A schematic diagram of a process of the method described in another embodiment;

[0066] Fig.24 A schematic diagram of a process of the method described in another embodiment;

[0067] Fig.25 is a structural block diagram of a device described in an embodiment;

[0068] Fig.26 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] With the rapid development of artificial intelligence and big data technologies, neural network models are increasingly used in image recognition, natural language processing and other fields. However, these complex neural network models place extremely high demands on computing resources and memory bandwidth, resulting in the gradual emergence of memory bottleneck problems. Traditional fixed-architecture processors are often limited by their fixed hardware structure when processing these memory-intensive tasks, and it is difficult to efficiently cope with dynamically changing computing needs. As a flexible and programmable processor architecture, coarse-grained reconfigurable processors (CGRP) can meet diverse application requirements to a certain extent by dynamically reconfiguring their hardware structure. However, current coarse-grained reconfigurable processors still have significant limitations in neural network computing, such as Figure 1 As shown in the figure, the current operator operation mode is usually "single operator mode", that is, each operator is an independently running kernel function, whose input comes from the CPU master control, and the input data is moved from the CPU memory to the accelerator memory for calculation inside the accelerator. After the calculation is completed, the output data is moved from the accelerator memory to the CPU memory. The memory interaction between the CPU and the accelerator is the sum of the input and output data of each operator, and the memory data bandwidth of the CPU and accelerator interaction is the lowest, which will bring considerable bandwidth consumption and transportation time overhead. Therefore, the existing coarse-grained reconfigurable processor architecture has the problem of low data processing efficiency when processing memory interactions inside and outside the chip.

[0071] The present application provides a coarse-grained reconfigurable processor operator memory data reuse design method, aiming to solve the above-mentioned technical problems. The following embodiments will specifically illustrate the coarse-grained reconfigurable processor operator memory data reuse design method described in the present application.

[0072] The coarse-grained reconfigurable processor operator memory data reuse design method provided in the embodiment of the present application can be applied to Figure 2In the application environment shown, the application environment includes a central processing unit 10 and a coarse-grained reconfigurable processor operator memory architecture 20, and the coarse-grained reconfigurable processor operator memory architecture 20 includes a processor 201 (i.e., a coarse-grained reconfigurable acceleration processor). The processor 201 is provided with a global memory 2011 and an on-chip memory 2012. The global memory 2011 and the on-chip memory 2012 correspond to two data transfer buses respectively. The on-chip memory 2012 is separately designed in the processing element array (Processing Element Array, PEA for short) of the processor 201. Each PEA has an independent on-chip memory inside, which has a smaller capacity and a faster reading and writing speed.

[0073] The processor 201 is used to fuse all operators in the model to be processed and process the fused operators on the on-chip memory 2012 .

[0074] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the application environment to which the present application scheme is applied. The specific application environment may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0075] In one embodiment, Figure 3 As shown in the figure, a coarse-grained reconfigurable processor operator memory data reuse design method is provided. Figure 2 The processor in the example is used to illustrate, including the following steps:

[0076] S201, obtaining operator-related information of all operators in the model to be processed.

[0077] The model to be processed is a neural network model. The operator is a computing task in the neural network model. The operator-related information includes at least one of the execution order of the operator, the life cycle of the operator, and the input-output dependency of the operator. The life cycle of the operator represents the time period from when its input data is ready to when its output data is used by the next operator.

[0078] In an embodiment of the present application, the processor can actively obtain the model to be processed from a preset path. Optionally, the processor can receive the model to be processed input by the user on the user interface. This embodiment does not limit the method of obtaining the model to be processed. After obtaining the model to be processed, the processor can parse the calculation graph file of the model to be processed through the API provided by the corresponding framework, and then extract the operator-related information of all operators. Specifically, the execution order of the operator can be obtained by using depth-first search (DFS) or breadth-first search (BFS), the life cycle of the operator can be determined by analyzing the input and output dependencies of the operator, and can be determined by analyzing the input and output tensors of the operator.

[0079] S202: Determine an operator fusion strategy based on operator-related information of all operators.

[0080] Among them, the operator fusion strategy is used to fuse all operators.

[0081] In an embodiment of the present application, after obtaining the operator-related information of all operators, the processor can determine the resources required to process each operator based on the operator-related information of all operators, and then determine the fusion strategy based on the resources required for each operator and the on-chip memory capacity of the processor.

[0082] S203: Fuse all operators according to the operator fusion strategy, and process the fused operators.

[0083] In an embodiment of the present application, after the processor determines the operator fusion strategy based on the above steps, it can fuse all operators or part of the operators according to the operator fusion strategy to obtain fused operators, and then uniformly process the fused operators in the on-chip memory.

[0084] The coarse-grained reconfigurable processor operator memory data reuse design method provided in the embodiment of the present application obtains the operator-related information of all operators in the model to be processed, and then determines the operator fusion strategy based on the operator-related information of all operators, and finally fuses all operators according to the operator fusion strategy, and processes the fused operators. In the above method, by uniformly processing the fused operators on the on-chip memory, or reusing the on-chip memory to process the intermediate data in the memory more quickly, the repeated transfer of the intermediate data can be reduced, the data transfer power consumption of the overall network operation can be reduced, the utilization rate of the computing unit is improved, the overall running time is reduced, and the data processing efficiency is improved.

[0085] In one embodiment, the above operator fusion strategy includes a complete memory fusion strategy and a block memory fusion strategy, and also provides a specific implementation method for determining the operator fusion strategy, such as Figure 4As shown, the "determining the operator fusion strategy according to the operator-related information of all operators" in the above step S202 includes:

[0086] S301, determining the data interaction amount between all operators and the central processing unit according to operator-related information of all operators.

[0087] Among them, the operator fusion strategy includes the full memory fusion strategy and the block memory fusion strategy. The full memory fusion strategy means that for some operators with relatively small input and output data volumes, their calculation logic and loop structure can be ignored, and multiple operators can be completely and directly fused in memory on a faster memory. The block memory fusion strategy means that many operators in the model to be processed cannot be fully fused in memory. In order to store as much intermediate data as possible in a faster memory, block fusion can be performed for operators that meet the calculation locality.

[0088] In the embodiment of the present application, after obtaining the operator-related information of all operators in the model to be processed, the processor can determine the amount of data interaction between all operators and the central processing unit (CPU) to obtain the memory size ultimately required by these operators, so as to determine whether these operators can be fused. Figure 5 As shown, the processor can first determine the execution order of operators according to the operator dependency relationship. The execution order of the operators can be (conv-conv-add-conv-pool-reduce). The execution order can be marked with the serial numbers in the figure 012... Figure 6 As shown, the life cycle of the operator is determined based on the execution order of the operator and the input-output dependency. The start and end of these life cycles are closures. Some algorithms are used to arrange the memory layout of the input and output of these operators, and actively reuse the memory where the invalid data is located. Assuming that no other algorithms are used, the static memory planning is performed simply according to the execution order and life cycle of the operator. The logic is as follows: (1) As Figure 7 As shown in the figure, the input of the conv op with execution order 0 needs to occupy 2KB of memory, which can be released after the conv is executed. (2) Figure 8 As shown in the figure, the output of the conv op with execution order 0 needs to occupy 2KB of memory, and its life cycle is [0, 2]. It is also the input of the add op with execution order 2, and the memory it occupies can only be released after the execution is completed. (3) Fig. 9 As shown in the figure, the input of the conv op with execution order 1 needs to occupy 2KB of memory, and its life cycle is [1, 1]. It can be released when the conv is executed. At this time, the memory of input_0 has been released, and the size is just enough for input_1 to use. Input_1 reuses this memory. (4) Fig.10As shown in the figure, the output (output_1) of the conv op with execution order 1 needs to occupy 2KB of memory, and its life cycle is [1, 2]. It is also the input of the add op with execution order 2, and the memory it occupies can only be released after the execution is completed. (5) Fig.11 As shown in the figure, the input of the add operator with execution order 2 is ready (output_0 and output_1), and its output (output_2) has a life cycle of [2, 3]. It is also the input of the convop with execution order 3, and the memory it occupies can only be released after the execution is completed. (6) Fig.12 As shown in the figure, the input of the conv operator with execution order 3 is ready (output_2), and its output (output_3) has a life cycle of [3, 4]. It is also the input of the pool op with execution order 4, and its memory can only be released after the execution is completed. At this time, the execution of the previous add operator has ended, and its input memory is released (output_0 and output_1). Output_3 can directly reuse its memory. (7) Fig.13 As shown in the figure, the input of the pool operator with execution order 4 is ready (output_3), and its output (output_4) has a life cycle of [4, 5]. It is also the input of the reduce op with execution order 5, and the memory it occupies can only be released after the execution is completed. At this time, although the memory address space of 0-2k is free, it cannot accommodate the 8K output_4, so the starting memory address of output_4 needs to start from 6K. At this time, the memory overhead of the entire graph reaches the maximum, requiring a total of 14K of space, of which 2K of space is wasted. (8) As shown in the figure, the input of the pool operator with execution order 4 is ready (output_3), and the output_4 has a life cycle of [4, 5]. Fig.14 As shown in the figure, the input of the reduce operator with execution order 5 is ready (output_4), and its output (output_5) can reuse the address space of 0-6K. This is a simple example. When using the above method in static memory planning, the life cycle and memory requirements of each operator's input and output are actually just fixed to the start-end range, with a length of a square of its size, but it can slide on the entire memory address on the fixed start-end timeline. The memory layout can be optimized by considering memory alignment, so that the memory used by the entire operator fusion range is minimized and more efficient.

[0089] For example, Fig.15As shown in the figure, assuming that the on-chip memory of the device is 128K, the input and output tensor data of these continuous operators are very small, and even if the intermediate data exists in the on-chip memory at the same time, it will not overflow. Then the calculation can be carried out as follows: each operator is independently and completely calculated in sequence. After the calculation is completed, the output data remains in the on-chip memory, and the next operator directly obtains its input from the on-chip memory. In this way, the intermediate data can be completely eliminated. In other words, the amount of data interaction between the CPU and the device side is directly reduced from the input and output of all operators ((2k+2k)+(2k+2k)+(2k+2k+2k)+(2k+4k)+(4k+8k)+(8k+0.5k) = 40.5k) to the input and output of the fused subgraph operator (the operators in this subgraph are fused as a single operator) (2k+2k+0.5k=4.5k), which is reduced to 1 / 9 of the original. In this case, the data transfer solution is as follows Fig.16 shown.

[0090] Optional, such as Fig.17 As shown, the above step S301 of "determining the amount of data interaction between all operators and the central processor according to the operator-related information of all operators" includes:

[0091] S3011, determining the input data volume and output data volume between each operator and the central processing unit according to the operator-related information of each operator.

[0092] In the embodiment of the present application, after obtaining the operator-related information of each operator, the processor may determine the input data volume and output data volume between each operator and the central processor based on the operator-related information.

[0093] S3012, determining the amount of data interaction between all operators and the central processing unit according to the amount of input data and the amount of output data between each operator and the central processing unit.

[0094] The data interaction volume is the sum of the input data volume and the output data volume corresponding to each operator.

[0095] In an embodiment of the present application, after the processor obtains the input data volume and output data volume between each operator and the central processing unit based on the above steps, the input data volume and output data volume between each operator and the central processing unit can be summed up to obtain the data interaction volume between all operators and the central processing unit.

[0096] S302: If the data interaction amount is not greater than the preset interaction amount threshold, the operator fusion strategy is determined to be a complete memory fusion strategy.

[0097] The preset interaction threshold is determined by the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture.

[0098] In the embodiment of the present application, the preset interaction threshold can be determined in advance according to the capacity of the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture. After the processor obtains the data interaction amount based on the above steps, the data interaction amount can be compared with the preset interaction threshold. If the data interaction amount is not greater than the preset interaction threshold, the operator fusion strategy is determined to be a complete memory fusion strategy.

[0099] S303: If the data interaction amount is greater than a preset interaction amount threshold, the operator fusion strategy is determined to be a block memory fusion strategy.

[0100] In the embodiment of the present application, if the data interaction amount is greater than a preset interaction amount threshold, the operator fusion strategy is determined to be a block memory fusion strategy.

[0101] The method described in the embodiment of the present application can achieve the memory reuse effect through two operator fusion strategies. The above method can be applied to many operator combinations of many neural networks, which can reduce the interaction between on-chip and off-chip memories, reduce the data transfer load, thereby improving the network operation rate and reducing the network operation power consumption.

[0102] In one embodiment, a specific implementation method for processing the fused operator is also provided when the operator fusion strategy is a complete memory fusion strategy, such as Fig.18 As shown, the above step S203 of “fusing all operators according to the operator fusion strategy and processing the fused operators” includes:

[0103] S401, sort all operators, and generate a first fused operator according to the sorted operators.

[0104] The first fused operator represents the operator fused according to the complete memory fusion strategy.

[0105] In an embodiment of the present application, when the processor determines that the operator fusion strategy is a complete memory fusion strategy, it can sort all operators and generate a first fused operator based on the sorted operators.

[0106] S402, storing the output data of the preceding operator in the first fused operator in an on-chip memory in the operator memory architecture of the coarse-grained reconfigurable processor, and obtaining the input data of the succeeding operator in the fused operator from the on-chip memory.

[0107] In the embodiment of the present application, after obtaining the first fused operator, the processor can store the output data of the previous operator in the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtain the input data of the next stage of the fused operator from the on-chip memory. For example, the output data of the first-stage operator is stored in the on-chip memory, and the second-stage operator directly obtains the input data from the on-chip memory, which can completely eliminate the intermediate data.

[0108] In one embodiment, a specific implementation method for processing the fused operators is provided when the operator fusion strategy is the block memory fusion strategy, such as Fig.19 As shown, the above step S203 of “fusing all operators according to the operator fusion strategy and processing the fused operators” includes:

[0109] S501, dividing all operators according to a preset interaction threshold to obtain multiple operator blocks.

[0110] The resources required by each operator block do not exceed a preset interaction threshold.

[0111] In an embodiment of the present application, when the processor determines that the operator fusion strategy is a block memory fusion strategy, the resource corresponding to the preset interaction amount threshold can be used as the maximum resource of the operator block, and all operators can be divided to obtain multiple operator blocks.

[0112] S502, sort all operators in any operator block, and generate a second fused operator according to the sorted operators.

[0113] The second fused operator represents the operator fused according to the block memory fusion strategy.

[0114] In an embodiment of the present application, when the processor determines that the operator fusion strategy is a complete memory fusion strategy, it can sort all operators in any operator block and generate a second fused operator based on the sorted operators.

[0115] S503, storing the output data of the previous stage operator in the second fused operator in the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtaining the input data of the latter stage in the fused operator from the on-chip memory.

[0116] In the embodiment of the present application, after obtaining the second fused operator, the processor can store the output data of the previous operator in the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtain the input data of the next stage of the fused operator from the on-chip memory. For example, the output data of the first-stage operator is stored in the on-chip memory, and the second-stage operator directly obtains the input data from the on-chip memory, which can completely eliminate the intermediate data.

[0117] For example, for the full memory fusion strategy, the full fusion of operators directly reduces the moving in and out of intermediate data without additional overhead, and is the best solution for operator memory reuse. However, current neural network models are increasingly biased towards processing large feature map feature inputs. For some target recognition networks, the amount of data for their inference parameters, such as the Weight parameter of the convolution operator, is also increasing. Many operators in most models cannot be fully fused in memory. At this time, in order to store as much intermediate data as possible in faster memory, block fusion can be performed for operators that meet computational locality.

[0118] like Fig. 20 As shown in , suppose there is such an operator model whose input is 64K. During its execution, it requires at least 128K of memory, while the on-chip memory can only support 64K of memory. Fig.21 As shown in the figure, by utilizing the computational locality of multiple consecutive operators, the operator that originally computes the complete input and output is equivalently divided into N small operators. Each operator is only responsible for a part of the original output calculation, and the subsequent small operators use the result of the previous operator as input and continue to calculate and output the input result required by the next operator.

[0119] like Fig. 22 As shown in the figure, the total on-chip memory occupied by inter-operator data in single-operator mode: the input generates output after passing through operator 1, and its output blocks are moved to off-chip memory after being output from on-chip memory; the output of operator 1 is the input of operator 2, so it is moved into on-chip memory in blocks, and the output of the blocks is calculated by operator 2, and then moved to off-chip.... Therefore, for the single-operator mode, in addition to the input and output of the entire network, the intermediate output will occupy on-chip memory twice the amount of its data. After the operators that meet the local computing characteristics are split and then merged, the on-chip memory occupied by the intermediate data is no longer twice the total amount, because the intermediate data no longer needs to be repeatedly moved in and out.

[0120] like Fig.23As shown in the figure, the processor cuts the input and output into half and fuses them separately along with the operator network. According to the computational locality of these operators, assuming that the output of prelu is calculated at 32K at a time, the corresponding required input (the output of add) is also 32K. Working backwards, it is finally calculated that the original input required is half of the original conv complete input.

[0121] like Fig.24 As shown in the figure, the subgraph composed of multiple operators is regarded as a complete operator, and it is divided into n blocks according to the output. Then at least the intermediate data can be completely eliminated, although it may be necessary to increase the logic related to data movement. If the two kernels (operator networks) can be completely parallel, then the parallelism can be doubled and the intermediate data can be completely eliminated. If the two kernels are executed completely serially, at least the intermediate data can be completely eliminated. In the above example, the original memory transfer capacity required is 64K*2*4 = 512K, and the memory transfer capacity after tile + fusion is (32K+32K)*2 = 128K, saving 384K of data transfer, accounting for 75% of the original transfer capacity.

[0122] In one embodiment, the above-mentioned coarse-grained reconfigurable processor operator memory data reuse design method also includes: dividing all operators in the network to be processed into multiple sub-networks according to network information of the network to be processed, and taking any sub-network as a model to be processed.

[0123] Among them, the network to be processed is a large model to be processed.

[0124] In the embodiment of the present application, after obtaining the network to be processed, the processor can divide all operators in the network to be processed into multiple sub-networks, which can be seen in Fig.23 and Fig.24 The kernel in the embodiment is selected, and any sub-network is used as a model to be processed, and then the method described in any of the above embodiments is executed.

[0125] The method described in the embodiments of the present application can eliminate the repeated movement of intermediate data in the neural network, reduce bandwidth occupancy and data transportation overhead, increase computing unit utilization, increase parallelism, and reduce the running time of the entire network by dividing all operators in the network to be processed into multiple sub-networks.

[0126] In one embodiment, a coarse-grained reconfigurable processor operator memory architecture is also provided, the architecture comprising: a processor, a global memory and an on-chip memory are provided on the processor, and the processor is used to execute the steps of the method of any of the above embodiments.

[0127] Based on all the above embodiments, a method for designing memory data reuse of operators in a coarse-grained reconfigurable processor is also provided. The method comprises:

[0128] S601, dividing all operators in the network to be processed into multiple sub-networks according to network information of the network to be processed, and taking any sub-network as a model to be processed.

[0129] S602, obtaining operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency.

[0130] S603: Determine the input data volume and output data volume between each operator and the central processor according to the operator-related information of each operator.

[0131] S604: Determine the amount of data interaction between all operators and the central processing unit according to the amount of input data and the amount of output data between each operator and the central processing unit.

[0132] S605: If the data interaction amount is not greater than a preset interaction amount threshold, the operator fusion strategy is determined to be a complete memory fusion strategy, wherein the preset interaction amount threshold is determined by an on-chip memory in the operator memory architecture of the coarse-grained reconfigurable processor.

[0133] S606: If the data interaction amount is greater than the preset interaction amount threshold, determine that the operator fusion strategy is a block memory fusion strategy.

[0134] S607, the operator fusion strategy is a complete memory fusion strategy, all operators are sorted, and a first fused operator is generated based on the sorted operators, the output data of the previous operator in the first fused operator is stored in the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and the input data of the latter stage in the fused operator is obtained from the on-chip memory.

[0135] S608, the operator fusion strategy is a block memory fusion strategy, which divides all operators according to a preset interaction threshold to obtain multiple operator blocks, sorts all operators in any operator block, and generates a second fused operator according to the sorted operators, stores the output data of the previous operator in the second fused operator in the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtains the input data of the latter stage in the fused operator from the on-chip memory.

[0136] The methods described in the above steps are all described in the above embodiments. Please refer to the above description for details and will not be repeated here.

[0137] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a coarse-grained reconfigurable processor operator memory data reuse design device for implementing the above-mentioned coarse-grained reconfigurable processor operator memory data reuse design method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more coarse-grained reconfigurable processor operator memory data reuse design device embodiments provided below can refer to the limitations of the coarse-grained reconfigurable processor operator memory data reuse design method in the above text, and will not be repeated here.

[0139] In one embodiment, Fig.25 As shown, a coarse-grained reconfigurable processor operator memory data reuse design device is provided, comprising:

[0140] The acquisition module 11 is used to acquire operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle and input-output dependency.

[0141] The determination module 12 is used to determine the operator fusion strategy according to the operator related information of all operators.

[0142] The processing module 13 is used to fuse all operators according to the operator fusion strategy and process the fused operators.

[0143] In one embodiment, the determination module 12 includes:

[0144] The first determining unit is used to determine the data interaction amount between all operators and the central processing unit according to the operator related information of all operators.

[0145] The second determination unit is used to determine that the operator fusion strategy is a complete memory fusion strategy if the data interaction amount is not greater than a preset interaction amount threshold; the preset interaction amount threshold is determined by the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture.

[0146] The third determining unit is used to determine that the operator fusion strategy is a block memory fusion strategy if the data interaction amount is greater than a preset interaction amount threshold.

[0147] In one embodiment, the first determining unit includes:

[0148] The first determining subunit is used to determine the input data volume and the output data volume between each operator and the central processing unit according to the operator related information of each operator.

[0149] The second determining subunit is used to determine the amount of data interaction between all operators and the central processing unit according to the amount of input data and the amount of output data between each operator and the central processing unit.

[0150] In one embodiment, the processing module 13 includes:

[0151] The first sorting unit is used to sort all operators and generate a first fused operator according to the sorted operators.

[0152] The first fusion unit is used to store the output data of the previous stage operator in the first fused operator into the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtain the input data of the next stage in the fused operator from the on-chip memory.

[0153] In one embodiment, the processing module 13 includes:

[0154] The fourth determining unit is used to divide all operators according to a preset interaction amount threshold to obtain multiple operator blocks.

[0155] The second sorting unit is used to sort all operators in any operator block and generate a second fused operator according to the sorted operators.

[0156] The second fusion unit is used to store the output data of the previous stage operator in the second fused operator into the on-chip memory in the coarse-grained reconfigurable processor operator memory architecture, and obtain the input data of the latter stage in the fused operator from the on-chip memory.

[0157] In one embodiment, the above-mentioned coarse-grained reconfigurable processor operator memory data reuse design device also includes: a classification module, which is used to divide all operators in the network to be processed into multiple sub-networks according to the network information of the network to be processed, and use any sub-network as a model to be processed.

[0158] Each module in the above coarse-grained reconfigurable processor operator memory data reuse design device can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0159] In one embodiment, a computer device is provided. The computer device may be a terminal or a server. The internal structure diagram thereof may be as follows: Fig.26 As shown, the computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a coarse-grained reconfigurable processor operator memory data reuse design method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0160] Those skilled in the art will understand that Fig.26 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0161] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the coarse-grained reconfigurable processor operator memory data reuse design method described in any of the above embodiments are implemented.

[0162] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for designing operator memory data reuse of a coarse-grained reconfigurable processor described in any of the above embodiments are implemented.

[0163] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the method for designing operator memory data reuse of a coarse-grained reconfigurable processor described in any of the above embodiments.

[0164] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0165] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A coarse-grained reconfigurable processor operator memory data reuse design method, the method is applied to the coarse-grained reconfigurable processor operator memory architecture, characterized in that: The method comprises: Obtain operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle, and input-output dependency; Determine an operator fusion strategy according to operator-related information of all the operators; All the operators are fused according to the operator fusion strategy, and the fused operators are processed.

2. The method according to claim 1, characterized in that The operator fusion strategy includes a complete memory fusion strategy and a block memory fusion strategy. The operator fusion strategy is determined according to the operator related information of all the operators, including: Determining the amount of data interaction between all the operators and the central processing unit according to the operator-related information of all the operators; If the data interaction amount is not greater than a preset interaction amount threshold, determining that the operator fusion strategy is the complete memory fusion strategy; the preset interaction amount threshold is determined by an on-chip memory in the operator memory architecture of the coarse-grained reconfigurable processor; If the data interaction amount is greater than the preset interaction amount threshold, the operator fusion strategy is determined to be the block memory fusion strategy.

3. The method according to claim 2, characterized in that The determining, according to the operator-related information of all the operators, the amount of data interaction between all the operators and the central processor comprises: Determine the amount of input data and the amount of output data between each of the operators and the central processing unit according to the operator-related information of each of the operators; The amount of data interaction between all the operators and the central processing unit is determined according to the amount of input data and the amount of output data between each of the operators and the central processing unit.

4. The method according to claim 1, characterized in that: The operator fusion strategy is a complete memory fusion strategy, the fused operators include multi-level operators, and all the operators are fused according to the operator fusion strategy, and the fused operators are processed, including: Sort all operators, and generate the first fused operator based on the sorted operators; The output data of the preceding operator in the first fused operator is stored in an on-chip memory in the operator memory architecture of the coarse-grained reconfigurable processor, and the input data of the succeeding operator in the fused operator is obtained from the on-chip memory.

5. The method according to claim 1, characterized in that The operator fusion strategy is a block memory fusion strategy. The steps of fusing all operators according to the operator fusion strategy and processing the fused operators include: According to the preset interaction threshold, all operators are divided into multiple operator blocks; Sorting all operators in any of the operator blocks, and generating a second fused operator according to the sorted operators; The output data of the preceding operator in the second fused operator is stored in an on-chip memory in the operator memory architecture of the coarse-grained reconfigurable processor, and the input data of the succeeding operator in the fused operator is obtained from the on-chip memory.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: According to the network information of the network to be processed, all operators in the network to be processed are divided into a plurality of sub-networks, and any of the sub-networks is used as the model to be processed.

7. A coarse-grained reconfigurable processor operator memory architecture, characterized in that: The architecture comprises: a processor, on which a global memory and an on-chip memory are arranged, and the processor is used to execute the steps of the method according to any one of claims 1 to 6.

8. A coarse-grained reconfigurable processor operator memory data reuse design device, characterized in that: The device comprises: An acquisition module, used to acquire operator-related information of all operators in the model to be processed; the operator-related information includes at least one of execution order, life cycle and input-output dependency; A determination module, used to determine an operator fusion strategy according to operator-related information of all the operators; A processing module is used to fuse all the operators according to the operator fusion strategy and process the fused operators.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images