An optimization method for multi-chip in-memory computing hybrid DNN accelerator

Abstract

The present invention belongs to the technical field of deep neural network, and discloses an optimization method for a multi-core in-memory computing digital-analog hybrid DNN accelerator, including: step 1, model preprocessing: preprocessing the DNN model to be calculated, and confirming the calculation method of each layer of the deep neural network DNN model according to the digital core particles and in-memory computing core particles in the multi-core in-memory computing digital-analog hybrid DNN accelerator; step 2, construction and optimization of task graph: according to the resource constraints of computing resources and storage resources, the task load of the DNN model is divided into computing and storage tasks to form a task graph; step 3, converting the task graph formed in step 2 into an execution graph of actual operation, and optimizing the execution graph. The present invention can effectively reduce the delay of a single reasoning, while improving the throughput of the overall operation, reducing the overall communication volume, reducing the on-chip network congestion, and improving the operation performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep neural networks, and more specifically, relates to an optimization method for a multi-core in-memory computing digital-analog hybrid DNN accelerator. Background Art

[0002] Neural network technology has brought great breakthroughs to the development of artificial intelligence by continuously expanding its scale and optimizing its network structure. As these technologies have shown breakthrough results in tasks such as perception and recognition, their applications have also expanded to many fields such as driverless cars, medical image detection, and natural language processing. However, these endless innovations have also brought about ever-expanding computing scales, increasingly complex computing models, and greater challenges in storage and computing mapping design.

[0003] In order to further support the development of machine learning technology, DNN accelerators that can handle deep neural network models of various sizes are needed to achieve energy-efficient computing and alleviate the memory bottleneck problem caused by the increase in model size. At present, existing technicians mainly try to solve related problems from two technical directions, one is the in-memory computing (CIM) technology, and the other is the core-based integration technology. In the foreseeable future, DNN models will become more complex, so using core technology to integrate in-memory computing cores, namely CIM cores, that is, analog cores, with traditional digital cores to accelerate DNN model calculations is a reasonable idea and solution direction. For example, Chinese patent application CN118070858A discloses a multi-core-based in-memory computing digital-analog hybrid DNN accelerator, but at the same time, this method requires more complex and accurate optimization methods and systems to manage and optimize the overall calculation. This is because, in this type of chip, CIM and digital core particles must work together to perform reasoning tasks, which brings challenges such as more complex data flow, data storage schemes, and communication modes with heterogeneous chips; in addition, the different computing strategies of the two types of architectures have different performance when processing different DNN computing operations, which may lead to unbalanced workloads and uneven throughput between the two core particles; finally, since the CIM architecture is usually implemented with fixed and low precision, it is necessary to provide an optimization method to ensure the accuracy of operations when the data types between CIM and digital core particles are inconsistent. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an optimization method for a multi-chip in-memory computing hybrid DNN accelerator, which models the calculation and storage of the reasoning process of the deep neural network model DNN as nodes of a graph, and then constructs the data dependencies between the calculation and storage tasks as connections between nodes, establishes the execution process of the entire DNN reasoning task, and optimizes the entire calculation process based on the classification of DNN tasks using an optimization algorithm based on simulated annealing to improve computing efficiency.

[0005] In order to achieve the above object, the present invention is achieved through the following technical solutions:

[0006] The present invention is an optimization method for a multi-chip in-memory computing digital-analog hybrid DNN accelerator, which specifically includes the following steps:

[0007] Step 1, model preprocessing: preprocess the deep neural network DNN model to be calculated, and confirm the calculation method of each layer of the deep neural network DNN model according to the digital core particles and in-memory computing core particles in the multi-core in-memory computing digital-analog hybrid DNN accelerator, that is, the analog core particles, that is, whether the current layer is calculated and processed on the digital core particles or the in-memory computing core particles CIM or on both;

[0008] Step 2: Construction and optimization of task graph: According to the resource constraints of computing resources and storage resources, the task load of the deep neural network DNN model is divided into computing tasks and storage tasks. The task graph is constructed according to the connection of each node on the task graph, that is, the data dependency between tasks. The task graph is optimized by using task replication and the pipeline strategy of DNN accelerator.

[0009] Step 3: Execution graph construction and optimization: According to the topology of the multi-chip in-memory computing digital-analog hybrid DNN accelerator, the computing tasks and storage tasks are mapped to the digital cores and in-memory computing cores on the chip, and the task graph constructed in step 2 is converted into an execution graph for actual operations, and the execution graph is optimized.

[0010] A further improvement of the present invention is that in step 2, the construction of the task map specifically includes the following steps:

[0011] Step 2.1.1, task graph initialization: In the initialization stage, refer to the deep neural network DNN model preprocessed in step 1 for modeling, without considering the limitations of core particles and integrated chips;

[0012] Step 2.1.2, Task segmentation: Add the resource restrictions of the core particles to the task map created in step 2.1.1, segment the computing tasks and storage tasks, and continuously update the connection of the task map to ensure the correctness of data dependencies. After task segmentation, all tasks have met the restrictions of the corresponding core particles, that is, all tasks have met the requirements of being assigned to core particles according to the corresponding types;

[0013] Step 2.1.3, merging and splitting of task maps: Introduce the number of digital core particles and in-memory computing core particles of the integrated chip as well as the on-chip storage resources, merge and split the current task map to make it conform to the task mapping actually running on the integrated chip.

[0014] A further improvement of the present invention is that step 2.1.1 task graph initialization specifically includes the following steps:

[0015] Step 2.1.1.1: According to the accuracy requirements of each layer of the deep neural network DNN model, all the same type of calculations in a layer are divided into one computing task, and all storage processes are regarded as one storage task;

[0016] Step 2.1.1.2: For the layers with high sensitivity, all their calculations are regarded as a high-precision calculation task and executed on the digital core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with low sensitivity, all their calculations are regarded as a low-precision calculation task and executed on the in-memory computing core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with medium sensitivity, digital calculation tasks and in-memory computing tasks are established to represent the calculations of the layer on the digital core and the in-memory computing core respectively;

[0017] Step 2.1.1.3: Connect all task nodes in sequence according to the layer order to initially form a graph, which clarifies the data dependencies between tasks and the task type of each part of the calculation.

[0018] A further improvement of the present invention is that in step 2.1.2, the computing task includes a digital computing task and an in-memory computing task, wherein:

[0019] The division of digital computing tasks uses on-chip storage as the division basis, specifically, whether the minimum storage required by the current digital computing task can be met by the on-chip storage of a digital core particle. If not, the current digital computing task is divided into two until the divided digital computing task meets the conditions;

[0020] The in-memory computing task is divided according to the size of the computing array. That is, if the weight required by the current in-memory computing task cannot be completely stored on one in-memory computing core particle, the in-memory computing task is divided into two.

[0021] The division of storage tasks is based on whether the current storage task can be stored on one in-memory computing core particle. If not, the current storage task is divided into two until the storage task can be stored on one in-memory computing core particle.

[0022] A further improvement of the present invention is that in step 2.1.3,

[0023] The merging of task graphs refers to the layer-by-layer traversal of the segmented task graphs. That is, if the number of tasks in the current layer is less than the number of cores that the integrated chip can provide, then check the tasks in the next layer and try to merge the next layer with the current layer. If the merger succeeds, continue to try to merge the next layer. If the merger fails, return to the loop to check the next layer.

[0024] The task map is split if the number of tasks in the current layer is greater than the number of cores that the integrated chip can provide. In this case, the current layer of the task map is split.

[0025] A further improvement of the present invention is that the task graph is split in the order of depth traversal to ensure that an input and output path in the layer must be in a sub-layer, and the correctness of the data dependency relationship of the tasks placed in a sub-layer is ensured, which specifically includes the following steps:

[0026] Step 2.1.3.1. Put as many tasks as possible in this layer into the first sub-layer, and put all the remaining tasks into the second sub-layer;

[0027] Step 2.1.3.2: Judge the second sublayer. If it still exceeds the resource limit, continue splitting until all sublayers of the current layer meet the resource limit. If the subsequent sublayers do not exceed the resource limit, continue the layer-by-layer traversal and judgment from the current last sublayer.

[0028] A further improvement of the present invention is that the task duplication for optimizing the task graph in step 2 means, after the task graph is generated, detecting whether the current graph will cause waste of resources. If there are idle core particles, the current graph is detected cyclically, and the tasks with the longest calculation time are copied and split onto idle core particles in turn based on the execution time, until there are no more core particles available for allocation. The data dependencies between the copied tasks are recalculated and the task graph is updated.

[0029] A further improvement of the present invention is that the execution graph construction in step 3 is specifically to initialize the execution graph: traverse each layer of the task graph, randomly map each task in the current task load, traverse the three tasks included in the task load respectively, and randomly map the corresponding tasks to the corresponding core particles, that is, digital computing tasks are assigned to digital core particles, in-memory computing tasks are assigned to in-memory computing core particles CIM, and storage tasks are assigned to in-memory computing core particle groups or dynamic random access memory DRAM. The task graph is combined with the actual hardware through random mapping to form an initial execution graph.

[0030] A further improvement of the present invention is that performing graph optimization specifically includes the following steps:

[0031] Step 3.1: Depending on whether the weights of the deep neural network DNN model are stored on the integrated chip, choose to use a task mapping method based on traffic perception to optimize the execution graph or a task planning method based on transmission delay to optimize the execution graph. Specifically:

[0032] When all weights of the deep neural network DNN model are stored on the integrated chip, the execution graph is optimized using a task mapping method based on traffic awareness. A two-dimensional array is maintained for each execution graph to track link utilization to estimate the traffic of the integrated chip on-chip network, where each element represents an integrated chip on-chip network link. The source and destination nodes of each traffic flow are determined, and the exact path of each flow and the number of flows sharing the same link are calculated according to the routing algorithm. Assuming that there are P data flows sharing the same link, the integrated chip on-chip network bandwidth of each flow is 1⁄P of the link bandwidth. The communication delay is inferred based on the traffic volume, and the congestion and utilization of the entire network are evaluated. The communication cost, that is, the sum of the cumulative traffic transmitted on all links of all tasks in the execution graph, is used as part of the objective function to optimize the network and performance:

[0033] Communication Cost= ∑ i ∈ tasks [ Vol input i ×Dist input i +Vol filter i ×Dist filter i +Vol output i ×Dist output i ]

[0034] in, and They represent the data volume of the current task and the number of hops required to transmit the data of the current task. is an iterator representing a task selected from all tasks. Indicates the execution of all tasks in the graph. Represents the input data required for the current task, Represents the weight data required for the current task, Represents the output data calculated by the current task;

[0035] When the weights of a deep neural network (DNN) model cannot be fully stored on the chip, off-chip memory access is required. The execution graph is optimized using a task planning method based on transmission delay. The weight adjustment is overlapped by adjusting the execution time and order of tasks. The execution time and order of each task are calibrated using a delay estimation algorithm to obtain relatively accurate task calculation and data transmission delays. The task graph is then optimized using an execution graph optimizer.

[0036] Step 3.2: Replace the Alternatively, the delay estimation algorithm is introduced into the simulated annealing algorithm to optimize the execution graph, wherein the simulated annealing algorithm includes the following steps:

[0037] Step 3.2.1. In each iteration of the simulated annealing process, randomly select a task type: a storage task, a computation task in an in-memory computation core particle, or a computation task in a digital core particle;

[0038] Step 3.2.2, randomly select two task nodes in the task of this type in the current execution graph, and exchange the mapping of the two tasks, that is, map one task to the core particle of the other task;

[0039] Step 3.2.3: After the swap, estimate the reasoning time of the new mapping scheme and compare the reasoning time before and after the swap. If the time required for reasoning is reduced, accept the change; otherwise, accept the change with a certain probability.

[0040] A further improvement of the present invention is that in step 3.1, the delay estimation algorithm comprises the following steps:

[0041] Step 3.1.1, without considering traffic interference, calculate the time of calculation and data transmission, configure an initial start timestamp and end timestamp for each calculation and storage task to characterize its task execution time, and determine whether a task overlaps with another task by comparing the timestamps;

[0042] Step 3.1.2: If overlapping tasks involve data movement, their data transmission will interfere with each other, and the data transmission delay needs to be recalculated, the timestamps of related tasks need to be corrected, and the related subtasks need to update their own timestamps.

[0043] The beneficial effect of the present invention is that the present invention transforms the computing task of a deep neural network DNN model into a graph model that can reflect the computing, storage and data transmission and is more convenient to optimize. At the same time, the graph model also contains information such as hardware resource constraints and core particle topology, wherein:

[0044] The combination of task replication and hierarchical pipelining in the present invention can effectively reduce the latency of a single inference while improving the overall computing throughput;

[0045] The task mapping method based on communication volume perception in the present invention can reduce the overall communication volume, reduce on-chip network congestion, and improve computing performance by optimizing the location where tasks are executed on the chip;

[0046] The present invention overlaps off-chip memory access and calculation in a task planning method based on transmission delay, reduces the impact of off-chip memory access delay on overall calculation, and improves hardware utilization; and optimizes task mapping based on the overlapped delay to improve computing performance.

[0047] The present invention improves performance and energy efficiency without significantly reducing accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the optimization method of the present invention.

[0049] Figure 2 It is an execution flow chart of task load segmentation and merging of the present invention.

[0050] Figure 3 This is the task replication execution process of the present invention.

[0051] Figure 4 It is a schematic diagram of the hierarchical flow of the present invention.

[0052] Figure 5 It is a schematic diagram of executing the graph initialization of the present invention.

[0053] Figure 6 It is a schematic diagram of the evaluation of the task mapping results of the present invention.

[0054] Figure 7 It is a schematic diagram of the overlapping time of off-chip memory access in the present invention.

[0055] Figure 8 It is a flow chart of the delay estimation algorithm of the present invention.

[0056] Figure 9 It is a flow chart of the optimization algorithm of simulated annealing of the present invention.

[0057] Figure 10 It is a statistical graph of the inference delay of the deep neural network DNN model of the present invention.

[0058] Figure 11 It is a statistical diagram of core particle utilization of the present invention. DETAILED DESCRIPTION

[0059] The following will disclose the embodiments of the present invention with drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. That is to say, in some embodiments of the present invention, these practical details are not necessary.

[0060] The accelerator targeted by the method of the present invention includes multiple digital cores and multiple analog in-memory (CIM) cores, namely analog cores. The digital cores and CIM cores are integrated on one chip, and adjacent digital cores, adjacent CIM cores, and adjacent digital cores and CIM cores are interconnected through an on-chip network. The CIM cores and the digital cores each include an analog router connected to the on-chip network, and each router is responsible for transmitting data between digital cores and digital cores, between CIM cores and CIM cores, between digital and analog cores, and between the chip and an off-chip memory. In this type of accelerator, the digital cores can provide higher computing accuracy, and the CIM cores provide lower computing accuracy but higher computing efficiency and energy efficiency.

[0061] In order to better describe the optimization method of the present invention, some definitions are introduced:

[0062] Workload: For DNN models that need to be calculated, each layer is considered a workload. A layer of the task graph after processing may correspond to multiple layers or part of a layer of the DNN, but for the sake of description consistency, it is still called a workload.

[0063] Task: Each workload can be split into one or more tasks. In the present invention, there are three types of tasks, including computing tasks performed on digital core particles, computing tasks performed on computation-in-memory core particles (CIMs), and storage tasks involving on-chip or off-chip data movement.

[0064] Task Graph: The task graph is used to describe the data dependencies between tasks.

[0065] The tasks split from the workload are composed of task nodes. The connection between each node on the task graph is the data dependency between tasks.

[0066] Execution Graph: The execution graph describes how to map the task graph to the core grains of the integrated chip and when the task can be scheduled for execution.

[0067] like Figure 1 As shown, an optimization method for a multi-chip in-memory computing digital-analog hybrid DNN accelerator is characterized in that: the optimization method specifically includes the following steps:

[0068] Step 1, model preprocessing: preprocess the deep neural network DNN model to be calculated, and confirm the calculation method of each layer of the deep neural network DNN model according to the digital core particles and the in-memory computing core particles, i.e., the analog core particles in the in-memory computing digital-analog hybrid DNN accelerator of the multi-core particles, i.e., whether the current layer is calculated and processed on the digital core particles or the in-memory computing core particles CIM, or on both;

[0069] Step 2: Construction and optimization of task graph: According to the resource constraints of computing resources and storage resources, the task load of the deep neural network DNN model is divided into computing tasks and storage tasks. The task graph is constructed according to the connection of each node on the task graph, that is, the data dependency between tasks. The task graph is optimized by using task replication and the pipeline strategy of DNN accelerator.

[0070] Step 3: Execution graph construction and optimization: According to the topology of the multi-chip in-memory computing digital-analog hybrid DNN accelerator, the computing tasks and storage tasks are mapped to the digital cores and in-memory computing cores on the chip, and the task graph constructed in step 2 is converted into an execution graph for actual operations, and the execution graph is optimized.

[0071] According to the above three steps, the computing task of a deep neural network DNN model is converted into a graph model that can reflect computing, storage and data transmission and is more convenient to optimize. At the same time, the graph model also contains information such as hardware resource constraints and core particle topology.

[0072] The goal of the present invention is to improve performance and energy efficiency without significantly reducing accuracy. To achieve this goal, it is necessary to reasonably utilize the computational precision provided by the accelerator, namely low precision, high precision and mixed precision, which requires preprocessing the deep neural network DNN model before calculation.

[0073] like Figure 1 In stage 1, each layer of the deep neural network DNN model is quantized, and the weight accuracy is divided into high precision, medium precision and low precision. In step 1.2, the sensitivity of each layer of the deep neural network DNN model is divided into three types: high sensitivity, medium sensitivity and low sensitivity. The sensitivity of each layer of the deep neural network DNN model to the calculation weight accuracy determines whether the workload is mapped to the digital core or the CIM core. If reducing the weight accuracy has a small impact on the final reasoning accuracy, the layer uses the low-precision mode and is mapped to the CIM core for calculation. If reducing the weight accuracy has a large impact on the final reasoning accuracy, the layer uses the high-precision mode and is mapped to the digital core for calculation. Otherwise, a mixed mode is used and mapped to both the digital core and the CIM core for collaborative calculation.

[0074] This application introduces two thresholds and Each workload is classified. After reducing the computational precision of the current workload, if the loss in inference accuracy is less than (2%), the workload is considered to be insensitive to low precision and is mapped entirely to the CIM core to increase computation speed. If the loss in accuracy is greater than If the layer is less than 5%, the layer is considered to be highly sensitive to precision and is mapped to digital core particles to ensure computational accuracy. Otherwise, the layer is considered to be moderately sensitive to precision and is mapped to digital and CIM core particles to improve computational efficiency while ensuring accuracy.

[0075] In the case of medium sensitivity, the variance of all filters in the layer, that is, the weights of a DNN layer are composed of several filters, is first calculated, and then the filters are sorted according to the variance of the filters. The top N% filters with larger weight variances are the key filters that need to be calculated with high precision. This is because the quantization of weights from high precision to low precision will introduce errors, and filters with larger variance values ​​tend to have a wider weight distribution, so they are more prone to errors during the quantization process. Therefore, this part of the key filters that have a greater impact on the calculation accuracy are assigned to the digital core, and the rest of the weights are sent to the analog core for calculation.

[0076] The whole quantization process is as follows: first, the deep neural network DNN model is trained with full precision. After the training is completed, the weight of each layer of the deep neural network DNN model is quantized to 8-bit high precision, and the data precision is maintained for retraining to obtain a high-precision deep neural network DNN model; then, based on the obtained high-precision deep neural network DNN model, the sensitivity of each layer of the deep neural network DNN model is divided; then, each task load of the deep neural network DNN model is requantized according to the divided sensitivity, that is, the high-sensitivity layer keeps the accuracy unchanged, the low-sensitivity layer is quantized to low precision, and the weight of 1 / 16 of the medium-sensitivity layer is quantized to high precision and the other part is quantized to low precision, so as to obtain a sensitivity-quantized deep neural network DNN model. Finally, the sensitivity-quantized deep neural network DNN model is retrained with the current precision to restore the accuracy of the sensitivity-quantized deep neural network DNN model. It should be noted that in the training process of the final model, each round of training needs to select the key filter of the medium-sensitivity layer, because the weight situation may change after each round of training. When changes occur, it is necessary to quantize the high-precision filter with increased variance to low-precision, and dequantize the low-precision filter with reduced variance back to high-precision.

[0077] In the first phase, the sensitivity of each layer of the model has been obtained, and the key filters and non-sensitive filters of moderately sensitive task loads have been identified, providing data support for the establishment and optimization of the task map in the second phase.

[0078] In step 2, the task graph is constructed based on the preprocessing model. The task graph is to divide the calculation and storage of each layer of the DNN model into smaller tasks according to the hardware resource limitations, and further merge and query the tasks to ensure that the graph can be mapped to the hardware without hardware conflicts.

[0079] The specific steps include:

[0080] Step 2.1.1, Task Graph Initialization: In the initialization stage, refer to the deep neural network DNN model preprocessed in step 1 for modeling, without considering the limitations of core particles and integrated chips. Specifically, the following steps are included:

[0081] Step 2.1.1.1: In the initialization phase, only the pre-processed deep neural network DNN model is used for modeling, without considering the physical limitations of the core particles and integrated chips. According to the accuracy requirements of each layer of the deep neural network DNN model, all the same type of calculations in a layer are divided into a calculation task, and all storage processes are regarded as a storage task.

[0082] Step 2.1.1.2, Figure 1 As shown, for the layers with high sensitivity, all its calculations are regarded as a high-precision calculation task, which is executed on the digital core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with low sensitivity, all its calculations are regarded as a low-precision calculation task, which is executed on the in-memory computing core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with medium sensitivity, digital calculation tasks and in-memory computing tasks are established to represent the calculations of the layer on the digital core and the in-memory computing core respectively;

[0083] Step 2.1.1.3: Connect all task nodes in sequence according to the layer order to initially form a graph, which clarifies the data dependencies between tasks and the task type of each part of the calculation.

[0084] Step 2.1.2, task division: Add the resource restrictions of the core particles to the task map created in step 2.1.1, and divide the computing tasks and storage tasks. The computing tasks here include digital computing tasks and in-memory computing tasks, where: the division of digital computing tasks uses on-chip storage as the division basis, specifically based on whether the minimum storage required by the current digital computing task can be met by the on-chip storage of a digital core particle. If not, the current digital computing task is divided into two until the divided digital computing task meets the conditions. The reason for using on-chip storage as the division basis here is that the digital core particle itself is slower than the in-memory computing speed. If it is necessary to frequently and repeatedly read and write other on-chip storage or off-chip storage, it will inevitably become a bottleneck in the entire operation, especially in pipeline computing. Frequent reading and writing slows down the overall running time, which is unacceptable. The division of in-memory computing tasks is based on the size of the computing array. That is, if the weight required by the current in-memory computing task cannot be completely stored on an in-memory computing core particle, the in-memory computing task is divided into two. The division of storage tasks is based on whether the current storage task can be stored on one in-memory computing core particle. If not, the current storage task is divided into two until the storage task can be stored on one in-memory computing core particle.

[0085] The reason why in-memory computing tasks are divided according to the array size is that writing to the ReRAM array takes a long time. In order to improve operating efficiency, it is necessary to ensure that the in-memory computing core particles in the current task are not written multiple times. In addition, when the storage space required for storage tasks is greater than the additional storage that can be provided by an in-memory computing node, it is also necessary to split them. It should also be noted that each task split will cause the nodes in the task map to change, which requires continuous updating of the connection of the task map to ensure correct data dependencies. After the task is split, all tasks have met the restrictions of the corresponding core particles, that is, all tasks have met the requirements of being assigned to core particles according to the corresponding type.

[0086] Step 2.1.3: Merge and split task graphs.

[0087] After the previous step, each task in the task map can meet the constraints of hardware resources, but the current task map is still arranged completely in layer order. The number of tasks in each layer of the deep neural network DNN model, that is, each layer of the task map, is not balanced. Some layers have more tasks, and the integrated chip cannot provide enough cores at one time to meet their computing needs. Some layers have fewer tasks, and executing only the current layer on the integrated chip at a time will waste hardware resources. Therefore, in this step, it is necessary to introduce the number of digital cores and in-memory computing cores of the integrated chip and on-chip storage resources, merge and split the current task map, and make it conform to the task mapping actually running on the integrated chip.

[0088] like Figure 2 As shown, in step 2.1.3, the merging of the task graph refers to the layer-by-layer traversal of the segmented task graph, that is, if the number of tasks in the current layer is less than the number of cores that the integrated chip can provide, then check the tasks in the next layer and try to merge the next layer with the current layer. If the merger succeeds, continue to try to merge the next layer. If the merger fails, return to the loop to check the next layer. The splitting of the task graph is to split the current layer of the task graph if the number of tasks in the current layer is more than the number of cores that the integrated chip can provide.

[0089] Since the task graph is a directed acyclic graph, a layer of the task graph is also directed acyclic. Therefore, there must be one or more unidirectional paths between the starting node and the ending node of the layer. There must be data dependencies between the nodes on these paths. However, when only considering the task nodes of this layer, there is no data dependency between these paths. Therefore, when splitting the task graph, it is carried out in the order of depth traversal to ensure that an input and output path in the layer must be in a sublayer, and to ensure the correctness of the data dependency of the tasks placed in a sublayer. When splitting, first put as many tasks as possible in this layer into the first sublayer to improve utilization, and put all the remaining tasks into the second sublayer; in this way, the first sublayer must meet the hardware limit, and then judge the second sublayer. If it still exceeds the resource limit, continue to split until all sublayers of the current layer meet the resource limit; if the subsequent sublayers do not exceed the resource limit, then continue the layer-by-layer traversal and judgment from the current last sublayer.

[0090] As shown in the figure, it is expressed as:

[0091] Step 2.1.3-1, Initialization , task_graph[i] Represents the first layer;

[0092] Step 2.1.3-2, if , then execute step 2.1.3-3, otherwise end;

[0093] Step 2.1.3-3. Comparison task_graph[i] The required core particles and the number of core particles that the accelerator can provide, if it does not exceed the accelerator core particle limit, jump to step 2.1.3-4, otherwise jump to step 2.1.3-5;

[0094] Step 2.1.3-4, Initialization ,if , then go to step 2.1.3-5, otherwise end;

[0095] Step 2.1.3-5. Comparison task_graph[i] and task_graph[i+1] The required core particles and the number of core particles that the accelerator can provide, if it does not exceed the accelerator core particle limit, jump to step 2.1.3-6, otherwise jump to step 2.1.3-13;

[0096] Step 2.1.3-6, Merge task_graph[i] and task_graph[i+1] ;

[0097] Step 2.1.3-7, update data dependencies and jump to step 2.1.3-4;

[0098] Step 2.1.3-8, Initialization ,Will task_graph[i] The tasks in the deep traversal are divided into several paths. task_graph[i][j] Represents a path, creating and As a temporary layer;

[0099] Step 2.1.3-9, if task_graph[i][j] Put in Will not cause If the resource requirement exceeds the accelerator limit, execute step 2.1.3-10, otherwise execute step 2.1.3-11;

[0100] Step 2.1.3-10: task_graph[i][j] join in , , jump to step 2.1.3-9;

[0101] Step 2.1.3-11, task_graph[i] The remaining paths are added to ;

[0102] Step 2.1.3-11, replace task_graph[i] , insert task_graph[i] and task_graph[i+1] between;

[0103] Step 2.1.3-12, update data dependency, jump to 6;

[0104] Step 2.1.3-13, , skip to step 2.1.3-2.

[0105] Step 2.2.1, task graph optimization: The present invention optimizes the task graph using task replication and a pipeline strategy of a DNN accelerator.

[0106] Some deep neural network (DNN) models may not be able to fully utilize the hardware resources on the integrated chip. In this case, task replication can be used to improve resource utilization and reasoning performance.

[0107] like Figure 3 As shown in the figure, it is assumed that there is a 4-CIM / 4-Digital accelerator and a deep neural network DNN model that only requires three in-memory computing cores and three digital cores. Specifically, the first layer of the model is highly sensitive and needs to be executed on one digital core; the second layer is of low sensitivity and needs to be executed on one in-memory computing core; the third and fourth layers are of medium sensitivity, each requiring one in-memory computing core and one digital core. Before task replication, the second and fourth layers, especially the fourth layer, are the bottleneck of the entire computing process and take longer to execute. After replication, the weight of the C0 task of the second layer is replicated to two in-memory computing cores, namely C0 and C1, while the weight of the D2 task of the fourth layer is replicated to two digital cores, namely D2 and D3. Since the calculation is performed in parallel on multiple cores after task replication, the calculation delay of the two layers can be reduced by nearly half.

[0108] Therefore, in order to improve hardware utilization and reasoning performance, after generating the task graph, check whether the current graph will cause resource waste. If there are free cores, then loop through the current graph and copy and split the tasks with the longest computation time to free cores in turn based on the execution time until there are no more cores available for allocation. The task execution time is obtained from the architecture simulator (see the experimental section). During the copying process, it must be ensured that the core type matches the task type, that is, in-memory computing tasks must be copied to in-memory computing cores, and digital tasks must be copied to digital cores. Note that the data dependencies between the copied tasks need to be recalculated and the task graph updated.

[0109] In addition, the pipeline strategy of the DNN accelerator, namely the hierarchical pipeline, can be used for optimization, allowing multiple inference jobs to be executed simultaneously in a pipeline manner to make the most of the hardware.

[0110] like Figure 4As shown in the figure, after task replication, the number of tasks and hardware resources are more logically matched. However, the operation of the deep neural network DNN model is sequential, that is, the output of the previous layer will be used as the input of the next layer to start the calculation of the next layer after the input of the previous layer is calculated. Therefore, if the next DNN operation is performed after one DNN operation is completed, the hardware utilization will be very low. For example, when it is executed only once, the second layer calculation only uses two in-memory calculation core particles, and the other core particles are waiting / have completed the calculation. Sequential pipelining allows these core particles that have completed the task to calculate the next job instead of waiting all the time. When enough jobs are executed sequentially in a sequential pipelining manner, the accelerator can achieve a high hardware utilization rate most of the time. At the same time, the longest level in the sequential pipelining determines the inference throughput, which is the 4th level in this example. Task replication can potentially reduce the duration of the key pipeline stage, that is, the longest pipeline level, and improve the throughput of the accelerator.

[0111] In summary, the combination of task replication and hierarchical pipelining can effectively reduce the latency of a single inference while improving the overall computing throughput.

[0112] After the task graph step is completed, the entire graph satisfies the hardware constraints of the core particles and each layer satisfies the resource constraints of the accelerator on the basis of satisfying the quantization results. However, the current graph has no actual relationship with the hardware, so it is necessary to further map the tasks of each layer of the task graph to each specific core particle in the accelerator. Therefore, the execution graph is further introduced here. The execution graph describes the specific mapping and allocation relationship between tasks and core particles, as well as the execution order and start time of tasks. Only when the execution graph is determined can the DNN model actually run in the accelerator and further optimize the operation process.

[0113] The process of initializing the execution map is specifically to traverse each layer of the task map first. Since the restrictions of the core particles and the accelerator have been met, each layer of the task load of the task map can be executed on the accelerator. Here, each task in the current task load is randomly mapped, and the three tasks contained in the task load are traversed respectively, and the corresponding tasks are randomly mapped to the corresponding core particles, that is, digital computing tasks are assigned to digital core particles, in-memory computing tasks are assigned to in-memory computing core particles CIM, and storage tasks are assigned to in-memory computing core particle groups or dynamic random access memory DRAM. The task map is combined with the actual hardware through random mapping to form an initial execution map. In this way, all hardware restrictions and integrated chip on-chip network (NoP) restrictions can be reflected in the execution map, which is a graph model, and paves the way for subsequent optimization. The mapping of tasks to hardware will affect the performance of the accelerator. The present invention can solve the various problems caused by the lack of CIM core particles / digital core particles resources and optimize the on-chip and off-chip data transmission overhead based on the execution map.

[0114] like Figure 5 As shown, the specific process of initializing the execution graph is:

[0115] Step 1. Initialization , task_graph[i] Represents the first layer;

[0116] Step 2: If , then execute step 3, otherwise the graph initialization ends;

[0117] Step 3: Initialization , task_graph[i] .DigitalTasks[j] Indicates the first Layer A digital computing task;

[0118] Step 4: If j < len(task_graph[i].DigitalTasks[j]) , then go to step 5, otherwise go to step 7;

[0119] Step 5: Randomly select one from all the free digital chips and task_graph[i].DigitalTasks[j] Mapping to the digital core particle completes the binding of the digital core particle and the digital computing task;

[0120] Step 6 , jump to step 4;

[0121] Step 7: Initialization , task_graph[i] .CIMTasks[j] Indicates the first Layer an in-memory computing task;

[0122] Step 8: If j < len(task_graph[i].CIMTasks[j]) , then go to step 9, otherwise go to step 11;

[0123] Step 9: Randomly select one of the free CIM cores and task_graph[i].CIMTasks[j] Map it to the CIM core particle to complete the binding between the CIM core particle and the in-memory computing task;

[0124] Step 10 , jump to step 8;

[0125] Step 11. Initialization , task_graph[i] .StorageTasks[j] Indicates the first Layer storage tasks;

[0126] Step 12: If j < len(task_graph[i].StorageTasks[j]) , then go to step 13, otherwise go to step 17;

[0127] Step 13: If the current chip still has on-chip storage space, execute step 14; otherwise, execute step 15;

[0128] Step 14: Randomly select a CIM chip with free storage space and task_graph[i].StorageTasks[j] Map to the CIM core particle, complete the binding of the CIM core particle and the storage task, and jump to step 16;

[0129] Step 15: task_graph[i].StorageTasks[j] Map to off-chip storage and jump to step 16;

[0130] Step 16 , jump to step 12;

[0131] Step 17 , skip to step 2.

[0132] However, the current execution graph is only a random mapping of tasks to core particles, and further optimization is needed to improve the hardware execution efficiency. The present invention uses the inference time determined by the task calculation and data transmission delay as the objective function. The goal of optimizing the execution graph is to find an excellent mapping method of the task graph to the integrated chip under the given objective function, so that the overall delay is the shortest. The challenge of this optimization is that tasks can be executed in parallel on different types of core particles, and the communication delay is non-deterministic and affected by the task placement and the network traffic pattern on the integrated chip. Therefore, it is necessary to find the critical path of the entire calculation process to properly estimate the delay of the inference job.

[0133] In the execution graph, due to the size of the deep neural network (DNN) model and the on-chip storage space, some storage tasks may be mapped to the off-chip main memory. Therefore, the execution graph optimization specifically includes two execution graph methods: task mapping based on communication volume awareness or task planning based on transmission delay.

[0134] The following steps are involved in performing graph optimization:

[0135] Step 3.1: Depending on whether the weights of the deep neural network DNN model are stored on the integrated chip, choose to use a task mapping method based on traffic perception to optimize the execution graph or a task planning method based on transmission delay to optimize the execution graph. Specifically:

[0136] When all weights of a deep neural network (DNN) model are stored on an integrated chip, there is frequent data movement between the in-memory computing core and the digital core due to factors such as data dependency and weight reading. If tasks are mapped considering the data dependency between tasks and the correlation of the integrated chip topology, such a task mapping algorithm may cause on-chip network congestion and affect overall performance. In order to minimize network congestion and data transmission distance, the communication bandwidth requirements of the integrated chip on-chip network topology and the model must be considered during the task mapping process.

[0137] Figure 6 The “good” and “bad” mappings of a 4-CIM / 4-Digital system are shown. In this example, assume that a precision-sensitive layer is computed on a 4-CIM / 4-Digital accelerator. This layer uses D0~3 digital cores for computation, and weights and intermediate data are stored on CIM cores. In the “good” mapping example, the (critical) filter and the input of the i-th layer are stored in C0 and C2 cores, respectively, making full use of multiple data paths and relatively balanced on-chip network traffic. In the “bad” mapping example, the i-th layer is stored in C0 and C2 cores, making full use of multiple data paths and relatively balanced on-chip network traffic. The filters and inputs of the layer are mapped to the same CIM chip C1. The digital cores all read the weights from C1 and transfer the calculated intermediate data to C1 for storage. Such a mapping will cause serious congestion in the network link between C1 and D1, and the path taken by each integrated chip on-chip network data packet is longer than in the "good" mapping example. When the workload requires multiple rounds of digital core calls, the communication overhead will accumulate and eventually affect performance. In order to minimize the on-chip network congestion and data transmission distance, the execution graph optimizer needs to understand the communication volume of the integrated chip on-chip network and its impact on the inference time.

[0138] In order to estimate the traffic on the integrated chip on-chip network, the present invention uses a task mapping method based on traffic perception to optimize the execution graph, maintains a two-dimensional array for each execution graph to track link utilization to estimate the traffic on the integrated chip on-chip network, wherein each element represents an integrated chip on-chip network link, the source and destination nodes of each traffic flow are determined, and the exact path of each flow and the number of flows sharing the same link are calculated according to the routing algorithm. Assuming that there are P data flows sharing a certain same link, the integrated chip on-chip network bandwidth of each flow is 1⁄P of the link bandwidth, and the communication delay is inferred based on the traffic volume, and the congestion and utilization of the entire network are evaluated. The communication cost, that is, the sum of the cumulatively transmitted traffic on all links of all tasks in the execution graph, is used as part of the objective function to optimize the network and performance:

[0139] Communication Cost= ∑ i ∈ tasks [ Vol input i ×Dist input i +Vol filter i ×Dist filter i +Vol output i ×Dist output i ]

[0140] in, and They represent the data volume of the current task and the number of hops required to transmit the data of the current task. is an iterator representing a task selected from all tasks. Indicates the execution of all tasks in the graph. Represents the input data required for the current task, Represents the weight data required for the current task, Represents the output data calculated by the current task;

[0141] When the weights of a deep neural network (DNN) model cannot be fully stored on the chip, off-chip memory access is inevitably required. In this type of calculation process, the weights of the first m layers that can be stored are first loaded from the main memory to the CIM core particles according to the size of the deep neural network (DNN) model and the available storage. After the calculation of the current m-layer DNN model is completed, the weights of the subsequent n layers are read from the memory to continue the calculation. Since the delay of off-chip memory access is much greater than the on-chip data transmission, and during the off-chip memory access to load weights, the calculation will be suspended until all data is loaded on the chip. Therefore, off-chip memory access is the biggest bottleneck for acceleration in this case.

[0142] In order to reduce the off-chip memory access overhead and stall cycles, the present invention proposes a time overlap solution, that is, the software needs to adjust the execution time and order of tasks to overlap the weight reading and calculation time as much as possible. Figure 7 It shows how to hide the overhead of weight reads through proper task mapping and scheduling. Assume that for a 4-CIM / 4-Digital accelerator, the data for the first four layers of a large model occupy all CIM cores C0-C3. Before the fifth layer starts, the weights must be read from memory. If the controller can know that the storage of C0 will be used up first, and the freed storage is large enough to accommodate the weights of the fifth layer, then the weights of the fifth layer can be read immediately after the first layer is completed. In this way, the data read time overlaps with the processing of layers 2-4. Similarly, for C1, the weight data of the sixth layer is read immediately after the calculation of the second layer is completed. In this way, the reading of the weights of the sixth layer overlaps with the processing of layers 3-5 (including the possible stall of the previous off-chip reading of the fifth layer). If the off-chip memory access and calculation are not overlapped, the subsequent weights need to be loaded uniformly after the fourth layer is completed. The delay caused by this is the time for the entire C0 and C1 to read the data, which is much longer than the current stall. However, this will make the data transmission of the entire accelerator more complicated, and it is difficult to determine the data communication cost and execution start time of the task.

[0143] To address this challenge, the present invention proposes a delay estimation algorithm to calibrate the timing and duration of each task. Figure 8 As shown, the delay estimation algorithm of the present invention is specifically:

[0144] Step 1: Calculate the on-chip network link status without considering the mutual interference of data transmission , data transmission delay of on-chip network and off-chip transmission delay ;

[0145] Step 2: Initialization ;

[0146] Step 3: If , then execute step 4, otherwise end. Represents the execution graph;

[0147] Step 4: Initialization ;

[0148] Step 5: If j < len(Latencies_ddr[i]) , then execute step 6, otherwise execute step 18, where latencies_ddr[i] Indicates Latency data of off-chip memory access in the layer execution graph;

[0149] Step 6: Initialization ;

[0150] Step 7: If k < len(Latencies[i]) , then execute step 8, otherwise execute step 11, where latencies[i] Indicates Latency data for on-chip data transfers in the layer execution graph;

[0151] Step 8: If and If there is overlap, go to step 9, otherwise go to step 10. Indicates The first layer in the execution graph Delay data for on-chip data transfer, latencies_ddr i [j] Indicates The first layer in the execution graph Delay data for off-chip memory access;

[0152] Step 9: Correction and Latencies[i][k] ;

[0153] Step 10 , jump to step 7;

[0154] Step 11. Initialization ;

[0155] Step 12: If k < len(Latencies[i + 1]) , then go to step 13, otherwise go to step 17;

[0156] Step 13: If and If the calculated tasks belong to the same layer in the original DNN model, execute step 14, otherwise execute step 15;

[0157] Step 14: Extension Latencies[i + 1][k] and Latencies_ddr[i][j] The time corresponding to the jump to step 16;

[0158] Step 15: If Latencies_ddr[i][j] and Latencies[i + 1][k] If there is time overlap, correct it and Latencies[i + 1][k] , otherwise go directly to step 16;

[0159] Step 16 , jump to 7.1;

[0160] Step 17 , jump to 6;

[0161] Step 18 , jump to 3.

[0162] like Figure 9 The simulated annealing optimization algorithm used by the present invention is simple and can find a suitable task mapping solution for the architecture.

[0163] Specifically, in each iteration of the simulated annealing process, a task type is randomly selected: a storage task, a computation task of an in-memory computing core particle, or a computation task in a digital core particle; two task nodes of this type of task are randomly selected in the current execution graph, and the mapping of these two tasks is exchanged, that is, one of the tasks is mapped to the core particle of the other task; this will change the traffic pattern of the integrated chip on-chip network and the critical path of the task execution process when the execution graph is calculated on the accelerator. After the exchange, the inference time of the new mapping scheme is estimated and compared with the inference time before and after the exchange. If the time required for inference is reduced, the change is accepted; otherwise, the inference time is reduced. The probability of accepting a change, where and They are the cost function calculation results of the new mapping scheme and the old mapping scheme respectively. The cost function here refers to the communication volume or delay.

[0164] like Figure 9 As shown, the simulated annealing optimization algorithm of the present invention is specifically:

[0165] Step 1: Initialize simulated annealing temperature ;

[0166] Step 2: If , execute step 3, otherwise end;

[0167] Step 3: Initialization ;

[0168] Step 4: If , then go to step 5, otherwise go to step 10, is the number of iterations;

[0169] Step 5: Randomly select a task type, randomly select two tasks of this type in the current execution graph and exchange the mapped core particles;

[0170] Step 6: Recalculate the running cost, where the cost refers to the communication volume or delay;

[0171] Step 7: Calculation ,if , then accept the change and execute step 9, otherwise execute step 8. and They are the cost function calculation results of the new mapping scheme and the old mapping scheme respectively. The cost function here refers to the communication volume or delay;

[0172] Step 8. Randomly select a number from 0 to 1. If , then accept the change, otherwise roll back;

[0173] Step 9 , jump to step 4;

[0174] Step 10 ,in is the annealing coefficient, jump to step 2.

[0175] The software simulator of the present invention evaluates the optimization method of the present invention to verify its effectiveness. The digital core particles are modeled based on the hardware parameters using the digital DNN simulator based on Timeloop, while the CIM core particles are modeled based on the adjusted NeuroSim simulator. The integrated chip on-chip network is modeled using BookSim. In the evaluation, the present invention uses DenseNet121, ResNet18, ResNet34, AlexNet, VGG8, VGG16, VGG19, BERT and GPT-1 as test cases.

[0176] Figure 10The overall performance of the accelerator is shown. The data in the figure is normalized based on the computing time of the initial task of each DNN model without optimization on the accelerator. The Ideal category is set here to be an ideal state of the integrated chip on-chip network, that is, the bandwidth is infinite and there is no congestion. According to the results in the figure, by splitting large tasks into multiple small tasks that can be executed in parallel to improve chip utilization and optimize on-chip / off-chip data transmission, the inference time can be significantly reduced. Compared with the test benchmark, this acceleration design can achieve a performance improvement of 30%~260%,

[0177] Figure 11 The utilization of each corelet during the computation of a portion of the model is shown as the total number of cycles to perform the computation divided by the total number of inference cycles. The normalized benchmark uses the utilization of the execution graph without any optimizations, so the utilization of the benchmark is unbalanced, with some corelets being busier than others. After a series of optimizations, namely task replication, communication-aware optimized mapping, and scheduling off-chip data transfers to hide data movement latency, the utilization of digital and CIM corelets was improved on all three models, with the optimized computations being able to nearly double the utilization of the corelets compared to the baseline.

[0178] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An optimization method for a multi-chip in-memory computing hybrid DNN accelerator, characterized by: The optimization method specifically comprises the following steps: Step 1, model preprocessing: preprocess the deep neural network DNN model to be calculated, and confirm the calculation method of each layer of the deep neural network DNN model according to the digital core particles and the in-memory computing core particles in the in-memory computing digital-analog hybrid DNN accelerator of the multi-core particles, and the calculation method includes three methods: calculation processing by digital core particles, calculation processing by in-memory computing core particles, and collaborative calculation processing by digital core particles and in-memory computing core particles; Step 2: Construction and optimization of task graph: According to the resource constraints of computing resources and storage resources, the task load of the deep neural network DNN model is divided into computing tasks and storage tasks. The task graph is constructed according to the connection of each node on the task graph, that is, the data dependency between tasks. The task graph is optimized by using task replication and the pipeline strategy of DNN accelerator. Step 3: Construction and optimization of execution graph: Based on the topology of the hybrid DNN accelerator for in-memory computing of multiple cores, the computing tasks and storage tasks are mapped to the digital cores and in-memory computing cores on the chip, and the task graph constructed in step 2 is converted into an execution graph for actual computing, and the execution graph is optimized; wherein, The following steps are involved in performing graph optimization: Step 3.1: Depending on whether the weights of the deep neural network DNN model are stored on the integrated chip, choose to use a task mapping method based on traffic perception to optimize the execution graph or a task planning method based on transmission delay to optimize the execution graph. Specifically: When all weights of the deep neural network DNN model are stored on the integrated chip, the execution graph is optimized using a task mapping method based on traffic awareness. A two-dimensional array is maintained for each execution graph to track link utilization to estimate the traffic of the integrated chip on-chip network, where each element represents an integrated chip on-chip network link. The source and destination nodes of each traffic flow are determined, and the exact path of each flow and the number of flows sharing the same link are calculated according to the routing algorithm. Assuming that there are P data flows sharing the same link, the integrated chip on-chip network bandwidth of each flow is 1 / P of the link bandwidth. The communication delay is inferred based on the traffic volume, and the congestion and utilization of the entire network are evaluated. The communication cost, that is, the sum of the cumulative traffic transmitted on all links of all tasks in the execution graph, is used as part of the objective function to optimize the network and performance: Where Vol(·) and Dist(·) represent the data volume of the current task and the number of hops required to transmit the data of the current task, respectively; i is an iterator, which represents a task selected from all tasks, tasks represents all tasks in the execution graph, and input i Indicates the input data required for the current task, filter i Represents the weight data required for the current task, output i Represents the output data calculated by the current task; When the weights of a deep neural network (DNN) model cannot be fully stored on the chip, off-chip memory access is required. The execution graph is optimized using a task planning method based on transmission delay. The weight adjustment is overlapped by adjusting the execution time and order of tasks. The execution time and order of each task are calibrated using a delay estimation algorithm to obtain relatively accurate task calculation and data transmission delays. The task graph is then optimized using an execution graph optimizer. Step 3.2: Bring the Communication Cost or delay estimation algorithm in step 3.1 into the simulated annealing algorithm to optimize the execution graph. The simulated annealing algorithm includes the following steps: Step 3.2.

1. In each iteration of the simulated annealing process, randomly select a task type: a storage task, a computation task in an in-memory computation core particle, or a computation task in a digital core particle; Step 3.2.2, randomly select two task nodes in the current type of task in the current execution graph, and exchange the mapping of the two tasks, that is, map one task to the core particle of the other task; Step 3.2.3: After the swap, estimate the reasoning time of the new mapping scheme and compare the reasoning time before and after the swap. If the reasoning time is reduced, accept the change; otherwise, replace the new mapping scheme with a probability of e. -1×(new _ cost-cost) / T Accept the change, where new_cost and cost are the cost function calculation results of the new mapping scheme and the old mapping scheme respectively. The cost function here refers to the communication volume or delay; In step 3.1, the delay estimation algorithm includes the following steps: Step 3.1.1, without considering traffic interference, calculate the time of calculation and data transmission, configure an initial start timestamp and end timestamp for each calculation and storage task to characterize its task execution time, and determine whether a task overlaps with another task by comparing the timestamps; Step 3.1.2: If overlapping tasks involve data movement, their data transmission will interfere with each other, and the data transmission delay needs to be recalculated, the timestamps of related tasks need to be corrected, and the related subtasks need to update their own timestamps.

2. The optimization method for a multi-chip in-memory computing hybrid DNN accelerator according to claim 1, characterized in that: In step 2, the construction of the task graph specifically includes the following steps: Step 2.1.1, task graph initialization: In the initialization stage, refer to the deep neural network DNN model preprocessed in step 1 for modeling, without considering the limitations of core particles and integrated chips; Step 2.1.2, Task segmentation: Add the resource restrictions of the core particles to the task map created in step 2.1.1, segment the computing tasks and storage tasks, and continuously update the connection of the task map to ensure the correctness of data dependencies. After task segmentation, all tasks have met the restrictions of the corresponding core particles, that is, all tasks have met the requirements of being assigned to core particles according to the corresponding types; Step 2.1.3, merging and splitting of task maps: Introduce the number of digital core particles and in-memory computing core particles of the integrated chip as well as the on-chip storage resources, merge and split the current task map to make it conform to the task mapping actually running on the integrated chip.

3. The optimization method for a multi-chip in-memory computing hybrid DNN accelerator according to claim 1, characterized in that: Step 2.1.1 Task graph initialization specifically includes the following steps: Step 2.1.1.1: According to the accuracy requirements of each layer of the deep neural network DNN model, all the same type of calculations in a layer are divided into one computing task, and all storage processes are regarded as one storage task; Step 2.1.1.2: For the layers with high sensitivity, all their calculations are regarded as a high-precision calculation task and executed on the digital core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with low sensitivity, all their calculations are regarded as a low-precision calculation task and executed on the in-memory computing core in the multi-core in-memory computing digital-analog hybrid DNN accelerator; for the layers with medium sensitivity, digital calculation tasks and in-memory computing tasks are established to represent the calculations of the layer on the digital core and the in-memory computing core respectively; Step 2.1.1.3: Connect all task nodes in sequence according to the layer order to initially form a graph, which clearly defines the data dependencies between tasks and the task type of each part of the calculation.

4. The optimization method for a multi-chip in-memory computing hybrid DNN accelerator according to claim 3, characterized in that: In step 2.1.2, the computing tasks include digital computing tasks and in-memory computing tasks, where: The division of digital computing tasks uses upper storage as the division basis, specifically, whether the minimum storage required by the current digital computing task can be met by the on-chip storage of a digital core particle. If not, the current digital computing task is divided into two until the divided digital computing task meets the conditions; The in-memory computing task is divided according to the size of the computing array. That is, if the weight required by the current in-memory computing task cannot be completely stored on one in-memory computing core particle, the in-memory computing task is divided into two. The division of storage tasks is based on whether the current storage task can be stored on one in-memory computing core particle. If not, the current storage task is divided into two until the storage task can be stored on one in-memory computing core particle.

5. The optimization method for a multi-chip in-memory computing hybrid DNN accelerator according to claim 2, characterized in that: In step 2.1.3, The merging of task graphs refers to the layer-by-layer traversal of the divided task graphs. That is, if the number of tasks in the current layer is less than the number of cores that the integrated chip can provide, the next layer of tasks will be checked and the next layer will be tried to be merged with the current layer. If the merger succeeds, the next layer will be tried again. If the merger fails, the next layer will be checked in a loop. The task map is split if the number of tasks in the current layer is greater than the number of cores that the integrated chip can provide. In this case, the current layer of the task map is split.

6. The optimization method for a multi-chip in-memory computing mixed analog-digital DNN accelerator according to claim 5, characterized in that: The task graph is split in the order of depth traversal to ensure that an input and output path in the layer is in a sub-layer and the data dependency of the tasks placed in a sub-layer is correct. The specific steps include: Step 2.1.3.

1. Put as many tasks as possible in this layer into the first sub-layer, and put all the remaining tasks into the second sub-layer; Step 2.1.3.2: Judge the second sublayer. If it still exceeds the resource limit, continue splitting until all sublayers of the current layer meet the resource limit. If the subsequent sublayers do not exceed the resource limit, continue the layer-by-layer traversal and judgment from the current last sublayer.

7. The optimization method for a multi-chip in-memory computing mixed analog-digital DNN accelerator according to claim 1, characterized in that: The task duplication for optimizing the task graph in step 2 means, after the task graph is generated, detecting whether the current graph will cause waste of resources. If there are idle core particles, the current graph is detected cyclically, and the tasks with the longest calculation time are copied and split onto idle core particles in turn based on the execution time, until there are no more core particles available for allocation. The data dependencies between the copied tasks are recalculated and the task graph is updated.

8. The optimization method for a multi-chip in-memory computing hybrid DNN accelerator according to claim 1, characterized in that: The execution graph construction in step 3 is specifically to initialize the execution graph: traverse each layer of the task graph, randomly map each task in the current task load, traverse the three tasks included in the task load respectively, and randomly map the corresponding tasks to the corresponding core particles, that is, digital computing tasks are assigned to digital core particles, in-memory computing tasks are assigned to in-memory computing core particles, and storage tasks are assigned to in-memory computing core particle groups or dynamic random access memory DRAM. The task graph is combined with the actual hardware through random mapping to form an initial execution graph.

Images