DNN inference acceleration system and method for resource-constrained computing and storage integrated chip

Abstract

The application discloses a DNN inference acceleration system and method for a resource-restricted storage and calculation integrated chip, which comprises a weight distribution module, a OU scheduling module, a weight programming module and an inference module.The weight distribution module is used for acquiring neural network information and ReRAM structure information in an offline state, and generating a mapping table of a weight block with an OU size to a physical OU.The OU scheduling module is used for inputting the neural network information, the ReRAM structure and the mapping table information to an OU scheduler in the offline state, and generating a scheduling table.The weight programming module is used for writing the weight block with the OU size into a designated physical OU according to the generated mapping table in an online state.The inference module is used for reading the scheduling table and activating the calculation on the OU with a corresponding input feature vector in the online state.The weight programming and inference are iteratively run until the DNN inference on all input data is completed.The programming delay of the ReRAM is optimized, and the DNN inference on the resource-restricted ReRAM chip is accelerated.

Description

Technical Field

[0001] This invention relates to the field of in-memory computing chip technology, and in particular to a DNN inference acceleration system and method for resource-constrained in-memory computing chips. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Deep Neural Networks (DNNs) have been widely used in image recognition, natural language processing, and object detection. However, while these DNNs offer high accuracy, they also require storing and computing hundreds of millions of weights. For example, VGG16 is a typical DNN model with a top-5 accuracy of 92.7%, but its weight size is approximately 548MB, requiring 1.5 × 10^6 weights per image recognition. 10 Multiple multiplications are required. For modern computer systems based on the traditional von Neumann architecture, running such a large-scale DNN model requires repeatedly transferring a large amount of data between storage and computing units, resulting in huge latency and energy consumption, thus limiting the efficiency of data processing.

[0004] Process-in-Memory (PIM) computing was proposed to overcome the computational limitations of the traditional von Neumann architecture. It eliminates the need to transfer data from memory to the processor, instead integrating computation directly into the memory array. This technology eliminates data transfer between memory and the processor, and therefore its applications in various fields of AI are being widely explored.

[0005] Novel non-volatile memory (NVM), including static random-access memory (SRAM), phase-change memory (PCM), magnetoresistive random access memory (MRAM), and resistive random access memory (ReRAM), can all be used for in-memory computing. ReRAM, in particular, offers access performance similar to DRAM and supports in-situ computation of matrix-vector multiplication (MVM). In recent years, several studies have explored ReRAM-based DNN accelerators.

[0006] As exploration of ReRAM-based DNN accelerators deepens, the potential of ReRAM for accelerating DNNs continues to be explored: from the design of ReRAM-based microarchitectures to the design of neural network inference and even training pipelines that conform to the characteristics of ReRAM, and further to the pruning, quantization, and reuse of DNN weights for ReRAM, optimization methods are emerging in increasing depth. However, these methods ideally assume that the entire crossbar can be computed within one clock cycle. In reality, the deviation generated by the current accumulation in each ReRAM cell can seriously affect the inference accuracy of the neural network. Therefore, some works have proposed a more practical structure that activates only a small region of each crossbar per cycle (also known as the Operation Unit, OU). Subsequently, authors further proposed pruning and reuse methods based on OU to reduce the area requirements and computational overhead of ReRAM based on this structure.

[0007] However, all previous work assumed that ReRAM resources were large enough that all DNN weights could be programmed into ReRAM at once with negligible programming latency. But recent research in the field of integrated circuit design shows that the chip capacity of ReRAM in area-constrained embedded or edge devices is far smaller than the current weight size of DNNs.

[0008] The inventors discovered that before performing DNN inference, it is impossible to deploy all the weights of the neural network onto the chip offline. Instead, it is necessary to deploy them online multiple times, with only a portion of the weights deployed each time. The overhead of deploying weights online multiple times may reduce the inference speed of the DNN in ReRAM. Summary of the Invention

[0009] To address the aforementioned issues, this invention proposes a DNN inference acceleration system and method for resource-constrained in-memory computing chips. By comprehensively considering the realistic ReRAM chip resources and the programming characteristics of the ReRAM chip itself, it greatly improves the inference speed of DNNs on ReRAM under resource constraints.

[0010] In some implementations, the following technical solutions are adopted:

[0011] A DNN inference acceleration system for resource-constrained in-memory computing chips includes:

[0012] The weight allocation module is used to acquire neural network information and ReRAM structure information in offline mode, and generate a mapping table from an OU-sized weight block to a physical OU. The mapping table is used to store the weight block information at each OU position when the neural network weights are deployed on ReRAM each time.

[0013] The OU scheduling module is used to input neural network information, ReRAM structure and mapping table information into the OU scheduler in offline mode to generate a scheduling table; the OU scheduler sorts the input feature vectors of each neural network layer and converts them into the input voltage of the corresponding crossbar; the scheduling table is used to store the number of times each OU runs and the running order, and to control the input of feature vectors and the output position of the results after running.

[0014] The weight programming module is used to write weight blocks of size OU into the specified physical OU based on the generated mapping table in online mode;

[0015] The inference module is used to read the scheduling table in an online state and activate the computation on OU with the corresponding input feature vector;

[0016] The weight programming module and inference module run iteratively until DNN inference is completed for all given input data.

[0017] Furthermore, the weight programming module programs neural network weights of the ReRAM device capacity onto the hardware device each time. The hardware device contains multiple crossbars, and each crossbar has a corresponding mapping table for each weight deployment stage.

[0018] The crossbars on ReRAM run in parallel, with only one OU being activated for computation at any given time for each crossbar, and each crossbar has a corresponding schedule table for each inference phase.

[0019] Each crossbar on the ReRAM reads data from the scheduling table to obtain the OU location to be activated and the input feature vector of that OU for this calculation. By converting the input feature vector into a voltage input to the bit line of that OU, the result of this calculation is obtained. The output feature vector information in the scheduling table is then obtained and the calculation result is transmitted to the corresponding location for storage.

[0020] In other embodiments, the following technical solutions are adopted:

[0021] A method for accelerating DNN inference in a resource-constrained in-memory computing chip-based DNN inference acceleration system includes:

[0022] Based on the ReRAM-based programming latency model, optimization strategies for programming latency are derived; these optimization strategies include: similarity matching strategy, balanced mapping strategy, and distance-aware strategy.

[0023] The weights of the trained neural network are quantized into integers, the neural network weights are divided into deployment layers of ReRAM size, and the weights of each deployment layer are divided into weight blocks in units of OU.

[0024] Based on similarity matching strategy, balanced mapping strategy and distance-aware strategy, a mapping table from each weight block to the physical OU is generated;

[0025] A scheduling table is generated based on the neural network structure, quantized weight information, ReRAM structure, and mapping table information.

[0026] The above process is implemented offline.

[0027] Furthermore, the programming latency model of the ReRAM is specifically as follows: the programming latency unit of the ReRAM is decomposed from small to large into single programming latency, single OU programming latency, single crossbar programming latency and the programming latency of the entire ReRAM;

[0028] Among them, the programming latency of a single OU is the sum of the programming latencies of all single operations in that OU, the programming latency of a single crossbar is the sum of the programming latencies of all OUs in that crossbar, and the programming latency of the entire ReRAM is the programming latency of the single crossbar with the largest programming latency.

[0029] The similarity matching strategy is as follows: reduce the number of global RESET operations by utilizing the similarity of OUs: when all units of parallel operations do not require a RESET operation from 1 to 0 in a single programming operation, this operation can be omitted.

[0030] The specific equalization mapping strategy is as follows:

[0031] By adjusting the similarity matching order of OUs, the programming latency of each crossbar can be balanced.

[0032] The distance-aware strategy specifically involves adjusting the position of OUs that have completed similarity matching, moving frequently operated OUs to positions within the crossbar where RESET programming latency is lower, thereby reducing single-crossbar programming latency.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] (1) This invention provides a DNN inference acceleration system for resource-constrained in-memory computing chips. By utilizing the information from the mapping table generated by the weight allocation module and the scheduling table generated by the OU scheduling module, the activation order, number of times, and transmission of input and output data of OUs are reasonably arranged, thereby realizing accurate and high-speed inference of DNNs on resource-constrained ReRAM chips.

[0035] (2) Deploying the entire DNN on a resource-constrained ReRAM accelerator is impractical. Inference of a single neural network requires multiple weight deployments, which are very time-consuming. Therefore, this invention decomposes the ReRAM programming latency, explores the factors affecting programming latency, and models the ReRAM programming latency. Based on the ReRAM programming latency modeling formula, a DNN inference acceleration method for resource-constrained in-memory computing chips is proposed. This method includes a similarity matching strategy, a balanced mapping strategy, and a distance-aware strategy. These three strategies optimize the programming latency of a single OU, a single crossbar, and the entire ReRAM, respectively, greatly accelerating DNN inference on resource-constrained ReRAM chips.

[0036] Other features and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the DNN inference acceleration system structure for resource-constrained in-memory computing chips in an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the DNN inference acceleration method in an embodiment of the present invention;

[0039] Figure 3 This is a flowchart of an example of the OU scheduler in an embodiment of the present invention. Detailed Implementation

[0040] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0042] Terminology Explanation:

[0043] Crossbar: One of the common architectures of ReRAM, and also the basic physical unit used for computation in ReRAM. A crossbar consists of word lines and bit lines, as well as resistors (also called cells) connecting the word lines and bit lines. By converting the input into a voltage input to the word line, the weights exist in the form of conductance (the reciprocal of resistance) in the resistors connected to the word and bit lines. Based on Ohm's law, a current is formed on the bit line, thus performing a multiplication operation. With resistors connected in parallel, the current on the bit lines accumulates, completing an addition operation. The crossbar is surrounded by peripheral circuits such as sample-and-hold circuits and shift-and-add circuits to assist in multi-digit multiplication and addition operations. A crossbar typically consists of 64*64 to 1024*1024 cells.

[0044] OU: Previous work has shown that the current deviation of each unit in the crossbar calculation process accumulates on the bit line, which will seriously affect the inference accuracy. Therefore, it is proposed to activate only a small part of the crossbar each time (usually between 4*4 and 32*32), that is, one OU (operation unit).

[0045] SET: In each cell of the crossbar, high resistance represents 0 and low resistance represents 1. The process of changing the cell from high resistance to low resistance by applying voltage is called SET operation.

[0046] RESET: The process of changing the resistance of a cell from low to high by applying voltage is called RESET operation.

[0047] Programming latency: During the inference process of a neural network on ReRAM, the weights of the neural network need to be written into each cell of the crossbar in the form of conductance. This writing process is called the programming process, and the latency required for the writing process is called the programming latency.

[0048] Example 1

[0049] In one or more embodiments, a DNN inference acceleration system for resource-constrained in-memory computing chips is disclosed, combining... Figure 1 It is divided into offline and online phases.

[0050] The offline phase mainly includes the weight allocator module and the OU scheduler module, as detailed below:

[0051] (1) Weight allocation module, used to obtain neural network information and ReRAM structure information in offline state, generate a mapping table of weight blocks of size OU to physical OU, the mapping table is used to store the weight block information of each OU position when the neural network weights are deployed on ReRAM each time;

[0052] The neural network information includes the trained neural network weights and the neural network structure; the ReRAM structure information includes information such as the crossbar size and OU size.

[0053] In resource-constrained ReRAM, the resources required for neural network deployment often exceed the hardware resources. Therefore, the neural network needs to be divided into multiple weight blocks and deployed multiple times to complete the inference process. During the deployment process, changes in the original data and new data in the hardware will cause different programming latencies. The task of the weight allocation module is to generate a mapping table from the OU size of the weight block to the physical OU with low overall programming latency. The mapping table is used to store the weight block information at each OU position when the neural network weights are deployed on ReRAM each time.

[0054] (2) The OU scheduling module is used to input the neural network structure, ReRAM structure, and mapping table information into the OU scheduler in offline mode to generate a scheduling table. The OU scheduler sorts the input feature vectors of each neural network layer and converts them into the input voltage of the corresponding crossbar. The scheduling table is used to store the number of runs and the running order of each OU, and to control the input of feature vectors and the output position of the results after the runs. In the mapping table generated by the weight allocator, we allow a weight block to be mapped to any physical OU to reduce the weight programming cost at runtime. To ensure the accurate and efficient operation of DNN inference, this embodiment designs an "OU scheduler" module in the offline stage of the proposed architecture.

[0055] In OU-based ReRAM, multiple OUs in a single crossbar run serially. The OU scheduler arranges the input feature vectors of each DNN layer in a reasonable manner and converts them into the input voltage of the corresponding crossbar. The scheduling table is used to store the number of runs and the running order of each OU, and to control the input of feature vectors and the output position of the results after the runs.

[0056] The online phase mainly includes a weighted programming module and an inference module, as detailed below:

[0057] (3) Weight programming module, used to write weight blocks of size OU into the specified physical OU in online state according to the generated mapping table.

[0058] Specifically, the weight programming module prepares weights for subsequent inference stages based on the ReRAM device's capacity. Each time, the weight programming module programs neural network weights equal to the ReRAM device's capacity onto the hardware device. The hardware device contains multiple crossbars, and each crossbar has a corresponding mapping table for each weight deployment stage. Based on the mapping table generated in the offline stage, weight blocks of size OU are written into the specified physical OU.

[0059] It is important to note that on resource-constrained ReRAM devices, the entire weights cannot be programmed into the hardware all at once before DNN inference; the cost of online weight programming must be factored into the end-to-end inference latency.

[0060] (4) Inference module, used to read the scheduling table in online state and activate the calculation on OU with the corresponding input feature vector;

[0061] During the inference phase, the crossbars on ReRAM run in parallel. Only one OU is activated for computation at any given time for each crossbar, and each crossbar has a corresponding scheduling table for each inference phase.

[0062] Each crossbar on the ReRAM reads data from the scheduling table to obtain the OU location to be activated and the input feature vector of that OU for this calculation. By converting the input feature vector into a voltage input to the bit line of that OU, the result of this calculation is obtained. The output feature vector information in the scheduling table is then obtained and the calculation result is transmitted to the corresponding location for storage.

[0063] The aforementioned weight programming and inference modules run iteratively until DNN inference is completed for all given input data. Considering the actual capacity of ReRAM and the size of the neural network weights, performing a single neural network inference requires splitting the neural network weights and deploying them multiple times on the hardware for computation. Intermediate computation results are temporarily stored to achieve a single, completed neural network inference process. Therefore, on resource-constrained ReRAM devices, completing a single neural network inference computation requires multiple iterations of the inference and weight programming phases. The specific number of iterations depends on the ReRAM capacity and the size of the neural network weights. For cases requiring inference of multiple input data, such as inferring multiple images, the inference and weight programming phases need to be iteratively executed until DNN inference is completed for all given input data.

[0064] Example 2

[0065] Based on the DNN inference acceleration system in Embodiment 1, this embodiment discloses a DNN inference acceleration method for resource-constrained in-memory computing chips, combined with... Figure 2 Specifically, it includes the following processes:

[0066] (1) Based on the ReRAM programming latency model, optimization strategies for programming latency are obtained; the optimization strategies include: similarity matching strategy, balanced mapping strategy and distance-aware strategy;

[0067] Specifically, by decomposing the ReRAM programming latency, we can explore the factors that affect the programming latency in depth and model the ReRAM programming latency. In this embodiment, the ReRAM programming latency is decomposed into single programming latency, single OU programming latency, single crossbar programming latency and the programming latency of the entire ReRAM from small to large.

[0068] The single programming delay includes the SET delay from 0 to 1 and the RESET delay from 1 to 0 during the programming process. SET and RESET are performed serially.

[0069] It should be noted that due to the influence of voltage drop and creep current, the RESET delay varies with the position of the programming cell in the crossbar. Specifically, cells closer to the crossbar voltage driver require a relatively shorter RESET delay during programming, while cells farther away from the crossbar voltage driver require a relatively longer RESET delay.

[0070] An OU contains multiple single-programming delays, and each single programming operation is performed serially. The specific number of single programming operations is affected by the size of the OU and the number of parallel units in a single programming operation. Therefore, the programming delay of a single OU is the sum of the single programming delays of all OUs.

[0071] The programming latency of a single crossbar is the sum of the programming latencies of all OUs in that crossbar, and multiple OUs in a single crossbar are programmed serially.

[0072] The entire ReRAM contains multiple crossbars, and each crossbar runs in parallel. Therefore, the programming latency of the entire ReRAM is equal to the programming latency of the crossbar with the longest programming latency.

[0073] The final modeling formula for ReRAM programming latency is:

[0074]

[0075] Where CB represents the number of crossbars, OU j This represents the OU at a specific position within the crossbar, where OU represents the number of OUs that the crossbar can hold. This represents the number of RESET operations required for the j-th OU of the i-th crossbar. This represents the number of SET operations required for the j-th OU of the i-th crossbar. Indicates in OU j The delay in performing a single RESET operation, Indicates in OU jThe delay in performing a single SET operation.

[0076] As shown in the formula above, the programming delay of a single OU depends on the number and location of SET and RESET operations. The delay of a RESET operation within an OU varies depending on the distance between the OU and the voltage driver. The programming delay of the entire crossbar is the sum of the programming delays of all its OUs. Finally, the programming delay of the entire weighted region depends on the slowest crossbar.

[0077] In this embodiment, based on the established ReRAM programming latency model and combined with the physical characteristics of ReRAM, optimization strategies are proposed in stages; the specific strategies are: similarity matching strategy, balanced mapping strategy, and distance-aware strategy.

[0078] The similarity matching strategy addresses the characteristic that RESET latency in ReRAM is much greater than SET latency and dominates the total programming latency. By utilizing the similarity of OUs, it reduces the number of global RESET operations, thereby reducing programming latency and accelerating the neural network inference process.

[0079] Specifically, when none of the units in a parallel operation require a RESET operation from 1 to 0 in a single programming operation, this operation can be omitted, thereby reducing the latency of that programming operation. By comparing with the original data in ReRAM, new data with fewer or no RESET operations can be matched. The programming latency of deploying the matched new data to the corresponding position of the original data is much lower than that of common sequential deployment.

[0080] When a new weighted block is programmed based on an old weighted block in an OU, the fewer RESET operations required, the higher the similarity; conversely, the more RESET operations required, the lower the similarity.

[0081] The balanced mapping strategy takes advantage of the fact that the programming latency of ReRAM depends on the programming latency of the crossbar with the largest programming latency. By adjusting the similarity matching order, the programming latency of each crossbar is balanced, thereby reducing the overall programming latency of ReRAM.

[0082] It's important to note that the OUs that complete similarity matching first have a larger matching space and are more likely to match new data with fewer RESET operations. Therefore, the similarity matching order affects the programming latency of a single crossbar, thus impacting the overall ReRAM programming latency. The balanced mapping strategy proposes a more reasonable similarity matching order, thereby making the programming latency of each crossbar more balanced and reducing the overall ReRAM programming latency.

[0083] This embodiment matches all logical OUs with suitable weight blocks based on similarity, achieving the optimal overall result. A greedy algorithm is used for similarity matching. The first logical OU to perform a similarity matching search can choose from all weight blocks in the next deployment layer, thus maximizing its search space. As the matching algorithm progresses, more and more matched weight blocks are removed from the search space (matched weight blocks cannot be matched with other OUs). Because the search space becomes increasingly smaller, it becomes difficult for the logical OU performing the matching search to find an ideal weight block that doesn't require a reset operation. The later the matching order, the worse the programming latency required for the matched weight blocks. Therefore, comparing the matching order significantly impacts the total programming latency of each crossbar.

[0084] If a naive similarity matching method is used, and all OUs in one crossbar are matched sequentially, and then all OUs in the next crossbar are matched sequentially, the programming latency of the last crossbar will be ten or even dozens of times that of the first crossbar. Since the weights of each neural network layer are evenly distributed among the crossbars, and each crossbar performs inference calculations in parallel, the large difference in programming latency between different crossbars will cause all crossbars with small programming latency to wait idly for the crossbars with large programming latency to complete their inference calculations, otherwise they will not be able to obtain the input of the next neural network layer. This will seriously affect the inference speed of DNN in ReRAM.

[0085] However, if we simply perform round-robin search matching on all crossbars, since the first crossbar is matched first in each round and the last crossbar is matched last, although the difference in programming latency between these two crossbars will be much smaller than that of the naive sequential method, there will still be a significant difference in programming latency between different crossbars.

[0086] To balance the programming latency of each crossbar and reduce its variance, this embodiment proposes a balanced mapping strategy. This strategy improves upon the simple round-robin search matching algorithm described above. Specifically, in odd-numbered rounds of round-robin search matching, a sequential round-robin strategy is used for all crossbars; in even-numbered rounds, a reverse-order round-robin strategy is used. Based on this method, the OU in the first crossbar is matched first in odd-numbered rounds, obtaining a larger search space, and is matched last in even-numbered rounds, obtaining a smaller search space. The programming latency of each crossbar will reach a relatively balanced state after every two rounds of round-robin matching.

[0087] The distance-aware strategy addresses the characteristic that RESET latency in ReRAM varies with the position of the programming unit in the crossbar. By adjusting the position of the OU that has completed similarity matching, the frequently operated OU is moved to a position in the crossbar where RESET programming latency is lower, thereby reducing the programming latency of a single crossbar.

[0088] Considering the significant differences in programming latency among OUs at different positions in the crossbar, the distance-aware strategy, after completing similarity matching, adjusts frequently operated OUs to positions in the crossbar with lower RESET programming latency. This reduces the programming latency of a single crossbar while achieving perfect compatibility with the two optimization strategies mentioned above.

[0089] (2) Quantize the weights of the trained neural network into integers, divide the neural network weights into deployment layers of ReRAM size, and divide the weights of each deployment layer into weight blocks in units of OU;

[0090] The peripheral circuitry of common ReRAM architectures makes ReRAM more suitable for integer operations. Therefore, neural network weights need to be quantized into integers. It should be noted that although quantization can reduce the amount of computation while being suitable for ReRAM calculations, it will also lead to a loss of neural network inference accuracy. In our experiments, we used post-training quantization to quantize the weights into 8-bit integers and kept the accuracy loss within 2%.

[0091] The number of bits in a quantized integer depends on the quantization method, neural network, and dataset used. Theoretically, under the same conditions, considering the loss of precision, the lower the number of bits, the faster the inference speed.

[0092] (3) Based on similarity matching strategy, balanced mapping strategy and distance-aware strategy, generate a mapping table from each weight block to the physical OU;

[0093] To minimize the total weight programming overhead during neural network runtime, this embodiment proposes a search strategy to find a near-optimal weight block mapped to OU in the weight allocation module. This module obtains weight data information and ReRAM structure from the trained DNN model.

[0094] The weight allocation module mainly includes the three optimization strategies mentioned above. By inputting the neural network architecture and specific ReRAM parameters, the neural network weights are divided into multiple deployment layers of ReRAM size, and the weights of each deployment layer are divided into weight blocks in units of OU. The deployment layer in which the weight block is located determines its deployment number.

[0095] The data in the first deployment layer of ReRAM will be temporarily deployed sequentially. Then, for each deployed OU, a similarity search and matching will be performed in the weighted blocks of the second deployment layer. It should be noted that the weighted blocks in the second deployment layer that have been matched by the previous OUs will not participate in the similarity matching process of subsequent OUs.

[0096] To achieve a balanced mapping strategy, the similarity matching process will follow this order:

[0097] Suppose there are four crossbars in the ReRAM, numbered 1, 2, 3, and 4. The similarity matching order is 1, 2, 3, 4, 4, 3, 2, 1, and so on, in a loop. Specifically, the first OU in the first crossbar is matched first, then the first OU in the second crossbar, and so on. This matching order can make the programming latency of each crossbar more balanced, thus accelerating the overall programming latency of the ReRAM.

[0098] After applying a similarity matching strategy to the weight blocks to be deployed at each layer, an ordered weight block group is generated for each OU location. Each weight block group contains the weight blocks that each deployment layer needs to deploy to that OU location. The distance-aware strategy calculates the total programming latency required for each weight block group and, in crossbar units, adjusts weight block groups with more RESET operations to positions closer to the voltage driver, i.e., positions with lower RESET operation latency; conversely, it places weight block groups with fewer RESET operations at positions with higher RESET operation latency.

[0099] All the above operations will be completed offline. After completion, a mapping table will be generated. This mapping table indicates the weighted blocks deployed by each OU during online inference. It's important to note that each deployment layer of each crossbar in ReRAM has a mapping table, as shown in Table 1. The mapping table uses...

[0100] <filter_id,channel_id,block_id> This indicates the filter number, channel number, and block number to which the weight block belongs.

[0101] Table 1. Examples of mapping representations

[0102] Filter_id Channel_id Block_id 1 2 6 1 1 0 8 4 12 3 1 6 … … …

[0103] (4) Generate a scheduling table based on the neural network structure, quantized weight information, ReRAM structure, and mapping table information;

[0104] To ensure the correctness of DNN inference, we designed an "OU scheduler" module in the offline phase of the proposed architecture. The OU scheduler works as follows: Figure 3As shown in Table 2, the input to the "OU scheduler" is the input feature vector and a "mapping table" containing the weight blocks to OU mappings in each inference iteration. Based on this information, the "OU scheduler" module generates a scheduling table to guide the activation order and number of OUs during runtime, the selection of corresponding input feature maps, and the storage location of the computation results, as shown in Table 2. The activation number, OU information, input feature map information, and output feature map information are respectively generated by...<Activation,OU_id,IFM_id,OFM_id> To express.

[0105] Table 2. Examples of Scheduling

[0106]

[0107] The above processes are all implemented offline. Guided by information in the mapping table and scheduling table, the online phase can continuously iterate the weight programming and inference processes until the entire DNN inference process is completed. This embodiment fully utilizes the ReRAM programming features and significantly improves the DNN inference speed on resource-constrained in-memory computing chips by optimizing programming latency.

[0108] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A DNN inference acceleration system for resource-constrained in-memory computing chips, characterized in that, include: The weight allocation module is used to acquire neural network information and ReRAM structure information in offline mode, and generate a mapping table from an OU-sized weight block to a physical OU. The mapping table is used to store the weight block information at each OU position when the neural network weights are deployed on ReRAM each time. The OU scheduling module is used to input neural network information, ReRAM structure and mapping table information into the OU scheduler in offline mode to generate a scheduling table; the OU scheduler sorts the input feature vectors of each neural network layer and converts them into the input voltage of the corresponding crossbar; the scheduling table is used to store the number of times each OU runs and the running order, and to control the input of feature vectors and the output position of the results after running. The weight programming module is used to write weight blocks of size OU into the specified physical OU based on the generated mapping table in online mode; The inference module is used to read the scheduling table in an online state and activate the computation on OU with the corresponding input feature vector; The weight programming module and inference module run iteratively until DNN inference is completed for all given input data; The weight programming module programs neural network weights of the ReRAM device capacity onto the hardware device each time. The hardware device contains multiple crossbars, and each crossbar has a corresponding mapping table for each weight deployment stage. The crossbars on ReRAM run in parallel, with only one OU being activated for computation at any given time for each crossbar, and each crossbar has a corresponding schedule table for each inference phase.

2. The DNN inference acceleration system for resource-constrained in-memory computing chips as described in claim 1, characterized in that, Each crossbar on the ReRAM reads data from the scheduling table to obtain the OU location to be activated and the input feature vector of that OU for this calculation. By converting the input feature vector into a voltage input to the bit line of that OU, the result of this calculation is obtained. The output feature vector information in the scheduling table is then obtained and the calculation result is transmitted to the corresponding location for storage.

3. A DNN inference acceleration method for a DNN inference acceleration system for resource-constrained in-memory computing chips as described in any one of claims 1-2, characterized in that, include: Based on the ReRAM-based programming latency model, optimization strategies for programming latency are derived. The optimization strategies include: similarity matching strategy, balanced mapping strategy, and distance-aware strategy; The weights of the trained neural network are quantized into integers, the neural network weights are divided into deployment layers of ReRAM size, and the weights of each deployment layer are divided into weight blocks in units of OU. Based on similarity matching strategy, balanced mapping strategy and distance-aware strategy, a mapping table from each weight block to the physical OU is generated; A scheduling table is generated based on the neural network structure, quantized weight information, ReRAM structure, and mapping table information. The above process is implemented offline.

4. The DNN inference acceleration method as described in claim 3, characterized in that, The ReRAM programming latency model is specifically as follows: the ReRAM programming latency unit is decomposed from small to large into single programming latency, single OU programming latency, single crossbar programming latency, and the programming latency of the entire ReRAM; Among them, the programming latency of a single OU is the sum of the programming latencies of all single operations in that OU, the programming latency of a single crossbar is the sum of the programming latencies of all OUs in that crossbar, and the programming latency of the entire ReRAM is the programming latency of the single crossbar with the largest programming latency.

5. The DNN inference acceleration method as described in claim 3, characterized in that, The similarity matching strategy is as follows: reduce the number of global RESET operations by utilizing the similarity of OUs: when all units of parallel operations do not require a RESET operation from 1 to 0 in a single programming operation, this operation can be omitted.

6. The DNN inference acceleration method as described in claim 3, characterized in that, The specific equalization mapping strategy is as follows: By adjusting the similarity matching order of OUs, the programming latency of each crossbar can be balanced.

7. The DNN inference acceleration method as described in claim 3, characterized in that, The distance-aware strategy specifically involves adjusting the position of OUs that have completed similarity matching, moving frequently operated OUs to positions within the crossbar where RESET programming latency is lower, thereby reducing single-crossbar programming latency.

Images