A Hardware-Aware CNN Hierarchical Search Analysis and Modeling Method
By employing a hardware-aware CNN hierarchical search analysis modeling method, combined with mixed precision quantization and data movement analysis, the configuration of each layer and the cyclic sorting of the CNN network are optimized. This solves the problem of high computational cost and hardware overhead of CNN models on CPU platforms, and improves inference efficiency and code performance.
Patent Information
- Application Number
- CN202311448391.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-02
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-11-02
AI Technical Summary
Existing technologies have high computational and hardware overhead in CNN models, low inference efficiency, and have failed to effectively optimize code performance, especially on CPU hardware platforms, ignoring the impact of hardware conditions.
We employ a hardware-aware CNN hierarchical search analysis modeling method, combined with mixed precision quantization and data movement analysis. By optimizing the configuration of each layer of the CNN network and cyclic sorting, we construct an objective function to balance precision loss and total data movement, and generate code with the optimal configuration.
With limited CPU hardware resources, the computational load and hardware overhead of CNN models are reduced, while inference efficiency and code performance are improved, resulting in more efficient model inference.
Smart Images

Figure CN117574964B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of design space exploration, and more specifically to a hardware-aware CNN hierarchical search analysis and modeling method. Background Technology
[0002] Convolutional Neural Networks (CNNs) have already impacted numerous fields, including image / video classification, language processing, and gene analysis. CNNs are computationally demanding. Therefore, there is significant interest in optimizing the implementation of CNN stages required in deep neural network (DNN) pipelines. Even within a single DNN pipeline, CNN stages of varying shapes and sizes are needed. Since the cost of data movement dominates the cost of floating-point arithmetic computations on all current hardware platforms, loop tiling is a key transformation in developing optimized CNN code. However, several challenges exist. A fundamental challenge is the size of the possible tiling space of loop variables in CNN computation; another challenge is that, due to the varying sensitivities of CNN layers, layer-by-layer quantization is required to better compress the network and reduce data movement while minimizing precision loss; yet another challenge is that, although mixed-precision quantization is widely used on accelerators, directly applying mixed-precision quantized CNN models to CPUs is not feasible.
[0003] In their paper "TVM: An Automated End-to-End Optimizing Compiler for Deep Learning", Tianqi Chen et al. proposed TVM, an open-source graphics-level compiler and operator-level optimizer that provides efficient portability across multiple applications to hardware backends for deep learning workloads. It employs a custom, learning-based cost model approach for rapidly exploring code optimizations.
[0004] In their paper "Optimizing Memory Efficiency for Deep Convolutional Neural Networks on GPUs," Chao Li et al. investigated the memory efficiency of different CNN layers and revealed the impact of data layout and memory access patterns on performance. Experiments show that the proposed optimizations have general effectiveness on single layers and various networks.
[0005] In their paper "Optimizing CNN model inference on CPUs," Yizhi Liu et al. introduced NeoCPU, a comprehensive approach for CNN model inference on CPUs that employs complete stack and system optimization schemes. Experiments show that NeoCPU achieves 3.45 times lower latency for CNN model inference than state-of-the-art implementations on various popular CPUs.
[0006] In their paper “Analytical Modeling Is Enough for High-Performance BLIS”, Tze Meng Low et al. presented a BLAS-like Library Instantiation Software (BLIS) framework that provides a more detailed layering of the GotoBLAS (now maintained as OpenBLAS) implementation, showing how analysis can determine the tuning parameters of high-level instantiation of matrix-matrix multiplication.
[0007] In their paper "Analytical Characterization and Design SpaceExploration for Optimization of CNNs", Rui Li et al. developed an analytical modeling method for finding the optimal loop-level optimization configuration for CNN networks on multi-core CPUs.
[0008] The optimization methods for CNN layer analysis and modeling mentioned above are all based on the CPU architecture and are performed under the condition of fixed bit width. They do not take into account whether the computational and memory overhead of the entire CNN model is optimal under this fixed precision. In other words, the analysis is performed without considering the hardware conditions, which results in the entire model still having large computational and hardware overhead, low efficiency, and affecting code performance and inference speed. Summary of the Invention
[0009] This invention addresses the problems of high computational and hardware overhead, low inference efficiency, and potential for further code performance optimization in existing technologies. It provides a hardware-aware CNN hierarchical search analysis and modeling method, which combines mixed precision quantization and data movement analysis and modeling in each layer of the CNN network to collaboratively process the CNN model. By balancing the precision loss after quantization with the total amount of data movement, the computational and hardware overhead of the CNN model is reduced, inference efficiency is accelerated, and the overall inference performance of the model is improved.
[0010] To achieve the above objectives, the present invention employs the following technical solution:
[0011] A hardware-aware CNN hierarchical search analysis and modeling method includes:
[0012] The configuration for mixed-precision quantization of a CNN network is generated to obtain the network-level quantization configuration. The generation of the network-level quantization configuration is based on mixed-precision quantization of the CNN network. The accuracy of the network after mixed-precision quantization is compared with that of the CNN network to obtain the accuracy loss of the quantized network.
[0013] For the network layer quantization configuration of the CNN network, a corresponding mixed precision quantization CNN network is obtained based on different schemes in the configuration. The quantization CNN network is cyclically sorted and the block size is explored for each layer. The configuration combination of the quantization network is obtained as the design space, and the optimal network configuration is determined within the design space. The network configuration includes the total amount of data movement of the network.
[0014] In the case of multi-level CPU caching, the total amount of data movement of the CNN network under different configurations is modeled with the accuracy loss of the quantized network to construct an objective function; the configuration search is performed to minimize the objective function to obtain the optimal configuration that balances the accuracy loss and the total amount of data movement of the entire network.
[0015] Code is generated based on the optimal configuration and corresponding microkernel found in the search, and the code is verified to determine the optimal configuration.
[0016] Furthermore, on a CNN network with L layers, under the constraint of a bit-width candidate set space, a random generation method is used to search for N different network layer quantization configurations. And the Kth configuration is The candidate set of bit widths for the l-th quantization layer is q. l ∈{bw1,bw2,…,bw Q}, bw Q This indicates that there are a total of Q possible bit widths in the candidate set, and these Q values represent the data bit widths to be quantized.
[0017] Furthermore, for a CNN network with L layers, the cyclic sorting and block size of each layer can be used to calculate the amount of data movement in each layer based on the quantization configuration, cyclic sorting, and block size, thereby obtaining the total amount of data movement in the network.
[0018] Based on N different network layer quantization configurations of CNN networks, configuration combinations of quantized networks are obtained. Within the design space, the network configuration with the best performance is searched. The network configuration includes network layer quantization configurations. Circular sort Block size and total network data movement DM total .
[0019] Furthermore, the design space is clipped to reduce the search range; simultaneously, a block search is also required.
[0020]
[0021] Where C represents the cache level capacity constraint. Different CPUs have different cache level capacities, and this constraint limits the block size. Space occupied This indicates that the block size is When constrained by C, the goal is to minimize the amount of data movement. This allows us to calculate the data movement DM for each layer of the CNN network.
[0022]
[0023] Where T num This refers to the number of blocks. The number of blocks needs to be determined based on the current convolution calculation. After calculating the data movement amount DM for each layer, the data movement amounts of all L layers are added together to obtain the total amount of data movement DM for the entire network. total :
[0024] DM total =∑ L DM.
[0025] Furthermore, the objective function is:
[0026]
[0027] Among them, quantitative configuration Quantize the configuration for the network layer A certain configuration in the file, cyclic sorting As a candidate set for cyclic sorting, the block size is... This is the configuration for all chunk sizes. A specific block size in the, and It is all satisfied The data set with the constraint block size, then This means quantitative allocation Circular sort and block size The amount of data movement in a single layer of a CNN network under certain configurations, while This represents the total amount of data movement across the entire network under the same configuration, where β is a weighting factor. For the sake of accuracy loss;
[0028] The search is performed by minimizing the objective function O, calculating all possible configurations according to the objective function, and then finding the optimal result and its corresponding configuration. This is the optimal configuration that balances the loss of accuracy with the total amount of data movement across the entire network.
[0029] Furthermore, for different platforms, appropriate microkernels need to be designed. The implementation code of these microkernels is designed based on the CPU's avx2 instruction set architecture. Then, for the optimal configuration, the microkernel is used to generate the final C / C++ code. Running this code calculates the total amount of data movement and the performance improvement of the corresponding configuration to see if the actual effect of this optimal configuration meets the optimization goal. Finally, a code for the optimal configuration that balances the loss of precision and the total amount of data movement is obtained.
[0030] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the hardware-aware CNN hierarchical search analysis modeling method.
[0031] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the hardware-aware CNN hierarchical search analysis modeling method.
[0032] This invention has the following technical features:
[0033] 1. Compared with traditional fixed-precision CNN network analysis and modeling methods, this invention combines mixed-precision quantization with CNN network analysis and modeling under CPU hardware resource conditions, resulting in lower hardware overhead, higher inference efficiency, and better optimized code performance for CNN models.
[0034] Traditional methods for analyzing and modeling CNN models suffer from high hardware overhead and computational complexity, resulting in low inference efficiency and long overall inference times. For example, the BLIS framework, a library-like software framework similar to BLAS, can analyze and determine high-level instantiation tuning parameters for matrix multiplication, enabling more detailed layering. MXNet and Intel OpenVINO typically treat CNN models as graphs, using high-performance libraries to perform graph operations, but lack flexibility. RuiLi et al.'s approach, which analyzes and models data movement based on layer-by-layer slice cycles and block sizes in CNN networks, suffers from long inference times and low efficiency. Compared to traditional methods, this invention employs mixed-precision quantization, achieving a trade-off between CNN model accuracy and total data movement within CPU hardware resources. This reduces computational overhead, hardware costs, and overall model size. Furthermore, the compressed CNN model exhibits higher inference efficiency and better code performance when analyzing and modeling data movement.
[0035] 2. Compared with traditional technical methods, this invention, under the condition of multi-level CPU caching, considers the mutual influence between mixed precision quantization and data movement analysis and modeling of CNN networks, weighs the precision loss of the quantization scheme and the total amount of data movement of the CNN model, and obtains a CNN model quantization implementation under CPU hardware resources, which is more in line with the actual situation of CPU hardware resources and obtains a better actual solution for CNN model analysis and modeling.
[0036] Traditional fixed-precision CNN model analysis and modeling either only considers layer-by-layer optimization, neglecting end-to-end global optimization, or only considers the impact of slice loops and block size on data movement, ignoring the different sensitivities of different network layers to bit width. The resulting network models inevitably exhibit suboptimal performance. In contrast to traditional methods, this invention proposes a data movement analysis and modeling approach for CNN models based on mixed-precision quantization. Furthermore, considering the interaction between mixed-precision quantization and network layer slice loops and block size, it introduces the precision loss metric to balance network precision with the total amount of model data movement, thereby obtaining a more practical optimal configuration considering multiple factors. Attached Figure Description
[0037] Figure 1 This is the overall process framework of the present invention;
[0038] Figure 2 This is a schematic diagram of the seven-layer recurrent convolutional layer of a CNN network;
[0039] Figure 3 These are the 8 optimal sorting methods obtained by trimming the cyclic sorting in this invention;
[0040] Figure 4This is a diagram illustrating the code generation process of this invention. Detailed Implementation
[0041] This invention considers the mutual influence on CPU performance between mixed-precision quantization and data movement analysis and modeling of CNN networks. It introduces a precision loss metric, weighting two important model metrics: the precision loss of the quantized model and the total data movement, to obtain the optimal configuration that balances both. Under CPU-constrained hardware resources, it achieves network analysis and modeling on a compressed CNN model based on mixed-precision quantization, combining iterative optimization with mixed-precision quantization. When analyzing and modeling data movement in the CNN model, the compressed network after mixed-precision quantization is utilized, effectively reducing computational load and hardware overhead. Without significantly impacting the overall network performance, mixed-precision quantization accelerates the model inference process.
[0042] Referring to the accompanying drawings, the present invention provides a hardware-aware CNN hierarchical search analysis and modeling method, comprising:
[0043] Step 1: Select a specific CNN network to generate a mixed precision quantization configuration to obtain a network-level quantization configuration. The generation of the network-level quantization configuration is based on performing mixed precision quantization on the CNN network. The accuracy of the network obtained after mixed precision quantization is compared with that of the CNN network to obtain the accuracy loss of the quantized network.
[0044] Starting with a mixed-precision quantization search, based on the selected CNN network (such as ResNet, VGG16, or GoogLeNet, among other popular convolutional neural networks), different layers have varying sensitivities and require different data bit widths for quantization. These different bit widths affect the data representation and consequently, the network's accuracy. Here, mixed-precision quantization uses fixed-point quantization. Therefore, a mixed-precision quantization search is needed to generate N quantization configurations for different network layers under mixed-precision quantization conditions. And the Kth configuration is The candidate set of bit widths for the l-th quantization layer is q. l ∈{bw1,bw2,…,bw Q}, bw Q This indicates that there are a total of Q selectable bit widths in the candidate set, and these Q values represent the data bit width for quantization. The mixed-precision quantization search utilizes a random generation method to generate multiple network layer quantization configurations with L layers. For example, if a ResNet has 18 layers, multiple permutations of 18 layers are randomly generated, and all these generated permutations are used as the network layer quantization configurations.
[0045] For example: the bit width candidate set can be set to q. l For ResNet18, the network depth is L = 18 layers, and each layer can be selected from this candidate set q. l Choosing a value from {8, 16, 32} determines the bit width of the data representation for that layer; this is called quantization for a specific layer. Determining the bit width of each of the 18 layers in the entire network results in 18 possible combinations of 8, 16, and 32. Because quantization is performed at the layer level, this combination is called the network layer quantization configuration. The configuration here consists of two parts: one is the quantization bit width of each layer obtained through permutations and combinations. For an 18-layer network, there are 18 corresponding bit width configurations. The other part is the accuracy, which can be obtained by quantizing the network under this configuration after training. If all layers of the corresponding network are quantized one by one using this mixed bit width configuration, this operation is called mixed precision quantization. That is, different layers use different bit widths to represent data, and the process of obtaining N configurations (where N can be set by the user; in experiments, it can be set to 50 or more as input datasets) is the search process. If every layer of this network is quantized, and the quantized values are not exactly the same, then this network is called a mixed precision quantized network.
[0046] That is, on a common CNN network with L layers, in a candidate set space q with a bit width l Under constraints, a random generation method is used to search for N different network layer quantization configurations. These schemes serve as input datasets for subsequent use. The configuration schemes obtained here will perform mixed-precision quantization on the corresponding networks. Quantization adopts a post-training quantization method (after the network is trained, the network data is directly truncated; for example, for 8-bit quantization, the 32-bit floating-point representation of the training layer is directly truncated into 8-bit fixed-point data, and other layers are quantized according to the corresponding configuration. By quantizing all layers according to the configuration, a quantized CNN network, specifically a mixed-precision quantized CNN network, can be obtained). The accuracy of the network is directly obtained. The accuracy of the network under different configurations is different. The accuracy of the network under the mixed-precision quantization configuration scheme is compared with the accuracy of the full-precision network (the original CNN network), and the difference is used as the accuracy loss of the quantized model. As one of the indicators to be considered later.
[0047] Step 2: For the network layer quantization configuration of the CNN network (N mixed precision quantization configuration schemes obtained in Step 1), the corresponding mixed precision quantized CNN network is obtained based on the different schemes in the configuration. The quantized CNN network is cyclically sorted and the block size is explored to obtain the configuration combination of the quantized network as the design space. The optimal network configuration is determined within the design space. The network configuration includes the total amount of data movement of the network.
[0048] like Figure 2 As shown, since a convolutional layer in a CNN network can be represented by seven recurrent layers, there are 7! = 5040 possible combinations of these seven layers. If the network has L layers, then the entire network will have 5040 possible combinations. L There are several combinations, each corresponding to the data movement overhead of the quantization network layers. If the total number of layers in the network is determined, along with the cyclic sorting and block size of each layer, the data movement of each layer can be calculated based on the quantization configuration, cyclic sorting, and block size, thus obtaining the total data movement of the network. As mentioned earlier, a network searches for N mixed-precision quantization configurations. Since the searches for network layer quantization configurations, cyclic sorting, and block sizes are independent, a total of N*5040 quantization configurations can be obtained. L The space of possible quantization network configurations is called the design space. Our task is to search within this space for the network configuration that achieves the best performance. This configuration includes network-level quantization configurations. Circular sort Block size and total network data movement DM total .
[0049] The hierarchical ordering of CNN networks can be simplified by pruning the design space. Figure 3 The eight sorting scenarios shown have arbitrarily adjustable hierarchical cycles within the curly braces. Since the impact on data movement is the same after any adjustment within the braces, they are grouped into the same category. Through pruning, eight hierarchical cyclic sorting schemes most likely to produce the minimum data movement are obtained, and the quantization network configuration combinations can be reduced to N*8. L This significantly reduces the design space required for the search, whereas here we still need to search the blocks:
[0050]
[0051] Where C represents the cache level capacity constraint. Different CPUs have different cache level capacities, and this constraint limits the block size. (This is affected by quantization; if the bit width of the data representation is smaller, then the same amount of space can be used to represent more data.) Space occupied This indicates that the block size is When constrained by C, the goal is to minimize the amount of data movement. This allows us to calculate the data movement DM for each layer of the CNN network.
[0052]
[0053] Where T num This refers to the number of blocks, which needs to be determined based on the current convolution calculation. For example, if the input feature map size is N*N and the block size is m*m, then the number of blocks is (N-m+1)*(N-m+1). After calculating the data movement amount DM for each layer, summing up the data movement amounts for each layer (there are L layers in total) gives the total data movement amount DM for the entire network. total :
[0054]
[0055] By calculating the corresponding cyclic sorting and block partitioning schemes, the overall data movement cost of the network can be obtained, thus providing the necessary data indicators for subsequent search of the optimal configuration.
[0056] Step 3: In the case of multi-level CPU caching, the total amount of network data movement of the CNN network under different configurations is modeled with the accuracy loss of the quantized network to construct an objective function; the configuration search is performed to minimize the objective function to obtain the optimal configuration that balances the accuracy loss and the total amount of network data movement.
[0057] The specific modeling method is as follows: using the metric obtained in step 1, the accuracy loss of the CNN network. The model is constructed for the total data movement parameters under different mixed-precision quantization configuration schemes, and the objective function O is:
[0058]
[0059] Quantitative configuration This refers to the network layer quantization configuration mentioned earlier. A certain configuration in the file, cyclic sorting Given one of eight candidate sets for cyclic sorting, the block size is... This is the configuration for all chunk sizes. A specific block size in the, and It is all satisfied The data set with the constraint block size, then This means quantitative allocation Circular sort and block size The amount of data movement in a single layer of a CNN network under certain configurations, while This represents the total amount of data movement across the entire network under the same configuration. β is a weighting factor (the weighting factor ranges from β∈[0,1]), a hyperparameter that is adjusted based on experimental observations. The accuracy loss calculated in step 1 is used to search for a solution that minimizes the objective function O. All possible configurations are calculated according to the objective function, and then the optimal result and its corresponding configuration are found. This allows us to derive the optimal configuration that balances accuracy loss with the total amount of data movement across the entire network.
[0060] For example, consider a ResNet18 network with a depth of L=18 layers. Given a data representation with a certain bit width, a seven-layer cyclic sort, and a block size, the O value of the l-th (l∈L) layer can be calculated using the aforementioned objective function. A single network layer quantization configuration in ResNet18 will correspond to 18 layers with varying numerical values. The configuration that minimizes the objective function value is then selected as the optimal configuration. In summary, the search process involves calculating the objective function O value under all possible combinations of configurations (including network layer configuration, cyclic sort, block size, and precision loss) using the formula. The process finds all configurations with the smallest O value. Since there will be different network layer configurations, different cyclic sorts, and different block sizes, the search will find different O values under different combinations of these metrics. The optimal configuration is then selected, ending the search and returning the best configuration.
[0061] At this stage, it is necessary to consider the accuracy loss given in step 1. The total data movement overhead DM of the mixed-precision quantization network given in step 2 total These two metrics, balancing accuracy loss and total data movement, are used as the objective function to calculate the optimal configuration. This achieves the best cyclic sorting and block size configuration after trade-offs while ensuring minimal accuracy loss, in order to minimize the total data movement.
[0062] Step 4: Generate code for the best searched configuration and corresponding microkernel, verify the code, and determine the code for the best configuration.
[0063] For different platforms, appropriate microkernels need to be designed. The implementation code of these microkernels is designed based on the CPU's AVX2 (or AVX512) instruction set architecture. Then, based on the optimal configuration given in step 3, the microkernel is used to generate the final C / C++ code, such as... Figure 4 As shown, running this code calculates the total amount of data moved and the performance improvement of the corresponding configuration, in order to see if the actual effect of this optimal configuration meets the optimization goal, and finally obtains the code of the optimal configuration that balances the loss of accuracy and the total amount of data moved.
[0064] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A hardware-aware CNN hierarchical search analysis and modeling method, characterized in that, include: The configuration for mixed-precision quantization of a CNN network is selected to generate the network layer quantization configuration; The generation of the network layer quantization configuration is based on performing mixed precision quantization on the CNN network. The accuracy of the network obtained after mixed precision quantization is compared with that of the CNN network to obtain the accuracy loss of the quantized network. For the network layer quantization configuration of the CNN network, a corresponding mixed precision quantization CNN network is obtained based on different schemes in the configuration. The quantization CNN network is cyclically sorted and the block size is explored for each layer. The configuration combination of the quantization network is obtained as the design space, and the optimal network configuration is determined within the design space. The network configuration includes the total amount of data movement of the network. In the case of multi-level CPU caching, the total amount of data movement of the CNN network under different configurations is modeled with the accuracy loss of the quantized network to construct an objective function; the configuration search is performed to minimize the objective function to obtain the optimal configuration that balances the accuracy loss and the total amount of data movement of the entire network. The system generates instructions based on the optimal configuration and corresponding microkernel found in the search, verifies the instructions, and determines the instructions for the optimal configuration. The objective function is: ; Among them, quantitative configuration Quantize the configuration for the network layer A certain configuration in the file, cyclic sorting As a candidate set for cyclic sorting, the block size is... This is the configuration for all chunk sizes. A specific block size in the, and It is all satisfied The data set with the constraint block size, then This means quantitative allocation Circular sorting and block size The amount of data movement in a single layer of a CNN network under certain configurations, while This represents the total amount of data moved across the entire network under the same configuration. It is a weighting factor. For precision loss; C represents the capacity constraint of the cache level; To minimize the objective function Perform a search, calculate all possible configurations according to the objective function, and then find the optimal result and its corresponding configuration. This is the optimal configuration that balances the loss of accuracy with the total amount of data movement across the entire network.
2. The hardware-aware CNN hierarchical search analysis and modeling method according to claim 1, characterized in that, On a CNN network with L layers, under the constraint of a bit-width candidate set space, a randomized search is performed to obtain N different network layer quantization configurations. The Kth configuration is , of which The candidate set of bit widths for layer quantization is , This indicates that the total number of candidate sets is [number missing]. There are Q selectable bit widths; these Q values represent the size of the data bit width used for quantization.
3. The hardware-aware CNN hierarchical search analysis and modeling method according to claim 1, characterized in that, For a CNN network with L layers, the data movement amount of each layer is calculated based on the quantization configuration, cyclic sorting, and block size, thereby obtaining the total data movement amount of the network. Based on N different network layer quantization configurations of CNN networks, configuration combinations of quantized networks are obtained. Within the design space, the network configuration with the best performance is searched. The network configuration includes network layer quantization configurations. Circular sorting Block size and total network data movement .
4. The hardware-aware CNN hierarchical search analysis and modeling method according to claim 1, characterized in that, The design space is clipped to reduce the search range; simultaneously, the blocks also need to be searched. ; Where C represents the cache level capacity constraint. Different CPUs have different cache level capacities, and this constraint limits the block size. Space occupied , This indicates that the block size is When, under constraint C, minimize the amount of data movement. This allows the amount of data movement in each layer of the CNN network to be calculated. : ; in This refers to the number of blocks, which needs to be determined based on the current convolution calculation, calculating the amount of data movement at each layer. Then, by summing up the data movement amounts at layer L, we can obtain the total amount of mobile data in the entire network. : 。 5. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes a computer program, it implements the steps of the hardware-aware CNN hierarchical search analysis modeling method according to any one of claims 1-4.
6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the hardware-aware CNN hierarchical search analysis modeling method according to any one of claims 1-4.
Citation Information
Patent Citations
Rapid automatic compression method for deep convolutional neural network
CN111860779A
Multi-bit-width quantization method for deep convolutional neural network
WO2023015674A1
