Quantitative compression and computing power adaptive optimization method and system of neural network

Abstract

The invention provides a quantitative compression and computing power adaptive optimization method and system of a neural network, and relates to the technical field of neural networks, which comprises the steps of: calculating a weight distribution entropy value and an activation value distribution entropy value, constructing a neuron importance evaluation index, generating an interlayer error propagation evaluation matrix in combination with quantitative error data, and determining a bit width configuration scheme. And differentiated resource allocation is carried out based on the target device calculation load, so that efficient compression and deployment of the neural network on the premise of ensuring the performance are finally realized, the calculation resource consumption is effectively reduced, and the reasoning speed is improved.

Description

Technical Field

[0001] This invention relates to neural network technology, and more particularly to a method and system for quantization compression and adaptive optimization of computing power in neural networks. Background Technology

[0002] With the widespread application of deep learning technology, neural network models have achieved remarkable results in fields such as image recognition, natural language processing, and speech recognition. However, these high-precision neural network models typically have a large number of parameters and complex structures, leading to significant challenges in terms of computing resources and storage space for model deployment and real-time inference, especially on resource-constrained edge devices. To address this issue, quantization compression techniques for neural networks have gradually become a research hotspot. This technique reduces model size and computational complexity by lowering network parameters and computational precision, while maintaining model performance as much as possible.

[0003] Existing neural network quantization compression techniques primarily employ fixed-bit-width quantization strategies, applying a uniform quantization standard to the entire network. This fails to differentiate quantization based on the characteristics of different layers, resulting in significant loss of quantization accuracy. Current quantization methods lack precise mechanisms for assessing the importance of neurons, failing to identify and protect neurons and layers that critically impact the final result, potentially leading to the loss of important information during compression. Furthermore, existing quantization methods rarely consider the actual computing power and load of the target hardware, failing to dynamically adjust the quantization strategy according to the resource constraints of different devices. This results in compressed models struggling to achieve optimal performance-efficiency balance across various deployment environments.

[0004] In practical applications, due to the lack of adaptive optimization mechanisms, existing technologies struggle to achieve a good balance between model accuracy and computational efficiency, especially in scenarios with dynamically changing computing resources, where compressed models fail to achieve optimal utilization of computing power. Therefore, developing an adaptive quantization compression method that comprehensively considers neuron importance, inter-layer error propagation, and the computational characteristics of the target device is of great significance for improving the deployment efficiency of neural networks in resource-constrained environments. Summary of the Invention

[0005] This invention provides a method and system for quantization compression and adaptive optimization of computing power in neural networks, which can solve the problems in the prior art.

[0006] A first aspect of this invention provides a method for quantization compression and adaptive computational power optimization of neural networks, comprising:

[0007] Obtain a pre-trained neural network, which contains multiple convolutional layers and fully connected layers;

[0008] For each convolutional layer and fully connected layer in the neural network, the weight distribution entropy value and the activation value distribution entropy value are calculated. Based on the weight distribution entropy value and the activation value distribution entropy value, an importance evaluation index for neurons is constructed. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization tests with different bit widths are performed on the convolutional layers and fully connected layers to obtain quantization error data. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix. Based on the inter-layer error propagation evaluation matrix, a bit width configuration scheme is determined.

[0009] Obtain the computing load data of the target device; based on the computing load data and the bit width configuration scheme, perform differentiated allocation of computing unit mapping and storage strategy for the network layer;

[0010] The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, thereby generating an optimized neural network.

[0011] In one optional implementation, the step of calculating the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, and constructing an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy includes:

[0012] Obtain the weight data and activation value data of each convolutional layer and fully connected layer in the neural network, discretize them to obtain the weight probability density distribution and activation value probability density distribution, and calculate the corresponding entropy value;

[0013] Similarity analysis is performed on the input and output features of neurons to identify neuron groups with similar response patterns; the distribution difference within each neuron group and the distribution difference between groups are calculated, and the contribution ratio of each neuron to the network function is determined based on the magnitude of the difference; based on the contribution ratio, a corresponding first balance factor and a second balance factor are assigned to each neuron group; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain the hierarchical information flow rate;

[0014] Calculate the mutual information between adjacent layers in the neural network;

[0015] The importance assessment index of a neuron is obtained by weighting and combining the hierarchical information flow with the mutual information; the importance assessment index is verified in multiple batches based on the network output error, and the final importance assessment index is determined when the index fluctuation is less than a preset fluctuation threshold.

[0016] In one alternative implementation, the step of calculating the mutual information between adjacent layers in the neural network includes:

[0017] Multi-scale decomposition of feature data from adjacent layers in a neural network yields feature representations at multiple scale levels. Feature probability distributions are calculated for each scale level feature representation. Based on the feature probability distributions at each scale level, the information entropy of the previous layer and the information entropy of the next layer are calculated respectively, and the combination yields mutual information at multiple scales.

[0018] The temporal feature sequences of adjacent layers are obtained, the feature probability distribution is temporally smoothed, and the temporal mutual information is calculated; the conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated; the multiple scale mutual information, temporal mutual information, and conditional mutual information are weighted and combined to obtain the initial mutual information.

[0019] A causal graph of inter-layer feature transmission is constructed to identify key transmission paths and analyze the causal effect strength of different transmission channels. Based on the causal effect strength and initial mutual information, a loss matrix for feature transmission is constructed, which characterizes the degree of information retention of features during transmission.

[0020] The connection weights of the corresponding channels are adjusted according to the loss values ​​of each transmission channel in the loss matrix, and the loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold, thereby obtaining the mutual information between adjacent layers.

[0021] In one optional implementation, the step of performing quantization tests on the convolutional layer and the fully connected layer with different bit widths to obtain quantization error data includes:

[0022] Feature importance analysis is performed on the parameters of the convolutional layer and the fully connected layer, and the parameter space is divided into critical parameter domains and non-critical parameter domains;

[0023] Quantization processing is performed in the critical parameter field using a first quantization bit width configuration, and in the non-critical parameter field using a second quantization bit width configuration smaller than the first quantization bit width configuration, to obtain adaptive quantization parameters.

[0024] The adaptive quantization parameters and the original floating-point parameters are used to calculate the error, and the parameter domain quantization error of different parameter domains is obtained. The influence of the quantization error of different parameter domains on the inter-layer propagation is analyzed, and the quantization sensitivity of each layer is calculated. The parameter domain quantization error is weighted according to the quantization sensitivity to obtain the quantization error data.

[0025] In one optional implementation, the step of weighting and combining the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determining the bit width configuration scheme based on the inter-layer error propagation evaluation matrix includes:

[0026] The local features between adjacent layers, the mid-range features between consecutive multi-layers, and the global features of the entire network are calculated respectively. The local features are obtained by weighted combination of quantization error data and importance evaluation index of adjacent layers. The mid-range features are obtained by weighted combination of local features of each layer within a feature block. The global features are obtained by weighted combination of mid-range features of each layer.

[0027] Construct an inter-layer error propagation evaluation matrix based on local features, mid-range features, and global features;

[0028] The error compensation value for each layer is calculated based on the inter-layer error propagation evaluation matrix. The error compensation value is determined by the weighted sum of the quantization error of the current layer and the propagation error of the previous layer. The bit width configuration scheme is obtained by adaptively adjusting the reference bit width of each layer based on the error compensation value.

[0029] Calculate the network performance evaluation value, update the feature combination weight coefficients in the construction process of the inter-layer error propagation evaluation matrix based on the network performance evaluation value, and reconstruct the inter-layer error propagation evaluation matrix until the network performance evaluation value meets the preset optimization objective.

[0030] In one optional implementation, the step of performing differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme includes:

[0031] The computational load data is combined using a time-series weighted method to obtain dynamic load characteristic values; the bit-width sensitivity is obtained by calculating the rate of change of resource requirements for each layer of the neural network based on the bit-width configuration scheme.

[0032] A resource allocation priority matrix is ​​constructed based on the load dynamic characteristic value and the bit width sensitivity. Each element of the resource allocation priority matrix is ​​obtained by weighted combination of the bit width sensitivity of the corresponding network layer and the load dynamic characteristic value of the corresponding computing unit, which is used to determine the mapping relationship between the network layer and the computing unit.

[0033] Importance analysis is performed on intermediate feature maps of the neural network, and an importance score is obtained based on the information distribution of the feature maps in the spatial and channel dimensions. Based on the importance score, the feature maps are divided into multiple importance levels, and a differentiated storage strategy is adopted for feature maps of different importance levels.

[0034] The mapping relationship and the storage strategy are jointly optimized based on a weighted combination of computational efficiency, storage efficiency, and communication efficiency.

[0035] In an optional implementation, the step of constructing a resource allocation priority matrix based on the load dynamic characteristic value and the bit width sensitivity includes:

[0036] The computing characteristic matching degree is calculated based on the hardware characteristic parameters of the computing unit and the computing characteristic parameters of the network layer; the computing unit characteristic value is obtained by weighted combination of the load dynamic characteristic value and the computing characteristic matching degree; the network layer characteristic value is obtained by weighted combination of the bit width sensitivity and the resource utilization of the neural network layer; the affinity between the network layer and the computing unit is calculated based on the computing unit characteristic value and the network layer characteristic value, and the affinity is used to characterize the degree of matching between the network layer and the computing unit.

[0037] Construct predictive performance metrics, including computational latency, resource utilization, and energy efficiency; establish a mapping optimization objective function based on the affinity and the predictive performance metrics; obtain an initial mapping scheme by iteratively optimizing the mapping optimization objective function under the conditions of satisfying the resource capacity constraints of computing units and task latency constraints; collect real-time execution data, adjust the mapping scheme online based on the execution data, and determine the resource allocation priority matrix.

[0038] A second aspect of the present invention provides a quantization compression and adaptive computing power optimization system for neural networks, comprising:

[0039] The first unit is used to obtain a pre-trained neural network, which includes multiple convolutional layers and fully connected layers;

[0040] The second unit is used to calculate the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, construct an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy, the importance evaluation index characterizing the information flow and inter-layer dependency of neurons; perform quantization tests with different bit widths on the convolutional layers and fully connected layers to obtain quantization error data; weight and combine the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determine the bit width configuration scheme based on the inter-layer error propagation evaluation matrix;

[0041] The third unit is used to acquire the computing load data of the target device; and to perform differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme.

[0042] The fourth unit is used to adjust the importance evaluation index according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, and generate the optimized neural network.

[0043] A third aspect of the present invention provides an electronic device, comprising:

[0044] processor;

[0045] Memory used to store processor-executable instructions;

[0046] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0048] The quantization compression and adaptive optimization method for neural networks provided by this invention constructs an importance evaluation index for neurons by calculating the weight distribution entropy and activation value distribution entropy, and generates an inter-layer error propagation evaluation matrix by combining quantization error data. This achieves precise configuration of different bit widths for each layer of the neural network, effectively reducing model storage space and computational resource consumption while maintaining the model's inference accuracy.

[0049] This invention maps computing units and allocates storage strategies differently based on the computing load data of the target device, which enhances the adaptability and execution efficiency of the model on different hardware platforms. It enables the model to adaptively adjust according to the computing power conditions of the actual deployment environment, thereby improving resource utilization.

[0050] This invention also continuously adjusts network parameters until a preset optimization goal is met by iteratively optimizing the importance evaluation index, forming a closed-loop optimization mechanism. This enables the neural network to achieve a balance between performance and resource consumption while maintaining functional integrity, making it suitable for resource-constrained edge computing scenarios and mobile device applications. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating the quantization compression and adaptive optimization method for neural networks according to an embodiment of the present invention.

[0052] Figure 2 Flowchart for parameter domain adaptive quantization and error propagation analysis. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0055] Figure 1 This is a flowchart illustrating the quantization compression and adaptive optimization method for neural networks according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0056] Obtain a pre-trained neural network, which contains multiple convolutional layers and fully connected layers;

[0057] For each convolutional layer and fully connected layer in the neural network, the weight distribution entropy value and the activation value distribution entropy value are calculated. Based on the weight distribution entropy value and the activation value distribution entropy value, an importance evaluation index for neurons is constructed. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization tests with different bit widths are performed on the convolutional layers and fully connected layers to obtain quantization error data. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix. Based on the inter-layer error propagation evaluation matrix, a bit width configuration scheme is determined.

[0058] Obtain the computing load data of the target device; based on the computing load data and the bit width configuration scheme, perform differentiated allocation of computing unit mapping and storage strategy for the network layer;

[0059] The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, thereby generating an optimized neural network.

[0060] For example, consider an object detection neural network consisting of 6 convolutional layers and 4 fully connected layers. The first convolutional layer has 3 input channels, 64 output channels, a 3×3 kernel size, and a stride of 1. The second convolutional layer has 64 input channels, 128 output channels, a 3×3 kernel size, and a stride of 2. The third convolutional layer has 128 input channels, 256 output channels, a 3×3 kernel size, and a stride of 1. The fourth convolutional layer has 256 input channels, 256 output channels, a 3×3 kernel size, and a stride of 2. The fifth convolutional layer has 256 input channels, 512 output channels, a 3×3 kernel size, and a stride of 1. The sixth convolutional layer has 512 input channels, 512 output channels, a 3×3 kernel size, and a stride of 2. The first fully connected layer has an input dimension of 25088 and an output dimension of 4096; the second fully connected layer has an input dimension of 4096 and an output dimension of 4096; the third fully connected layer has an input dimension of 4096 and an output dimension of 1000; and the fourth fully connected layer has an input dimension of 1000 and an output dimension of 5. This network was pre-trained on a standard object detection dataset, achieving an accuracy of 92.8% and a model size of 552MB.

[0061] The weight distribution entropy is calculated by analyzing the statistical distribution characteristics of the parameters in each layer. Taking the second convolutional layer as an example, this layer contains 73,728 parameters. The parameter value range is divided into 256 uniform intervals, and the number of parameters in each interval is counted to obtain a parameter distribution histogram. The proportion of the number of parameters in each interval to the total number of parameters in the histogram is calculated to obtain the probability distribution. The information entropy of this probability distribution is calculated, and the weight distribution entropy value is 5.86. The higher the weight distribution entropy value, the more dispersed the weight distribution and the greater the information content. The weight distribution entropy values ​​are calculated for all layers, resulting in the following entropy value sequence: 5.32 for the first convolutional layer, 5.86 for the second convolutional layer, 6.12 for the third convolutional layer, 5.74 for the fourth convolutional layer, 6.35 for the fifth convolutional layer, 5.92 for the sixth convolutional layer, 6.48 for the first fully connected layer, 6.21 for the second fully connected layer, 5.89 for the third fully connected layer, and 4.76 for the fourth fully connected layer.

[0062] The activation value distribution entropy is calculated by analyzing the statistical distribution characteristics of the output feature maps of each layer. Taking the second convolutional layer as an example, 1000 images are used as input to record the output feature map of this layer, with a feature map size of 56×56×128. The range of activation values ​​in the feature map is divided into 256 uniform intervals, and the number of activation values ​​in each interval is counted to obtain an activation value distribution histogram. The proportion of activation values ​​in each interval to the total number of activation values ​​in the histogram is calculated to obtain the probability distribution. The information entropy of this probability distribution is calculated, and the activation value distribution entropy value is 4.92. The higher the activation value distribution entropy value, the more dispersed the activation value distribution and the greater the information content. The activation value distribution entropy was calculated for all layers, resulting in the following entropy sequence: 4.53 for the 1st convolutional layer, 4.92 for the 2nd convolutional layer, 5.31 for the 3rd convolutional layer, 4.85 for the 4th convolutional layer, 5.42 for the 5th convolutional layer, 5.08 for the 6th convolutional layer, 5.63 for the 1st fully connected layer, 5.36 for the 2nd fully connected layer, 4.95 for the 3rd fully connected layer, and 3.82 for the 4th fully connected layer.

[0063] The importance evaluation index is obtained by weighted combination of the weight distribution entropy and the activation value distribution entropy, representing the information flow of neurons and the degree of inter-layer dependency. The weight of the weight distribution entropy is set to 0.6, and the weight of the activation value distribution entropy is set to 0.4. Taking the second convolutional layer as an example, the importance evaluation index is calculated as 5.86 × 0.6 + 4.92 × 0.4 = 5.48. The importance evaluation index is calculated for all layers, resulting in the following sequence: Convolutional Layer 1: 5.00; Convolutional Layer 2: 5.48; Convolutional Layer 3: 5.80; Convolutional Layer 4: 5.38; Convolutional Layer 5: 5.98; Convolutional Layer 6: 5.59; Fully Connected Layer 1: 6.13; Fully Connected Layer 2: 5.87; Fully Connected Layer 3: 5.52; Fully Connected Layer 4: 4.38. A higher importance evaluation index indicates greater importance of the layer in the network and a more significant impact on the overall network performance.

[0064] Quantization error data was obtained by performing quantization tests with different bit widths on convolutional and fully connected layers. Quantization tests were performed on each layer at 8-bit, 7-bit, 6-bit, 5-bit, and 4-bit widths. Taking the second convolutional layer as an example, 1000 images were used as input, and the layer was quantized with different bit widths. The difference in the output feature maps before and after quantization was compared, and the root mean square error (RMS) was calculated. The RMS error for 8-bit quantization was 0.0042, for 7-bit quantization it was 0.0083, for 6-bit quantization it was 0.0165, for 5-bit quantization it was 0.0312, and for 4-bit quantization it was 0.0596. Similar tests were performed on all layers to obtain a quantization error data matrix. The quantization error data was weighted and combined with an importance evaluation metric to generate an inter-layer error propagation evaluation matrix. The weight of the quantization error data was set to 0.7, and the weight of the importance evaluation metric was set to 0.3. Taking the 7-bit quantization of the second convolutional layer as an example, the error propagation evaluation value is calculated as 0.0083×0.7+5.48×0.3 / 10=0.170. The error propagation evaluation value is calculated for all bit width configurations of all layers, and an inter-layer error propagation evaluation matrix is ​​constructed.

[0065] An error propagation evaluation threshold of 0.25 was set. Layer-bit width combinations with evaluation values ​​below the threshold were considered feasible; those with evaluation values ​​above the threshold were considered to lead to significant accuracy loss. The algorithm iterated from high to low bit widths, selecting the lowest bit width that met the threshold as the bit width configuration for that layer. Considering the overall model size constraint, a compression ratio target of 25% of the original model size was set. Based on these principles, the bit width configuration scheme was determined as follows: 8 bits for the first convolutional layer, 7 bits for the second convolutional layer, 6 bits for the third convolutional layer, 8 bits for the fourth convolutional layer, 7 bits for the fifth convolutional layer, 6 bits for the sixth convolutional layer, 7 bits for the first fully connected layer, 6 bits for the second fully connected layer, 7 bits for the third fully connected layer, and 8 bits for the fourth fully connected layer. With this bit width configuration, the model size is 142MB, which is 25.7% of the original model size, and the average accuracy decreases by 2.3% to 90.5%.

[0066] The target device is an embedded computing platform comprising one high-performance processing unit, two general-purpose computing units, and four low-power computing units. Load data, including compute utilization, memory usage, and power consumption, is collected from each computing unit during different tasks using monitoring tools. For the high-performance processing unit, the average compute utilization is 76%, the average memory usage is 62%, and the average power consumption is 5.8W; for the general-purpose computing units, the average compute utilization is 68%, the average memory usage is 54%, and the average power consumption is 3.2W; and for the low-power computing units, the average compute utilization is 58%, the average memory usage is 45%, and the average power consumption is 1.5W. Based on the compute load data and bit width configuration, differentiated allocation of computing units and storage strategies is performed on the network layer. According to the computational characteristics and bit width requirements of the network layer, layers with high importance evaluation metrics and high bit width requirements are mapped to the high-performance processing unit; layers with medium importance evaluation metrics and medium bit width requirements are mapped to the general-purpose computing units; and layers with low importance evaluation metrics and low bit width requirements are mapped to the low-power computing units.

[0067] Taking the feature map output from the third convolutional layer as an example, its shape is 56×56×128. Information entropy analysis is performed on the feature map in both spatial and channel dimensions to calculate its importance score. The average spatial region importance score is 0.67; the average channel importance score is 0.59. Based on the importance score, the feature map is divided into multiple importance levels. Importance thresholds of 0.7 and 0.4 are set, classifying the feature map into three levels: high importance, medium importance, and low importance. Differentiated storage strategies are adopted for feature maps of different importance levels. High-importance regions and channels are stored using lossless compression, maintaining 8-bit precision; medium-importance regions and channels are quantized with low precision, reduced to 4-bit precision; low-importance regions and channels are sparsified, retaining only the most significant 20% of elements and setting the rest to zero. This differentiated storage strategy significantly reduces memory usage while maintaining the integrity of key information.

[0068] The importance evaluation metrics are adjusted based on the performance parameters of the neural network until they meet the preset optimization objectives, generating an optimized neural network. The preset optimization objectives are: model size not exceeding 30% of the original model, accuracy decrease not exceeding 3%, and inference speed improvement not less than 200%. The optimal bit width configuration and mapping strategy are found by iteratively adjusting the weight configuration of the importance evaluation metrics. In the initial configuration, the model size is 142MB, accuracy is 90.5%, and inference speed is 15fps. The weight distribution entropy value is adjusted to 0.55, and the activation value distribution entropy value is adjusted to 0.45. The importance evaluation metrics are recalculated to obtain an updated bit width configuration. The weight of quantization error data is adjusted to 0.65, and the weight of the importance evaluation metrics is adjusted to 0.35. The inter-layer error propagation evaluation matrix is ​​regenerated to obtain an updated bit width configuration. After optimization, the model size is 138MB, accuracy is 90.2%, and inference speed is 32fps, meeting the preset optimization objectives. The final optimized neural network is suitable for resource-constrained embedded devices, significantly reducing storage requirements and computational overhead while maintaining high accuracy.

[0069] In one optional implementation, the step of calculating the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, and constructing an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy includes:

[0070] Obtain the weight data and activation value data of each convolutional layer and fully connected layer in the neural network, discretize them to obtain the weight probability density distribution and activation value probability density distribution, and calculate the corresponding entropy value;

[0071] Similarity analysis is performed on the input and output features of neurons to identify neuron groups with similar response patterns; the distribution difference within each neuron group and the distribution difference between groups are calculated, and the contribution ratio of each neuron to the network function is determined based on the magnitude of the difference; based on the contribution ratio, a corresponding first balance factor and a second balance factor are assigned to each neuron group; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain the hierarchical information flow rate;

[0072] Calculate the mutual information between adjacent layers in the neural network;

[0073] The importance assessment index of a neuron is obtained by weighting and combining the hierarchical information flow with the mutual information; the importance assessment index is verified in multiple batches based on the network output error, and the final importance assessment index is determined when the index fluctuation is less than a preset fluctuation threshold.

[0074] For example, weight data can be directly extracted from the trained model, specifically the weight matrices of each layer. For instance, for convolutional layers, the weight shape is (number of output channels, number of input channels, kernel height, kernel width); for fully connected layers, the weight shape is (number of output neurons, number of input features). Activation value data is obtained by recording the feature maps or vectors output by each layer during the forward propagation of the neural network. This data is typically a four-dimensional tensor (batch size, number of channels, feature map height, feature map width) or a two-dimensional matrix (batch size, number of features).

[0075] After data acquisition, discretization is performed, dividing continuous weight values ​​and activation values ​​into several intervals. The numerical range is divided into 128 intervals on average, and the frequency of values ​​within each interval is counted and normalized to obtain a probability density distribution. Taking weight discretization as an example, assuming the weight value range of a certain convolutional layer is [-0.5, 0.5], and the interval width is set to 0.0078, the number of weight values ​​in each interval is counted and divided by the total number of weights to obtain the probability distribution. For a certain convolutional layer, a 128-dimensional probability distribution vector similar to [0.001, 0.003, 0.008, ..., 0.002] is obtained.

[0076] Based on the discretized probability distribution, the entropy values ​​of the weight distribution and the activation value distribution are calculated. For the weight distribution of a certain layer, assuming the probability vector is [p1, p2, ..., p128], the entropy value is calculated using the information entropy formula: H = -Σ(pi × log2(pi)), where pi is the probability value of the i-th interval, and the summation is performed over all non-zero probabilities. For example, for the probability vector [0.001, 0.003, 0.008, ..., 0.002], substituting into the entropy formula and taking the negative sign, the entropy value is 3.742. The same calculation method is used for the activation value distribution entropy value, i.e., H = -Σ(pi × log2(pi)), where pi is the probability value of the i-th interval in the probability vector of the activation value distribution, and the calculated value is 4.213.

[0077] In the neuron similarity analysis phase, the input connection weights and output response features are extracted for each neuron. For example, for the j-th neuron in the i-th layer, its input weight vector is [-0.03, 0.15, 0.08, -0.12, ...], and its output response features are [0.23, 0.45, 0.10, 0.35, ...]. The similarity between neurons is assessed by calculating cosine similarity or Euclidean distance. When the similarity between two neurons exceeds 0.85, they are grouped into the same group.

[0078] In one embodiment, for a fully connected layer containing 1024 neurons, similarity analysis identified 87 neuron groups, with the largest group containing 35 neurons and the smallest group containing only a single neuron. For these groups, the average variance of the neuron responses within each group (e.g., 0.032) was calculated as the internal distribution dissimilarity, and the average distance between the response centers of different groups (e.g., 0.567) was calculated as the inter-group distribution dissimilarity.

[0079] Based on the above variability data, the contribution ratio of each neuron to network function was determined. The smaller the variability within a group and the larger the variability between groups, the more specialized the function of the neurons in that group is, and the higher their contribution ratio. For example, a neuron group with an internal variability of 0.025 and an inter-group variability of 0.612 was assigned a relatively high contribution ratio of 0.078.

[0080] The first and second balance factors are allocated based on their contribution proportions. The first balance factor is used for the weight distribution entropy value, and its calculation takes into account the size and internal variability of the neuron group. The second balance factor is used for the activation value distribution entropy value, mainly considering the inter-group variability and neuronal response intensity. In the example, a certain neuron group obtains a first balance factor of 0.65 and a second balance factor of 0.82.

[0081] The hierarchical information flow is obtained by multiplying the weight distribution entropy value by the first balancing factor and the activation value distribution entropy value by the second balancing factor, and then combining them through a nonlinear mapping function. The nonlinear mapping function can be an sigmoid function, specifically f(x) = 1 / (1 + e^(-k×(x-x0))), where k controls the slope of the function and x0 is the center point position. In implementation, k = 0.5 and x0 = 5 can be set to ensure the output value is within the range [0,1]. In practice, the weight distribution entropy value of a certain layer is 3.742, the first balancing factor is 0.65, and the product is 2.432; the activation value distribution entropy value is 4.213, the second balancing factor is 0.82, and the product is 3.455; after nonlinear mapping, the hierarchical information flow is 0.765.

[0082] Mutual information calculation between adjacent layers is a crucial step in evaluating information transfer efficiency. By analyzing the relationship between the joint probability distribution of the output of layer i and the input of layer i+1 and their respective marginal distributions, the degree of information transfer is quantified. The mutual information calculation method involves taking the logarithm of the ratio of the joint probability to the marginal probability, multiplying it by the joint probability, and summing the results. The joint probability represents the probability that two variables simultaneously take a specific value, while the marginal probability represents the probability that a single variable takes a specific value. In practical calculations, these probability distributions can be estimated using a histogram method. The output of layer i and the input of layer i+1 are divided into m and n intervals respectively, constructing an m×n joint distribution matrix, and then the mutual information calculation is performed. For example, the calculated mutual information between layers 3 and 4 is 1.856.

[0083] Finally, the importance assessment index of neurons is obtained by weighting and combining hierarchical information flow and mutual information. The weighting coefficients can be determined experimentally; for example, hierarchical information flow can be weighted at 0.6, and mutual information at 0.4. For a neuron in a certain layer, the hierarchical information flow is 0.765, and the average mutual information with adjacent layers is 1.856. The final importance assessment index is calculated as 0.765 × 0.6 + 1.856 × 0.4 = 1.201.

[0084] To ensure the stability of the evaluation metrics, the importance evaluation metrics were validated in multiple batches based on network output error. Different batches of validation data (e.g., 5 batches, 100 samples per batch) were used to observe metric changes. The final importance evaluation metric was confirmed when the metric fluctuation in three consecutive batches was less than a preset fluctuation threshold of 0.05. For example, if the importance metrics of a neuron in the 5 batches were [1.201, 1.185, 1.196, 1.202, 1.193], with a maximum fluctuation of 0.017, which is less than the threshold of 0.05, the final importance evaluation metric was determined to be 1.195 (the average of the 5 batches).

[0085] This invention accurately assesses the actual contribution of neurons to network function by performing similarity analysis on neurons and identifying neuron groups, combined with weight distribution entropy and activation value distribution entropy. It considers not only the characteristics of individual neurons but also the overall effect of neuron groups, and ensures the stability of the assessment indicators through multiple batch validations, thus improving the accuracy and reliability of neuron importance assessment.

[0086] In one alternative implementation, the step of calculating the mutual information between adjacent layers in the neural network includes:

[0087] Multi-scale decomposition of feature data from adjacent layers in a neural network yields feature representations at multiple scale levels. Feature probability distributions are calculated for each scale level feature representation. Based on the feature probability distributions at each scale level, the information entropy of the previous layer and the information entropy of the next layer are calculated respectively, and the combination yields mutual information at multiple scales.

[0088] The temporal feature sequences of adjacent layers are obtained, the feature probability distribution is temporally smoothed, and the temporal mutual information is calculated; the conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated; the multiple scale mutual information, temporal mutual information, and conditional mutual information are weighted and combined to obtain the initial mutual information.

[0089] A causal graph of inter-layer feature transmission is constructed to identify key transmission paths and analyze the causal effect strength of different transmission channels. Based on the causal effect strength and initial mutual information, a loss matrix for feature transmission is constructed, which characterizes the degree of information retention of features during transmission.

[0090] The connection weights of the corresponding channels are adjusted according to the loss values ​​of each transmission channel in the loss matrix, and the loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold, thereby obtaining the mutual information between adjacent layers.

[0091] For example, we can obtain feature data from adjacent layers in a neural network. Taking a deep neural network with 5 layers as an example, we select the 3rd and 4th layers as the research objects and extract the activation values ​​of these two layers when processing 1000 samples as feature data. The feature dimension of the 3rd layer output is 256, and the feature dimension of the 4th layer output is 128.

[0092] The acquired feature data is decomposed at multiple scales. Wavelet transform is used to decompose the features into different frequency components. A three-level decomposition is used to obtain feature representations at three scale levels: high frequency, mid frequency, and low frequency. For the 256-dimensional features at the third level, the decomposition yields 64-dimensional features at scale 1 (high frequency), 96-dimensional features at scale 2 (mid frequency), and 96-dimensional features at scale 3 (low frequency). Similarly, the 128-dimensional features at the fourth level are decomposed to obtain 32-dimensional features at scale 1, 48-dimensional features at scale 2, and 48-dimensional features at scale 3.

[0093] For each scale level, a kernel density estimation method is used to construct a probability density function for the feature vector at each scale. Taking high-frequency features as an example, a Gaussian kernel function with a bandwidth of 0.15 is used to generate a smooth probability distribution in the feature space. For the high-frequency features of 1000 samples, the probability that their distribution falls within 10 equally divided intervals is calculated, resulting in a discretized probability distribution vector.

[0094] Based on the calculated feature probability distribution, information entropy and mutual information at each scale level are calculated. Taking high-frequency features as an example, the information entropy value of the high-frequency features in the third layer is 4.32, the information entropy value of the high-frequency features in the fourth layer is 3.85, and the mutual information value of the high-frequency features between the two layers is 2.76. The mutual information is calculated by statistically analyzing the joint occurrence frequency of the two layers of features in each interval, constructing a joint probability table, then taking the logarithm of the ratio of the joint probability to the individual marginal probabilities, multiplying it by the joint probability, and summing the results. Similarly, the mutual information of mid-frequency features is calculated to be 3.12, and the mutual information of low-frequency features is 3.45. Combining the mutual information of the three scales, a weighted average method is used to calculate the multi-scale mutual information, with weights set to 0.3, 0.3, and 0.4 respectively, resulting in a multi-scale mutual information value of 3.14.

[0095] When processing continuous input data (such as video frame sequences), activation values ​​of adjacent layers at 100 consecutive time points are extracted to form a temporal feature sequence. For each time point, feature vectors from layers 3 and 4 are extracted to construct a feature matrix in the temporal dimension. Temporal smoothing is applied to the feature probability distribution, and temporal mutual information is calculated using the sliding window method. In the specific implementation, the window size is set to 5, and the temporal features are slid across the window to calculate the feature probability distribution within each window. For the sequence of 100 time points, probability distributions of 96 sliding windows are generated. The similarity of feature distributions of adjacent layers within the same time window is compared, and the temporal mutual information value is calculated to be 2.87. The calculation method treats the features within the time window as a whole, calculates the joint probability and marginal probability at the window level, and then evaluates them according to the mutual information calculation principle.

[0096] Conditional feature information from adjacent layers is obtained, and conditional mutual information is calculated. Input conditions for the neural network are selected (e.g., images of different categories), and feature distributions of adjacent layers are calculated under these conditions. In the specific implementation, 1000 samples are divided into 10 categories, and the feature probability distribution under each category condition is calculated. For each category, the conditional mutual information between layers 3 and 4 is calculated, and the average conditional mutual information value is 3.05. Conditional mutual information calculation evaluates the degree of information dependence between features of two layers under input conditions. The calculation method is to calculate the mutual information under each condition separately and then take a weighted average according to the probability of the condition occurring.

[0097] The initial mutual information is obtained by weighting and combining the multi-scale mutual information, temporal mutual information, and conditional mutual information. The weight parameters are set to 0.5, 0.3, and 0.2, and the calculated initial mutual information value is 3.06. The specific calculation method is 3.14×0.5+2.87×0.3+3.05×0.2=3.06.

[0098] A causal graph for inter-layer feature propagation is constructed, treating the 256-dimensional features of layer 3 and the 128-dimensional features of layer 4 as nodes. Correlation analysis is used to establish connections between these nodes. A correlation threshold of 0.4 is set; a connection is established when the correlation coefficient between two feature dimensions exceeds this threshold. The correlation coefficient is calculated by multiplying the covariance of each feature dimension across all samples by its respective standard deviation. The final result is a causal graph containing 384 nodes and approximately 15,000 edges.

[0099] Random walk sampling was performed on the causal graph to identify the most influential transmission paths. The third-layer feature nodes were randomly perturbed, and the changes in the fourth-layer feature nodes were measured to calculate the causal effect strength. The causal effect strength was calculated as the ratio of the change in the fourth-layer feature nodes before and after the perturbation to the perturbation in the third layer. After 10,000 simulations, 50 critical transmission paths were identified, with an average causal effect strength of 0.72.

[0100] A loss matrix for feature propagation is constructed, a 256×128 matrix representing the information propagation loss from each feature dimension in layer 3 to each feature dimension in layer 4. For connections on critical propagation paths, lower loss values ​​are set based on their causal effect strength; for non-critical paths, higher loss values ​​are set. Specifically, an initial mutual information value of 3.06 is first used as a global baseline, and a normalization coefficient is calculated as the reciprocal of the initial mutual information (approximately 0.327). Then, the causal effect strength of each connection is multiplied by this normalization coefficient to ensure that the overall information flow matches the initial mutual information. Finally, the loss value is calculated by subtracting the adjusted causal effect strength from 1. Furthermore, for feature dimension pairs that contribute significantly to mutual information (identified based on the joint probability distribution), their loss value is further reduced by 10% to preserve critical information channels. By combining initial mutual information and causal effect strength, the loss matrix reflects the causal relationships of local connections while maintaining consistency with the global information flow. The average loss value of the initial loss matrix is ​​0.35.

[0101] For connections with a loss value higher than 0.5, their weights are reduced by 20%; for connections with a loss value lower than 0.2, their weights are increased by 15%. Specifically, the formula for adjusting the weights is: New weight = Original weight × (1 - Adjustment coefficient × (Loss value - Baseline loss value)), where the baseline loss value is set to 0.35 and the adjustment coefficient is set to 0.5. After adjustment, the loss matrix is ​​recalculated, and the difference is compared with the previous iteration.

[0102] The loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold of 0.01. The fluctuation value is calculated by taking the square root of the sum of the squares of the differences between the elements of the loss matrix in the current iteration and the previous iteration. In the actual implementation, after 8 iterations, the fluctuation value of the loss matrix drops to 0.009, satisfying the convergence condition. The final average value of the loss matrix is ​​0.28, indicating that the information transmission efficiency has been improved.

[0103] Subtracting the average loss value from 1 yields an information retention rate of 0.72. Multiplying this retention rate by the initial mutual information yields a mutual information value of 2.20 between adjacent layers, representing the effective information transfer between layers 3 and 4 of the neural network. This mutual information value can be used to evaluate the efficiency of information flow between network layers, providing a basis for subsequent network structure optimization.

[0104] This invention employs a combination of multi-scale decomposition and temporal smoothing to calculate inter-layer mutual information, comprehensively capturing the information transmission characteristics of features across different scales and temporal dimensions. By constructing a causal graph of inter-layer feature transmission, analyzing the causal effect strength of different transmission channels, and establishing a feature transmission loss matrix, the degree of information retention during feature transmission can be accurately characterized. Dynamically adjusting the connection weights of transmission channels based on the loss values ​​ensures the accuracy and adaptability of mutual information calculation.

[0105] In one optional implementation, the step of performing quantization tests on the convolutional layer and the fully connected layer with different bit widths to obtain quantization error data includes:

[0106] Feature importance analysis is performed on the parameters of the convolutional layer and the fully connected layer, and the parameter space is divided into critical parameter domains and non-critical parameter domains;

[0107] Quantization processing is performed in the critical parameter field using a first quantization bit width configuration, and in the non-critical parameter field using a second quantization bit width configuration smaller than the first quantization bit width configuration, to obtain adaptive quantization parameters.

[0108] The adaptive quantization parameters and the original floating-point parameters are used to calculate the error, and the parameter domain quantization error of different parameter domains is obtained. The influence of the quantization error of different parameter domains on the inter-layer propagation is analyzed, and the quantization sensitivity of each layer is calculated. The parameter domain quantization error is weighted according to the quantization sensitivity to obtain the quantization error data.

[0109] Combination Figure 2The flowchart for parameter domain adaptive quantization and error propagation analysis is used to illustrate this, for example, obtaining the parameters of the convolutional and fully connected layers of a pre-trained neural network model. Taking an image classification network as an example, this network contains 5 convolutional layers and 3 fully connected layers. The parameter matrix of the second convolutional layer is extracted, with a shape of 64×32×3×3, representing 64 output channels, 32 input channels, and a 3×3 convolutional kernel size; the parameter matrix of the first fully connected layer is extracted, with a shape of 512×4096, representing 512 output neurons and 4096 input features.

[0110] Feature importance analysis was performed on these parameters, dividing the parameter space into critical parameter domains and non-critical parameter domains. The feature importance analysis employed a parameter sensitivity evaluation method, specifically involving adding a small perturbation to each parameter and measuring the magnitude of change in the network output. For the parameter matrix of the second convolutional layer, 1000 parameter locations were randomly selected. At each location, the parameter value was increased by 1% of its original value, while keeping other parameters constant. The average rate of change in the network output was calculated using a validation dataset. Parameter sensitivity was defined as the ratio of the network output rate of change to the parameter rate of change. For example, the sensitivity distribution of the parameters in the second convolutional layer ranged from 0.01 to 3.56, with an average of 0.75. Parameter locations with a sensitivity greater than 1.5 were classified as critical parameter domains, accounting for approximately 20% of the total parameters; parameter locations with a sensitivity less than or equal to 1.5 were classified as non-critical parameter domains, accounting for approximately 80% of the total parameters. Similarly, sensitivity analysis was performed on the 512×4096 parameter matrix of the first fully connected layer, with a sensitivity distribution ranging from 0.02 to 2.86 and an average of 0.68. Parameters with a sensitivity greater than 1.2 were classified as critical parameter domains, accounting for approximately 25% of the total parameters; parameters with a sensitivity less than or equal to 1.2 were classified as non-critical parameter domains, accounting for approximately 75% of the total parameters. For sensitivity analysis, 1000 validation samples were used, with each sample calculated five times and the average value taken to reduce the impact of randomness.

[0111] Differential quantization is performed on the predefined parameter domains. The first quantization bit width is configured as 8 bits, applied to the critical parameter domain; the second quantization bit width is configured as 4 bits, applied to the non-critical parameter domains. For the second convolutional layer, the critical parameter domain contains approximately 11,500 parameters, using 8-bit quantization; the non-critical parameter domain contains approximately 46,000 parameters, using 4-bit quantization. Specifically, the quantization process involves determining the numerical range of each parameter domain and then mapping the floating-point numbers to the corresponding integer range. For example, in the second convolutional layer, the numerical range of the critical parameter domain is [-0.35, 0.42], which is uniformly mapped to the integer range [-128, 127] represented by 8 bits; the numerical range of the non-critical parameter domain is [-0.28, 0.31], mapped to the integer range [-8, 7] represented by 4 bits.

[0112] For the first fully connected layer, the critical parameter domain contains approximately 524,000 parameters, quantized using 8-bit quantization; the non-critical parameter domain contains approximately 1,572,000 parameters, quantized using 4-bit quantization. The numerical range of the critical parameter domain is [-0.12, 0.15], and the numerical range of the non-critical parameter domain is [-0.09, 0.11]. Considering the numerical distribution characteristics, uniform quantization is used, with a quantization step size of 0.00224 for the critical parameter domain and 0.0125 for the non-critical parameter domain.

[0113] After quantization, adaptive quantization parameters are obtained, and their error is calculated compared with the original floating-point parameters to obtain the parameter domain quantization error for different parameter domains. The calculation method is to calculate the mean squared error between the quantized parameters and the original parameters. For the second convolutional layer, the mean squared error of quantization in the critical parameter domain is 0.000125, and the mean squared error of quantization in the non-critical parameter domain is 0.000728. For the first fully connected layer, the mean squared error of quantization in the critical parameter domain is 0.000089, and the mean squared error of quantization in the non-critical parameter domain is 0.000475.

[0114] Considering the inter-layer connections in the network structure, an error propagation model is constructed. Through forward propagation analysis, the impact of quantization error on the activation values ​​of subsequent layers is measured. Taking the second convolutional layer as an example, the quantization parameters of this layer are applied to the network, while other layers retain their original floating-point parameters. The average error of the network output is calculated on 1000 validation samples. Then, the second convolutional layer is restored to floating-point parameters, the third convolutional layer is quantized, and the network output error is recalculated. The output errors in the two cases are compared to evaluate the impact of quantization error propagation.

[0115] For example, after quantization of the second convolutional layer, the average relative error of the network output is 1.8%; after quantization of the third convolutional layer, the average relative error of the network output is 2.1%; and after quantization of the first fully connected layer, the average relative error of the network output is 3.5%. Based on these data, the quantization sensitivity of each layer is calculated, defined as the ratio of the relative error of the network output caused by the quantization of that layer to the average relative error of all layers. The quantization sensitivity of the second convolutional layer is 0.72, the quantization sensitivity of the third convolutional layer is 0.84, and the quantization sensitivity of the first fully connected layer is 1.40.

[0116] The quantization errors in the parameter domains are weighted according to quantization sensitivity to obtain the final quantization error data. The weighting method is to multiply the quantization error in the parameter domain of each layer by the corresponding quantization sensitivity. For the second convolutional layer, the weighted quantization error in the critical parameter domain is 0.000125 × 0.72 = 0.000090, and the weighted quantization error in the non-critical parameter domain is 0.000728 × 0.72 = 0.000524. For the first fully connected layer, the weighted quantization error in the critical parameter domain is 0.000089 × 1.40 = 0.000125, and the weighted quantization error in the non-critical parameter domain is 0.000475 × 1.40 = 0.000665. These weighted quantization error data comprehensively consider parameter importance and layer sensitivity, and can more accurately reflect the actual impact of different quantization strategies on network performance.

[0117] The differentiated quantization method of this invention distinguishes the importance of parameters and uses different bit width configurations for different parameter domains, thereby effectively reducing storage space and computational resource consumption while maintaining model accuracy. Combined with inter-layer error propagation analysis and quantization sensitivity assessment, it provides a more accurate quantization error assessment mechanism, offering theoretical basis and practical guidance for model compression and hardware deployment, effectively balancing inference performance and resource utilization efficiency.

[0118] In one optional implementation, the step of weighting and combining the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determining the bit width configuration scheme based on the inter-layer error propagation evaluation matrix includes:

[0119] The local features between adjacent layers, the mid-range features between consecutive multi-layers, and the global features of the entire network are calculated respectively. The local features are obtained by weighted combination of quantization error data and importance evaluation index of adjacent layers. The mid-range features are obtained by weighted combination of local features of each layer within a feature block. The global features are obtained by weighted combination of mid-range features of each layer.

[0120] Construct an inter-layer error propagation evaluation matrix based on local features, mid-range features, and global features;

[0121] The error compensation value for each layer is calculated based on the inter-layer error propagation evaluation matrix. The error compensation value is determined by the weighted sum of the quantization error of the current layer and the propagation error of the previous layer. The bit width configuration scheme is obtained by adaptively adjusting the reference bit width of each layer based on the error compensation value.

[0122] Calculate the network performance evaluation value, update the feature combination weight coefficients in the construction process of the inter-layer error propagation evaluation matrix based on the network performance evaluation value, and reconstruct the inter-layer error propagation evaluation matrix until the network performance evaluation value meets the preset optimization objective.

[0123] For example, when generating the inter-layer error propagation evaluation matrix by weighting and combining quantization error data with importance assessment indicators, a multi-level feature analysis method is employed. Taking a 15-layer image recognition neural network as an example, the network can be divided into 3 feature blocks, each containing 5 layers. When calculating the local features between adjacent layers, the quantization error data of layers 3 and 4 are extracted first. The quantization error of layer 3 in the key parameter domain is 0.000112, the quantization error in the non-key parameter domain is 0.000683, the quantization sensitivity is 0.79, and the weighted quantization error data is 0.000629. The quantization error of layer 4 in the key parameter domain is 0.000098, the quantization error in the non-key parameter domain is 0.000572, the quantization sensitivity is 0.81, and the weighted quantization error data is 0.000543. The importance assessment indicators for the two layers are extracted: 0.68 for layer 3 and 0.71 for layer 4, reflecting the functional contribution of neurons in each layer.

[0124] The calculation of local features employs an exponential decay weighted method, setting the decay coefficient to 0.7. The quantization error data of the current layer is multiplied by 0.7, added to the product of the quantization error data of the previous layer and the importance evaluation index, and then multiplied by 0.3 to obtain the inter-layer local features. Specifically, the local feature calculation between layers 3 and 4 is 0.000629 × 0.7 + 0.000543 × 0.71 × 0.3 = 0.000556. Similarly, the local features of all adjacent layer pairs in the network are calculated to construct a local feature matrix.

[0125] When calculating the mid-range features between consecutive multilayers, the interaction effects of each layer within the feature block are considered. For the first feature block (layers 1 to 5), local features of each layer pair are extracted, and distance decay weights are set: a layer pair with a distance of 1 has a weight of 0.5, a layer pair with a distance of 2 has a weight of 0.3, a layer pair with a distance of 3 has a weight of 0.15, and a layer pair with a distance of 4 has a weight of 0.05. The mid-range features are calculated with layer 3 as the center, comprehensively considering the influence of layers 1 to 5 on layer 3, and the local features of each layer pair are weighted and combined. The calculation results show that the mid-range feature value of layer 3 is 0.000612, reflecting the comprehensive quantization sensitivity of layer 3 within the feature block.

[0126] After the mid-range features of each layer within a feature block are calculated, a mid-range feature transfer model is constructed based on the hierarchical transfer relationship. Considering the nonlinear dependencies between distant layers in deep networks, an adaptive weight allocation strategy is adopted. Transfer weights are assigned according to the position and functional importance of each layer in the network. Layers closer to the input have lower weights, set to 0.2; hidden layers in the middle have moderate weights, set to 0.5; and layers closer to the output have higher weights, set to 0.8. Taking the mid-range feature transfer of the second feature block as an example, the mid-range features of layers 6 to 10 are weighted according to their position, resulting in a feature block-level transferred feature of 0.000576.

[0127] When calculating the global features of the entire network, the mid-range features of all feature blocks are integrated. An importance-driven hierarchical aggregation method is employed, assigning aggregation weights based on the degree of influence of each feature block on the network output. Through perturbation analysis, the influence weights of the first feature block are determined to be 0.25, the second feature block 0.35, and the third feature block 0.40. The mid-range features of the three feature blocks are then weighted and summed to obtain the global feature value of 0.000603, reflecting the overall network's quantization sensitivity.

[0128] An inter-layer error propagation evaluation matrix is ​​constructed based on local, mid-range, and global features. A 15×15 matrix is ​​constructed, where each element represents the intensity of error propagation from the corresponding row layer to the corresponding column layer. For directly connected layer pairs, local eigenvalues ​​are used for padding; for non-directly connected layer pairs within a feature block, the product of the mid-range feature and the distance attenuation coefficient is used for padding; for layer pairs spanning feature blocks, the product of the global feature and the inter-block distance attenuation coefficient is used for padding. Taking the error propagation impact from layer 3 to layer 7 as an example, since they belong to different feature blocks and are not directly connected, their evaluation value is 0.000603×0.4=0.000241, where 0.4 is the inter-block distance attenuation coefficient.

[0129] The error compensation value for each layer is calculated based on the inter-layer error propagation evaluation matrix. The error compensation value is determined by the weighted sum of the current layer's own quantization error and the propagation error of the preceding layers. For layer 7, its own quantization error is 0.000527, and the propagation error of the preceding layers is the cumulative value of the quantization errors of each preceding layer after attenuation through the propagation matrix, calculated to be 0.000382. Setting the weight of the current layer's own error to 0.6 and the weight of the propagation error to 0.4, the error compensation value for layer 7 is 0.000527 × 0.6 + 0.000382 × 0.4 = 0.000469.

[0130] The baseline bit width for each layer is adaptively adjusted based on the error compensation value. The baseline bit width is set to 8 bits, and the bit width is adjusted according to the comparison between the error compensation value and a reference threshold. If the error compensation value is less than 0.000300, the bit width is reduced by 2 bits; if the error compensation value is between 0.000300 and 0.000500, the bit width is reduced by 1 bit; if the error compensation value is between 0.000500 and 0.000700, the baseline bit width remains unchanged; if the error compensation value is greater than 0.000700, the bit width is increased by 1 bit. Taking layer 7 as an example, its error compensation value is 0.000469, which falls between 0.000300 and 0.000500, therefore the bit width is adjusted to 7 bits. In this way, an appropriate bit width configuration is determined for each layer in the network.

[0131] Calculate the network performance evaluation value and optimize the evaluation matrix based on the evaluation results. A comprehensive evaluation index is adopted, including the accuracy loss of the quantized model on the validation set (weight 0.5), the model storage space reduction rate (weight 0.3), and the inference computation reduction rate (weight 0.2). Taking the above bit-width configuration scheme as an example, tested on 1000 validation samples, the accuracy loss of the quantized model is 0.76%, the storage space reduction rate is 58.3%, and the inference computation reduction rate is 42.1%, resulting in a comprehensive performance evaluation value of 0.682.

[0132] If the performance evaluation value does not reach the preset optimization target of 0.75, the feature combination weight coefficients in the evaluation matrix construction process are updated. The attenuation coefficient in local feature calculation is increased from 0.7 to 0.75, the distance weight in mid-range feature calculation is adjusted from [0.5, 0.3, 0.15, 0.05] to [0.55, 0.25, 0.15, 0.05], and the feature block weight in global feature calculation is adjusted from [0.25, 0.35, 0.40] to [0.20, 0.35, 0.45]. The inter-layer error propagation evaluation matrix is ​​reconstructed using the updated weight coefficients, and the above bit-width configuration process is repeated until the network performance evaluation value reaches the preset optimization target.

[0133] This invention accurately captures the propagation pattern of quantization error in the network by constructing an inter-layer error propagation evaluation matrix, and allocates differentiated bit width configurations to different layers, achieving an optimal balance between accuracy loss and model compression rate. This effectively improves the deployment efficiency of quantization models on resource-constrained devices while maintaining the stability of model performance.

[0134] In one optional implementation, the step of performing differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme includes:

[0135] The computational load data is combined using a time-series weighted method to obtain dynamic load characteristic values; the bit-width sensitivity is obtained by calculating the rate of change of resource requirements for each layer of the neural network based on the bit-width configuration scheme.

[0136] A resource allocation priority matrix is ​​constructed based on the load dynamic characteristic value and the bit width sensitivity. Each element of the resource allocation priority matrix is ​​obtained by weighted combination of the bit width sensitivity of the corresponding network layer and the load dynamic characteristic value of the corresponding computing unit, which is used to determine the mapping relationship between the network layer and the computing unit.

[0137] Importance analysis is performed on intermediate feature maps of the neural network, and an importance score is obtained based on the information distribution of the feature maps in the spatial and channel dimensions. Based on the importance score, the feature maps are divided into multiple importance levels, and a differentiated storage strategy is adopted for feature maps of different importance levels.

[0138] The mapping relationship and the storage strategy are jointly optimized based on a weighted combination of computational efficiency, storage efficiency, and communication efficiency.

[0139] For example, when differentiating the allocation of computing unit mapping and storage strategies in the network layer based on computing load data and bit width configuration schemes, the computing load data on the device is first obtained. Taking a heterogeneous computing platform as an example, this platform includes one high-performance processing unit, two general-purpose computing units, and four low-power computing units. Load data of each computing unit performing different tasks is collected using monitoring tools, including computing utilization, memory usage, and power consumption data. For the high-performance processing unit, load data is sampled every 10 seconds for 5 consecutive minutes, resulting in 30 sampling points. The computing utilization ranges from 45% to 92%, with an average of 76%; the memory usage ranges from 38% to 85%, with an average of 62%; and the power consumption ranges from 3.5W to 8.2W, with an average of 5.8W. Similarly, load data for the general-purpose computing units and low-power computing units is collected.

[0140] The dynamic characteristic value of the load is obtained by time-weighted combination of the computational load data. A time-decay weighting method is used to process the load data, with higher weights for recent sampling points and lower weights for older sampling points. The decay factor is set to 0.95. For the computational utilization of high-performance processing units, the weight of the most recent sampling point is 1, the weight of the previous sampling point is 0.95, the weight of the sampling point before that is 0.95², and so on. The weighted average load is calculated, yielding a time-adjusted computational utilization of 79%, a memory utilization of 65%, and a power consumption of 6.1W. Combining these three indicators with weights of 0.4, 0.3, and 0.3 respectively, the dynamic characteristic value of the load for the high-performance processing unit is 0.704. Similarly, the dynamic characteristic values ​​of the load for the two general-purpose computing units are calculated to be 0.623 and 0.587, respectively, and the dynamic characteristic values ​​of the load for the four low-power computing units are 0.412, 0.435, 0.398, and 0.426, respectively.

[0141] Taking a 10-layer object detection network as an example, this network contains 6 convolutional layers and 4 fully connected layers. According to the bit-width configuration scheme obtained in the previous steps, the first convolutional layer has a bit width of 8 bits, the second convolutional layer has a bit width of 7 bits, the third convolutional layer has a bit width of 6 bits, the fourth convolutional layer has a bit width of 8 bits, the fifth convolutional layer has a bit width of 7 bits, the sixth convolutional layer has a bit width of 6 bits, the first fully connected layer has a bit width of 7 bits, the second fully connected layer has a bit width of 6 bits, the third fully connected layer has a bit width of 7 bits, and the fourth fully connected layer has a bit width of 8 bits. For each layer, the change in computational resource requirements under different bit-width configurations is calculated. Taking the second convolutional layer as an example, its number of parameters is 73,728. Reducing the quantization from 8 bits to 7 bits saves 1 bit per parameter, resulting in a total saving of 73,728 bits of storage space, approximately 9.216 KB. The rate of change in computational resource requirements is 12.5%, representing the proportion of resource requirements reduced due to the reduction in bit width. Considering the computational characteristics of different layers, the bit-width sensitivity of a convolutional layer is also related to the kernel size and the number of input and output channels. The second convolutional layer has a 3×3 kernel, 64 input channels, and 128 output channels; its bit-width sensitivity is calculated as 0.125×(3×3×64×128) / (73,728)=0.125×1=0.125. Similarly, the bit-width sensitivity of all layers is calculated to obtain the bit-width sensitivity vector.

[0142] A resource allocation priority matrix is ​​constructed based on load dynamic feature values ​​and bit width sensitivity. A 10×7 matrix is ​​constructed, where rows represent the 10 layers of the network and columns represent the 7 computational units. Each element in the matrix is ​​obtained by a weighted combination of the bit width sensitivity of the corresponding network layer and the load dynamic feature value of the corresponding computational unit. The weight of bit width sensitivity is set to 0.6, and the weight of load dynamic feature value is set to 0.4. Taking the mapping priority between the second convolutional layer and the high-performance processing unit as an example, 0.125×0.6+0.704×0.4=0.3566 is calculated. The specific combination method is to multiply the bit width sensitivity by its weight of 0.6, add the load dynamic feature value multiplied by its weight of 0.4, and obtain the weighted sum as the priority value. Similarly, the mapping priority between all layers and all computational units is calculated, and the priority matrix is ​​filled. The higher the priority value, the higher the efficiency of the layer in executing on the computational unit. Based on the priority matrix, the mapping relationship between network layers and computational units is determined. A greedy algorithm is used, selecting the layer-unit pair with the highest priority for mapping each time, until all layers are allocated to computational units. In the actual mapping, the 1st and 4th convolutional layers and the 4th fully connected layer are mapped to high-performance processing units; the 2nd and 5th convolutional layers and the 1st and 3rd fully connected layers are mapped to two general-purpose computing units; and the 3rd and 6th convolutional layers and the 2nd fully connected layer are mapped to four low-power computing units.

[0143] Importance analysis is performed on the intermediate feature maps of the neural network. Taking the feature map output from the 3rd convolutional layer as an example, its shape is 56×56×128, representing a feature map height of 56 pixels, a width of 56 pixels, and 128 channels. Information entropy analysis is performed on the feature map in the spatial dimension, dividing the 56×56 spatial region into a 4×4 grid, with each grid region being 14×14 pixels. The distribution entropy of pixel values ​​within each grid region is calculated, resulting in an entropy matrix of 16 spatial regions. Specifically, the pixel value distribution within each grid region is discretized into 32 uniform intervals, the number of pixels in each interval is counted, the normalized probability distribution is calculated, and then the information entropy of this distribution is calculated. Regions with higher entropy values ​​contain more information and are more important. Simultaneously, information entropy analysis is performed on the 128 channels, calculating the entropy value for each channel. The calculation method involves analyzing the pixel value distribution of each channel across the entire feature map space and calculating its information entropy. When obtaining the importance score based on the information distribution, a two-dimensional weighting strategy is used, combining the spatial dimension entropy value and the channel dimension entropy value. Specifically, for each spatial region and channel intersection, its importance score is calculated as a weighted sum of spatial entropy multiplied by 0.6 and channel entropy multiplied by 0.4, then normalized to a range of 0 to 1. This scoring method comprehensively considers the importance distribution of features in both spatial location and channel semantics, thus more comprehensively reflecting the distribution patterns of information in the feature map. The final spatial region importance scores ranged from 0.35 to 0.92, with an average of 0.67; the channel importance scores ranged from 0.28 to 0.85, with an average of 0.59.

[0144] Feature maps are divided into multiple importance levels based on importance scores. Importance thresholds of 0.7 and 0.4 are set, classifying feature maps into three levels: high importance (score ≥ 0.7), medium importance (0.4 ≤ score < 0.7), and low importance (score < 0.4). For the feature map of the third convolutional layer, 5 grids in the spatial region are considered high importance, 8 grids are medium importance, and 3 grids are low importance; in the channels, 42 are high importance, 65 are medium importance, and 21 are low importance. Different storage strategies are adopted for feature maps of different importance levels. High importance regions and channels are stored using lossless compression, maintaining 8-bit precision; medium importance regions and channels are quantized with low precision, reducing to 4-bit precision; low importance regions and channels are sparsified, retaining only the most significant 20% of elements and setting the rest to zero. Sparsification is achieved by sorting feature values, retaining the top 20% of elements with the largest absolute value, setting other elements to zero, and then using run-length encoding to compress and store the zero element position information. This differentiated storage strategy significantly reduces memory usage while maintaining the integrity of critical information.

[0145] In the joint optimization step, computational efficiency is weighted at 0.5, storage efficiency at 0.3, and communication efficiency at 0.2. The initial mapping scheme is evaluated, yielding computational efficiency of 0.82, storage efficiency of 0.75, and communication efficiency of 0.68, with a weighted combination value of 0.772. Computational efficiency is measured by the ratio of the execution time of each layer on the allocated computing units to the theoretical optimal execution time; storage efficiency is measured by the ratio of actual memory usage to the original memory requirement; and communication efficiency is measured by the ratio of inter-layer data transfer volume to the theoretical minimum transfer volume. Overall efficiency is improved by adjusting the mapping relationships and storage strategies. For example, the second convolutional layer is remapped from a general-purpose computing unit to a high-performance processing unit, while the fourth convolutional layer is remapped from a high-performance processing unit to a general-purpose computing unit; the storage strategy of the feature map in the third convolutional layer is adjusted, reducing the number of medium-importance channels from 65 to 50 and increasing the number of low-importance channels. The specific adjustment process involves iterating through the mapping combinations and storage configurations, calculating the efficiency index for each combination, and selecting the most efficient scheme. After optimization, the computational efficiency was improved to 0.87, the storage efficiency to 0.81, the communication efficiency to 0.73, and the weighted combination value to 0.823, meeting the performance requirements.

[0146] This invention captures the dynamic characteristics of hardware through time-weighted analysis, constructs a priority mapping by combining the network layer width sensitivity, and performs importance-driven storage optimization on the feature map. This achieves the optimal matching between computing resources and neural network layers, effectively balancing inference accuracy, computing efficiency and storage overhead, and provides an efficient solution for deploying neural networks on resource-constrained devices.

[0147] In one optional implementation, the step of constructing a resource allocation priority matrix based on the load dynamic characteristic value and the bit width sensitivity includes:

[0148] The computing characteristic matching degree is calculated based on the hardware characteristic parameters of the computing unit and the computing characteristic parameters of the network layer; the computing unit characteristic value is obtained by weighted combination of the load dynamic characteristic value and the computing characteristic matching degree; the network layer characteristic value is obtained by weighted combination of the bit width sensitivity and the resource utilization of the neural network layer; the affinity between the network layer and the computing unit is calculated based on the computing unit characteristic value and the network layer characteristic value, and the affinity is used to characterize the degree of matching between the network layer and the computing unit.

[0149] Construct predictive performance metrics, including computational latency, resource utilization, and energy efficiency; establish a mapping optimization objective function based on the affinity and the predictive performance metrics; obtain an initial mapping scheme by iteratively optimizing the mapping optimization objective function under the conditions of satisfying the resource capacity constraints of computing units and task latency constraints; collect real-time execution data, adjust the mapping scheme online based on the execution data, and determine the resource allocation priority matrix.

[0150] For example, when constructing a resource allocation priority matrix based on load dynamic feature values ​​and bit width sensitivity, the matching degree of computational characteristics is calculated based on the hardware characteristic parameters of the computing units and the computational characteristic parameters of the network layers. Taking a heterogeneous computing platform containing 7 computing units as an example, the hardware characteristic parameters of each computing unit are obtained, including processing power, memory bandwidth, and support for dedicated instruction sets. The high-performance processing unit has a processing power of 2.4GHz, a memory bandwidth of 12.8GB / s, and supports matrix multiplication acceleration instructions; the general-purpose computing unit has a processing power of 1.8GHz, a memory bandwidth of 6.4GB / s, and supports basic vector operation instructions; the low-power computing unit has a processing power of 1.2GHz, a memory bandwidth of 3.2GB / s, and supports integer operation instructions. Simultaneously, the computational characteristic parameters of the 10 layers of the neural network are obtained, including computational density, memory access mode, and operation type. The computational density of the second convolutional layer is 256 FLOPS / Byte, the memory access mode is high spatial locality, and the operation type is mainly floating-point matrix multiplication; the computational density of the third fully connected layer is 32 FLOPS / Byte, the memory access mode is low spatial locality, and the operation type is mainly vector addition.

[0151] Based on these parameters, the matching degree of computational characteristics is calculated using a feature vector similarity method. Hardware characteristic parameters and network layer computational characteristic parameters are normalized into feature vectors, and the similarity between the two vectors is calculated. Processing power corresponds to computational density, memory bandwidth corresponds to memory access mode, and instruction set support corresponds to operation type. During normalization, processing power in the range of 0.8GHz to 3GHz is mapped to a range of 0 to 1, and memory bandwidth in the range of 2GB / s to 16GB / s is mapped to a range of 0 to 1. Instruction set support is quantified into a numerical value of 0 to 1 by analyzing the matching degree between the instruction set and the operation type; for example, the support degree of matrix multiplication instructions for convolution operations is 0.9, and the support degree of fully connected operations is 0.8. Taking the high-performance processing unit and the second convolutional layer as an example, the matching degree of processing capability is 0.85, the matching degree of memory bandwidth is 0.78, and the matching degree of instruction set is 0.92. When calculating the overall matching degree, the weights of the three are set to 0.4, 0.3, and 0.3, respectively, resulting in an overall computational characteristic matching degree of 0.85×0.4+0.78×0.3+0.92×0.3=0.850. Similarly, the computational characteristic matching degree between all computing units and all network layers is calculated, constructing a 7×10 matching degree matrix.

[0152] The load dynamic feature value weight is set to 0.45, and the computational characteristic matching degree weight is set to 0.55. Taking the high-performance processing unit as an example, its load dynamic feature value is 0.704, and its computational characteristic matching degree with the second convolutional layer is 0.850. The calculated feature value of the computational unit is 0.704×0.45+0.850×0.55=0.783. Different feature values ​​of the computational unit are calculated for the same computational unit combined with different network layers. The network layer feature value is obtained by weighting the bit width sensitivity and the resource utilization of the neural network layer. Taking the second convolutional layer as an example, its bit width sensitivity is 0.125, and its resource utilization is 0.72. The bit width sensitivity weight is set to 0.4, and the resource utilization weight is set to 0.6. The calculated feature value of the network layer is 0.125×0.4+0.72×0.6=0.482. Resource utilization is obtained by analyzing the execution of the network layer on a standard test platform. Specifically, the layer is run 100 times, and the average CPU / GPU utilization, memory usage, and cache hit rate of the computing unit are recorded during the execution process. The resource utilization efficiency index is obtained by combining these data to indicate the degree of effective utilization of computing resources by the layer during the execution process.

[0153] Affinity is calculated using a weighted product method, with the feature value of the computational unit set to a weight of 0.65 and the feature value of the network layer set to a weight of 0.35. Taking the high-performance processing unit and the second convolutional layer as an example, the feature value of the computational unit is 0.783, and the feature value of the network layer is 0.482. The affinity is calculated as 0.783^0.65 × 0.482^0.35 = 0.650. Specifically, the affinity value is obtained by multiplying the 0.65 power of the feature value of the computational unit by the 0.35 power of the feature value of the network layer. This calculation method can balance the influence of the two feature values ​​and maintain stable results across different orders of magnitude. A higher affinity value indicates a better match between the network layer and the computational unit. The affinity between all network layers and all computational units is calculated, constructing a 10×7 affinity matrix.

[0154] Predicted performance metrics include computation latency, resource utilization, and energy efficiency. Computation latency is obtained by estimating the execution time of a network layer on a specific computing unit, considering computational load, memory access overhead, and the processing power of the computing unit. Specifically, the estimation method divides the theoretical computational load (floating-point operations) of the network layer by the processing speed of the computing unit (floating-point operations per second), then considers the impact of memory access latency and data transfer latency, and predicts the actual execution time using a linear regression model. Taking the execution of the second convolutional layer on a high-performance processing unit as an example, its computational load is 3.6 GFLOPS. Considering the theoretical performance of the high-performance processing unit (2.8 TFLOPS) and its actual efficiency of 80%, plus memory access overhead, the estimated latency is 1.25 ms. Resource utilization is obtained by estimating the proportion of computing unit resources used during network layer execution, including computational resource utilization and memory resource utilization. Taking the execution of the second convolutional layer on a high-performance processing unit as an example, the computational resource utilization rate is 82%, the memory resource utilization rate is 68%, and the overall resource utilization rate, calculated with weights of 0.6 and 0.4, is 82% × 0.6 + 68% × 0.4 = 76.6%. Energy efficiency is obtained by estimating the computational throughput per unit of energy consumption, expressed as the number of operations processed per watt per second. Taking the execution of the second convolutional layer on a high-performance processing unit as an example, its power consumption is 6.2W, its computational throughput is 23.6 GOPS, and its energy efficiency level is 3.8 GOPS / W.

[0155] The mapping optimization objective function adopts a multi-objective weighted combination form, setting the affinity weight to 0.4, the computation delay weight to 0.3, the resource utilization weight to 0.2, and the energy efficiency weight to 0.1. For each mapping combination of network layer and computation unit, its objective function value is calculated. To make different indicators comparable, each indicator needs to be normalized first. The computation delay normalization method is the ratio of the inverse of the delay to the inverse of the maximum delay; resource utilization is already a normalized value; and energy efficiency is normalized by the ratio to the maximum energy efficiency level. Taking the high-performance processing unit and the second convolutional layer as an example, the affinity is 0.650, the normalized computation delay value is 0.85 (the shorter the delay, the higher the normalized value), the resource utilization is 0.766, the normalized energy efficiency value is 0.76, and the objective function value is 0.650×0.4+0.85×0.3+0.766×0.2+0.76×0.1=0.751. A higher objective function value indicates a better mapping combination.

[0156] Under the constraints of computing unit resource capacity and task latency, an initial mapping scheme is obtained by iteratively optimizing the objective function of the mapping. Resource capacity constraints include the upper limit of the computing unit's processing power and memory capacity; task latency constraints include the end-to-end processing latency of the entire network not exceeding a preset threshold of 20ms. A genetic algorithm is used to solve this optimization problem, with a population size of 100, 500 iterations, a crossover probability of 0.8, and a mutation probability of 0.1. The algorithm first randomly generates 100 initial mapping schemes that satisfy the constraints, each scheme being a feasible allocation scheme from a network layer to a computing unit. Then, in each iteration, the fitness of each scheme is calculated based on the objective function value; schemes with higher fitness have a greater probability of being selected. The selected schemes generate new schemes through a crossover operation, which randomly selects two parent schemes and swaps some mapping relationships to generate offspring schemes. Simultaneously, a mutation operation is performed on the schemes with a probability of 0.1, randomly changing the mapping objective of a certain network layer. In each iteration, the individual with the highest objective function value that satisfies the constraints is selected as a candidate solution. After 500 iterations, the optimal mapping scheme was obtained: the 1st and 4th convolutional layers and the 4th fully connected layer are mapped to high-performance processing units; the 2nd and 5th convolutional layers and the 1st fully connected layer are mapped to the first general-purpose computing unit; the 3rd fully connected layer is mapped to the second general-purpose computing unit; and the 3rd and 6th convolutional layers and the 2nd fully connected layer are mapped to four low-power computing units. The average objective function of this scheme is 0.782, and the end-to-end processing latency is 16.8ms, satisfying the constraints.

[0157] Actual execution data is collected every 15 seconds, including the actual load of each computing unit, the actual execution time of each network layer, and energy consumption data. Data collection is achieved through a performance monitoring module embedded in the system. The monitoring module records the start and end times of each network layer and calculates the actual execution latency; it also records the resource utilization of each computing unit, including computing resource utilization, memory utilization, and power consumption data. When a significant difference is found between the actual execution performance and the predicted performance, the mapping scheme is adjusted. A difference threshold of 15% is set. If the actual execution time exceeds 1.15 times the predicted time, or the actual resource utilization is less than 0.85 times the predicted utilization, a significant difference is considered to exist. The adjustment method is to recalculate the affinity and objective function and select the locally optimal adjustment scheme. To avoid fluctuations caused by frequent adjustments, a minimum adjustment interval of 30 seconds is set, meaning there must be at least a 30-second interval between two adjustments. For example, after running for 30 seconds, it was found that the actual execution time of the second convolutional layer on the first general computing unit was 2.1ms, which was 1.17 times the predicted time of 1.8ms. This triggered an adjustment, remapping the second convolutional layer to the high-performance processing unit, and simultaneously mapping the fourth convolutional layer, which was originally on the high-performance processing unit, to the second general computing unit.

[0158] Based on the adjusted mapping scheme, a resource allocation priority matrix is ​​determined. The priority matrix has a dimension of 10×7, with rows representing network layers and columns representing computational units. The element values ​​in the matrix represent the mapping priority between the corresponding network layer and computational unit, ranging from [0,1], with larger values ​​indicating higher priority. The priority matrix is ​​constructed by first setting basic priorities based on the final determined mapping relationships: for final determined mapping relationships, the corresponding element value is set to 1; for alternative mapping relationships, values ​​between 0.5 and 0.9 are set based on their objective function values; and for unsuitable mapping relationships, values ​​between 0 and 0.4 are set. Then, historical execution performance is considered. If a network layer has a good historical execution record with a certain computational unit, its priority is appropriately increased; if it has a poor historical execution record, its priority is appropriately decreased. For example, the priority of the second convolutional layer with the high-performance processing unit is set to 1, the priority with the first general-purpose computational unit is set to 0.8, and the priority with other computational units is set to 0.3. This priority matrix not only reflects the current optimal mapping scheme but also includes information on alternative schemes, providing flexibility for dynamic adjustment.

[0159] This invention achieves precise matching between neural networks and heterogeneous computing resources by comprehensively considering the matching relationship between hardware characteristics and network computing characteristics, and combining load dynamic characteristics and bit width sensitivity. At the same time, it supports dynamic adjustment based on real-time execution data, effectively improving overall execution efficiency and resource utilization, and providing an adaptive optimization strategy for neural network deployment in resource-constrained scenarios.

[0160] A second aspect of the present invention provides a quantization compression and adaptive computing power optimization system for neural networks, comprising:

[0161] The first unit is used to obtain a pre-trained neural network, which includes multiple convolutional layers and fully connected layers;

[0162] The second unit is used to calculate the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, construct an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy, the importance evaluation index characterizing the information flow and inter-layer dependency of neurons; perform quantization tests with different bit widths on the convolutional layers and fully connected layers to obtain quantization error data; weight and combine the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determine the bit width configuration scheme based on the inter-layer error propagation evaluation matrix;

[0163] The third unit is used to acquire the computing load data of the target device; and to perform differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme.

[0164] The fourth unit is used to adjust the importance evaluation index according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, and generate the optimized neural network.

[0165] A third aspect of the present invention provides an electronic device, comprising:

[0166] processor;

[0167] Memory used to store processor-executable instructions;

[0168] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0169] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0170] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for quantization compression and adaptive optimization of computing power in neural networks, characterized in that, include: Obtain a pre-trained neural network, which contains multiple convolutional layers and fully connected layers; For each convolutional layer and fully connected layer in the neural network, the weight distribution entropy value and the activation value distribution entropy value are calculated. Based on the weight distribution entropy value and the activation value distribution entropy value, an importance evaluation index for neurons is constructed. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization error data is obtained by performing quantization tests with different bit widths on the convolutional layer and the fully connected layer. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix, and the bit width configuration scheme is determined based on the inter-layer error propagation evaluation matrix. Obtain the computing load data of the target device; Based on the computational load data and the bit width configuration scheme, the network layer performs differentiated allocation of computational unit mapping and storage strategies; The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, thereby generating an optimized neural network.

2. The method according to claim 1, characterized in that, The steps of calculating the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, and constructing an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy, include: Obtain the weight data and activation value data of each convolutional layer and fully connected layer in the neural network, discretize them to obtain the weight probability density distribution and activation value probability density distribution, and calculate the corresponding entropy value; Similarity analysis is performed on the input and output features of neurons to identify neuron groups with similar response patterns; the distribution difference within each neuron group and the distribution difference between groups are calculated, and the contribution ratio of each neuron to the network function is determined based on the difference ratio; based on the contribution ratio, a corresponding first balance factor and a second balance factor are assigned to each neuron group; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain the hierarchical information flow rate; Calculate the mutual information between adjacent layers in the neural network; The importance assessment index of a neuron is obtained by weighting and combining the hierarchical information flow with the mutual information; the importance assessment index is verified in multiple batches based on the network output error, and the final importance assessment index is determined when the index fluctuation is less than a preset fluctuation threshold.

3. The method according to claim 2, characterized in that, The steps for calculating the mutual information between adjacent layers in the neural network include: Multi-scale decomposition of feature data from adjacent layers in a neural network yields feature representations at multiple scale levels. Feature probability distributions are calculated for each scale level feature representation. Based on the feature probability distributions at each scale level, the information entropy of the previous layer and the information entropy of the next layer are calculated respectively, and the combination yields mutual information at multiple scales. The temporal feature sequences of adjacent layers are obtained, the feature probability distribution is temporally smoothed, and the temporal mutual information is calculated; the conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated; the multiple scale mutual information, temporal mutual information, and conditional mutual information are weighted and combined to obtain the initial mutual information. A causal graph of inter-layer feature transmission is constructed to identify key transmission paths and analyze the causal effect strength of different transmission channels. Based on the causal effect strength and initial mutual information, a loss matrix for feature transmission is constructed, which characterizes the degree of information retention of features during transmission. The connection weights of the corresponding channels are adjusted according to the loss values ​​of each transmission channel in the loss matrix, and the loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold, thereby obtaining the mutual information between adjacent layers.

4. The method according to claim 1, characterized in that, The steps for obtaining quantization error data by performing quantization tests with different bit widths on the convolutional layer and the fully connected layer include: Feature importance analysis is performed on the parameters of the convolutional layer and the fully connected layer, and the parameter space is divided into critical parameter domains and non-critical parameter domains; Quantization processing is performed in the critical parameter field using a first quantization bit width configuration, and in the non-critical parameter field using a second quantization bit width configuration smaller than the first quantization bit width configuration, to obtain adaptive quantization parameters. The adaptive quantization parameters and the original floating-point parameters are used to calculate the error, and the parameter domain quantization error of different parameter domains is obtained. The influence of the quantization error of different parameter domains on the inter-layer propagation is analyzed, and the quantization sensitivity of each layer is calculated. The parameter domain quantization error is weighted according to the quantization sensitivity to obtain the quantization error data.

5. The method according to claim 1, characterized in that, The steps of weighting and combining the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determining the bit width configuration scheme based on the inter-layer error propagation evaluation matrix include: The local features between adjacent layers, the mid-range features between consecutive multi-layers, and the global features of the entire network are calculated respectively. The local features are obtained by weighted combination of quantization error data and importance evaluation index of adjacent layers. The mid-range features are obtained by weighted combination of local features of each layer within a feature block. The global features are obtained by weighted combination of mid-range features of each layer. Construct an inter-layer error propagation evaluation matrix based on local features, mid-range features, and global features; The error compensation value for each layer is calculated based on the inter-layer error propagation evaluation matrix. The error compensation value is determined by the weighted sum of the quantization error of the current layer and the propagation error of the previous layer. The bit width configuration scheme is obtained by adaptively adjusting the reference bit width of each layer based on the error compensation value. Calculate the network performance evaluation value, update the feature combination weight coefficients in the construction process of the inter-layer error propagation evaluation matrix based on the network performance evaluation value, and reconstruct the inter-layer error propagation evaluation matrix until the network performance evaluation value meets the preset optimization objective.

6. The method according to claim 1, characterized in that, The steps for performing differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme include: The computational load data is combined using a time-series weighted method to obtain dynamic load characteristic values; the bit-width sensitivity is obtained by calculating the rate of change of resource requirements for each layer of the neural network based on the bit-width configuration scheme. A resource allocation priority matrix is ​​constructed based on the load dynamic characteristic value and the bit width sensitivity. Each element of the resource allocation priority matrix is ​​obtained by weighted combination of the bit width sensitivity of the corresponding network layer and the load dynamic characteristic value of the corresponding computing unit, which is used to determine the mapping relationship between the network layer and the computing unit. Importance analysis is performed on intermediate feature maps of the neural network, and an importance score is obtained based on the information distribution of the feature maps in the spatial and channel dimensions. Based on the importance score, the feature maps are divided into multiple importance levels, and a differentiated storage strategy is adopted for feature maps of different importance levels. The mapping relationship and the storage strategy are jointly optimized based on a weighted combination of computational efficiency, storage efficiency, and communication efficiency.

7. The method according to claim 6, characterized in that, The steps for constructing a resource allocation priority matrix based on the load dynamic characteristic value and the bit width sensitivity include: The computing characteristic matching degree is calculated based on the hardware characteristic parameters of the computing unit and the computing characteristic parameters of the network layer; the computing unit characteristic value is obtained by weighted combination of the load dynamic characteristic value and the computing characteristic matching degree; the network layer characteristic value is obtained by weighted combination of the bit width sensitivity and the resource utilization of the neural network layer; the affinity between the network layer and the computing unit is calculated based on the computing unit characteristic value and the network layer characteristic value, and the affinity is used to characterize the degree of matching between the network layer and the computing unit. Construct predictive performance metrics, including computational latency, resource utilization, and energy efficiency; establish a mapping optimization objective function based on the affinity and the predictive performance metrics; obtain an initial mapping scheme by iteratively optimizing the mapping optimization objective function under the conditions of satisfying the resource capacity constraints of computing units and task latency constraints; collect real-time execution data, adjust the mapping scheme online based on the execution data, and determine the resource allocation priority matrix.

8. A quantization compression and adaptive optimization system for neural networks, used to implement the method of any one of claims 1-7, characterized in that, include: The first unit is used to obtain a pre-trained neural network, which includes multiple convolutional layers and fully connected layers; The second unit is used to calculate the weight distribution entropy value and the activation value distribution entropy value for each convolutional layer and fully connected layer in the neural network, and to construct an importance evaluation index for neurons based on the weight distribution entropy value and the activation value distribution entropy value. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization error data is obtained by performing quantization tests with different bit widths on the convolutional layer and the fully connected layer. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix, and the bit width configuration scheme is determined based on the inter-layer error propagation evaluation matrix. The third unit is used to acquire the computing load data of the target device; Based on the computational load data and the bit width configuration scheme, the network layer performs differentiated allocation of computational unit mapping and storage strategies; The fourth unit is used to adjust the importance evaluation index according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, and generate the optimized neural network.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images