Quantization compression and computing power adaptive optimization method and system of neural network

Abstract

The application provides a neural network quantization compression and computing power adaptive optimization method and system, relates to the technical field of neural networks, and comprises the following steps: calculating weight distribution entropy and activation value distribution entropy, constructing a neuron importance evaluation index, generating an interlayer error propagation evaluation matrix to determine a bit width configuration scheme in combination with quantization error data, and performing differentiated resource allocation based on target device computing load, so that efficient compression and deployment of the neural network under the premise of ensuring performance are finally realized, and the consumption of computing resources is effectively reduced and the inference speed is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to neural network technology, in particular to a neural network quantization compression and computing power adaptive optimization method and system. BACKGROUND

[0002] With the wide application of deep learning technology, neural network models have achieved remarkable results in image recognition, natural language processing, speech recognition and other fields. However, these high-precision neural network models usually have a large parameter size and complex structure, which leads to huge computing resource and storage space challenges for model deployment and real-time inference, especially on resource-constrained edge devices. In order to solve this problem, neural network quantization compression technology has gradually become a research hotspot. This technology reduces model size and computational complexity by reducing network parameters and computational precision, while maintaining model performance as much as possible.

[0003] Existing neural network quantization compression techniques mainly use fixed bit-width quantization strategies, applying a unified quantization standard to the entire network, which cannot be differentiated according to the characteristics of different layers, resulting in a large loss of quantization accuracy. Existing quantization methods lack accurate evaluation mechanisms for neuron importance, and cannot identify and protect neurons and layers that have a key impact on the final result in the network, so important information may be lost during compression. Existing quantization methods rarely consider the actual computing power and load status of target hardware devices, and cannot dynamically adjust the quantization strategy according to the resource constraints of different devices, resulting in compressed models that are difficult to achieve the best performance-efficiency balance in various deployment environments.

[0004] In practical applications, due to the lack of adaptive optimization mechanisms, existing technologies are difficult to achieve a good balance between model accuracy and computational efficiency, especially in scenarios where computing resources change dynamically, making it difficult for compressed models to achieve optimal utilization of computing power. Therefore, developing an adaptive quantization compression method that takes into account neuron importance, inter-layer error propagation, and target device computing characteristics is of great significance for improving the deployment efficiency of neural networks in resource-constrained environments. SUMMARY

[0005] The neural network quantization compression and computing power adaptive optimization method and system provided by the embodiments of the present application can solve the problems in the prior art.

[0006] In a first aspect, the present application provides a neural network quantization compression and computing power adaptive optimization method, comprising:

[0007] obtaining a pre-trained neural network, the neural network comprising a plurality of convolutional layers and fully connected layers;

[0008] For each convolutional layer and fully connected layer in the neural network, the weight distribution entropy value and the activation value distribution entropy value are calculated. Based on the weight distribution entropy value and the activation value distribution entropy value, an importance evaluation index for neurons is constructed. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization tests with different bit widths are performed on the convolutional layers and fully connected layers to obtain quantization error data. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix. Based on the inter-layer error propagation evaluation matrix, a bit width configuration scheme is determined.

[0009] Obtain the computing load data of the target device; based on the computing load data and the bit width configuration scheme, perform differentiated allocation of computing unit mapping and storage strategy for the network layer;

[0010] The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, thereby generating an optimized neural network.

[0011] In one optional implementation, the step of calculating the weight distribution entropy and activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, and constructing an importance evaluation index for neurons based on the weight distribution entropy and activation value distribution entropy includes:

[0012] Obtain the weight data and activation value data of each convolutional layer and fully connected layer in the neural network, discretize them to obtain the weight probability density distribution and activation value probability density distribution, and calculate the corresponding entropy value;

[0013] Similarity analysis is performed on the input and output features of neurons to identify neuron groups with similar response patterns; the distribution difference within each neuron group and the distribution difference between groups are calculated, and the contribution ratio of each neuron to the network function is determined based on the magnitude of the difference; based on the contribution ratio, a corresponding first balance factor and a second balance factor are assigned to each neuron group; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain the hierarchical information flow rate;

[0014] Calculate the mutual information between adjacent layers in the neural network;

[0015] The importance assessment index of a neuron is obtained by weighting and combining the hierarchical information flow with the mutual information; the importance assessment index is verified in multiple batches based on the network output error, and the final importance assessment index is determined when the index fluctuation is less than a preset fluctuation threshold.

[0016] In one alternative implementation, the step of calculating the mutual information between adjacent layers in the neural network includes:

[0017] performing multi-scale decomposition on the feature data of adjacent layers in the neural network to obtain feature representations of multiple scale levels, calculating a feature probability distribution for each scale level of the feature representations; calculating an information entropy of a previous layer and an information entropy of a next layer based on the feature probability distribution of each scale level, and combining to obtain multiple scale mutual information;

[0018] obtaining a time sequence feature sequence of adjacent layers, performing time sequence smoothing processing on the feature probability distribution, and calculating time sequence mutual information; obtaining conditional feature information of adjacent layers, and calculating conditional mutual information; and weighting and combining the multiple scale mutual information, the time sequence mutual information, and the conditional mutual information to obtain initial mutual information;

[0019] constructing a causal graph of inter-layer feature transmission, identifying a key transmission path, and analyzing a causal effect strength of different transmission channels; constructing a loss matrix of feature transmission based on the causal effect strength and the initial mutual information, the loss matrix representing an information retention degree of features in a transmission process;

[0020] adjusting a connection weight of a corresponding channel according to a loss value of each transmission channel in the loss matrix, and iteratively updating the loss matrix until a fluctuation value thereof is less than a preset threshold value, to obtain mutual information between adjacent layers.

[0021] In an optional implementation, the step of performing quantization tests of different bit widths on the convolutional layer and the fully connected layer to obtain quantization error data includes:

[0022] performing feature importance analysis on parameters of the convolutional layer and the fully connected layer, and dividing a parameter space into a key parameter domain and a non-key parameter domain;

[0023] performing quantization processing on the key parameter domain using a first quantization bit width configuration, and performing quantization processing on the non-key parameter domain using a second quantization bit width configuration smaller than the first quantization bit width configuration, to obtain adaptive quantization parameters;

[0024] performing error calculation on the adaptive quantization parameters and original floating-point parameters to obtain parameter domain quantization errors of different parameter domains; analyzing an influence degree of the parameter domain quantization errors in inter-layer transmission, calculating quantization sensitivities of layers, and weighting the parameter domain quantization errors according to the quantization sensitivities to obtain quantization error data.

[0025] In an optional implementation, weighting and combining the quantization error data and the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determining a bit width configuration scheme based on the inter-layer error propagation evaluation matrix includes:

[0026] local features between adjacent layers in the neural network, mid-range features between multiple consecutive layers, and global features of the whole network are calculated respectively, the local features are obtained by weighted combination of quantization error data and importance evaluation indexes of adjacent layers, the mid-range features are obtained by weighted combination of local features of layers within a feature block, and the global features are obtained by weighted combination of mid-range features of layers;

[0027] An inter-layer error propagation evaluation matrix is constructed based on the local features, the mid-range features, and the global features.

[0028] An error compensation value of each layer is calculated according to the inter-layer error propagation evaluation matrix, the error compensation value is determined by a weighted sum of quantization error of a current layer and propagated error of a previous layer, and a bit width configuration scheme is obtained by adaptively adjusting a reference bit width of each layer based on the error compensation value.

[0029] A network performance evaluation value is calculated, feature combination weight coefficients in the inter-layer error propagation evaluation matrix construction process are updated and the inter-layer error propagation evaluation matrix is reconstructed according to the network performance evaluation value, until the network performance evaluation value meets a preset optimization target.

[0030] In an optional implementation, the step of performing computation unit mapping and differentiated allocation of storage strategies for network layers based on the computation load data and the bit width configuration scheme comprises:

[0031] A load dynamic feature value is obtained by time series weighted combination of the computation load data, and a bit width sensitivity is obtained by calculating a resource demand change rate of each layer of the neural network based on the bit width configuration scheme.

[0032] A resource allocation priority matrix is constructed according to the load dynamic feature value and the bit width sensitivity, each matrix element of the resource allocation priority matrix is obtained by weighted combination of the bit width sensitivity of a corresponding network layer and the load dynamic feature value of a corresponding computation unit, and is used to determine a mapping relationship between the network layer and the computation unit.

[0033] An importance analysis is performed on intermediate feature maps in the neural network, an importance score is obtained according to information distribution of the feature maps in spatial dimensions and channel dimensions, and the feature maps are divided into multiple importance levels based on the importance score, and differentiated storage strategies are adopted for feature maps of different importance levels.

[0034] The mapping relationship and the storage strategies are jointly optimized according to a weighted combination value of computation efficiency, storage efficiency, and communication efficiency.

[0035] In an optional implementation, the step of constructing a resource allocation priority matrix according to the load dynamic feature value and the bit width sensitivity comprises:

[0036] The calculation characteristic matching degree is calculated based on the hardware characteristic parameter of the calculation unit and the calculation characteristic parameter of the network layer;The calculation unit characteristic value is obtained by weighting and combining the load dynamic characteristic value and the calculation characteristic matching degree, and the network layer characteristic value is obtained by weighting and combining the bit width sensitivity and the resource utilization of the neural network layer;The affinity between the network layer and the calculation unit is calculated based on the calculation unit characteristic value and the network layer characteristic value, and the affinity is used to represent the matching degree of the network layer and the calculation unit;

[0037] A prediction performance index is constructed, the prediction performance index including calculation delay, resource utilization and energy efficiency level;A mapping optimization objective function is established based on the affinity and the prediction performance index;An initial mapping scheme is obtained by iteratively optimizing the mapping optimization objective function under the condition of meeting the calculation unit resource capacity constraint and the task time delay constraint;Real-time execution data is collected, and the mapping scheme is adjusted online according to the execution data to determine the resource allocation priority matrix.

[0038] In a second aspect of the embodiment of the present application, a neural network quantization compression and computing power adaptive optimization system is provided, comprising:

[0039] A first unit is configured to obtain a pre-trained neural network, wherein the neural network comprises a plurality of convolutional layers and fully connected layers;

[0040] A second unit is configured to calculate a weight distribution entropy value and an activation value distribution entropy value for each convolutional layer and fully connected layer in the neural network, construct an importance evaluation index of neurons based on the weight distribution entropy value and the activation value distribution entropy value, wherein the importance evaluation index represents the information flow and the inter-layer dependence degree of the neurons;The convolutional layers and the fully connected layers are subjected to quantization tests of different bit widths to obtain quantization error data;The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix, and a bit width configuration scheme is determined based on the inter-layer error propagation evaluation matrix;

[0041] A third unit is configured to obtain calculation load data of a target device;The network layer is subjected to calculation unit mapping and differential allocation of storage strategies based on the calculation load data and the bit width configuration scheme;

[0042] A fourth unit is configured to adjust the importance evaluation index according to the performance parameter of the neural network until the performance parameter meets a preset optimization target, and generate an optimized neural network.

[0043] In a third aspect of the embodiment of the present application, an electronic device is provided, comprising:

[0044] A processor;

[0045] A memory for storing processor-executable instructions.

[0046] The processor is configured to invoke instructions stored in the memory to perform the method described above.

[0047] In a fourth aspect, the application provides a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the method described above.

[0048] The neural network quantization compression and computing power adaptive optimization method provided by the application constructs an importance evaluation index of neurons by calculating weight distribution entropy and activation value distribution entropy, generates an inter-layer error propagation evaluation matrix in combination with quantization error data, realizes accurate configuration of different bit widths of each layer of the neural network, effectively reduces model storage space and computing resource consumption, and maintains the inference accuracy of the model.

[0049] The application performs computing unit mapping and differential allocation of storage strategies based on the computing load data of the target device, enhances the adaptability and execution efficiency of the model on different hardware platforms, enables the model to adaptively adjust according to the computing power conditions of the actual deployment environment, and improves resource utilization.

[0050] The application also continuously adjusts network parameters until the preset optimization target is met by iteratively optimizing the importance evaluation index, forms a closed-loop optimization mechanism, balances the performance and resource consumption of the neural network while maintaining the functional integrity, and is suitable for resource-constrained edge computing scenarios and mobile device applications. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The figure is a flowchart of the neural network quantization compression and computing power adaptive optimization method of the embodiment of the application.

[0052] Figure 2 The figure is a flowchart of parameter domain adaptive quantization and error propagation analysis. DETAILED DESCRIPTION

[0053] To make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme of the embodiments of the application will be described in detail below with reference to the drawings of the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0054] The technical scheme of the application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in some embodiments.

[0055] Figure 1 A flowchart of a quantization compression and computing power adaptive optimization method for an embodiment of the neural network is shown in FIG. 1. As shown in FIG. 1, the method comprises the following steps. Figure 1

[0056] obtaining a pre-trained neural network, the neural network comprising a plurality of convolutional layers and fully connected layers;

[0057] calculating a weight distribution entropy value and an activation value distribution entropy value for each convolutional layer and fully connected layer in the neural network, constructing an importance evaluation index of neurons based on the weight distribution entropy value and the activation value distribution entropy value, the importance evaluation index representing information flow and inter-layer dependence of the neurons, performing quantization tests of different bit widths on the convolutional layers and fully connected layers to obtain quantization error data, combining the quantization error data and the importance evaluation index by weighting to generate an inter-layer error propagation evaluation matrix, and determining a bit width configuration scheme based on the inter-layer error propagation evaluation matrix;

[0058] obtaining computing load data of a target device, and performing differential allocation of computing unit mapping and storage strategies based on the computing load data and the bit width configuration scheme;

[0059] adjusting the importance evaluation index according to performance parameters of the neural network until the performance parameters meet a preset optimization target, and generating an optimized neural network.

[0060] For example, a target detection neural network is taken as an example, which is composed of 6 convolutional layers and 4 fully connected layers. The first convolutional layer has an input channel number of 3, an output channel number of 64, a convolution kernel size of 3x3, and a step size of 1; the second convolutional layer has an input channel number of 64, an output channel number of 128, a convolution kernel size of 3x3, and a step size of 2; the third convolutional layer has an input channel number of 128, an output channel number of 256, a convolution kernel size of 3x3, and a step size of 1; the fourth convolutional layer has an input channel number of 256, an output channel number of 256, a convolution kernel size of 3x3, and a step size of 2; the fifth convolutional layer has an input channel number of 256, an output channel number of 512, a convolution kernel size of 3x3, and a step size of 1; and the sixth convolutional layer has an input channel number of 512, an output channel number of 512, a convolution kernel size of 3x3, and a step size of 2. The first fully connected layer has an input dimension of 25088 and an output dimension of 4096; the second fully connected layer has an input dimension of 4096 and an output dimension of 4096; the third fully connected layer has an input dimension of 4096 and an output dimension of 1000; and the fourth fully connected layer has an input dimension of 1000 and an output dimension of 5. The network is pre-trained on a standard target detection data set, has an accuracy of 92.8%, and a model size of 552 MB.

[0061] ​The weight distribution entropy value is calculated by analyzing the statistical distribution characteristics of the parameters of each layer. Taking the second convolutional layer as an example, the layer includes 73728 parameters, the parameter value range is divided into 256 uniform intervals, the number of parameters in each interval is counted, and a parameter distribution histogram is obtained. The proportion of the number of parameters in each interval in the total number of parameters in the statistical histogram is obtained to obtain the probability distribution. The information entropy of the probability distribution is calculated to obtain the weight distribution entropy value of 5.86. The higher the weight distribution entropy value, the more dispersed the weight distribution, and the greater the information quantity. The weight distribution entropy values of all layers are calculated to obtain the entropy value sequence: 5.32 for the first convolutional layer, 5.86 for the second convolutional layer, 6.12 for the third convolutional layer, 5.74 for the fourth convolutional layer, 6.35 for the fifth convolutional layer, 5.92 for the sixth convolutional layer, 6.48 for the first fully connected layer, 6.21 for the second fully connected layer, 5.89 for the third fully connected layer, and 4.76 for the fourth fully connected layer.

[0062] The activation value distribution entropy value is calculated by analyzing the statistical distribution characteristics of the output feature maps of each layer. Taking the second convolutional layer as an example, 1000 images are used as input, and the output feature maps of the layer are recorded, with a feature map size of 56x56x128. The activation value range in the feature map is divided into 256 uniform intervals, the number of activation values in each interval is counted, and an activation value distribution histogram is obtained. The proportion of the number of activation values in each interval in the total number of activation values in the statistical histogram is obtained to obtain the probability distribution. The information entropy of the probability distribution is calculated to obtain the activation value distribution entropy value of 4.92. The higher the activation value distribution entropy value, the more dispersed the activation value distribution, and the greater the information quantity. The activation value distribution entropy values of all layers are calculated to obtain the entropy value sequence: 4.53 for the first convolutional layer, 4.92 for the second convolutional layer, 5.31 for the third convolutional layer, 4.85 for the fourth convolutional layer, 5.42 for the fifth convolutional layer, 5.08 for the sixth convolutional layer, 5.63 for the first fully connected layer, 5.36 for the second fully connected layer, 4.95 for the third fully connected layer, and 3.82 for the fourth fully connected layer.

[0063] The importance evaluation index is obtained by weighted combination of the weight distribution entropy value and the activation value distribution entropy value, and represents the information flow and the inter-layer dependence degree of the neuron. The weight of the weight distribution entropy value is set to 0.6, and the weight of the activation value distribution entropy value is set to 0.4. Taking the second convolution layer as an example, the importance evaluation index is calculated as 5.86*0.6+4.92*0.4=5.48. The importance evaluation index of all layers is calculated to obtain the index sequence: 5.00 for the first convolution layer, 5.48 for the second convolution layer, 5.80 for the third convolution layer, 5.38 for the fourth convolution layer, 5.98 for the fifth convolution layer, 5.59 for the sixth convolution layer, 6.13 for the first full connection layer, 5.87 for the second full connection layer, 5.52 for the third full connection layer, and 4.38 for the fourth full connection layer. The higher the importance evaluation index, the greater the importance of the layer in the network, and the more significant the influence on the overall performance of the network.

[0064] The quantization error data is obtained by performing quantization tests with different bit widths on the convolution layers and the full connection layers. The quantization tests with 8-bit, 7-bit, 6-bit, 5-bit and 4-bit are performed on each layer. Taking the second convolution layer as an example, 1000 images are used as input, and the layer is quantized with different bit widths, the differences between the output feature maps before and after quantization are compared, and the root mean square error is calculated. The root mean square error of 8-bit quantization is 0.0042, the root mean square error of 7-bit quantization is 0.0083, the root mean square error of 6-bit quantization is 0.0165, the root mean square error of 5-bit quantization is 0.0312, and the root mean square error of 4-bit quantization is 0.0596. Similar tests are performed on all layers to obtain a quantization error data matrix. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix. The weight of the quantization error data is set to 0.7, and the weight of the importance evaluation index is set to 0.3. Taking the 7-bit quantization of the second convolution layer as an example, the error propagation evaluation value is calculated as 0.0083*0.7+5.48*0.3 / 10=0.170. The error propagation evaluation values of all bit width configurations of all layers are calculated to construct the inter-layer error propagation evaluation matrix.

[0065] The error propagation evaluation threshold is set to 0.25, and for layer-bit width combinations with evaluation values lower than the threshold, the bit width configuration is considered feasible; for layer-bit width combinations with evaluation values higher than the threshold, the bit width configuration is considered to cause greater precision loss. From high bit width to low bit width, the lowest bit width that meets the threshold condition is selected as the bit width configuration of the layer. At the same time, considering the overall model size constraint, the compression ratio target is set to 25% of the original model size. According to the above principles, the bit width configuration scheme is determined: the first convolution layer bit width is 8 bits, the second convolution layer bit width is 7 bits, the third convolution layer bit width is 6 bits, the fourth convolution layer bit width is 8 bits, the fifth convolution layer bit width is 7 bits, the sixth convolution layer bit width is 6 bits, the first full connection layer bit width is 7 bits, the second full connection layer bit width is 6 bits, the third full connection layer bit width is 7 bits, and the fourth full connection layer bit width is 8 bits. Under the bit width configuration scheme, the model size is 142MB, which is 25.7% of the original model size, the average accuracy decreases by 2.3% and reaches 90.5%.

[0066] The target device is an embedded computing platform, which includes 1 high-performance processing unit, 2 general-purpose computing units, and 4 low-power computing units. Load data of each computing unit when executing different tasks is collected through a monitoring tool, including computing utilization, memory occupancy, and power consumption data. For the high-performance processing unit, the average value of computing utilization is 76%, the average value of memory occupancy is 62%, and the average value of power consumption is 5.8W; for the general-purpose computing unit, the average value of computing utilization is 68%, the average value of memory occupancy is 54%, and the average value of power consumption is 3.2W; for the low-power computing unit, the average value of computing utilization is 58%, the average value of memory occupancy is 45%, and the average value of power consumption is 1.5W. Based on the computing load data and the bit width configuration scheme, the network layer is mapped to the computing unit and the storage strategy is differentially allocated. According to the computing characteristics and bit width requirements of the network layer, the layers with high importance evaluation indicators and high bit width requirements are mapped to the high-performance processing unit, the layers with medium importance evaluation indicators and medium bit width requirements are mapped to the general-purpose computing unit, and the layers with low importance evaluation indicators and low bit width requirements are mapped to the low-power computing unit.

[0067] Taking the feature map output by the third convolutional layer as an example, the shape of the feature map is 56x56x128. The importance scores of the feature map are calculated by performing information entropy analysis on the feature map in the spatial dimension and the channel dimension. The average value of the spatial region importance score is 0.67, and the average value of the channel importance score is 0.59. The feature map is divided into multiple importance levels based on the importance scores. The importance threshold is set to 0.7 and 0.4, and the feature map is divided into three levels of high importance, medium importance and low importance. Different storage strategies are used for feature maps of different importance levels. The high importance region and channel are stored in a lossless compression manner, maintaining 8-bit precision; the medium importance region and channel are quantized to 4-bit precision; and the low importance region and channel are sparsified, retaining only the most significant 20% elements, and the rest are set to zero. This differentiated storage strategy significantly reduces memory usage while maintaining the integrity of key information.

[0068] The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, and an optimized neural network is generated. The preset optimization target is: the model size is not more than 30% of the original model, the accuracy rate decreases by not more than 3%, and the inference speed is improved by not less than 200%. By iteratively adjusting the weight configuration of the importance evaluation index, the optimal bit width configuration and mapping strategy are found. Under the initial configuration, the model size is 142MB, the accuracy rate is 90.5%, and the inference speed is 15fps. Adjust the weight distribution entropy weight to 0.55, and the activation value distribution entropy weight to 0.45. Recalculate the importance evaluation index to get the updated bit width configuration scheme. Adjust the quantization error data weight to 0.65, and the importance evaluation index weight to 0.35. Recalculate the inter-layer error propagation evaluation matrix to get the updated bit width configuration scheme. After optimization, the model size is 138MB, the accuracy rate is 90.2%, and the inference speed is 32fps, meeting the preset optimization target. The final generated optimized neural network is suitable for resource-constrained embedded devices, while maintaining high accuracy, significantly reducing storage requirements and computational overhead.

[0069] In an optional implementation, the steps of calculating the weight distribution entropy and the activation value distribution entropy for each convolutional layer and fully connected layer in the neural network, and constructing an importance evaluation index of neurons based on the weight distribution entropy and the activation value distribution entropy include:

[0070] The weight data and activation value data of each convolutional layer and fully connected layer in the neural network are obtained, and the weight probability density distribution and the activation value probability density distribution are discretized respectively, and the corresponding entropy values are calculated;

[0071] The input and output characteristics of neurons are analyzed for similarity, and a group of neurons with similar response patterns is identified; the distribution difference within the group of neurons and the distribution difference between groups are calculated, and the contribution of each neuron to network function is determined according to the difference size; based on the contribution ratio, a corresponding first balance factor and a second balance factor are assigned to each group of neurons; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain a hierarchical information flow;

[0072] The mutual information between adjacent layers in the neural network is calculated.

[0073] The hierarchical information flow and the mutual information are combined to obtain an importance evaluation index of neurons; the importance evaluation index is verified in multiple batches based on network output error, and when the index fluctuation is less than a preset fluctuation threshold, the final importance evaluation index is determined.

[0074] For example, for a convolutional layer, the weight shape is (output channel number, input channel number, convolution kernel height, convolution kernel width); for a fully connected layer, the weight shape is (output neuron number, input feature number). The activation value data is obtained by recording the feature map or vector output of each layer during the forward propagation of the neural network. These data are usually four-dimensional tensors (batch size, channel number, feature map height, feature map width) or two-dimensional matrices (batch size, feature number).

[0075] After data acquisition, discretization processing is performed, and continuous weight values and activation values are divided into several intervals. The numerical range is evenly divided into 128 intervals, the frequency of values in each interval is counted, and the probability density distribution is obtained by normalization. Taking weight discretization as an example, assuming that the weight value range of a convolutional layer is [-0.5, 0.5], the interval width is set to 0.0078, the number of weight values in each interval is counted and divided by the total weight number to obtain the probability distribution. For a certain convolutional layer, a 128-dimensional probability distribution vector similar to [0.001, 0.003, 0.008,..., 0.002] is obtained.

[0076] Based on the discretized probability distribution, the weight distribution entropy value and the activation value distribution entropy value are calculated. For the weight distribution of a certain layer, assuming the probability vector is [p1, p2,..., p128], the entropy value is calculated using the information entropy formula: H = -∑(pi x log2(pi)), where pi is the probability value of the i-th interval, and the sum is calculated for all non-zero probabilities. For example, for the probability vector [0.001, 0.003, 0.008,..., 0.002], the entropy value is calculated by substituting the formula, taking the negative sign, and obtaining the entropy value result of 3.742. For the activation value distribution entropy value, the same calculation method is used, i.e. H = -∑(pi x log2(pi)), where pi is the probability value of the i-th interval in the probability vector of the activation value distribution, and the calculated value is, for example, 4.213.

[0077] In the neuron similarity analysis stage, the input connection weight and output response feature of each neuron are extracted. For example, for the j-th neuron in the i-th layer, its input weight vector is [-0.03, 0.15, 0.08, -0.12,...], and the output response feature is [0.23, 0.45, 0.10, 0.35,...]. The similarity between neurons is evaluated by calculating the cosine similarity or Euclidean distance, and when the similarity of two neurons exceeds 0.85, they are classified into the same group.

[0078] In an embodiment, for a fully connected layer containing 1024 neurons, 87 neuron groups are identified through similarity analysis, of which the largest group contains 35 neurons and the smallest group contains only a single neuron. For these groups, the average variance of the neuron response within the group (such as 0.032) is calculated as the internal distribution difference, and the average distance of the response centers between different groups (such as 0.567) is calculated as the inter-group distribution difference.

[0079] Based on the above difference data, the contribution ratio of each neuron to the network function is determined. The smaller the internal difference of the group and the greater the inter-group difference, the more specialized the function of the neuron group and the higher the contribution ratio. For example, for a neuron group with an internal difference of 0.025 and an inter-group difference of 0.612, a higher contribution ratio of 0.078 is allocated.

[0080] According to the contribution ratio, a first balance factor and a second balance factor are allocated. The first balance factor is used for the weight distribution entropy value, and its calculation takes into account the size and internal difference of the neuron group; the second balance factor is used for the activation value distribution entropy value, and mainly takes into account the inter-group difference and the neuron response intensity. In an example, a certain neuron group obtains a first balance factor of 0.65 and a second balance factor of 0.82.

[0081] The weight distribution entropy value is multiplied by a first balance factor, the activation value distribution entropy value is multiplied by a second balance factor, and the hierarchical information flow is obtained by combining through a nonlinear mapping function. The nonlinear mapping function can adopt an S-shaped function, and the specific form is f(x) = 1 / (1 + e^(-k x (x-x0))), wherein k controls the function slope, and x0 is the center point position, and in the implementation, k can be set to 0.5, and x0 can be set to 5, so as to ensure that the output value is in the range of [0, 1]. In practice, the weight distribution entropy value of a certain layer is 3.742, the first balance factor is 0.65, the product is 2.432; the activation value distribution entropy value is 4.213, the second balance factor is 0.82, the product is 3.455; and the hierarchical information flow obtained after nonlinear mapping is 0.765.

[0082] Adjacent layer mutual information calculation is an important link for evaluating information transmission efficiency. By analyzing the relationship between the joint probability distribution of the i-th layer output and the i+1-th layer input and the respective edge distribution, the information transmission degree is quantified. The mutual information calculation method is to take the logarithm of the ratio of the joint probability and the edge probability, and then multiply the joint probability and sum. The joint probability represents the probability that two variables take a certain value at the same time, and the edge probability represents the probability that a single variable takes a certain value. In actual calculation, the histogram method can be used to estimate these probability distributions, the i-th layer output and the i+1-th layer input are divided into m and n intervals respectively, an m*n joint distribution matrix is constructed, and then the mutual information calculation method is used for calculation. For example, the mutual information calculation value between the 3rd layer and the 4th layer is 1.856.

[0083] Finally, the hierarchical information flow and the mutual information are weighted and combined to obtain the neuron importance evaluation index. The weighting coefficients can be determined through experiments, for example, the hierarchical information flow is given a weight of 0.6, and the mutual information is given a weight of 0.4. For a certain layer neuron, the hierarchical information flow is 0.765, and the average mutual information with the adjacent layer is 1.856, and the final importance evaluation index calculation result is 0.765x0.6+1.856x0.4=1.201.

[0084] In order to ensure the stability of the evaluation index, the importance evaluation index is verified in multiple batches based on the network output error. Different verification data batches (such as 5 batches, 100 samples per batch) are used to observe the change of the index. When the index fluctuation of 3 consecutive batches is less than the preset fluctuation threshold 0.05, the final importance evaluation index is confirmed. For example, the importance indexes of a certain neuron in 5 batches are [1.201, 1.185, 1.196, 1.202, 1.193], the maximum fluctuation is 0.017, which is less than the threshold 0.05, and therefore the final importance evaluation index is determined to be 1.195 (the average value of the 5 batches).

[0085] This invention accurately assesses the actual contribution of neurons to network function by performing similarity analysis on neurons and identifying neuron groups, combined with weight distribution entropy and activation value distribution entropy. It considers not only the characteristics of individual neurons but also the overall effect of neuron groups, and ensures the stability of the assessment indicators through multiple batch validations, thus improving the accuracy and reliability of neuron importance assessment.

[0086] In one alternative implementation, the step of calculating the mutual information between adjacent layers in the neural network includes:

[0087] Multi-scale decomposition of feature data from adjacent layers in a neural network yields feature representations at multiple scale levels. Feature probability distributions are calculated for each scale level feature representation. Based on the feature probability distributions at each scale level, the information entropy of the previous layer and the information entropy of the next layer are calculated respectively, and the combination yields mutual information at multiple scales.

[0088] The temporal feature sequences of adjacent layers are obtained, the feature probability distribution is temporally smoothed, and the temporal mutual information is calculated; the conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated; the multiple scale mutual information, temporal mutual information, and conditional mutual information are weighted and combined to obtain the initial mutual information.

[0089] A causal graph of inter-layer feature transmission is constructed to identify key transmission paths and analyze the causal effect strength of different transmission channels. Based on the causal effect strength and initial mutual information, a loss matrix for feature transmission is constructed, which characterizes the degree of information retention of features during transmission.

[0090] The connection weights of the corresponding channels are adjusted according to the loss values ​​of each transmission channel in the loss matrix, and the loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold, thereby obtaining the mutual information between adjacent layers.

[0091] For example, we can obtain feature data from adjacent layers in a neural network. Taking a deep neural network with 5 layers as an example, we select the 3rd and 4th layers as the research objects and extract the activation values ​​of these two layers when processing 1000 samples as feature data. The feature dimension of the 3rd layer output is 256, and the feature dimension of the 4th layer output is 128.

[0092] The acquired feature data is decomposed at multiple scales. Wavelet transform is used to decompose the features into different frequency components. A three-level decomposition is used to obtain feature representations at three scale levels: high frequency, mid frequency, and low frequency. For the 256-dimensional features at the third level, the decomposition yields 64-dimensional features at scale 1 (high frequency), 96-dimensional features at scale 2 (mid frequency), and 96-dimensional features at scale 3 (low frequency). Similarly, the 128-dimensional features at the fourth level are decomposed to obtain 32-dimensional features at scale 1, 48-dimensional features at scale 2, and 48-dimensional features at scale 3.

[0093] For each scale level, the kernel density estimation method is used to construct the probability density function of the feature vector at each scale. Taking the high-frequency features as an example, a Gaussian kernel function with a bandwidth of 0.15 is used to generate a smooth probability distribution in the feature space. For the high-frequency features of 1000 samples, the probability of their distribution falling within 10 equal intervals is calculated to obtain the discretized probability distribution vector.

[0094] Based on the calculated feature probability distribution, the information entropy and mutual information at each scale level are calculated. Taking the high-frequency features as an example, the information entropy value of the 3rd layer high-frequency features is 4.32, and the information entropy value of the 4th layer high-frequency features is 3.85. The high-frequency feature mutual information value between the two layers is 2.76. The calculation method of mutual information is to calculate the joint occurrence frequency of the two layers of features in each interval, construct a joint probability table, then take the logarithm of the ratio of the joint probability and the respective marginal probability, multiply it by the joint probability, and sum it up. Similarly, the medium-frequency feature mutual information is 3.12, and the low-frequency feature mutual information is 3.45. By integrating the mutual information of the three scales, a weighted average method is used to calculate the multi-scale mutual information, with weights of 0.3, 0.3 and 0.4 respectively, and the multi-scale mutual information value is 3.14.

[0095] When processing continuous input data (such as video frame sequences), the activation values of adjacent layers at 100 consecutive time points are extracted to form a time series of features. For each time point, the feature vectors of the 3rd and 4th layers are extracted to construct a feature matrix in the time dimension. The time series smoothing processing is performed on the feature probability distribution, and the sliding window method is used to calculate the time series mutual information. In specific implementation, the window size is set to 5, and the time series features are slid to calculate the feature probability distribution in each window. For a sequence of 100 time points, 96 probability distributions of sliding windows are generated. The similarity of the feature distributions of adjacent layers in the same time window is compared, and the time series mutual information value is calculated as 2.87. The calculation method is to regard the features in the time window as a whole, and then calculate the joint probability and marginal probability at the window level, and then evaluate according to the mutual information calculation principle.

[0096] The conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated. The input conditions of the neural network (such as images of different categories) are selected, and the feature distributions of adjacent layers under these conditions are calculated. In specific implementation, 1000 samples are divided into 10 categories, and the feature probability distribution under each category condition is calculated. For each category, the conditional mutual information between the 3rd and 4th layers is calculated, and the average conditional mutual information value is 3.05. The calculation of conditional mutual information is to evaluate the information dependence between the two layers of features under the input condition. The calculation method is to calculate the mutual information under each condition, and then perform a weighted average according to the probability of the condition.

[0097] The initial mutual information is obtained by weighted combination of multi-scale mutual information, time series mutual information and conditional mutual information. The weight parameters are set to 0.5, 0.3 and 0.2, and the initial mutual information value is calculated to be 3.06. The specific calculation method is 3.14*0.5+2.87*0.3+3.05*0.2=3.06.

[0098] A causal graph for inter-layer feature transmission is constructed, and the 256-dimensional features of the 3rd layer and the 128-dimensional features of the 4th layer are regarded as nodes. The connection relationship between nodes is established through correlation analysis. The correlation threshold is set to 0.4, and the connection is established when the correlation coefficient between two feature dimensions exceeds the threshold. The correlation coefficient is obtained by calculating the product of the covariance of the two feature dimensions on all samples and the standard deviation of each. Finally, a causal graph containing 384 nodes and about 15000 edges is obtained.

[0099] Random walk sampling is performed on the causal graph to identify the most influential transmission path. The 3rd layer feature nodes are randomly disturbed, and the change response of the 4th layer feature nodes is measured to calculate the causal effect strength. The causal effect strength calculation method is the ratio of the change of the 4th layer features before and after the disturbance to the disturbance of the 3rd layer. After 10000 times of simulation experiment, 50 key transmission paths are determined, and the average causal effect strength of these paths is 0.72.

[0100] A loss matrix for feature transmission is constructed, and a 256*128 matrix is constructed to represent the information transmission loss of each feature dimension of the 3rd layer to each feature dimension of the 4th layer. For the connections on the key transmission paths, a lower loss value is set according to the causal effect strength; for non-key paths, a higher loss value is set. Specifically, first, use the initial mutual information value 3.06 as a global reference to calculate the normalization coefficient as the inverse of the initial mutual information (about 0.327). Then, multiply the causal effect strength of each connection by this normalization coefficient to ensure that the overall information flow matches the initial mutual information. Finally, the loss value is calculated by subtracting the adjusted causal effect strength from 1. In addition, for feature dimensions with greater mutual information contribution (identified according to joint probability distribution), the loss value is additionally reduced by 10% to preserve key information channels. Through this method of combining initial mutual information and causal effect strength, the loss matrix not only reflects the causal relationship of local connections, but also maintains consistency with the global information flow. The average loss value of the initial loss matrix is 0.35.

[0101] For connections with a loss value higher than 0.5, reduce their weight by 20%; for connections with a loss value lower than 0.2, increase their weight by 15%. In specific implementation, the formula for adjusting the weight is: new weight=original weight*(1-adjustment coefficient*(loss value-reference loss value)), where the reference loss value is set to 0.35 and the adjustment coefficient is set to 0.5. After adjustment, the loss matrix is recalculated and the difference with the previous iteration is compared.

[0102] The loss matrix is iteratively updated until its fluctuation value is less than a preset threshold 0.01. The fluctuation value is calculated by summing the square of the difference between each element of the current iteration and the loss matrix of the last iteration, and then taking the square root. In actual implementation, after 8 iterations, the fluctuation value of the loss matrix is reduced to 0.009, satisfying the convergence condition. The average value of the final loss matrix is 0.28, indicating that the information transmission efficiency is improved.

[0103] By subtracting the average loss value from 1, the information retention rate is obtained as 0.72. Multiplying this retention rate by the initial mutual information, the mutual information value between adjacent layers is obtained as 2.20, representing the effective information transmission amount between the third layer and the fourth layer of the neural network. This mutual information value can be used to evaluate the information flow efficiency between network layers, providing a basis for subsequent network structure optimization.

[0104] The present application calculates the inter-layer mutual information by combining multi-scale decomposition and time series smoothing processing, comprehensively capturing the information transmission characteristics of features in different scales and time series dimensions. By constructing a causal graph of inter-layer feature transmission, analyzing the causal effect strength of different transmission channels, and establishing a feature transmission loss matrix, the information retention degree of features in the transmission process can be accurately described. Based on the loss value, the connection weights of the transmission channels are dynamically adjusted, ensuring the accuracy and adaptability of the mutual information calculation.

[0105] In an optional embodiment, the step of performing quantization error data testing on the convolutional layer and the fully connected layer includes:

[0106] Performing feature importance analysis on the parameters of the convolutional layer and the fully connected layer divides the parameter space into a key parameter domain and a non-key parameter domain;

[0107] In the key parameter domain, a first quantization bit width configuration is used to perform quantization processing, and in the non-key parameter domain, a second quantization bit width configuration smaller than the first quantization bit width configuration is used to perform quantization processing, to obtain adaptive quantization parameters;

[0108] Error calculation is performed on the adaptive quantization parameters and the original floating-point parameters to obtain parameter domain quantization errors in different parameter domains. The influence degree of the different parameter domain quantization errors in inter-layer transmission is analyzed, and the quantization sensitivity of each layer is calculated. The parameter domain quantization errors are weighted according to the quantization sensitivity to obtain quantization error data.

[0109] In combination with Figure 2The parameter domain adaptive quantization and error propagation analysis flowchart is described, and the pre-trained neural network model is obtained. For example, the parameters of the convolutional layer and the fully connected layer of an image classification network are extracted. The network contains 5 convolutional layers and 3 fully connected layers. The parameter matrix of the second convolutional layer is extracted, with a shape of 64x32x3x3, representing 64 output channels, 32 input channels, and a 3x3 convolution kernel size. The parameter matrix of the first fully connected layer is extracted, with a shape of 512x4096, representing 512 output neurons and 4096 input features.

[0110] The feature importance analysis of these parameters divides the parameter space into key parameter domains and non-key parameter domains. The feature importance analysis uses a parameter sensitivity evaluation method. The specific process is to add a small perturbation to each parameter and measure the change in network output. For the parameter matrix of the second convolutional layer, 1000 parameter positions are randomly selected, and each time a position is selected, the parameter value at that position is increased by 1% of the original value, and the other parameters remain unchanged. The average change rate of the network output is calculated using the validation dataset. The parameter sensitivity is defined as the ratio of the network output change rate to the parameter change rate. For example, the sensitivity distribution of the second convolutional layer parameters ranges from 0.01 to 3.56, with an average value of 0.75. Parameters with a sensitivity greater than 1.5 are classified as key parameter domains, accounting for about 20% of the total number of parameters; parameters with a sensitivity less than or equal to 1.5 are classified as non-key parameter domains, accounting for about 80% of the total number of parameters. Similarly, the sensitivity analysis of the 512x4096 parameter matrix of the first fully connected layer is performed, with a sensitivity distribution ranging from 0.02 to 2.86 and an average value of 0.68. Parameters with a sensitivity greater than 1.2 are classified as key parameter domains, accounting for about 25% of the total number of parameters; parameters with a sensitivity less than or equal to 1.2 are classified as non-key parameter domains, accounting for about 75% of the total number of parameters. During the sensitivity analysis, 1000 validation samples were used, and each sample was calculated 5 times to reduce the influence of randomness.

[0111] The differential quantization processing is performed on the divided parameter domain. The first quantization bit width configuration is set to 8-bit fixed point number, which is applied to the key parameter domain; the second quantization bit width configuration is set to 4-bit fixed point number, which is applied to the non-key parameter domain. For the second convolutional layer, the key parameter domain contains about 11,500 parameters, which uses 8-bit quantization; the non-key parameter domain contains about 46,000 parameters, which uses 4-bit quantization. The specific quantization process is to determine the numerical range of each parameter domain, and then map the floating point number to the integer range of the corresponding bit width. Taking the second convolutional layer as an example, the numerical range of the key parameter domain is [-0.35, 0.42], which is uniformly mapped to the [-128, 127] integer range represented by 8-bit fixed point number; the numerical range of the non-key parameter domain is [-0.28, 0.31], which is mapped to the [-8, 7] integer range represented by 4-bit fixed point number.

[0112] For the first fully connected layer, the key parameter domain contains about 524,000 parameters, which uses 8-bit quantization; the non-key parameter domain contains about 1,572,000 parameters, which uses 4-bit quantization. The numerical range of the key parameter domain is [-0.12, 0.15], and the numerical range of the non-key parameter domain is [-0.09, 0.11]. When quantizing, the numerical distribution characteristics are considered, and the uniform quantization method is adopted, the quantization step of the key parameter domain is 0.00224, and the quantization step of the non-key parameter domain is 0.0125.

[0113] After completing the quantization processing, the adaptive quantization parameters are obtained, and the error calculation is performed with the original floating point parameters to obtain the parameter domain quantization error of different parameter domains. The calculation method is to calculate the mean square error of the quantization parameters and the original parameters. For the second convolutional layer, the quantization mean square error of the key parameter domain is 0.000125, and the quantization mean square error of the non-key parameter domain is 0.000728. For the first fully connected layer, the quantization mean square error of the key parameter domain is 0.000089, and the quantization mean square error of the non-key parameter domain is 0.000475.

[0114] Considering the inter-layer connection relationship in the network structure, an error propagation model is constructed. Through forward propagation analysis, the influence of quantization error on subsequent layer activation values is measured. Taking the second convolutional layer as an example, the quantization parameters of this layer are applied to the network, and the original floating point parameters are used in other layers, the average error of the network output is calculated on 1000 validation samples. Then the second convolutional layer is restored to floating point parameters, the third convolutional layer is quantized, and the network output error is recalculated. The output errors in the two cases are compared to evaluate the influence of quantization error transmission.

[0115] For example, the average relative error of the network output after quantization of the second convolution layer is 1.8%; the average relative error of the network output after quantization of the third convolution layer is 2.1%; and the average relative error of the network output after quantization of the first full connection layer is 3.5%. Based on these data, the quantization sensitivity of each layer is calculated, which is defined as the ratio of the relative error of the network output caused by the quantization of the layer to the average relative error of all layers. The quantization sensitivity of the second convolution layer is 0.72, the quantization sensitivity of the third convolution layer is 0.84, and the quantization sensitivity of the first full connection layer is 1.40.

[0116] The parameter domain quantization error is weighted according to the quantization sensitivity to obtain the final quantization error data. The weighting method is to multiply the parameter domain quantization error of each layer by the corresponding quantization sensitivity. For the second convolution layer, the weighted quantization error of the key parameter domain is 0.000125 x 0.72 = 0.000090, and the weighted quantization error of the non-key parameter domain is 0.000728 x 0.72 = 0.000524. For the first full connection layer, the weighted quantization error of the key parameter domain is 0.000089 x 1.40 = 0.000125, and the weighted quantization error of the non-key parameter domain is 0.000475 x 1.40 = 0.000665. These weighted quantization error data take into account the importance of parameters and the sensitivity of layers, and can more accurately reflect the actual impact of different quantization strategies on network performance.

[0117] The differentiated quantization method of the application realizes effective reduction of storage space and consumption of computing resources while maintaining model accuracy by distinguishing the importance of parameters and adopting different bit width configurations for different parameter domains. Combined with inter-layer error propagation analysis and quantization sensitivity evaluation, a more accurate quantization error evaluation mechanism is provided, which provides a theoretical basis and practical guidance for model compression and hardware deployment, effectively balancing inference performance and resource utilization efficiency.

[0118] In an optional implementation, the quantization error data and the importance evaluation index are combined by weighting to generate an inter-layer error propagation evaluation matrix, and the step of determining a bit width configuration scheme based on the inter-layer error propagation evaluation matrix comprises:

[0119] Local features between adjacent layers, mid-range features between consecutive multiple layers, and global features of the whole network in the neural network are calculated respectively, the local features are obtained by weighted combination of the quantization error data and the importance evaluation index of adjacent layers, the mid-range features are obtained by weighted combination of the local features of each layer in a feature block, and the global features are obtained by weighted combination of the mid-range features of each layer;

[0120] An inter-layer error propagation evaluation matrix is constructed based on the local features, the mid-range features, and the global features;

[0121] According to the inter-layer error propagation evaluation matrix, an error compensation value of each layer is calculated, the error compensation value is determined by a weighted sum of quantization error of a current layer and propagated error of a previous layer, and a reference bit width of each layer is adaptively adjusted based on the error compensation value to obtain a bit width configuration scheme.

[0122] A network performance evaluation value is calculated, and a feature combination weight coefficient in the inter-layer error propagation evaluation matrix construction process is updated and the inter-layer error propagation evaluation matrix is reconstructed according to the network performance evaluation value until the network performance evaluation value meets a preset optimization target.

[0123] For example, when the quantization error data and the importance evaluation index are weighted and combined to generate the inter-layer error propagation evaluation matrix, a multi-level feature analysis method is adopted. Taking an image recognition neural network containing 15 layers as an example, the network can be divided into 3 feature blocks, and each feature block contains 5 layers. When calculating the local feature between adjacent layers, the quantization error data of the 3rd layer and the 4th layer are extracted first. The key parameter domain quantization error of the 3rd layer is 0.000112, the non-key parameter domain quantization error is 0.000683, the quantization sensitivity is 0.79, and the weighted quantization error data is 0.000629. The key parameter domain quantization error of the 4th layer is 0.000098, the non-key parameter domain quantization error is 0.000572, the quantization sensitivity is 0.81, and the weighted quantization error data is 0.000543. The importance evaluation index of the two layers is extracted, which is 0.68 for the 3rd layer and 0.71 for the 4th layer, reflecting the functional contribution of each layer of neurons.

[0124] The calculation of the local feature adopts an exponential decay weighting method, and the decay coefficient is set to 0.7. The quantization error data of the current layer is multiplied by 0.7, and the product of the quantization error data of the previous layer and the importance evaluation index is added to the product of 0.3 to obtain the inter-layer local feature. In specific implementation, the local feature calculation between the 3rd layer and the 4th layer is 0.000629*0.7+0.000543*0.71*0.3=0.000556. Similarly, the local features of all adjacent layer pairs in the network are calculated to construct a local feature matrix.

[0125] When calculating the mid-range feature between continuous multiple layers, the interaction between layers in a feature block is considered. For the first feature block (1st layer to 5th layer), the local features of each layer pair are extracted, and a distance decay weight is set. The weight of the layer pair with a distance of 1 is 0.5, the weight of the layer pair with a distance of 2 is 0.3, the weight of the layer pair with a distance of 3 is 0.15, and the weight of the layer pair with a distance of 4 is 0.05. The mid-range feature is calculated with the 3rd layer as the center, and the influence of the 1st layer to the 5th layer on the 3rd layer is considered, and the local features of each layer pair are weighted and combined. The calculation result shows that the mid-range feature value of the 3rd layer is 0.000612, which reflects the comprehensive quantization sensitivity of the 3rd layer in the feature block.

[0126] After the mid-range feature of each layer in the feature block is calculated, the mid-range feature propagation model is constructed through the hierarchical transmission relationship. Considering the nonlinear dependence relationship between distant layers in the deep network, an adaptive weight allocation strategy is adopted. According to the position and functional importance of each layer in the network, the transmission weight is allocated. The weight of the layer close to the input is low, which is set to 0.2; the weight of the middle hidden layer is moderate, which is set to 0.5; the weight of the layer close to the output is high, which is set to 0.8. Taking the mid-range feature transmission of the second feature block as an example, the mid-range features of the 6th layer to the 10th layer are weighted according to the position weight, and the transmission feature of the feature block level is 0.000576.

[0127] When calculating the global feature of the overall network, the mid-range features of all feature blocks are integrated. An importance-driven hierarchical aggregation method is adopted, and the aggregation weight is allocated according to the influence degree of each feature block on the network output. Through perturbation analysis, the influence weight of the first feature block is 0.25, the influence weight of the second feature block is 0.35, and the influence weight of the third feature block is 0.40. The mid-range features of the three feature blocks are weighted and summed according to the weight, and the network global feature value is 0.000603, which reflects the quantization sensitivity feature of the overall network.

[0128] Based on the local feature, the mid-range feature and the global feature, an inter-layer error propagation evaluation matrix is constructed. A 15x15 matrix is constructed, and each element in the matrix represents the error propagation influence intensity from the row corresponding layer to the column corresponding layer. For the directly connected layer pair, the local feature value is filled; for the non-directly connected layer pair within the feature block, the product of the mid-range feature and the distance attenuation coefficient is filled; for the layer pair across the feature block, the product of the global feature and the inter-block distance attenuation coefficient is filled. Taking the error propagation influence of the 3rd layer to the 7th layer as an example, since they belong to different feature blocks and are not directly connected, their evaluation value is 0.000603x0.4=0.000241, wherein 0.4 is the inter-block distance attenuation coefficient.

[0129] According to the inter-layer error propagation evaluation matrix, the error compensation value of each layer is calculated. The error compensation value is determined by the weighted sum of the quantization error of the current layer itself and the propagation error of the previous layer. For the 7th layer, its own quantization error is 0.000527, and the propagation error of the previous layer is the cumulative value of the quantization error of each previous layer after attenuation through the propagation matrix, which is calculated to be 0.000382. The weight of the self-error is set to 0.6, and the weight of the propagation error is set to 0.4. The error compensation value of the 7th layer is 0.000527x0.6+0.000382x0.4=0.000469.

[0130] The reference bit width of each layer is adaptively adjusted based on the error compensation value. The reference bit width is set to 8 bits, and the bit width is adjusted according to the comparison result of the error compensation value and the reference threshold value. If the error compensation value is less than 0.000300, the bit width is reduced by 2 bits; if the error compensation value is between 0.000300 and 0.000500, the bit width is reduced by 1 bit; if the error compensation value is between 0.000500 and 0.000700, the reference bit width is kept unchanged; and if the error compensation value is greater than 0.000700, the bit width is increased by 1 bit. Taking the 7th layer as an example, the error compensation value is 0.000469, which is between 0.000300 and 0.000500, so the bit width is adjusted to 7 bits. In this way, an appropriate bit width configuration is determined for each layer in the network.

[0131] The network performance evaluation value is calculated, and the evaluation matrix is optimized based on the evaluation result. A comprehensive evaluation index is adopted, including the accuracy loss of the quantized model on the validation set (weight 0.5), the storage space reduction rate of the model (weight 0.3), and the inference calculation amount reduction rate (weight 0.2). Taking the above bit width configuration scheme as an example, the accuracy loss of the quantized model is 0.76%, the storage space reduction rate is 58.3%, and the inference calculation amount reduction rate is 42.1% on 1000 validation samples. The comprehensive performance evaluation value is 0.682.

[0132] If the performance evaluation value does not reach the preset optimization target 0.75, the feature combination weight coefficients in the evaluation matrix construction process are updated. The attenuation coefficient in the local feature calculation is increased from 0.7 to 0.75, the distance weight in the middle-range feature calculation is adjusted from [0.5, 0.3, 0.15, 0.05] to [0.55, 0.25, 0.15, 0.05], and the feature block weight in the global feature calculation is adjusted from [0.25, 0.35, 0.40] to [0.20, 0.35, 0.45]. The inter-layer error propagation evaluation matrix is reconstructed using the updated weight coefficients, and the above bit width configuration process is repeated until the network performance evaluation value reaches the preset optimization target.

[0133] The present application accurately captures the transmission rule of quantization error in the network by constructing the inter-layer error propagation evaluation matrix, allocates different bit width configurations for different layers, realizes the optimal balance between precision loss and model compression rate, effectively improves the deployment efficiency of the quantized model on resource-limited devices, and at the same time maintains the stability of the model performance.

[0134] In an optional implementation, the step of differentially allocating the computing unit mapping and storage strategy to the network layer based on the computing load data and the bit width configuration scheme comprises:

[0135] combining the load dynamic characteristic values obtained from the time-series weighted combination of the computing load data; and calculating a resource demand change rate of each layer of the neural network based on the bit-width configuration scheme to obtain a bit-width sensitivity;

[0136] constructing a resource allocation priority matrix based on the load dynamic characteristic values and the bit-width sensitivity, each matrix element of the resource allocation priority matrix being obtained by weighted combination of the bit-width sensitivity of a corresponding network layer and the load dynamic characteristic value of a corresponding computing unit, for determining a mapping relationship between the network layer and the computing unit;

[0137] performing importance analysis on the intermediate feature maps of the neural network, obtaining an importance score based on information distribution of the feature maps in spatial dimensions and channel dimensions; and dividing the feature maps into multiple importance levels based on the importance score, and adopting a differentiated storage strategy for the feature maps of different importance levels;

[0138] jointly optimizing the mapping relationship and the storage strategy based on a weighted combination value of computing efficiency, storage efficiency and communication efficiency.

[0139] For example, when mapping the network layer to the computing unit and differentially allocating the storage strategy based on the computing load data and the bit-width configuration scheme, first, the computing load data on the device is obtained. Taking a heterogeneous computing platform as an example, the platform includes 1 high-performance processing unit, 2 general-purpose computing units and 4 low-power computing units. The load data of each computing unit when executing different tasks is collected through a monitoring tool, including computing utilization, memory occupancy and power consumption data. For the high-performance processing unit, the load data of 30 sampling points is obtained by sampling every 10 seconds within 5 minutes, the computing utilization ranges from 45% to 92% with an average of 76%, the memory occupancy ranges from 38% to 85% with an average of 62%, and the power consumption ranges from 3.5 W to 8.2 W with an average of 5.8 W. Similarly, the load data of the general-purpose computing units and the low-power computing units is collected.

[0140] The load dynamic characteristic values are obtained by time series weighted combination of the calculation load data. The load data is processed by using time decay weighting method, and the weight of recent sampling points is higher and the weight of long-term sampling points is lower. The decay factor is set to 0.95, and for the calculation utilization rate of the high-performance processing unit, the weight of the recent sampling point is 1, the weight of the previous sampling point is 0.95, the weight of the second previous sampling point is 0.95 2, and so on. The weighted average load is calculated, and the time series adjusted calculation utilization rate is 79%, the memory occupancy rate is 65%, and the power consumption is 6.1 W. The three indexes are combined according to the weights of 0.4, 0.3 and 0.3, and the load dynamic characteristic value of the high-performance processing unit is 0.704. Similarly, the load dynamic characteristic values of the two general-purpose computing units are 0.623 and 0.587 respectively, and the load dynamic characteristic values of the four low-power computing units are 0.412, 0.435, 0.398 and 0.426 respectively.

[0141] Taking a target detection network containing 10 layers as an example, the network contains 6 convolutional layers and 4 fully connected layers. According to the bit width configuration scheme obtained by the foregoing steps, the bit width of the first convolutional layer is 8 bits, the bit width of the second convolutional layer is 7 bits, the bit width of the third convolutional layer is 6 bits, the bit width of the fourth convolutional layer is 8 bits, the bit width of the fifth convolutional layer is 7 bits, the bit width of the sixth convolutional layer is 6 bits, the bit width of the first fully connected layer is 7 bits, the bit width of the second fully connected layer is 6 bits, the bit width of the third fully connected layer is 7 bits, and the bit width of the fourth fully connected layer is 8 bits. For each layer, the change of the calculation resource requirement under different bit width configurations is calculated. Taking the second convolutional layer as an example, the number of parameters is 73,728, which is reduced from 8-bit quantization to 7-bit quantization, and each parameter saves 1 bit, a total of 73,728 bits of storage space, about 9.216 KB. The calculation resource requirement change rate is 12.5%, which represents the proportion of resource requirement reduction caused by bit width reduction. Considering the calculation characteristics of different layers, the bit width sensitivity of the convolutional layer is also related to the convolution kernel size, the input and output channel number. The convolution kernel of the second convolutional layer is 3x3, the input channel is 64, and the output channel is 128. The bit width sensitivity is 0.125x(3x3x64x128) / (73,728)=0.125x1=0.125. Similarly, the bit width sensitivity of all layers is calculated to obtain the bit width sensitivity vector.

[0142] According to the load dynamic characteristic value and the bit width sensitivity, a resource allocation priority matrix is constructed. A 10x7 matrix is constructed, with the rows representing the 10 layers of the network and the columns representing the 7 computing units. Each element in the matrix is obtained by weighting combination of the bit width sensitivity of the corresponding network layer and the load dynamic characteristic value of the corresponding computing unit. The bit width sensitivity weight is set to 0.6, and the load dynamic characteristic value weight is set to 0.4. Taking the mapping priority of the 2nd convolution layer and the high-performance processing unit as an example, 0.125x0.6+0.704x0.4=0.3566 is calculated. The specific combination method is to multiply the bit width sensitivity by its weight 0.6, add the load dynamic characteristic value multiplied by its weight 0.4, and obtain the weighted sum as the priority value. Similarly, the mapping priority between all layers and all computing units is calculated, and the priority matrix is filled. The higher the priority value, the higher the efficiency of the layer executing on the computing unit. According to the priority matrix, the mapping relationship between the network layers and the computing units is determined. A greedy algorithm is used, and each time the layer-unit pair with the highest priority is selected for mapping until all layers are allocated to computing units. In actual mapping, the 1st, 4th convolution layers and the 4th fully connected layer are mapped to the high-performance processing unit; the 2nd, 5th convolution layers and the 1st, 3rd fully connected layers are mapped to two general-purpose computing units; the 3rd, 6th convolution layers and the 2nd fully connected layer are mapped to four low-power computing units.

[0143] The importance of the intermediate feature map in the neural network is analyzed. Taking the feature map output by the third convolutional layer as an example, its shape is 56x56x128, representing a feature map with a height of 56 pixels, a width of 56 pixels, and 128 channels. The information entropy of the feature map in the spatial dimension is analyzed, and the 56x56 spatial region is divided into a 4x4 grid, with each grid region being 14x14 pixels. The distribution entropy of the pixel values in each grid region is calculated to obtain an entropy value matrix of 16 spatial regions. The specific calculation method is to discretize the pixel value distribution in each grid region into 32 uniform intervals, count the number of pixels in each interval, calculate the normalized probability distribution, and then calculate the information entropy of the distribution. The region with high entropy value contains more information and is more important. At the same time, the information entropy of the 128 channels is analyzed, and the entropy value of each channel is calculated. The calculation method is to analyze the pixel value distribution of each channel in the entire feature map space and calculate its information entropy. When the importance score is obtained based on the information distribution, a two-dimensional weighting strategy is adopted to combine the spatial dimension entropy value and the channel dimension entropy value. Specifically, for each intersection of a spatial region and a channel, the importance score is calculated as the weighted sum of the spatial entropy value multiplied by 0.6 and the channel entropy value multiplied by 0.4, and then normalized to the range of 0 to 1. This scoring method considers the importance distribution of features in spatial location and channel semantics, and can more comprehensively reflect the distribution rule of information in the feature map. The final spatial region importance score ranges from 0.35 to 0.92, with an average of 0.67; the channel importance score ranges from 0.28 to 0.85, with an average of 0.59.

[0144] Based on the importance score, the feature map is divided into multiple importance levels. Set the importance threshold to 0.7 and 0.4, and divide the feature map into three levels: high importance (score ≥ 0.7), medium importance (0.4 ≤ score < 0.7), and low importance (score < 0.4). For the feature map of the third convolutional layer, there are 5 grids in the spatial region belonging to high importance, 8 grids belonging to medium importance, and 3 grids belonging to low importance; there are 42 channels belonging to high importance, 65 channels belonging to medium importance, and 21 channels belonging to low importance. Different storage strategies are adopted for feature maps of different importance levels. High importance regions and channels are stored using lossless compression, maintaining 8-bit precision; medium importance regions and channels are quantized to low precision, reducing to 4-bit precision; low importance regions and channels are sparsified, retaining only the most significant 20% elements, with the rest set to zero. Sparse processing retains the top 20% largest absolute value elements by sorting the feature values, and sets the rest to zero, then uses run-length encoding to compress the zero element position information. This differential storage strategy significantly reduces memory usage while maintaining the integrity of key information.

[0145] In the step of joint optimization, the calculation efficiency weight is set to 0.5, the storage efficiency weight is set to 0.3, and the communication efficiency weight is set to 0.2. The initial mapping scheme is evaluated, the calculation efficiency is 0.82, the storage efficiency is 0.75, the communication efficiency is 0.68, and the weighted combination value is 0.772. The calculation efficiency is evaluated by measuring the execution time of each layer on the allocated calculation unit, and the ratio of the theoretical optimal execution time represents the efficiency; the storage efficiency is evaluated by the ratio of the actual memory occupation to the original memory demand; the communication efficiency is evaluated by the ratio of the inter-layer data transmission amount to the theoretical minimum transmission amount. By adjusting the mapping relationship and the storage strategy, the overall efficiency is improved. For example, the second convolutional layer is remapped from the general-purpose calculation unit to the high-performance processing unit, and the fourth convolutional layer is mapped from the high-performance processing unit to the general-purpose calculation unit; the storage strategy of the third convolutional layer feature map is adjusted, and the number of important channels is reduced from 65 to 50, and the number of low importance channels is increased. The specific adjustment process is to traverse the mapping combination and the storage configuration, calculate the efficiency index of each combination, and select the scheme with the highest efficiency. After optimization, the calculation efficiency is improved to 0.87, the storage efficiency is improved to 0.81, the communication efficiency is improved to 0.73, and the weighted combination value is improved to 0.823, which meets the performance requirements.

[0146] The application captures the dynamic characteristics of hardware through timing weighted analysis, constructs priority mapping combined with network layer bit width sensitivity, and implements importance-driven storage optimization on feature maps, thereby realizing optimal matching between calculation resources and neural network layers, effectively balancing inference accuracy, calculation efficiency and storage overhead, and providing a solution for efficient neural network deployment on resource-constrained devices.

[0147] In an optional implementation, the step of constructing a resource allocation priority matrix according to the load dynamic characteristic value and the bit width sensitivity comprises:

[0148] The calculation characteristic matching degree is calculated based on the hardware characteristic parameters of the calculation unit and the calculation characteristic parameters of the network layer; the calculation unit characteristic value is obtained by weighting and combining the load dynamic characteristic value and the calculation characteristic matching degree; the network layer characteristic value is obtained by weighting and combining the bit width sensitivity and the resource utilization rate of the neural network layer; the affinity between the network layer and the calculation unit is calculated based on the calculation unit characteristic value and the network layer characteristic value, and the affinity is used to represent the matching degree of the network layer and the calculation unit;

[0149] The prediction performance index is constructed, and the prediction performance index includes a calculation delay, resource utilization, and energy efficiency level; a mapping optimization objective function is established based on the affinity and the prediction performance index; an initial mapping scheme is obtained by iteratively optimizing the mapping optimization objective function under the condition of satisfying the calculation unit resource capacity constraint and the task time delay constraint; real-time execution data is collected, and the mapping scheme is adjusted online according to the execution data to determine the resource allocation priority matrix.

[0150] For example, when the resource allocation priority matrix is constructed according to the load dynamic characteristic value and the bit width sensitivity, the calculation characteristic matching degree is calculated based on the hardware characteristic parameters of the calculation unit and the calculation characteristic parameters of the network layer. Taking a heterogeneous computing platform including 7 calculation units as an example, the hardware characteristic parameters of each calculation unit are obtained, including processing capability, memory bandwidth, and support of special instruction set, etc. The processing capability of the high-performance processing unit is 2.4 GHz, the memory bandwidth is 12.8 GB / s, and the matrix multiplication acceleration instruction is supported; the processing capability of the general-purpose computing unit is 1.8 GHz, the memory bandwidth is 6.4 GB / s, and the basic vector operation instruction is supported; the processing capability of the low-power consumption computing unit is 1.2 GHz, the memory bandwidth is 3.2 GB / s, and the integer operation instruction is supported. At the same time, the calculation characteristic parameters of the 10 layers of the neural network are obtained, including the calculation density, the memory access mode, and the operation type. The calculation density of the 2nd convolution layer is 256 FLOPS / Byte, the memory access mode is high spatial locality, and the operation type is mainly floating-point matrix multiplication; the calculation density of the 3rd fully connected layer is 32 FLOPS / Byte, the memory access mode is low spatial locality, and the operation type is mainly vector addition.

[0151] Based on these parameters, the calculation characteristic matching degree is calculated, and the feature vector similarity method is adopted. The hardware characteristic parameters and network layer calculation characteristic parameters are normalized into feature vectors respectively, and the similarity of the two vectors is calculated. The processing capacity corresponds to the calculation density, the memory bandwidth corresponds to the memory access mode, and the instruction set support corresponds to the operation type. In the normalization process, the processing capacity is mapped to the range of 0 to 1 in the range of 0.8GHz to 3GHz, and the memory bandwidth is mapped to the range of 0 to 1 in the range of 2GB / s to 16GB / s. The instruction set support is quantified as a value from 0 to 1 by analyzing the matching degree of the instruction set and the operation type, for example, the support degree of the matrix multiplication instruction for the convolution operation is 0.9, and the support degree for the full connection operation is 0.8. Taking the high-performance processing unit and the second convolution layer as an example, the processing capacity matching degree is 0.85, the memory bandwidth matching degree is 0.78, and the instruction set matching degree is 0.92. When calculating the comprehensive matching degree, the weights of the three are set to 0.4, 0.3 and 0.3 respectively, and the comprehensive calculation characteristic matching degree is 0.85x0.4+0.78x0.3+0.92x0.3=0.850. Similarly, the calculation characteristic matching degrees between all calculation units and all network layers are calculated, and a 7x10 matching degree matrix is constructed.

[0152] The load dynamic characteristic value weight is set to 0.45, and the calculation characteristic matching degree weight is set to 0.55. Taking the high-performance processing unit as an example, the load dynamic characteristic value is 0.704, and the calculation characteristic matching degree with the second convolution layer is 0.850, and the calculation unit characteristic value is calculated as 0.704x0.45+0.850x0.55=0.783. For the combination of the same calculation unit and different network layers, different calculation unit characteristic values are calculated. The bit width sensitivity and the resource utilization rate of the neural network layer are combined to obtain the network layer characteristic value. Taking the second convolution layer as an example, the bit width sensitivity is 0.125, and the resource utilization rate is 0.72. The bit width sensitivity weight is set to 0.4, and the resource utilization rate weight is set to 0.6. The network layer characteristic value is calculated as 0.125x0.4+0.72x0.6=0.482. The resource utilization rate is obtained by analyzing the execution of the network layer on the standard test platform. Specifically, 100 times of calculation of the layer are run, and the average CPU / GPU utilization rate, memory usage rate and cache hit rate of the calculation unit during execution are recorded, and the resource utilization efficiency index is obtained, which represents the effective utilization degree of the calculation resource during execution of the layer.

[0153] The affinity is calculated by using the weighted product method, and the characteristic value weight of the calculation unit is set to 0.65, and the characteristic value weight of the network layer is set to 0.35. Taking the high-performance processing unit and the second convolution layer as an example, the characteristic value of the calculation unit is 0.783, the characteristic value of the network layer is 0.482, and the affinity calculation is 0.783^0.65x0.482^0.35=0.650. Specifically, the 0.65 power of the calculation unit characteristic value is multiplied by the 0.35 power of the network layer characteristic value to obtain the affinity value. This calculation method can balance the influence of the two characteristic values and maintain stable results on different orders of magnitude. The higher the affinity value, the better the matching degree of the network layer and the calculation unit. The affinity between all network layers and all calculation units is calculated to construct a 10x7 affinity matrix.

[0154] The predicted performance indicators include calculation delay, resource utilization, and energy efficiency level. The calculation delay is obtained by estimating the execution time of the network layer on the specific calculation unit, considering the calculation amount, memory access amount, and calculation unit processing capacity. The specific estimation method is to divide the theoretical calculation amount (floating point operation) of the network layer by the processing speed (floating point operation per second) of the calculation unit, and then consider the influence of memory access delay and data transmission delay, and predict the actual execution time by using a linear regression model. Taking the execution of the second convolution layer on the high-performance processing unit as an example, the calculation amount is 3.6 GFLOPS, considering the theoretical performance of 2.8 TFLOPS of the high-performance processing unit and the actual efficiency of 80%, and adding the memory access overhead, the estimated delay is 1.25 ms. The resource utilization is obtained by estimating the proportion of the calculation unit resources occupied by the network layer during execution, including the calculation resource utilization and the memory resource utilization. Taking the execution of the second convolution layer on the high-performance processing unit as an example, the calculation resource utilization is 82%, the memory resource utilization is 68%, and the comprehensive resource utilization is calculated according to the weights of 0.6 and 0.4 as 82% x 0.6 + 68% x 0.4 = 76.6%. The energy efficiency level is obtained by estimating the calculation throughput per unit energy consumption, expressed as the number of operations per second per watt. Taking the execution of the second convolution layer on the high-performance processing unit as an example, the power consumption is 6.2 W, the calculation throughput is 23.6 GOPS, and the energy efficiency level is 3.8 GOPS / W.

[0155] The mapping optimization objective function adopts a multi-objective weighted combination form, the affinity weight is set to 0.4, the calculation delay weight is set to 0.3, the resource utilization rate weight is set to 0.2, and the energy efficiency level weight is set to 0.1. For each mapping combination of the network layer and the calculation unit, the objective function value is calculated. In order to make different indexes comparable, it is necessary to normalize each index. The delay normalization method is to calculate the ratio of the delay reciprocal to the maximum delay reciprocal, the resource utilization rate is already a normalized value, and the energy efficiency level normalization method is to calculate the ratio to the maximum energy efficiency level. Taking the high-performance processing unit and the second convolution layer as an example, the affinity is 0.650, the calculation delay normalized value is 0.85 (the shorter the delay, the higher the normalized value), the resource utilization rate is 0.766, the energy efficiency level normalized value is 0.76, and the objective function value is 0.650*0.4+0.85*0.3+0.766*0.2+0.76*0.1=0.751. The higher the objective function value, the better the mapping combination.

[0156] Under the condition of meeting the calculation unit resource capacity constraint and the task delay constraint, the initial mapping scheme is obtained by iteratively optimizing the mapping optimization objective function. The resource capacity constraint includes the upper limit of the processing capacity and the upper limit of the memory capacity of the calculation unit; the task delay constraint includes that the end-to-end processing delay of the entire network does not exceed a preset threshold of 20 ms. The genetic algorithm is used to solve this optimization problem, the population size is set to 100, the iteration number is set to 500, the crossover probability is set to 0.8, and the mutation probability is set to 0.1. The algorithm first randomly generates 100 initial mapping schemes that meet the constraint conditions, and each scheme is a feasible network layer to calculation unit allocation scheme. Then in each iteration, the fitness of each scheme is calculated according to the objective function value, and the scheme with higher fitness is selected with a higher probability. The selected scheme generates a new scheme through crossover operation, and the crossover operation is to randomly select two parent schemes and exchange part of the mapping relationship to generate a child scheme. At the same time, the scheme is subjected to mutation operation with a probability of 0.1, and the mapping target of a certain network layer is randomly changed. In each iteration, the individual with the highest objective function value and meeting the constraint condition is selected as the candidate solution. After 500 iterations, the optimal mapping scheme is obtained: the first, fourth convolution layers and the fourth fully connected layer are mapped to the high-performance processing unit; the second, fifth convolution layers and the first fully connected layer are mapped to the first general-purpose calculation unit; the third fully connected layer is mapped to the second general-purpose calculation unit; and the third, sixth convolution layers and the second fully connected layer are mapped to the four low-power calculation units. The average value of the objective function of the scheme is 0.782, the end-to-end processing delay is 16.8 ms, and the constraint condition is met.

[0157] Actual execution data is collected every 15 seconds, including the actual load of each computing unit, the actual execution time of each network layer, and energy consumption data. Data collection is achieved through a performance monitoring module embedded in the system. The monitoring module records the start and end execution times of each network layer to calculate the actual execution delay. It also records the resource utilization of each computing unit, including computing resource occupancy, memory occupancy, and power consumption data. When a significant difference is found between actual and predicted performance, the mapping scheme is adjusted. The difference threshold is set at 15%. If the actual execution time exceeds 1.15 times the predicted time, or the actual resource utilization is less than 0.85 times the predicted utilization, a significant difference is considered to exist. The adjustment method is to recalculate the affinity and objective function and select a locally optimal adjustment scheme. To avoid fluctuations caused by frequent adjustments, a minimum adjustment interval of 30 seconds is set, i.e., at least 30 seconds between two adjustments. For example, after running for 30 seconds, it is found that the actual execution time of the 2nd convolution layer on the first general-purpose computing unit is 2.1 ms, which is 1.17 times the predicted time of 1.8 ms, triggering adjustment. The 2nd convolution layer is remapped to the high-performance processing unit, and the 4th convolution layer originally on the high-performance processing unit is mapped to the second general-purpose computing unit.

[0158] According to the adjusted mapping scheme, a resource allocation priority matrix is determined. The priority matrix has a dimension of 10x7, with rows representing network layers and columns representing computing units. The element values in the matrix represent the mapping priority of the corresponding network layer and computing unit, with values ranging from 0 to 1. The higher the value, the higher the priority. The priority matrix is constructed as follows: first, set the base priority according to the final determined mapping relationship: for the final determined mapping relationship, set the corresponding element value to 1; for the alternative mapping relationship, set a value between 0.5 and 0.9 according to its objective function value; for the unsuitable mapping relationship, set a value between 0 and 0.4. Then consider the historical execution performance: if a certain network layer has good historical execution records on a certain computing unit, appropriately increase its priority; if it has poor historical execution records, appropriately decrease its priority. For example, the priority of the 2nd convolution layer and the high-performance processing unit is set to 1, the priority of the first general-purpose computing unit is set to 0.8, and the priority of other computing units is set to 0.3. This priority matrix not only reflects the current optimal mapping scheme, but also contains information about alternative schemes, providing flexibility for dynamic adjustment.

[0159] The present application considers the matching relationship between hardware characteristics and network computing characteristics, and combines load dynamic characteristics and bit width sensitivity, to achieve accurate matching of neural networks and heterogeneous computing resources, while supporting dynamic adjustment based on real-time execution data, effectively improving overall execution efficiency and resource utilization, and providing an adaptive optimization strategy for neural network deployment in resource-constrained scenarios.

[0160] In a second aspect, the present application provides a neural network quantization compression and computing power adaptive optimization system, comprising:

[0161] A first unit is configured to obtain a pre-trained neural network, wherein the neural network comprises a plurality of convolutional layers and fully connected layers.

[0162] A second unit is configured to calculate a weight distribution entropy value and an activation value distribution entropy value for each convolutional layer and fully connected layer in the neural network, construct an importance evaluation index of neurons based on the weight distribution entropy value and the activation value distribution entropy value, wherein the importance evaluation index represents information flow and inter-layer dependence degree of the neurons, perform quantization tests with different bit widths on the convolutional layers and the fully connected layers to obtain quantization error data, combine the quantization error data and the importance evaluation index by weighting to generate an inter-layer error propagation evaluation matrix, and determine a bit width configuration scheme based on the inter-layer error propagation evaluation matrix.

[0163] A third unit is configured to obtain computing load data of a target device, and perform computing unit mapping and differential allocation of storage strategies on network layers based on the computing load data and the bit width configuration scheme.

[0164] A fourth unit is configured to adjust the importance evaluation index according to a performance parameter of the neural network until the performance parameter meets a preset optimization target, and generate an optimized neural network.

[0165] In a third aspect, the present application provides an electronic device, comprising:

[0166] A processor;

[0167] A memory configured to store processor-executable instructions;

[0168] The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0169] In a fourth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions are executed by a processor to implement the method described above.

[0170] The present application can be a method, device, system and / or computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein, which are used to perform various aspects of the present application.

[0171] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for quantization compression and adaptive optimization of computing power in neural networks, characterized in that, include: Obtain a pre-trained neural network, which contains multiple convolutional layers and fully connected layers; For each convolutional layer and fully connected layer in the neural network, the weight distribution entropy and activation value distribution entropy are calculated. Based on these entropy values, an importance evaluation index for neurons is constructed. This importance evaluation index characterizes the information flow and inter-layer dependency of neurons. Specifically, this includes: obtaining the weight data and activation value data of each convolutional layer and fully connected layer in the neural network, discretizing them to obtain the weight probability density distribution and activation value probability density distribution respectively, and calculating the corresponding entropy values; performing similarity analysis on the input and output features of neurons to identify neuron groups with similar response patterns; calculating the distribution difference within and between neuron groups, and determining the contribution ratio of each neuron to the network function based on the difference magnitude; and allocating [resources / information] to each neuron group based on the contribution ratio. The corresponding first and second balance factors are used; the product of the weight distribution entropy value and the first balance factor and the product of the activation value distribution entropy value and the second balance factor are combined through a nonlinear mapping function to obtain the hierarchical information flow; the mutual information between adjacent layers in the neural network is calculated; the hierarchical information flow and the mutual information are weighted and combined to obtain the importance evaluation index of the neuron; the importance evaluation index is verified in multiple batches based on the network output error, and the final importance evaluation index is determined when the index fluctuation is less than a preset fluctuation threshold; quantization tests with different bit widths are performed on the convolutional layer and the fully connected layer to obtain quantization error data; the quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix, and the bit width configuration scheme is determined based on the inter-layer error propagation evaluation matrix; Obtain the computing load data of the target device; based on the computing load data and the bit width configuration scheme, perform differentiated allocation of computing unit mapping and storage strategy for the network layer; The importance evaluation index is adjusted according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, thereby generating an optimized neural network.

2. The method according to claim 1, characterized in that, The steps for calculating the mutual information between adjacent layers in the neural network include: Multi-scale decomposition of feature data from adjacent layers in a neural network yields feature representations at multiple scale levels. Feature probability distributions are calculated for each scale level feature representation. Based on the feature probability distributions at each scale level, the information entropy of the previous layer and the information entropy of the next layer are calculated respectively, and the combination yields mutual information at multiple scales. The temporal feature sequences of adjacent layers are obtained, the feature probability distribution is temporally smoothed, and the temporal mutual information is calculated; the conditional feature information of adjacent layers is obtained, and the conditional mutual information is calculated; the multiple scale mutual information, temporal mutual information, and conditional mutual information are weighted and combined to obtain the initial mutual information. A causal graph of inter-layer feature transmission is constructed to identify key transmission paths and analyze the causal effect strength of different transmission channels. Based on the causal effect strength and initial mutual information, a loss matrix for feature transmission is constructed, which characterizes the degree of information retention of features during transmission. The connection weights of the corresponding channels are adjusted according to the loss values ​​of each transmission channel in the loss matrix, and the loss matrix is ​​iteratively updated until its fluctuation value is less than a preset threshold, thereby obtaining the mutual information between adjacent layers.

3. The method according to claim 1, characterized in that, The steps for obtaining quantization error data by performing quantization tests with different bit widths on the convolutional layer and the fully connected layer include: Feature importance analysis is performed on the parameters of the convolutional layer and the fully connected layer, and the parameter space is divided into critical parameter domains and non-critical parameter domains; Quantization processing is performed in the critical parameter field using a first quantization bit width configuration, and in the non-critical parameter field using a second quantization bit width configuration smaller than the first quantization bit width configuration, to obtain adaptive quantization parameters. The adaptive quantization parameters and the original floating-point parameters are used to calculate the error, and the parameter domain quantization error of different parameter domains is obtained. The influence of the quantization error of different parameter domains on the inter-layer propagation is analyzed, and the quantization sensitivity of each layer is calculated. The parameter domain quantization error is weighted according to the quantization sensitivity to obtain the quantization error data.

4. The method according to claim 1, characterized in that, The steps of weighting and combining the quantization error data with the importance evaluation index to generate an inter-layer error propagation evaluation matrix, and determining the bit width configuration scheme based on the inter-layer error propagation evaluation matrix include: The local features between adjacent layers, the mid-range features between consecutive multi-layers, and the global features of the entire network are calculated respectively. The local features are obtained by weighted combination of quantization error data and importance evaluation index of adjacent layers. The mid-range features are obtained by weighted combination of local features of each layer within a feature block. The global features are obtained by weighted combination of mid-range features of each layer. Construct an inter-layer error propagation evaluation matrix based on local features, mid-range features, and global features; The error compensation value for each layer is calculated based on the inter-layer error propagation evaluation matrix. The error compensation value is determined by the weighted sum of the quantization error of the current layer and the propagation error of the previous layer. The bit width configuration scheme is obtained by adaptively adjusting the reference bit width of each layer based on the error compensation value. Calculate the network performance evaluation value, update the feature combination weight coefficients in the construction process of the inter-layer error propagation evaluation matrix based on the network performance evaluation value, and reconstruct the inter-layer error propagation evaluation matrix until the network performance evaluation value meets the preset optimization objective.

5. The method according to claim 1, characterized in that, The steps for performing differentiated allocation of computing unit mapping and storage strategies for the network layer based on the computing load data and the bit width configuration scheme include: The computational load data is combined using a time-series weighted method to obtain dynamic load characteristic values; the bit-width sensitivity is obtained by calculating the rate of change of resource requirements for each layer of the neural network based on the bit-width configuration scheme. A resource allocation priority matrix is ​​constructed based on the load dynamic characteristic value and the bit width sensitivity. Each element of the resource allocation priority matrix is ​​obtained by weighted combination of the bit width sensitivity of the corresponding network layer and the load dynamic characteristic value of the corresponding computing unit, which is used to determine the mapping relationship between the network layer and the computing unit. Importance analysis is performed on intermediate feature maps of the neural network, and an importance score is obtained based on the information distribution of the feature maps in the spatial and channel dimensions. Based on the importance score, the feature maps are divided into multiple importance levels, and a differentiated storage strategy is adopted for feature maps of different importance levels. The mapping relationship and the storage strategy are jointly optimized based on a weighted combination of computational efficiency, storage efficiency, and communication efficiency.

6. The method according to claim 5, characterized in that, The steps for constructing a resource allocation priority matrix based on the load dynamic characteristic value and the bit width sensitivity include: The computing characteristic matching degree is calculated based on the hardware characteristic parameters of the computing unit and the computing characteristic parameters of the network layer; the computing unit characteristic value is obtained by weighted combination of the load dynamic characteristic value and the computing characteristic matching degree; the network layer characteristic value is obtained by weighted combination of the bit width sensitivity and the resource utilization of the neural network layer; the affinity between the network layer and the computing unit is calculated based on the computing unit characteristic value and the network layer characteristic value, and the affinity is used to characterize the degree of matching between the network layer and the computing unit. Construct predictive performance metrics, including computational latency, resource utilization, and energy efficiency; establish a mapping optimization objective function based on the affinity and the predictive performance metrics; obtain an initial mapping scheme by iteratively optimizing the mapping optimization objective function under the conditions of satisfying the resource capacity constraints of computing units and task latency constraints; collect real-time execution data, adjust the mapping scheme online based on the execution data, and determine the resource allocation priority matrix.

7. A quantization compression and adaptive optimization system for neural networks, used to implement the method of any one of claims 1-6, characterized in that, include: The first unit is used to obtain a pre-trained neural network, which includes multiple convolutional layers and fully connected layers; The second unit is used to calculate the weight distribution entropy value and the activation value distribution entropy value for each convolutional layer and fully connected layer in the neural network, and to construct an importance evaluation index for neurons based on the weight distribution entropy value and the activation value distribution entropy value. The importance evaluation index characterizes the information flow of neurons and the degree of inter-layer dependence. Quantization error data is obtained by performing quantization tests with different bit widths on the convolutional layer and the fully connected layer. The quantization error data and the importance evaluation index are weighted and combined to generate an inter-layer error propagation evaluation matrix, and the bit width configuration scheme is determined based on the inter-layer error propagation evaluation matrix. The third unit is used to acquire the computing load data of the target device; Based on the computational load data and the bit width configuration scheme, the network layer performs differentiated allocation of computational unit mapping and storage strategies; The fourth unit is used to adjust the importance evaluation index according to the performance parameters of the neural network until the performance parameters meet the preset optimization target, and generate the optimized neural network.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images