Self-learning-based self-adaptive mixed precision quantization network generation method

Abstract

The invention relates to the technical field of model quantification, and discloses a self-learning-based self-adaptive mixed precision quantification network generation method, which comprises the following steps: obtaining a teacher bit width set and a to-be-trained student element network, the teacher bit width set comprising a plurality of candidate teachers with high bit widths, and the to-be-trained student element network comprising a plurality of candidate teachers with high bit widths; the student element network shares a full-precision weight and supports various bit width configurations; for any bit width configuration, determining a corresponding target teacher according to an interlayer Manhattan distance between the student element network and each candidate teacher and an entropy value of prediction probability distribution of each candidate teacher; performing joint training on all bit width configurations of the student element network to obtain a self-adaptive bit width student element network; determining a target bit width of each layer of network by using a bit width search strategy based on an inter-layer importance evaluation result; and based on the target bit width of each layer of network and the self-adaptive bit width student element network, a target mixed precision quantization network is generated, and the technical problem of low diagnosis accuracy of a related model in power equipment state monitoring is solved.

Description

Technical Field

[0001] The present application relates to the field of model quantization technology, and in particular to a method for generating an adaptive mixed-precision quantization network based on self-learning. Background Art

[0002] As power systems continue to expand in size and complexity, monitoring, warning, and addressing various power failures and safety hazards are becoming increasingly difficult. Deep learning technology, due to its powerful data processing and pattern recognition capabilities, has been widely applied in power system safety monitoring and fault diagnosis. For example, by building deep neural network models to analyze operational data from power equipment, potential equipment failures can be predicted in advance, ensuring reliable power system operation. However, the enormous amount of data generated by power systems and the presence of a large number of resource-constrained devices, such as edge computing nodes and smart meters distributed throughout the power network, place extremely high demands on the computational complexity and storage requirements of deep learning models. Network model quantization technology aims to reduce the computational complexity and storage requirements of deep neural networks to adapt to resource-constrained devices and scenarios. Traditional quantization methods train for a single fixed bit width. When quantized to other bit widths, performance degrades significantly, limiting the model's flexibility to meet varying accuracy requirements. To support multiple bit widths, multiple training iterations are typically required, with multiple copies of the model trained to different bit widths. This is not only time-consuming and labor-intensive, but also increases storage costs. Adaptive quantization methods have gained increasing attention in recent years, allowing for flexible selection of the bit width of deep neural networks during inference to meet dynamically changing requirements. However, existing adaptive bit-width quantization methods often use high bit widths during training, resulting in poor performance for low bit-width quantization, making it unable to meet the high-precision requirements of power equipment condition monitoring. Summary of the Invention

[0003] The present application provides a method for generating an adaptive mixed-precision quantization network based on self-learning, which solves the technical problem of low diagnostic accuracy of related lightweight models in power equipment status monitoring, and achieves the technical effect of improving the accuracy of power equipment status monitoring.

[0004] In order to achieve the above objectives, the main technical solutions adopted in this application include: In the first aspect, the embodiment of the present application provides a method for generating an adaptive mixed precision quantization network based on self-learning, which is applied to power equipment status monitoring. The method comprises: obtaining a teacher bit width set and a student meta-network to be trained, wherein the teacher bit width set includes multiple candidate teachers with high bit widths, and the student meta-network shares full-precision weights and supports multiple bit width configurations; for any bit width configuration, the corresponding target teacher is determined according to the inter-layer Manhattan distance between the student meta-network and each candidate teacher and the entropy value of the predicted probability distribution of each candidate teacher; all bit width configurations of the student meta-network are jointly trained to obtain an adaptive bit width student meta-network. The invention relates to a network, comprising: determining a training loss between a student meta-network and a target teacher for any bit-width configuration, obtaining a joint loss function by weighted summing the training losses under all bit-width configurations, and updating the shared full-precision weights based on the joint loss function; generating an inter-layer importance evaluation result of each layer of the network based on normalized parameter ratio, normalized entropy, normalized variance, and normalized activation sparsity; determining a target bit-width of each layer of the network based on real-time device status data and the inter-layer importance evaluation result; and generating a target mixed-precision quantization network responsive to the real-time device status based on the target bit-width of each layer of the network and the adaptive bit-width student meta-network.

[0005] The method for generating an adaptive mixed precision quantization network based on self-learning proposed in the embodiment of the present application is applied to the state monitoring of power equipment. By combining the joint loss of adaptive knowledge distillation training progress, a target teacher selection method based on the Manhattan distance between layers and the entropy value of the predicted probability distribution of each candidate teacher is proposed. The most suitable teacher network is adaptively selected for each bit width, which significantly improves the performance of low bit width quantization and optimizes the training process. By using the normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity to calculate the importance of each layer, an inter-layer importance evaluation scheme is formulated, and based on the inter-layer importance evaluation results and real-time equipment status data, the target bit width of each layer network is determined. Based on the target bit width of each layer network and the adaptive bit width student meta-network, a target mixed precision quantization network that responds to the real-time equipment status is generated, which solves the technical problem of low diagnostic accuracy of related lightweight models in power equipment state monitoring and achieves the technical effect of improving the accuracy of power equipment state monitoring.

[0006] Optionally, the method further includes: obtaining historical device status data, and extracting multidimensional features from the historical device status data, including device operating status indicators, abnormal feature patterns, and time-series change trends; generating a device status-bit width mapping strategy table based on the multidimensional features, wherein the device status-bit width mapping strategy table is used to characterize the configuration strategies of each layer of bit width corresponding to different power equipment states; matching the real-time device status data with the device status feature vector in the device status-bit width mapping strategy table to determine the target bit width of each layer of the network.

[0007] Optionally, the method also includes: analyzing the real-time indicators and fluctuation characteristics of historical equipment status data, and evaluating the real-time strength and abnormal fluctuation frequency of the data based on the real-time indicators and the fluctuation characteristics; if the real-time nature of the data is strong and the abnormal fluctuation frequency is high, shortening the probability and reducing the round interval to speed up the conversion rate; otherwise, extending the round interval to slow down the conversion rate, so as to dynamically adapt the training probability to the real-time demand of electricity.

[0008] Optionally, the training loss is a weighted combination of labeled data supervision loss and knowledge distillation loss, wherein the labeled data supervision loss is a cross entropy loss and the distillation loss is a KL divergence loss, and the network parameters are synchronously updated through gradient accumulation.

[0009] Optionally, the method further includes: dynamically adjusting the weight coefficients of the cross entropy loss and the KL divergence loss using a linear annealing strategy: focusing on the cross entropy loss to establish the category discrimination boundary at the beginning of training, and gradually increasing the weight coefficient of the KL divergence loss as the training progresses.

[0010] Optionally, the method further includes: dynamically balancing the weights of the entropy value and the inter-layer Manhattan distance through hyperparameters, and linearly adjusting the hyperparameters as the training progresses to determine a target teacher corresponding to the current bit width configuration.

[0011] Optionally, the method further includes: using a quantization function with a floor operator in the training process to learn quantization steps for weights and activations of each layer of the network respectively.

[0012] Optionally, the method also includes: independently learning a dedicated BN layer for each pair of adjacent layer bit width transition combinations; quantizing the distribution offset caused by the bit width change between consecutive layers by calculating the activation mean and variance difference before and after quantization; and compensating for the distribution offset by dynamically adjusting the scaling parameters and offset parameters of the BN layer.

[0013] Optionally, based on the inter-layer importance evaluation result, a bit width search strategy is used to determine the target bit width of each layer of the network, including: arranging the layers in descending order of layer importance according to the inter-layer importance evaluation result, and performing the following operations on each layer: initializing the bit width to the highest quantization bit width supported by the current layer, and iteratively reducing it to a lower bit width at a fixed step size while ensuring that the accuracy loss does not exceed a set threshold, so as to determine the target bit width of the current layer.

[0014] Optionally, the method also includes: performing a diagnostic accuracy test on the network with the current bit width configuration on the power equipment fault data set to obtain the accuracy loss of the current bit width; generating the set threshold value for the accuracy loss based on the safety level of the power equipment and the fault diagnosis accuracy requirements, and in the process of adjusting the bit width configuration from high to low, if the adjusted accuracy loss exceeds the set threshold value, the bit width reduction is terminated and the previous bit width configuration is retained.

[0015] In the second aspect, the embodiment of the present application provides a generation system of an adaptive mixed precision quantization network based on self-learning, which is applied to power equipment status monitoring. The system includes: an acquisition module for acquiring a teacher bit width set and a student meta-network to be trained, wherein the teacher bit width set includes multiple high-bit width candidate teachers, and the student meta-network shares full-precision weights and supports multiple bit width configurations; a training module for determining the corresponding target teacher for any bit width configuration based on the inter-layer Manhattan distance between the student meta-network and each candidate teacher and the entropy value of the predicted probability distribution of each candidate teacher; and for jointly training all bit width configurations of the student meta-network to obtain an adaptive bit width student network. The meta-network includes: for any bit-width configuration, determining the training loss between the student meta-network and the target teacher, weighted summing the training losses under all bit-width configurations to obtain a joint loss function, and updating the shared full-precision weights based on the joint loss function; a generation module for generating inter-layer importance evaluation results of each layer of the network based on normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity; determining the target bit-width of each layer of the network based on real-time device status data and the inter-layer importance evaluation results; and generating a target mixed-precision quantization network that responds to the real-time device status based on the target bit-width of each layer of the network and the adaptive bit-width student meta-network.

[0016] In a third aspect, an embodiment of the present application provides a computer device comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the computer instructions to execute the above-mentioned method for generating an adaptive mixed-precision quantization network based on self-learning.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above-mentioned method for generating an adaptive mixed-precision quantization network based on self-learning.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the above-mentioned method for generating an adaptive mixed-precision quantization network based on self-learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the specific implementation methods or the description of the prior art. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of a method for generating an adaptive mixed-precision quantization network based on self-learning provided in an embodiment of the present application; Figure 2 A schematic diagram of a system for generating an adaptive mixed-precision quantization network based on self-learning provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solutions, and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0022] In modern power systems, ensuring a secure and stable power supply is crucial. As power systems continue to expand in size and complexity, monitoring, warning, and addressing various power failures and safety hazards are becoming increasingly difficult. Deep learning technology, due to its powerful data processing and pattern recognition capabilities, has been widely used in areas such as power system safety monitoring and fault diagnosis. For example, by building deep neural network models to analyze the operating data of power equipment, potential equipment failures can be predicted in advance, ensuring the reliable operation of the power system. However, the amount of data generated by power systems is enormous, and the operating environment contains a large number of resource-constrained devices, such as edge computing nodes and smart meters distributed throughout the power network. This places extremely high demands on the computational complexity and storage requirements of deep learning models. The quantization technology of network models aims to reduce the computational complexity and storage requirements of deep neural networks to adapt to resource-constrained devices and scenarios. Traditional quantization methods are trained for a single fixed bit width. When quantized to other bit widths, the performance will drop significantly, which limits the flexibility of the model under different precision requirements. In order to support multiple bit widths, it is usually necessary to repeat the training multiple times and train multiple copies of the model into models with different bit widths. This is not only time-consuming and labor-intensive, but also increases the storage cost. In recent years, adaptive quantization methods have gradually attracted attention. They allow the bit width of deep neural networks to be flexibly selected during inference to meet dynamically changing requirements. In particular, after training, weights and activations can be freely quantized to various precision levels without additional fine-tuning or calibration. However, existing adaptive bit width quantization methods still have some problems: (1) When different bit widths share weights for joint training, it is impossible to balance the optimization difficulty between different precisions. High precision often dominates in training, resulting in poor performance of low bit width quantization and unable to meet the high precision requirements of power equipment status monitoring. (2) Although the current adaptive bit-width quantization method supports multi-precision switching, it requires that all layers of the network must adopt a unified bit-width configuration, and can only use the same bit-width throughout the network. If attempts are made to implement mixed-precision quantization (i.e., different layers use different bit-widths), the distribution shift will occur when the bit-width changes between consecutive layers, which limits its flexibility in practical scenarios. (3) Existing mixed-precision network quantization requires a complex and computationally expensive optimization process to determine the optimal bit-width for each layer, which is not suitable for adaptive bit-width quantization scenarios.

[0023] In recent years, with the widespread application of deep learning in fields such as computer vision and image processing, model optimization has become a research hotspot. Among numerous optimization techniques, quantization has attracted considerable attention for its ability to effectively reduce model storage requirements and computational complexity, thereby improving model efficiency. To further improve quantization, researchers have explored various approaches and techniques to reduce quantization error, employing diverse methods and techniques from various perspectives. Mixed-precision quantization (MPQ), a key branch of quantization, aims to reduce model size and computational requirements while maintaining or even improving model accuracy by applying different bit widths to weights and activations. Early research relied on empirical, manual determination of bit width combinations, but this approach has limitations. Consequently, mixed-precision quantization has emerged, and three main approaches have emerged: search-based methods combine neural architecture search (NAS), reinforcement learning (RL), or evolutionary algorithms to explore the optimal bit width for each layer; model-feature-based methods guide the quantization process by analyzing the importance of network layers or their sensitivity to quantization; and learning-based methods treat bit width as a learnable parameter, determining the optimal bit width for each layer through training and fine-tuning. These methods have achieved remarkable results in optimizing model performance and compression efficiency. In order to maximize the efficiency and deployability of deep neural networks in resource-constrained environments, the combination of quantization technology and other compression technologies has become a research trend. Weight sharing is combined with quantization to reduce the number of unique parameters in the model through clustering technology, and quantize the shared weights to fewer bits, thereby reducing storage requirements and computational workload. Knowledge distillation is combined with quantization to train a smaller "student" model to imitate the pre-trained "teacher" model. The student model uses lower-precision weights and activations while retaining most of the accuracy of the teacher model. In addition, the hybrid compression strategy combines multiple techniques such as pruning, weight sharing and quantization. The sparsity introduced by pruning ensures that the quantization operation is more efficient, resulting in faster inference time and reduced memory usage. While these methods have improved the quantization performance and compression efficiency of the models to a certain extent, existing research still faces some challenges. For example, mixed-precision quantization methods are computationally expensive when determining the optimal quantization bit width for each layer, and most feature extraction methods are based on fixed bit width configurations, ignoring the important connections between layers.

[0024] An embodiment of the present application provides a method for generating an adaptive mixed-precision quantization network based on self-learning. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] Please refer to Figure 1 , Figure 1A schematic diagram of a method for generating a self-learning adaptive mixed precision quantization network according to an embodiment of the present application is provided, which is applied to power equipment status monitoring, such as Figure 1 As shown, the following steps are included: Step S100: obtaining a teacher bit width set and a student meta-network to be trained, wherein the teacher bit width set includes a plurality of candidate teachers with high bit widths, and the student meta-network shares full-precision weights and supports a variety of bit width configurations.

[0026] Among them, the teacher bit width set consists of high-bit width candidate teachers with different bit widths, which are used to train and guide the low-precision student meta-network.

[0027] Step S300: For any bit-width configuration, determine the corresponding target teacher based on the inter-layer Manhattan distance between the student meta-network and each candidate teacher and the entropy value of the predicted probability distribution of each candidate teacher; jointly train all bit-width configurations of the student meta-network to obtain an adaptive bit-width student meta-network, including: for any bit-width configuration, determine the training loss between the student meta-network and the target teacher, weightedly sum the training losses under all bit-width configurations to obtain a joint loss function, and update the shared full-precision weight based on the joint loss function.

[0028] Specifically, the Manhattan distance between the candidate teacher and each layer of the student meta-network is calculated to measure the similarity between the candidate teacher and each layer of the network. The smaller the similarity, the closer the weight distribution of the two, and the higher the knowledge transfer efficiency. The entropy of the predicted probability distribution output by the candidate teacher can be used to assess the candidate teacher's prediction confidence; the smaller the entropy value, the higher the prediction confidence. A hyperparameter is used to balance the weights of the two (i.e., Manhattan distance and entropy of the predicted probability distribution) in the teacher selection process. This hyperparameter is dynamically adjusted according to the training progress to optimize the adaptive bitwidth network quantization training process and select a target teacher for the current bitwidth student meta-network. This allows each layer of the student meta-network to learn more adaptive knowledge during training, providing a basis for subsequent inter-layer importance assessment.

[0029] Step S500: Based on the normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity, generate the inter-layer importance evaluation results of each layer of the network; based on the real-time device status data and the inter-layer importance evaluation results, determine the target bit width of each layer of the network; based on the target bit width of each layer of the network and the adaptive bit width student meta-network, generate a target mixed precision quantization network that responds to the real-time device status.

[0030] Layer importance assessment comprehensively evaluates the importance of each layer using four metrics to avoid artificial bias. These metrics are normalized parameter ratio, normalized entropy, normalized variance, and normalized activation sparsity. The resulting layer importance score is not a simple weighted average, but is dynamically adjusted using a learnable weight multiplier. This weight multiplier is determined based on the importance of the knowledge learned by each layer during the self-learning process. Normalized parameter ratio refers to the ratio of the number of parameters in a layer to the total number of parameters in the model. Normalized entropy assesses the information richness of the layer's parameters. Higher entropy values indicate a more chaotic parameter distribution and a more sensitive layer (requiring a high bitwidth for accuracy). Normalized variance characterizes the feature diversity of the layer's parameters. Layers with higher variance carry more key features and are more sensitive to quantization. Normalized activation sparsity measures the redundancy of the layer's output. Layers with higher sparsity can tolerate low bitwidth quantization. Normalization essentially compresses the values of the four metrics to between 0 and 1, facilitating weighted summation. The bitwidth search strategy uses a greedy algorithm to generate the solution with the optimal bitwidth cost. Starting from the most important layer, while ensuring that the accuracy loss is within the threshold range, fine-tune from the maximum possible bit width to the lower bit width to determine the optimal bit width accuracy for each layer.

[0031] The method for generating an adaptive mixed precision quantization network based on self-learning proposed in the embodiment of the present application is applied to power equipment status monitoring. By combining the joint loss of knowledge distillation training progress adaptation, a target teacher selection method based on the Manhattan distance between layers and the entropy value of the predicted probability distribution of each candidate teacher is proposed. The most suitable teacher network is adaptively selected for each bit width, and the quantization strategy configuration is automatically generated using a dynamic learning process, which significantly improves the performance of low bit width quantization and optimizes the training process. By using normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity to calculate the importance of each layer, an inter-layer importance evaluation scheme is formulated, and based on the inter-layer importance evaluation results and real-time equipment status data, the target bit width of each layer network is determined. Based on the target bit width of each layer network and the adaptive bit width student meta-network, a target mixed precision quantization network that responds to the real-time equipment status is generated, and the optimal bit width is allocated to each layer of the neural network. While ensuring accuracy, the compression efficiency is maximized, solving the technical problem of low diagnostic accuracy of related lightweight models in power equipment status monitoring, and achieving the technical effect of improving the accuracy of power equipment status monitoring.

[0032] In some embodiments, historical device status data is obtained, and multidimensional features in the historical device status data are extracted, including device operating status indicators, abnormal feature patterns, and time-series change trends; based on the multidimensional features, a device status-bit width mapping strategy table is generated, and the device status-bit width mapping strategy table is used to characterize the configuration strategy of each layer of bit width corresponding to different power equipment states; the real-time device status data is matched with the device status feature vector in the device status-bit width mapping strategy table to determine the target bit width of each layer of the network.

[0033] In power system security monitoring and fault diagnosis, real-time analysis of equipment status data is crucial for fault early warning. Based on equipment operating status indicators, abnormal characteristics, and the temporal trends of equipment parameters, a pre-generated equipment status-bitwidth mapping strategy table is used to adjust the quantization bit widths of each layer in real time, achieving the goal of reducing costs in normal conditions and maintaining accuracy in abnormal conditions.

[0034] Specifically, historical device status data is first analyzed to construct a multidimensional feature library containing device operating status, abnormal characteristic patterns, and temporal change trends. A device status-bitwidth mapping strategy table is pre-generated to characterize the optimal quantization bitwidth configuration for each network layer under different operating scenarios. For example, during normal operation, a low bitwidth is used to reduce computational overhead; during abnormal fluctuations, a high bitwidth is automatically switched to capture detailed features. During real-time monitoring, the system collects current device data in real time and matches it with the mapping strategy table. Based on the matching results, the corresponding bitwidth configuration strategy is quickly invoked, dynamically adjusting the quantization bitwidth of each layer. This optimizes computing resource allocation while ensuring fault diagnosis accuracy, enabling rapid response to power failures and safety hazards.

[0035] In some embodiments, the real-time indicators and fluctuation characteristics of historical equipment status data are analyzed, and the real-time strength and abnormal fluctuation frequency of the data are evaluated based on the real-time indicators and the fluctuation characteristics; if the real-time nature of the data is strong and the abnormal fluctuation frequency is high, the probability is shortened and the round interval is reduced to speed up the conversion rate; otherwise, the round interval is extended to slow down the conversion rate, so as to dynamically adapt the training probability to the real-time demand of electricity.

[0036] In some embodiments, traditional quantization struggles to retain the characteristics of real-time data such as voltage and current collected from end-to-end grid devices, as well as information such as device status and fault alarms, due to the strong temporal nature and noise interference of this data. By introducing knowledge distillation, a trained high-precision model acts as a teacher, transferring its knowledge of voltage fluctuation trends and stability thresholds to the adaptive bitwidth quantization student model under training. This optimizes the training process and improves the model's adaptability to complex grid conditions.

[0037] In some embodiments, the training loss is a weighted combination of labeled data supervision loss and knowledge distillation loss, wherein the labeled data supervision loss is a cross entropy loss and the distillation loss is a KL divergence loss, and the network parameters are synchronously updated through gradient accumulation.

[0038] To minimize the quantization loss, QAT is defined as an optimization problem. The optimization goal is to minimize the discretization error and the difference between the model prediction and the true label, and achieve the optimal quantitative representation while ensuring the prediction accuracy. The optimization goal is defined as follows: In the above formula, is the training loss; is the original unquantized real-valued weight, which is the variable that needs to be adjusted during the optimization process; is a real-valued weight The quantized weights are obtained after the quantization operation. By adjusting the unquantized real-valued weights , so that the loss function To minimize the error, the model is optimally quantized while ensuring the prediction accuracy, thus reducing the discretization error and the difference between the model prediction and the true label.

[0039] The training loss can be a labeled cross entropy loss or other loss functions, such as distillation loss. Transferring the knowledge of a large parameter network to supervise the training of a small neural network through knowledge distillation can significantly improve the performance, sometimes even exceeding the performance of the large network. The training loss is defined as a weighted combination of the labeled data supervision loss and the knowledge distillation loss, with the parameter Control weights, the training loss in the equation is defined as: In the above formula, It can be a single distillation loss or a multi-distillation loss, which converts the cross entropy loss (called ) and the KL divergence loss between the candidate teacher and the student model (called ) into the equation, where quantifies the alignment between the predictions of the lower-order network and the true labels, while It measures the similarity between the lower network and the higher network: and are the soft labels provided by the teacher model and the output of the student model, respectively. During training, the teacher's weights are frozen and it only performs forward propagation. In the forward pass, the student's parameters are quantized, while the corresponding full-precision values are retained internally. During backpropagation, the student's retained full-precision values are gradually updated. After convergence, the student retains its full-precision weights and the parameters used in the quantizer. To achieve bitwidth adaptation, during joint training, given a bitwidth set , synchronously calculate each quantitative configuration The total loss is expressed as , bit width The precision loss is expressed as ,in Therefore, the joint training loss function can be obtained : In the above formula, Represents the probability distribution of the current model output under accuracy b.

[0040] In some embodiments, a linear annealing strategy is used to dynamically adjust the weight coefficients of the cross entropy loss and the KL divergence loss: in the early stages of training, emphasis is placed on the cross entropy loss to establish the category discrimination boundary, and as the training progresses, the weight coefficient of the KL divergence loss is gradually increased.

[0041] In the process of deep learning model training, the dynamic adjustment of the weight distribution strategy of the loss function has a significant impact on the optimization of model performance. In the early stage of training, when the model has not yet fully captured the data distribution characteristics, the cross entropy loss is preferentially used as the dominant optimization objective. To establish the initial category discrimination boundary, at this stage, the model parameters are in a rapid adjustment period, and by minimizing the prediction error, the model can quickly fit the overall distribution characteristics of the training data. When the model enters the late convergence stage, that is, close to or reaches the convergence state, relying solely on the prediction error for optimization may cause the model to fall into a local optimal solution or even overfitting. Therefore, the knowledge divergence loss is gradually increased during the training process. This method can not only improve the convergence efficiency of the model, but also effectively balance the model's fitting ability and generalization performance. Update, and set the total number of training rounds to , the cross entropy loss has a fixed weight of , using the linear annealing strategy adjustment, we have: in, , is a fixed weight for the cross entropy loss, is the total number of training rounds.

[0042] In some embodiments, the weights of the entropy value and the inter-layer Manhattan distance are dynamically balanced through hyperparameters, and the hyperparameters are linearly adjusted as the training progresses to determine the target teacher corresponding to the current bit width configuration.

[0043] In quantized neural network training, high-precision networks are able to capture richer input feature representations due to their larger model capacity, while low-precision networks trade computational precision for model compression advantages. However, when selecting a teacher, the more complex the model, the higher the performance. Specifically, the gap in representational capabilities between low-precision students and high-precision teachers leads to reduced knowledge transfer efficiency. This phenomenon has been verified in multiple studies and essentially violates the ability matching assumption in knowledge distillation, which states that the representation spaces of teachers and students must maintain a moderate overlap to achieve effective knowledge transfer. When the accuracy of the teacher network is much higher than that of the student model, the difference in feature space distribution between the two will significantly reduce the distillation efficiency. The difference in representational capabilities between the teacher and the student must satisfy the following requirements: In the above formula, 、 are the ability indicators of teachers and students respectively. is the threshold, otherwise the students cannot effectively imitate the teacher.

[0044] By dynamically selecting the optimal teacher network, knowledge is transferred from the existing high-precision teacher branch to the low-precision student branch, while jointly optimizing a single network with all bit widths. Specifically, during training, given a set of quantization bit widths: For bit width , by balancing the confidence of the prediction and the distance in the model space, adaptively selecting from the teacher bit width set according to the current input Choose the best teachers.

[0045] Given higher precision As bit width The closer the distance is to a candidate teacher, the higher the similarity between the two networks, and the more suitable it is to be the target teacher. The farther the distance is, the more difficult it is to effectively learn the knowledge of the teacher network. Although it is usually difficult to measure the distance in the model space, it is particularly easy for adaptive bit width networks due to shared weights. Inspired by physical distance, Manhattan distance is used to calculate the distance between each layer as an importance evaluation strategy for teacher selection. For the convolution layer, the weight tensors of the teacher network and a convolution layer of the student model are , ,in, is the number of input channels, is the number of output channels, Is the size of the convolution kernel, then calculate the student filter and teacher filters The weighted distances between are as follows: in, For filter No. A specific weight value for the channel weight matrix; and is the subscript number of the traversal filter matrix. Based on the above formula, the average distance of all filters in the same layer of the teacher network and the student model can be calculated As the weight distance of this layer, the closer the distance between layers, the higher the similarity between the two networks, and the more suitable it is to be used as a teacher network. The specific calculation is as follows: For the fully connected layer, the weight matrices of the teacher network and the student model are , ,in is the input feature dimension, is the number of output units, and the student neuron weight vector is calculated and the teacher neuron weight vector The weighted distances between are as follows: Based on the above formula, the average distance between all neurons in the same fully connected layer of the teacher network and the student model can be calculated As the weight distance of this layer, the closer the distance between layers, the higher the similarity between the two networks, and the more suitable it is to be used as a teacher network. The specific calculation is as follows: Assuming that the fully connected layer is equally important as the convolutional layer, the distance between two quantized networks is defined as: in, is the number of layers, and The smaller the distance between the two networks, the more closely the teacher network matches the student network in terms of spatial filtering characteristics, and the more effective its multi-scale feature transfer is. However, the model spatial distance alone is not enough to select the most suitable teacher. The entropy of the probability distribution of the teacher network after softmax is used. To evaluate the prediction confidence of the teacher model, the greater the entropy of the probability distribution of the model output, the more uncertain the prediction. There is a trade-off between prediction confidence and model distance. A network with higher accuracy generally has greater capacity and gives more reliable predictions, while being further away from a network with lower accuracy. Therefore, the optimal teacher for the current batch is selected as follows: in, Is a bit-wide The probability distribution of the teacher network, It is a hyperparameter for balancing trade-offs and is dynamically adjusted using a linear annealing strategy. The specific calculation formula is: in, is the maximum value of the hyperparameter, is the minimum value of the hyperparameter, is the current training round, is the total number of training rounds. When , the selection is biased towards the highest accuracy. When it is very large, it tends to choose the closest model. Experiments have found that the student model's preference for teachers also changes during training. Initially, the performance of higher precision models improves faster, and the student model tends to choose higher precision. As training progresses, the difference between different accuracies will be reduced, and it will favor closer accuracies. Therefore, the dynamic teacher selection strategy is more adaptable to the current input and training progress than manually fixed teachers. Dynamic teacher selection is based on the joint evaluation of model space distance and prediction confidence. For the target bit width of each training batch, the total distance between networks is obtained by calculating the cumulative distance of the weights of each layer from the teacher bit width set. Combined with the teacher's predicted entropy, the weights of the two are balanced by hyperparameters to dynamically select the optimal teacher. In the early stages of training, a teacher network with high bit width and high confidence is prioritized to quickly guide the low-precision network to establish initial performance. As training progresses, the model space distance and prediction confidence between the teacher network and the student model are gradually balanced, and a teacher network closer to the current student model bit width is selected to achieve efficient knowledge transfer. In the later stages of training, more emphasis is placed on model space distance, and a teacher network with a bit width similar to that of the student model is selected for fine-tuning, thereby achieving the minimum bit width requirement while maintaining high accuracy. This dynamic selection method can effectively improve the performance of the low-precision network, reduce the negative impact of the high-precision network on it, and achieve efficient knowledge transfer and model optimization.

[0046] In some embodiments, the training process uses a quantization function with a floor operator to learn the quantization step size for the weights and activations of each layer of the network.

[0047] The activation value distribution of power grid data varies significantly at different bit widths. For example, voltage data fluctuates slightly and has a concentrated distribution during normal operation, but changes dramatically and has a wide distribution during faults. In the embodiments of this application, low-precision quantization is used for basic data such as load forecasts, which change slowly, to reduce computational and storage overhead. High-precision quantization is used for real-time data such as current and voltage at the moment of a fault, capturing data details and achieving flexible and efficient mixed-precision quantization.

[0048] First, for a given layer , the weight and input activation The quantitative expression of and ,in Represents the bit width. The quantization function for the bit width is defined as: in is the floor operator, Represents a learnable scalar that defines the quantization step size, similar to the LSQ method. For each layer and bit width, two such scalars are learned, one for weights and the other for activations. The rounding function used in the LSQ method is Replace with round down. This allows you to directly get Get , without having to store full-precision weights, significantly reducing model storage requirements, whose size is only determined by the given highest-precision quantized network Size determines.

[0049] In some embodiments, a dedicated BN layer is independently learned for each pair of adjacent layer bit width transition combinations; the distribution offset caused by the bit width change between consecutive layers is quantized by calculating the activation mean and variance differences before and after quantization; and the distribution offset is compensated by dynamically adjusting the scaling parameters and offset parameters of the BN layer.

[0050] Based on the switchable batch normalization layer, a dedicated batch normalization layer (BN layer) is independently learned for each pair of adjacent layer bit width transition combinations to achieve dynamic alignment of feature distribution. Specifically, assuming there is a set of bit width candidates ,save The parameters and internal state copies of the BN layer are recorded as .if It is a layer The bit width, It is a layer bit width, learning parameter and .parameter and Still with layers Bit width are related because they depend only on the current quantization level and have nothing to do with the weights of the layer, as shown in the formula below: In the above formula, is the output activation value after the transition batch normalization layer; is the original activation value input to the current transition batch normalization layer; for The mean value in the current batch is used to center the input; for The variance in the current batch is used to measure the dispersion of the activation values.

[0051] The meta-network is trained using a two-stage approach. The process can be summarized as follows: Phase 1: In this phase, the network weights are kept real-valued. In each mini-batch, the quantized forward propagation of all bit widths is traversed, and the training losses of all bit widths are accumulated based on the knowledge distillation method. After that, the parameters are updated by backpropagation. The network trained by the adaptive bit width quantization study belongs to this phase. Phase 2: By continuing the training process from the previous checkpoint, the probability , trained in the same way as described in stage 1 (i.e. using the same bit width in all layers). , the bit width of each individual layer are randomly selected and independent of each other, thus forming a network with different layers using different bit widths. As training progresses, , effectively increasing the chances of training the meta-network with hierarchical random bit allocation. This process continues until ,in Usually .

[0052] In some embodiments, the importance of each layer is evaluated by analyzing the degree of influence of each layer on the calculation results, and different quantization precision combinations are tried to generate multiple mixed-precision sub-networks. Then, the historical data of power grid fault diagnosis is used for testing. A comprehensive evaluation is performed based on indicators such as fault identification accuracy and diagnosis speed, and finally the mixed-precision sub-network with the best performance under the average bit width is determined.

[0053] First, the number of parameters in each layer affects the overall model size. Layers with more parameters are prioritized for quantization to reduce model size without affecting accuracy. Considering the normalized parameter ratio, the calculation formula is: in, Indicates the The normalized parameter ratio of a layer is used to measure the proportion of the number of parameters of this layer in the total number of model parameters, reflecting the contribution of this layer to the model size. Indicates the The number of parameters of the layer, that is, the total number of learnable weight parameters in the neural network of this layer. It represents the total number of parameters of the entire model, that is, the sum of the number of parameters in all layers. It is used to normalize the number of parameters in each layer so that the parameter ratios of different layers are comparable.

[0054] Secondly, the entropy of the parameter affects the bit requirement. Higher entropy means more information, so more bit width is required. Considering the normalized entropy, the calculation formula is: in, Indicates the The normalized entropy of a layer is used to measure the degree of match between the information richness of the layer parameters and the quantization bit width. The normalized value can reflect the parameter distribution's demand for quantization accuracy. A higher value indicates that the parameter carries more information and requires a higher bit width to retain the information. Indicates the The zero-order entropy of the layer parameters, that is, the information entropy calculated by the distribution probability of the statistical parameter values, is used to quantify the information uncertainty of the layer parameters. Indicates the target bit width of the quantization model, which is used to normalize the zero-order entropy to make the entropy indicators under different bit widths comparable.

[0055] It is well established that parameter distribution drives bit accuracy requirements. Most layers in a DNN model follow a Gaussian distribution, and the variance represents compactness. Consider the normalized variance, which is calculated as: in, Indicates the The normalized variance of a layer measures the dispersion of its parameter distribution. A logarithmic transformation is used to map the variance ratio to a specific range, reflecting the parameter distribution's requirement for quantization bit width. Smaller values indicate a more concentrated parameter distribution, potentially minimizing accuracy loss during quantization and enabling adaptation to a lower bit width. Larger values indicate a more dispersed parameter distribution, requiring a higher bit width to preserve detail. Indicates the The variance of a layer parameter is calculated as the average of the squared deviations of all parameter values from the mean for that layer, reflecting the degree of fluctuation of the parameter values around the mean. A larger variance indicates a more dispersed parameter distribution; a smaller variance indicates a more compact parameter distribution. The maximum value of the variance of all parameters in the network, where k traverses all layers. This value is used to normalize the variance of each layer to make the variance of different layers comparable and ensure that the normalized results fall within a reasonable range.

[0056] Finally, activation sparsity indicates the criticality of a layer. Layers with higher zero or near-zero activations are likely to be less critical. Considering normalized activation sparsity, it is calculated as in, Indicates the The normalized activation sparsity of a layer measures the proportion of zero or near-zero activation values in that layer. This metric reflects the "criticality" of a layer. Higher values indicate a greater concentration of ineffective (or low-contribution) signals in the activations. Therefore, a lower bit width can be preferred during quantization for this layer to reduce computational overhead without significantly impacting model performance. Indicates the The number of zero or near-zero activations in the layer. This parameter is used to quantify the absolute number of sparse activations within the layer. Indicates the The total number of activations for the layer, i.e., the number of elements in the output feature map of the layer. This parameter is used to normalize the number of sparse activations so that the sparsity metrics of different layers are comparable.

[0057] The importance of the final layer combines these components: in It is a weight multiplier used to control the impact on the overall importance score and is adjusted according to the optimization goal. For scenarios with limited computing resources on edge devices, the parameter proportion weight is increased. Using compression models to improve activation sparsity weights for central control systems To accelerate computing, increase the entropy weight for transient feature extraction tasks and variance weights The optimization goal is to maximize grid data processing accuracy, minimize inference latency, and enhance model robustness within the constraints of device computing power.

[0058] In some embodiments, based on the inter-layer importance evaluation results, a bit width search strategy is used to determine the target bit width of each layer of the network, including: arranging the layers in descending order of layer importance according to the inter-layer importance evaluation results, and performing the following operations on each layer: initializing the bit width to the highest quantization bit width supported by the current layer, and iteratively reducing it to a lower bit width at a fixed step size while ensuring that the accuracy loss does not exceed a set threshold to determine the target bit width of the current layer.

[0059] In some embodiments, a diagnostic accuracy test is performed on a network with a current bit width configuration on a data set of faults in power equipment to obtain the accuracy loss of the current bit width; based on the safety level of the power equipment and the fault diagnosis accuracy requirements, a set threshold for the accuracy loss is generated; in the process of adjusting the bit width configuration from a high bit to a low bit, if the adjusted accuracy loss exceeds the set threshold, the bit width reduction is terminated and the previous bit width configuration is retained.

[0060] The layer width selection follows the above layer importance calculation to calculate the inter-layer importance evaluation results of each layer. Layer-Adaptive Bitwidth Student Meta-Network , if the average bit width of the mixed precision network is specified , then it must meet ,in For the The selected width of the layer, For the The number of parameters of the layer, is the total bit width cost of the required sub-network, since the cost Subnetwork set is finite, so the cost of using the greedy algorithm to generate the bit width is The best candidate for the bit width of each layer is determined through a search process to ensure that the minimum bit width is achieved while maintaining high accuracy. The layers are ranked by importance, and the search process starts from the most important layer. Each layer starts with the maximum possible bit width, and when fine-tuning towards the lower bit width, the accuracy loss is guaranteed to be within the threshold. within the range.

[0061] The method for generating an adaptive mixed-precision quantization network based on self-learning provided in the embodiments of the present application effectively overcomes many drawbacks of traditional quantization methods and existing adaptive bit-width quantization methods: First, in order to address the technical problems that traditional adaptive bitwidth network quantization has large quantization loss under low-precision settings and low-precision performance is affected by high-precision due to the shared weights of all bitwidths, the embodiment of the present application provides a method for generating an adaptive mixed-precision quantization network based on self-learning, combined with knowledge distillation to design a joint loss function that is adaptive to the training progress. At the same time, a method for selecting a target teacher based on model prediction confidence and the Manhattan distance between models is proposed. The network's own quantized copy is used as a teacher to adaptively select the most suitable teacher network for each bitwidth, significantly improving the performance of low-bitwidth quantization and optimizing the training process.

[0062] Secondly, the self-learning-based adaptive mixed-precision quantization network generation method provided in the embodiment of the present application adopts a specific quantization method to reduce the model storage requirement, and at the same time proposes a bit-width-sensitive transition batch normalization method. By independently learning an exclusive batch normalization layer for the bit-width transition combination of each pair of adjacent layers, it solves the problem of a sharp drop in model accuracy and learning process conflicts when dynamically changing the bit width due to differences in activation value distribution under different bit widths, and realizes flexible and efficient mixed-precision quantization, so that the model can run stably under different bit-width configurations. In addition, the embodiment of the present application provides a method for generating an adaptive mixed-precision quantization network based on self-learning. By using multiple statistical indicators to calculate the importance of each layer, an inter-layer importance evaluation scheme is formulated. Based on this scheme, a greedy algorithm is used to generate the best candidate for bit width cost. While ensuring that the accuracy loss is within the set threshold, the optimal bit precision of each layer is determined starting from the most important layer. Without additional training, the optimal mixed-precision subnetwork under a given average precision can be found relatively quickly, providing strong support for achieving a better trade-off between accuracy and speed in practical applications. In terms of technical effects, taking ResNet18 on the CIFAR-10 dataset as an example, when the target average bit width is 4-bit, the accuracy of the mixed-precision network searched by this mechanism reaches 93.38%, surpassing the 93.34% of the fixed 4-bit precision quantization baseline model; the accuracy of the MobileNet V2 model at a 4-bit average bit width is 91.69%, also exceeding the 91.61% of the fixed precision quantization baseline. On the ImageNet dataset, the ResNet18 model achieved a 6-bit average bit-width mixed-precision network accuracy of 68.88%, a 0.09% improvement over the single-precision 6-bit quantized model, while also reducing theoretical computational effort by approximately 33.3%. Compared to methods that do not employ inter-layer importance search, the accuracy loss at the same average bit-width is reduced by 40%-60%. For example, in the ResNet18 model on CIFAR-10, the mixed-precision network with randomly assigned bit-widths achieved an accuracy of 92.16%, an improvement of 1.22 percentage points under this mechanism, fully demonstrating its significant advantage in balancing computational resources and model performance.

[0063] Please refer to Figure 2 , Figure 2 A schematic diagram of a generation system of an adaptive mixed precision quantization network based on self-learning provided in an embodiment of the present application, such as Figure 2As shown, the embodiment of the present application also provides a generation system of an adaptive mixed precision quantization network based on self-learning, which is applied to power equipment status monitoring, and the system includes: an acquisition module for acquiring a teacher bit width set and a student meta-network to be trained, wherein the teacher bit width set includes multiple high-bit width candidate teachers, and the student meta-network shares full-precision weights and supports multiple bit width configurations; a training module for determining the corresponding target teacher for any bit width configuration according to the inter-layer Manhattan distance between the student meta-network and each candidate teacher and the entropy value of the predicted probability distribution of each candidate teacher; and for jointly training all bit width configurations of the student meta-network to obtain an adaptive bit width student The meta-network includes: for any bit-width configuration, determining the training loss between the student meta-network and the target teacher, weighted summing the training losses under all bit-width configurations to obtain a joint loss function, and updating the shared full-precision weights based on the joint loss function; a generation module for generating inter-layer importance evaluation results of each layer of the network based on normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity; determining the target bit-width of each layer of the network based on real-time device status data and the inter-layer importance evaluation results; and generating a target mixed-precision quantization network that responds to the real-time device status based on the target bit-width of each layer of the network and the adaptive bit-width student meta-network.

[0064] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0065] The generation system of the self-learning-based adaptive mixed-precision quantization network in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0066] See also Figure 3 , Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown in FIG. Figure 3As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 10 is taken as an example.

[0067] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0068] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0069] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0070] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0071] The computer device further includes a communication interface 30 for communicating with other devices or a communication network.

[0072] The embodiments of the present application also provide a computer-readable storage medium. The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0073] An embodiment of the present application provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a method according to any embodiment of the present application.

[0074] The systems or modules described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0075] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0080] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of other identical elements in the process, method, commodity, or apparatus comprising the element.

[0081] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0082] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

[0083] Although the embodiments of the present application have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present application, and such modifications and variations shall fall within the scope defined by the appended claims.

Claims

1. A method for generating an adaptive mixed-precision quantization network based on self-learning, applied to power equipment status monitoring, characterized in that: The method comprises: Obtaining a teacher bit width set and a student meta-network to be trained, wherein the teacher bit width set includes multiple candidate teachers with high bit widths, and the student meta-network shares full-precision weights and supports multiple bit width configurations; For any bit-width configuration, a corresponding target teacher is determined based on the inter-layer Manhattan distance between the student meta-network and each candidate teacher and the entropy of the predicted probability distribution of each candidate teacher; all bit-width configurations of the student meta-network are jointly trained to obtain an adaptive bit-width student meta-network, including: for any bit-width configuration, determining the training loss between the student meta-network and the target teacher, weighted summing the training losses under all bit-width configurations to obtain a joint loss function, and updating the shared full-precision weight based on the joint loss function; Based on the normalized parameter ratio, normalized entropy, normalized variance and normalized activation sparsity, the inter-layer importance evaluation results of each layer of the network are generated; based on the real-time device status data and the said inter-layer importance evaluation results, the target bit width of each layer of the network is determined; based on the target bit width of each layer of the network and the said adaptive bit width student meta-network, a target mixed precision quantization network that responds to the real-time device status is generated.

2. The method according to claim 1, characterized in that The method further comprises: Acquire historical equipment status data and extract multi-dimensional features from the historical equipment status data, including equipment operating status indicators, abnormal feature patterns, and time series change trends; Based on the multi-dimensional features, a device state-bit width mapping strategy table is generated, wherein the device state-bit width mapping strategy table is used to characterize the configuration strategy of each layer of bit width corresponding to different power device states; The real-time device status data is matched with the device status feature vector in the device status-bit width mapping strategy table to determine the target bit width of each layer of the network.

3. The method according to claim 2, characterized in that The method also includes: analyzing the real-time indicators and fluctuation characteristics of historical equipment status data, and evaluating the real-time strength and abnormal fluctuation frequency of the data based on the real-time indicators and the fluctuation characteristics; if the real-time nature of the data is strong and the abnormal fluctuation frequency is high, shortening the probability and reducing the round interval to speed up the conversion rate; conversely, extending the round interval to slow down the conversion rate, so as to dynamically adapt the training probability to the real-time demand of electricity.

4. The method according to claim 1, wherein The training loss is a weighted combination of the labeled data supervision loss and the knowledge distillation loss, wherein the labeled data supervision loss is the cross entropy loss and the distillation loss is the KL divergence loss. The network parameters are updated synchronously through gradient accumulation.

5. The method according to claim 4, characterized in that The method further includes: dynamically adjusting the weight coefficients of the cross entropy loss and the KL divergence loss using a linear annealing strategy: focusing on the cross entropy loss to establish a category discrimination boundary at the beginning of training, and gradually increasing the weight coefficient of the KL divergence loss as training progresses.

6. The method according to claim 1, characterized in that The method further includes: dynamically balancing the weights of the entropy value and the inter-layer Manhattan distance through a hyperparameter, and linearly adjusting the hyperparameter as the training progresses to determine a target teacher corresponding to the current bit width configuration.

7. The method according to claim 1, characterized in that The method also includes: using a quantization function with a floor operator in the training process to learn the quantization step size for the weights and activations of each layer of the network respectively.

8. The method according to claim 1, characterized in that The method also includes: independently learning a dedicated BN layer for each pair of adjacent layer bit width transition combinations; quantizing the distribution offset caused by the change in bit width between consecutive layers by calculating the activation mean and variance differences before and after quantization; and compensating for the distribution offset by dynamically adjusting the scaling parameters and offset parameters of the BN layer.

9. The method according to claim 1, characterized in that The method also includes: arranging the layers in descending order of layer importance according to the inter-layer importance evaluation result, and performing the following operations on each layer: determining the initialization bit width according to the real-time monitored device status and the highest quantization bit width supported by the current layer, and iteratively reducing the bit width from the initialization bit width to the lower bit width at a fixed step size while ensuring that the accuracy loss does not exceed a set threshold, so as to determine the target bit width of the current layer.

10. The method according to claim 9, characterized in that The method also includes: performing a diagnostic accuracy test on a network with a current bit width configuration on a data set of faults in an electric power equipment to obtain an accuracy loss of the current bit width; generating a set threshold for the accuracy loss based on the safety level of the electric power equipment and the fault diagnosis accuracy requirements; and during the process of adjusting the bit width configuration from a high bit to a low bit, if the adjusted accuracy loss exceeds the set threshold, terminating the bit width reduction and retaining the previous bit width configuration.

Images