Electric power scene-oriented heterogeneous feature decoupling distillation defect detection method and device and storage medium

Abstract

The invention discloses a heterogeneous feature decoupling distillation defect detection method and device for an electric power scene and a storage medium, and aims to solve the problems that in existing electric power inspection, the computing power of edge equipment is limited, high-resolution perception and real-time reasoning are difficult to consider at the same time, and a traditional lightweight model is poor in generalization ability under long-tail distribution and complex backgrounds. A cloud teacher-edge student cooperation normal form is constructed, a CLIP-ViT large model of frozen parameters is used as a teacher network, and a lightweight CNN is used as a student network; the high-resolution student stream is utilized to accurately capture insulator cracks and other tiny defect textures, and the low-resolution teacher stream provides global semantic guidance; introducing a linear manifold alignment mechanism without an activation function, and losslessly mapping student features to a high-dimensional semantic space of a teacher; and in combination with batch-based structured relation distillation, forcing the student model to duplicate the sample topology logic of the teacher.

Description

Technical Field

[0001] This application relates to the field of power equipment inspection technology, specifically to a method, device, and storage medium for detecting heterogeneous feature decoupling distillation defects in power scenarios. Background Technology

[0002] In recent years, computer vision technologies, represented by deep learning, have made groundbreaking progress, driving the rapid development of power line inspection towards automation and intelligence. Through drone inspection systems equipped with high-performance visible light or infrared cameras, and fixed intelligent monitoring terminals installed on power poles, massive amounts of image or video data of power equipment can be continuously and automatically collected. Subsequently, intelligent algorithm models deployed at the edge (such as drone-borne computing units or embedded industrial control computers on power poles) are used for real-time analysis and defect diagnosis. Currently, deep learning technologies suitable for power line defect detection mainly follow two development paths, but both face significant bottlenecks:

[0003] The first type of technical route is mainly based on lightweight convolutional neural networks (CNNs), such as the YOLO series, MobileNet series, ShuffleNet, etc. These models significantly reduce the number of model parameters and floating-point operations by introducing model compression and acceleration techniques such as depthwise separable convolution, channel attention, network pruning and quantization, so that they can achieve a high inference frame rate on resource-constrained edge hardware. Although it has an advantage in inference speed, the inherent local inductive bias of the CNN architecture makes it more inclined to capture local textures, edges and shape patterns in images, and it is difficult to model long-distance global contextual dependencies. This characteristic has caused a series of prominent problems in complex power inspection scenarios: (1) Failure problem under long-tail distribution: Real power inspection data exhibits a very typical long-tail distribution characteristic, that is, the number of samples representing the normal state of equipment is huge, while the samples representing various defects (such as micro-cracks in insulators, corrosion of hardware, and pin detachment) are extremely scarce. CNN models struggle to learn sufficiently robust and highly discriminative feature representations on a limited number of defect samples, resulting in low recall rates for tail defect categories and a high risk of missed detections. (2) Weak generalization ability in complex backgrounds: Transmission lines often traverse complex environments such as mountains, forests, and urban-rural fringe areas. Inspection images are prone to unstructured factors such as foliage occlusion, drastic changes in light and shadow, cloud and fog interference, and background texture confusion (such as bird nests and moss). CNN models are susceptible to such local texture interference, misjudging background artifacts as defects or misjudging real defects with complex textures as background, resulting in severely insufficient generalization ability and robustness of the model.

[0004] The second type of technical approach relies on large-scale pre-trained models with Transformer as the core architecture, especially the Contrastive Language-Image Pre-training (CLIP) model. This type of model learns powerful open-world semantic understanding capabilities and remarkable zero-shot generalization performance through comparative learning pre-training on massive image-text pairs on the Internet scale. Its core self-attention mechanism can capture the global dependencies between any pixel regions in the image, thereby achieving a deep understanding of the semantics of the entire image scene and having strong suppression and discrimination capabilities for complex backgrounds. However, deploying such large models with huge parameters and complex computations directly to the edge faces insurmountable obstacles: (1) Explosive computation and memory overhead: The computational complexity of the self-attention mechanism in Transformer is proportional to the square of the length of the input sequence. For high-resolution inspection images, the sequence length formed after dividing them into image blocks is extremely long, resulting in high model inference latency and huge memory consumption, which cannot meet the stringent requirements of edge devices for millisecond-level real-time response. (2) The contradiction between high-resolution input and computational efficiency: In order to control the amount of computation, the standard VisionTransformer usually downsamples the input image to a lower resolution. This forced downsampling operation is fatal for the pixel-level tiny defects that need to be detected in power line inspection (such as insulator cracks with a width of only a few pixels). A large amount of high-frequency detail information of the image is lost during the compression process, which makes the model essentially "unable to see" the target to be detected, and the detection accuracy is out of the question.

[0005] In summary, existing power vision inspection technologies are caught in a trilemma of "accuracy-speed-detail": lightweight CNNs run fast but are poorly understood and easily fooled, with limited accuracy in complex scenes and with small sample sizes; while large-scale Transformer models can understand the data but are too slow to run, failing to meet the real-time and high-resolution requirements of edge deployments. Therefore, the industry urgently needs an innovative technical framework that can break through these dilemmas, enabling edge-side models to inherit the deep semantic understanding and strong anti-interference capabilities of large models while maintaining efficient inference. Summary of the Invention

[0006] The main objective of this application is to provide a method for detecting heterogeneous feature decoupling distillation defects in power scenarios, comprising the following steps:

[0007] Step S10: Construct a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained. The teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network.

[0008] Step S20: Implement asymmetric dual-stream input decoupling, preprocess the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network;

[0009] Step S30: At the end of the student network, a linear projection layer without a non-linear activation function is constructed to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network;

[0010] Step S40: For the power inspection images in the same training batch, calculate the semantic affinity matrix between the corresponding samples of student features and teacher features respectively. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network.

[0011] Step S50: Construct a multi-task loss function and use the multi-task loss function to jointly optimize the training process of the student network. The multi-task loss function includes a classification loss for supervising prediction results, a feature alignment loss for aligning student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples.

[0012] Step S60: After the student network training is completed, the student network is deployed only to edge devices to detect defects in power equipment in real time.

[0013] In one embodiment, the step of constructing a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained, wherein the teacher network is a pre-trained large model based on the Transformer architecture and the student network is a convolutional neural network, includes:

[0014] The CLIP-ViT model was selected as the teacher network, and all parameters of the teacher network were set to a frozen state during the training process of the heterogeneous knowledge distillation system.

[0015] The lightweight convolutional neural network MobileNetV3 is selected as the student network, and during the training process of the heterogeneous knowledge distillation system, all parameters of the student network are set to a trainable state for optimization through backpropagation.

[0016] By using the feature comparison path from the feature output of the student network to the feature of the teacher network, a collaborative training framework is constructed in which the teacher network provides knowledge guidance to the student network and knowledge is transferred unidirectionally.

[0017] In one embodiment, the step of implementing asymmetric dual-stream input decoupling, preprocessing the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network, includes:

[0018] The power inspection images are downsampled to generate low-resolution teacher flow images;

[0019] The same original power inspection image is cropped and scaled to generate a high-resolution student stream image;

[0020] The low-resolution teacher stream image is input into the teacher network, and the high-resolution student stream image is input into the student network; wherein the size of the low-resolution teacher stream image includes, but is not limited to, 336×336 pixels, and the size of the high-resolution student stream image includes, but is not limited to, 448×448 pixels.

[0021] In one embodiment, the step of constructing a linear projection layer without a non-linear activation function at the end of the student network to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network includes:

[0022] After the last convolutional or pooling layer of the student network, a linear fully connected layer and a batch normalization layer are connected to construct the linear projection layer without nonlinear activation functions.

[0023] The original features extracted by the student network are input into the linear projection layer, and linear mapping and normalization are performed to obtain the projected student features, which are represented as follows:

[0024]

[0025] in, These are the original characteristics of the student network. and The learnable weights and biases of the linear fully connected layer. This indicates a batch normalization operation; The original characteristics of the student backbone network, and For learnable weights and biases, This indicates batch normalization operation.

[0026] The linear projection layer maps the feature distributions of the student network and the teacher network in a high-dimensional semantic space, thus preserving the semantic information carried by the negative activation values ​​in the teacher network features.

[0027] In one embodiment, the step of calculating the semantic affinity matrix between corresponding samples of student features and teacher features for the power inspection images within the same training batch, and minimizing the difference loss between the two semantic affinity matrices to enable the student network to learn the topological relationships between samples represented by the teacher network, includes:

[0028] Obtain the student feature matrix from training batch B after mapping by the linear projection layer. and the teacher feature matrix output by the teacher network ;

[0029] Calculate the student feature matrix separately Teacher Feature Matrix Semantic affinity matrix between corresponding samples and The calculation formula is:

[0030]

[0031] in, For temperature coefficient, The semantic affinity matrix for students represents the semantic affinity between samples within a batch. Teacher semantic affinity matrix ;

[0032] Based on the student semantic affinity matrix, The teacher semantic affinity matrix The KL divergence between samples forces the student network to learn the inter-sample topological relationships constructed by the teacher network within a batch.

[0033] In one embodiment, the step of constructing a multi-task loss function and jointly optimizing the training process of the student network using the multi-task loss function includes the steps of classification loss for supervising prediction results, feature alignment loss for aligning student features and teacher features, and relation distillation loss for constraining the topological relationships between samples, comprising:

[0034] Based on the predicted results and true labels output by the student network, the binary cross-entropy loss is calculated as the classification loss. ;

[0035] The student feature matrix and teacher feature matrix are both L2 normalized, and the mean squared error between them is calculated as the feature alignment loss. ;

[0036] Based on the student semantic affinity matrix and the teacher semantic affinity matrix, the KL divergence between them is calculated as the relation distillation loss. ;

[0037] The classification loss, feature alignment loss, and relation distillation loss are weighted and summed according to preset weight coefficients to obtain the total loss, and the model parameters of the student network are optimized through the backpropagation algorithm.

[0038] In one embodiment, the step of deploying the student network to edge devices only after the student network training is completed, and performing real-time detection of defects in power equipment, includes:

[0039] After training is completed, the model parameters of the student network are saved, and the teacher network is stripped from the heterogeneous knowledge distillation system.

[0040] The trained student network is converted into a format suitable for inference on the edge device and deployed to the edge device;

[0041] During the operation of the edge device, the power inspection images are acquired in real time and preprocessed, then input into the student network for forward inference, and real-time alarms are issued based on the output defect probability.

[0042] A heterogeneous feature decoupling distillation defect detection device for power applications, comprising:

[0043] The network construction module is used to construct a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained. The teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network.

[0044] The dual-stream preprocessing module is used to implement asymmetric dual-stream input decoupling, preprocess the power inspection images, and generate two power inspection images with different resolutions. The high-resolution image is input to the student network, and the low-resolution image is input to the teacher network.

[0045] The feature projection module constructs a linear projection layer without a non-linear activation function at the end of the student network, mapping the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network.

[0046] The heterogeneous feature decoupling distillation module is used to calculate the semantic affinity matrix between corresponding samples of student features and teacher features for the power inspection images in the same training batch. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network.

[0047] A multi-task loss construction module is used to construct a multi-task loss function and use the multi-task loss function to jointly optimize the training process of the student network. The multi-task loss function includes a classification loss for supervising prediction results, a feature alignment loss for aligning student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples.

[0048] The model deployment module is used to deploy the student network to edge devices only after the student network has been trained, so as to detect defects in power equipment in real time.

[0049] Therefore, this application has the following beneficial effects:

[0050] This application provides a heterogeneous feature decoupling distillation defect detection method for power scenarios, comprising the following steps: constructing a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained, wherein the teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network; implementing asymmetric dual-stream input decoupling, preprocessing power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network, and the low-resolution image is input to the teacher network; at the end of the student network, constructing a linear projection layer without a nonlinear activation function, mapping the features extracted by the student network to a high-dimensional model that matches the features of the teacher network. Semantic space; for the power inspection images in the same training batch, the semantic affinity matrix between the corresponding samples of student features and teacher features is calculated respectively. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network; a multi-task loss function is constructed, and the training process of the student network is jointly optimized using the multi-task loss function. The multi-task loss function includes a classification loss for supervising the prediction results, a feature alignment loss for aligning the student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples; after the student network is trained, only the student network is deployed to edge devices to detect defects in power equipment in real time. This application aims to address the limitations of edge devices in current power line inspection systems, which hinder the simultaneous achievement of high-resolution perception and real-time inference. Furthermore, traditional lightweight models exhibit poor generalization ability in long-tailed distributions and complex backgrounds. The solution constructs a collaborative paradigm of "cloud teacher-edge student," using a CLIP-ViT model with frozen parameters as the teacher network and a lightweight CNN as the student network. High-resolution student streams accurately capture the texture of minute defects such as insulator cracks, while low-resolution teacher streams provide global semantic guidance. A linear manifold alignment mechanism without activation functions is introduced to losslessly map student features to the high-dimensional semantic space of the teacher. Combined with batch-based structured relation distillation, the student model is forced to replicate the sample topological logic of the teacher. This application significantly reduces computational load and storage requirements while substantially improving the model's detection accuracy and robustness in small-sample and unstructured environments, making it suitable for real-time automated inspections using drones and tower-mounted monitoring terminals. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a system flowchart of a heterogeneous feature decoupling distillation defect detection method for power scenarios;

[0053] Figure 2 This is an algorithm architecture diagram of a heterogeneous feature decoupling distillation defect detection method for power scenarios;

[0054] Figure 3 This is a comparison chart of the accuracy of heterogeneous feature decoupling distillation defect detection methods for power scenarios under different data efficiencies;

[0055] Figure 4 This is a comparison chart showing the accuracy and efficiency of the method in this application with other mainstream methods in power defect detection tasks. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0058] To address the shortcomings of existing technologies, this application provides a heterogeneous feature decoupling distillation defect detection method for power scenarios. By innovatively integrating the knowledge capacity of the large Transformer model with the efficient inference capabilities of convolutional networks, it resolves the core contradiction in power defect detection where accuracy and speed are difficult to balance. This application constructs a heterogeneous distillation system consisting of a CLIP-ViT teacher network with frozen parameters and a MobileNetV3 student network. An asymmetric dual-stream input mechanism is designed: high-resolution images are input into the student network to preserve spatial details, while low-resolution images are input into the teacher network to extract global semantics. At the feature level, heterogeneous features are mapped to a unified semantic space through a linear projection layer, and a novel distillation of topological relationships between samples is introduced, enabling the student network not only to learn the feature representation of a single sample but also to understand the semantic relationships between different power defects.

[0059] A multi-task loss function collaboratively optimizes classification accuracy, feature alignment, and relation preservation. Relation distillation loss transfers the teacher network's deep understanding of complex power scenarios through the semantic affinity matrix. To address edge deployment requirements, hybrid precision quantization and channel pruning techniques are employed to compress the model size, achieving an inference speed of 23ms / frame. In a field test at a State Grid 220kV substation, this method achieved a detection accuracy of 90.8% for 12 types of defects, including insulator damage and hardware corrosion, a 7.2 percentage point improvement over traditional YOLOv5. Simultaneously, it achieved a real-time processing capability of 21 frames per second on a Jetson Nano device, significantly reducing 4G / 5G network bandwidth consumption by 87%. This solution overcomes the technical barriers of cloud-edge-device collaboration in intelligent power inspection, providing highly reliable and low-latency defect early warning assurance for the safe operation of power equipment.

[0060] This application provides a method for detecting heterogeneous feature decoupling distillation defects in power scenarios, including steps S10-S60, referring to... Figure 1 , Figure 1 This is a system flowchart of a heterogeneous feature decoupling distillation defect detection method for power scenarios.

[0061] Step S10: Construct a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained. The teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network.

[0062] Step S20: Implement asymmetric dual-stream input decoupling, preprocess the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network;

[0063] Step S30: At the end of the student network, a linear projection layer without a non-linear activation function is constructed to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network;

[0064] Step S40: For the power inspection images in the same training batch, calculate the semantic affinity matrix between the corresponding samples of student features and teacher features respectively. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network.

[0065] Step S50: Construct a multi-task loss function and use the multi-task loss function to jointly optimize the training process of the student network. The multi-task loss function includes a classification loss for supervising prediction results, a feature alignment loss for aligning student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples.

[0066] Step S60: After the student network training is completed, the student network is deployed only to edge devices to detect defects in power equipment in real time.

[0067] Specifically, this embodiment includes the following steps S10 to S60. This method, through a series of collaborative technical steps, aims to solve the core challenge of achieving high-precision detection of minute defects in edge devices with limited computing power.

[0068] Step S10: Construct a heterogeneous knowledge distillation system.

[0069] The core of this step is to establish an asymmetric teacher-student collaborative framework that combines the deep semantic understanding capabilities of large models with the efficient inference characteristics of lightweight models. In this implementation, the teacher network uses a large pre-trained model based on a Transformer architecture, adapted and fine-tuned for power line inspection data, such as the visual encoder part of CLIP. This model acts as a knowledge provider during training, with all its parameters set to a frozen state, meaning it does not participate in gradient updates but only performs forward computation to output stable and semantically rich features. The student network uses a lightweight convolutional neural network designed specifically for mobile and edge computing, such as MobileNetV3. All its parameters are trainable and optimizable during training. In this way, a heterogeneous knowledge distillation system with distinct structures and parameter states is constructed, laying the architectural foundation for subsequent cross-architecture knowledge transfer.

[0070] Step S20: Implement asymmetric dual-stream input decoupling.

[0071] This step aims to resolve the conflict between the need for high-resolution input and the computational burden of large models. For the same power line inspection image, two inputs with different resolutions are generated in parallel. One is a high-resolution image, processed to 448×448 pixels, which serves as the input to the student network. This high-resolution input can preserve the fine texture and spatial details of pixel-level defects such as microcracks on the insulator surface, small rust spots on hardware, or missing edges of pins, giving full play to the advantages of convolutional neural networks in local feature extraction and avoiding the loss of key information due to image compression. The other is a relatively low-resolution image, downsampled to 336×336 pixels, which serves as the input to the teacher network. This resolution conforms to the preset input size of most large-scale visual Transformer models, keeping the computational complexity and memory consumption of its forward propagation within an acceptable range. Despite the reduced resolution, the teacher network, thanks to its strong semantic priors obtained through pre-training on large-scale data, can still effectively extract macroscopic understandings about scene categories, global context, and background semantics from the low-frequency information of the image. This design achieves an input decoupling strategy where "students focus on detailed textures while teachers provide semantic guidance," optimizing the allocation of computing resources from the data source.

[0072] Step S30: Construct a linear projection layer without a non-linear activation function.

[0073] Because convolutional neural networks commonly use activation functions like ReLU, their output feature values ​​are truncated to the non-negative range, resulting in a relatively sparse feature distribution. In contrast, teacher networks based on the Transformer architecture typically have densely distributed internal features, containing a large amount of negative activation information crucial for distinguishing foreground from background and performing logical reasoning. To effectively align the student network's features to the teacher network's semantic space, this step adds a special linear projection layer at the end of the student network's feature extraction backbone. This layer has an extremely simple structure, consisting of only a learnable linear fully connected transformation unit and a batch normalization unit for stable training connected sequentially, explicitly excluding any non-linear activation functions. Its function is to directly map the original local features extracted by the student network to a high-dimensional semantic space that matches the feature dimension of the teacher network through a purely linear transformation. This untruncated linear mapping design allows the student network to fully receive and simulate the complex semantic patterns carried by negative values ​​in the teacher features, such as the ability to suppress interfering backgrounds, thus creating conditions for subsequent high-quality knowledge transfer.

[0074] Step S40: Calculate the semantic affinity matrix and perform structured relation distillation.

[0075] To enable the student network to learn higher-order knowledge beyond imitating individual sample features, this step introduces a batch-based sample relationship constraint mechanism. For a training batch containing several images, the system simultaneously obtains the student feature set after projection alignment and the teacher feature set output by the teacher network. Subsequently, the semantic affinity matrix between samples within these two feature sets is calculated. This matrix is ​​obtained by calculating the similarity between feature vectors and then normalizing it, where each element quantifies the semantic association strength between any two samples within the batch in the corresponding feature space. By constructing a loss function to minimize the difference between the student affinity matrix and the teacher affinity matrix, this step forces the student network not only to learn the correct feature representation of each sample itself but also to imitate the topological relationship logic between samples established by the teacher network. This means that the student network will implicitly learn high-level discriminative knowledge such as "which defective samples are semantically closer" and "how to distinguish defective samples from normal samples or complex background samples," greatly enhancing the model's robustness and generalization ability under complex data distributions.

[0076] Step S50: Construct a multi-task loss function and optimize it jointly.

[0077] The training process of the student network is guided and constrained by a composite multi-task loss function. This total loss function consists of a weighted sum of three core components.

[0078] The first part is the classification loss, which usually adopts the binary cross-entropy loss. This directly supervises whether the final classification output of the student network is consistent with the real defect label of the image. This is the fundamental way to ensure that the model has basic discrimination ability.

[0079] The second part is feature alignment loss, which aims to narrow the spatial distance between the projected features of the student network and the original features of the teacher network. It is often achieved by metrics such as mean squared error, ensuring that the student features numerically approximate the semantic representation of the teacher features.

[0080] The third part is the relation distillation loss generated in the preceding steps, used to align the semantic affinity relationships between samples constructed by the teacher-student networks. By assigning appropriate weights to these three losses and combining them into a total loss, the training process utilizes the backpropagation algorithm to iteratively optimize and update only the parameters of the student network and the linear projection layer. This joint optimization mechanism ensures that the student network simultaneously acquires accurate classification capabilities, feature representations similar to those of the teacher, and a deep understanding of sample relationships.

[0081] Step S60: Independent deployment and real-time monitoring of student networks.

[0082] After the model training phase is complete, the deployment and application phase begins. At this point, the large teacher network, having fulfilled its knowledge transfer mission, is completely removed from the system. Only a lightweight student network, fully trained and incorporating the essence of the teacher network's knowledge, is retained. This student network contains a convolutional neural network backbone, linear projection layers, and a final classifier. This student network model is further converted into an efficient inference format suitable for target edge devices (such as drone flight control boards and embedded industrial control computers) and integrated into the device software. In actual power line inspection operations, the edge device only needs to perform preprocessing (scaling to high resolution and normalization) consistent with the student stream on the real-time acquired equipment images. Then, it is input into the student network for a single forward inference calculation to quickly obtain the probability value of whether a defect exists. By comparing this probability with a preset alarm threshold, the system can automatically identify, locate, and report defects within milliseconds. This deployment scheme perfectly achieves both high-precision detection and real-time response—the dual industrial standards—under strictly limited edge resource conditions.

[0083] In one embodiment, the step of constructing a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained, wherein the teacher network is a pre-trained large model based on the Transformer architecture and the student network is a convolutional neural network, includes:

[0084] The CLIP-ViT model was selected as the teacher network, and all parameters of the teacher network were set to a frozen state during the training process of the heterogeneous knowledge distillation system.

[0085] The lightweight convolutional neural network MobileNetV3 is selected as the student network, and during the training process of the heterogeneous knowledge distillation system, all parameters of the student network are set to a trainable state for optimization through backpropagation.

[0086] By using the feature comparison path from the feature output of the student network to the feature of the teacher network, a collaborative training framework is constructed in which the teacher network provides knowledge guidance to the student network and knowledge is transferred unidirectionally.

[0087] Specifically, in this embodiment, the specific implementation process of constructing a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained includes:

[0088] First, regarding the selection and configuration of the teacher network, this embodiment uses the CLIP-ViT model. CLIP is a vision-language model pre-trained on large-scale internet graph-text pairs through contrastive learning. Its visual encoder part adopts the Vision Transformer architecture, possessing powerful open-world semantic understanding capabilities and zero-shot generalization potential. To better suit the power line inspection field, the model can be pre-tuned using collected power equipment images (which may contain a small amount of labeled data) to enhance its feature sensitivity to specific targets such as insulators, fittings, and towers. When formally constructing the heterogeneous distillation system and starting training, all parameters of the teacher network are set to a frozen state. In specific implementation, this means that within the deep learning framework, the attribute of each trainable tensor in the model is set to False, so that during backpropagation, the parameters of the teacher network will not receive gradient updates; its role is only as a stable feature extractor, providing consistent and semantically rich prior knowledge guidance for the entire heterogeneous knowledge distillation system.

[0089] Secondly, regarding the selection and configuration of the student network, this embodiment uses MobileNetV3 as the student network. MobileNetV3 is a lightweight convolutional neural network designed specifically for mobile and embedded vision applications. By introducing efficient depthwise separable convolutions, linear bottleneck structures, and network modules optimized based on neural architecture search, it significantly reduces the number of model parameters and computational complexity while maintaining high accuracy. This network naturally possesses a keen ability to capture local textures and spatial details in images, making it very suitable for handling tasks requiring high-resolution input, such as detecting minute defects in power equipment. In the heterogeneous distillation system, all parameters of the student network are set to a trainable state, i.e., the requires_grad attribute is kept True. During subsequent training, the parameters of the student network will be iteratively optimized through backpropagation algorithm based on the gradient calculated from the total loss function, allowing it to gradually learn and absorb knowledge from the teacher network.

[0090] Finally, in constructing the collaborative training framework, this embodiment achieves unidirectional knowledge transfer by establishing a feature comparison path from the student network to the teacher network. Specifically, during the forward propagation of training, the input image enters the student network and the teacher network respectively via a dual-stream path. The student network processes its high-resolution input to extract richly detailed features; the teacher network processes its low-resolution input to extract features containing global semantics. The system then guides the intermediate or terminal features of the student network (usually after processing through a specific alignment layer) to compare with the features output by the teacher network in a specified space. This comparison is quantified and optimized using a designed loss function (such as feature distance loss, relation distillation loss, etc.). The knowledge flow of the entire framework is unidirectional, meaning that knowledge flows only from the parameter-frozen teacher network to the parameter-trainable student network. The teacher network plays a static "mentor" role, while the student network acts as a dynamic "learner," constantly adjusting itself to mimic the teacher's feature representation and judgment logic. This unidirectional collaborative paradigm ensures the stability and efficiency of knowledge transfer, forming the basic architecture of the entire heterogeneous feature decoupling distillation method.

[0091] In one embodiment, the step of implementing asymmetric dual-stream input decoupling, preprocessing the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network, includes:

[0092] The power inspection images are downsampled to generate low-resolution teacher flow images;

[0093] The same original power inspection image is cropped and scaled to generate a high-resolution student stream image;

[0094] The low-resolution teacher stream image is input into the teacher network, and the high-resolution student stream image is input into the student network; wherein the size of the low-resolution teacher stream image includes, but is not limited to, 336×336 pixels, and the size of the high-resolution student stream image includes, but is not limited to, 448×448 pixels.

[0095] Specifically, in this embodiment, the specific operation process for implementing asymmetric dual-stream input decoupling is as follows:

[0096] First, the dual-stream image is generated and processed. After acquiring the original power inspection image, the heterogeneous knowledge distillation system starts two independent preprocessing pipelines in parallel. The first pipeline is responsible for generating low-resolution teacher stream images. Its main operation is to downsample the original image. Typically, image scaling algorithms such as bilinear interpolation or bicubic interpolation are used to reduce the original high-definition image (e.g., resolution may be 1920×1080 or higher) to a preset standard resolution. In this embodiment, the typical value of this standard resolution is 336×336 pixels. This size selection weighs the following factors: it needs to meet the input specifications of the selected teacher network (such as CLIP-ViT) during pre-training to ensure that the model can perform forward operations normally; at the same time, compared with the original resolution, this size can significantly reduce the computational complexity of the Transformer model's self-attention mechanism, keeping the time consumption and memory usage of a single forward propagation within an acceptable range, making it possible to use a large model as a teacher during the training phase.

[0097] The second pipeline is responsible for generating high-resolution student flow images. The core operation involves cropping and scaling the same original power inspection image. Specifically, to enhance the model's robustness and fully utilize image information, the original image is typically cropped randomly during the training phase, selecting an image region containing key equipment (such as insulator strings). This cropped region is then scaled to a higher target resolution. In this embodiment, this target resolution is typically 448×448 pixels. The core purpose of choosing a higher resolution is to maximize the preservation of high-frequency texture details and accurate spatial geometric information of the detected object. For pixel-level defects such as hairline cracks on the insulator surface, tiny rust spots on metal parts, or missing pins, high-resolution input ensures that these key features are not blurred or lost when entering the convolutional neural network. The local receptive field characteristics of the convolutional neural network enable it to effectively utilize this rich pixel-level information, thereby achieving a keen perception of minute defects.

[0098] Secondly, differentiated data normalization is implemented. After generating the two images, they are not simply input into the network, but a crucial normalization step is performed. Since teacher networks (such as CLIP-ViT) and student networks (such as MobileNetV3) are typically pre-trained on different datasets, their weight adaptation data distributions differ. Therefore, the low-resolution teacher stream image needs to be strictly normalized channel-by-channel according to the mean and standard deviation statistics used during the teacher model's pre-training. For example, for the CLIP model, its specific normalization parameters must be used. The high-resolution student stream image, on the other hand, needs to be normalized according to the general statistics of the student network's pre-training dataset (such as ImageNet). This differentiated processing at the source prevents the feature extraction capability of either network from decreasing due to data distribution shifts caused by uniform preprocessing, and is an important detail ensuring that heterogeneous models can each achieve their optimal performance.

[0099] Finally, the image input routing is completed. After the above preprocessing, the system routes the low-resolution teacher stream image to the input port of the teacher network. Despite the low resolution, thanks to the global modeling capabilities of the Transformer architecture and the semantic priors learned from massive amounts of data, the teacher network can robustly extract macroscopic semantic features about scene category, global object state, and background context from this image. Simultaneously, the high-resolution student stream image is routed to the input port of the student network. The student network leverages its convolutional structure to focus on capturing microscopic texture features and local patterns from this richly detailed image. This high-low combination, decoupling of detail and semantics, essentially constructs an efficient collaborative paradigm: the teacher network grasps the overall picture macroscopically, providing the student network with high-level semantic guidance and anti-interference cues; the student network, on the other hand, delves into the microscopic details of "where the defects are specifically" and "what the detailed morphology is like," ensuring the precision of the detection. Both are closely integrated during training through a knowledge distillation mechanism, ultimately enabling the student network to make judgments with both local accuracy and global robustness based solely on high-resolution input when running independently.

[0100] In one embodiment, the step of constructing a linear projection layer without a non-linear activation function at the end of the student network to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network includes:

[0101] After the last convolutional or pooling layer of the student network, a linear fully connected layer and a batch normalization layer are connected to construct the linear projection layer without nonlinear activation functions.

[0102] The original features extracted by the student network are input into the linear projection layer, and linear mapping and normalization are performed to obtain the projected student features, which are represented as follows:

[0103]

[0104] in, These are the original characteristics of the student network. and The learnable weights and biases of the linear fully connected layer. This indicates a batch normalization operation; The original characteristics of the student backbone network, and For learnable weights and biases, This indicates batch normalization operation.

[0105] The linear projection layer maps the feature distributions of the student network and the teacher network in a high-dimensional semantic space, thus preserving the semantic information carried by the negative activation values ​​in the teacher network features.

[0106] Specifically, in this embodiment, the implementation of constructing a linear projection layer without a nonlinear activation function is a key solution designed to address the fundamental differences in feature distribution between convolutional neural networks and Transformer models. Its purpose is to establish a lossless channel to achieve accurate alignment of heterogeneous feature spaces. The specific process is as follows:

[0107] First, the structure of the linear projection layer is constructed. At the end of the feature extraction backbone of the student network (e.g., MobileNetV3), i.e., after the last convolutional layer or global pooling layer that generates spatial features, this embodiment connects a layer consisting of two components in sequence: a fully connected linear layer followed by a batch normalization layer. This combination is explicitly defined as the linear projection layer. Crucially, the definition and implementation of this layer explicitly excludes and never introduces any non-linear activation functions, such as ReLU, GELU, or Sigmoid, during training. This is a core design feature that distinguishes it from conventional neural network modules. The fully connected linear layer is responsible for dimensional transformation and linear combination of features, while the batch normalization layer stabilizes the training process, accelerates convergence, and ensures a stable distribution of output features.

[0108] Secondly, feature mapping and normalization calculations are performed. The original feature tensors extracted from the student backbone network are then processed. The input is fed into the projection layer mentioned above. This feature is first transformed by a linear fully connected layer, i.e., with a learnable weight matrix. Multiply and add the bias vector This completes the linear mapping from the student's original feature space to the target high-dimensional space. The result is then fed into a batch normalization layer to adjust the mean and variance of its distribution, as shown in the formula:

[0109]

[0110] in, and These are the only trainable parameters for this projection layer, and they are optimized along with the student network via gradient descent during distillation training. Output This refers to the projected student features, whose dimensions are consistent with the feature dimension d output by the teacher's network.

[0111] Finally, feature distribution alignment is achieved while preserving semantic information. The purely linear transformation implemented by the linear projection layer acts as a linear bridge or semantic translator between the student CNN feature space and the teacher Transformer feature space. Because non-linear activation functions (especially functions like ReLU that set negative values ​​to zero) are abandoned, this mapping process does not truncate feature values. This allows negative activation values ​​that may exist in the student network features, and those that may be generated through linear combination, to be fully preserved and transmitted. In the teacher network features, negative activation values ​​often carry important semantic information, such as identifying background regions, expressing inhibitory signals, or forming a certain contrast relationship. By... With teacher characteristics Alignment is performed within the same high-dimensional semantic space (e.g., through feature alignment loss). The projection mechanism constructed in this embodiment ensures that the student network not only learns the numerical patterns of teacher features, but also more completely inherits the rich discriminative semantic topology contained in these positive and negative value distributions.

[0112] In one embodiment, the step of calculating the semantic affinity matrix between corresponding samples of student features and teacher features for the power inspection images within the same training batch, and minimizing the difference loss between the two semantic affinity matrices to enable the student network to learn the topological relationships between samples represented by the teacher network, includes:

[0113] Obtain the student feature matrix from training batch B after mapping by the linear projection layer. and the teacher feature matrix output by the teacher network ;

[0114] Calculate the student feature matrix separately Teacher Feature Matrix Semantic affinity matrix between corresponding samples and The calculation formula is:

[0115]

[0116] in, For temperature coefficient, The semantic affinity matrix for students represents the semantic affinity between samples within a batch. Teacher semantic affinity matrix ;

[0117] Based on the student semantic affinity matrix, The teacher semantic affinity matrix The KL divergence between samples forces the student network to learn the inter-sample topological relationships constructed by the teacher network within a batch.

[0118] Specifically, in this embodiment, for the power inspection images within the same training batch, a specific implementation is provided for calculating the semantic affinity matrix between corresponding samples of student features and teacher features. This aims to improve the representation learning ability of the student network by capturing and transferring higher-order semantic relationships between samples within the batch. The detailed process is as follows:

[0119] First, the batch feature matrix is ​​obtained. In one training iteration, the batch size is set to B. The system processes B pairs of images preprocessed using a two-stream algorithm in parallel. Through forward propagation, a student feature matrix of dimension [B, d] is obtained from the student pathway aligned by the linear projection layer. , where d is the dimension of the feature vector; simultaneously, a teacher feature matrix of the same dimension is obtained from the frozen teacher network. These two matrices represent the vectorized representations of all samples in the current batch in the common semantic space after being processed by their respective networks.

[0120] Secondly, the semantic affinity matrix is ​​calculated. This is the core operation of this step. The system calculates the semantic affinity matrix respectively. and Given the input, calculate its corresponding semantic affinity matrix. and The specific calculation formula is as follows:

[0121]

[0122] In this formula, The symbol "·" indicates the semantic affinity between samples within a batch, and "·" indicates matrix multiplication. express The transpose of . Its physical meaning is: first, calculate the pairwise dot product (or cosine similarity) between all eigenvectors within the same matrix to obtain an original similarity matrix [B, B]. Then, a temperature coefficient is introduced. ( >0) Scale the matrix. Temperature coefficient It is a key hyperparameter that controls the degree of "smoothing" of the output probability distribution; a higher level of smoothness is better. It will make the distribution more uniform and lower This makes the distribution sharper and focuses more on the most similar sample pairs. Finally, the Softmax function is applied to each row to transform the scaled similarity values ​​into a probability distribution. Therefore, the resulting semantic affinity matrix R (regardless of whether it is...) still In the i-th row and j-th column, the element , is interpreted as the probability that, given the i-th sample, the j-th sample is semantically related to it in the feature space. This matrix contains a complex, relative semantic topology among all samples in the current batch.

[0123] Finally, a relation distillation loss is constructed to drive learning. To transfer the more accurate and robust inter-sample relation knowledge established by the teacher network to the student network, this embodiment calculates... and The KL divergence between two probability distribution matrices is used to construct the relationship for distillation loss. KL divergence measures the distribution of relationships when using a student network. To approximate the relationship distribution of the teacher network The amount of information lost during this process. In the backpropagation phase of training, this is minimized. The gradient guides the adjustment of the trainable parameters of the student network (i.e., the parameters of the student network's CNN backbone and linear projection layers) to improve the output features. The constructed inter-sample relationships To approximate the characteristics of teachers' online networks as closely as possible The relationship constructed This means that the student network is not only learning "whether this image is defective," but also learning the deeper logic of "how to distinguish and associate this defective image with other normal images and other types of defective images in the batch." This imitation of the topological relationships between samples is a powerful form of regularization and knowledge transfer, which can significantly improve the student model's generalization ability and discrimination accuracy when faced with imbalanced data distribution (long-tail problem) and complex background interference.

[0124] In one embodiment, the step of constructing a multi-task loss function and jointly optimizing the training process of the student network using the multi-task loss function includes the steps of classification loss for supervising prediction results, feature alignment loss for aligning student features and teacher features, and relation distillation loss for constraining the topological relationships between samples, comprising:

[0125] Based on the predicted results and true labels output by the student network, the binary cross-entropy loss is calculated as the classification loss. ;

[0126] The student feature matrix and teacher feature matrix are both L2 normalized, and the mean squared error between them is calculated as the feature alignment loss. ;

[0127] Based on the student semantic affinity matrix and the teacher semantic affinity matrix, the KL divergence between them is calculated as the relation distillation loss. ;

[0128] The classification loss, feature alignment loss, and relation distillation loss are weighted and summed according to preset weight coefficients to obtain the total loss, and the model parameters of the student network are optimized through the backpropagation algorithm.

[0129] Specifically, in this embodiment, the construction of a multi-task loss function and the joint optimization of the student network training process using the multi-task loss function involves integrating three core supervision objectives into a differentiable optimization objective to comprehensively and balancedly guide the parameter learning of the student network. The detailed process is as follows:

[0130] First, calculate the classification loss. This loss is designed to ensure that the student network possesses basic defect detection capabilities. Specifically, after processing high-resolution input, the student network outputs a scalar from its terminal classifier (typically a linear layer followed by a sigmoid activation function), representing the predicted probability of a defect in the current image. The system compares this predicted probability with the image's true label (typically 0 for normal and 1 for defect). This embodiment uses binary cross-entropy loss as the classification loss. The calculation method is as follows. This loss function effectively measures the difference between the predicted probability distribution and the true distribution, and its gradient characteristics are beneficial to the convergence of the model on binary classification tasks. Minimize The mapping relationship that directly drives students' online learning from input images to final defect identification is the fundamental constraint that ensures the practicality of the model.

[0131] Secondly, calculate the feature alignment loss. This loss function aims to bridge the gap between the student and teacher networks in the feature semantic space, enabling direct transfer of heterogeneous knowledge. This is achieved by obtaining the student feature matrix after linear projection layer mapping. And the teacher feature matrix output by the teacher network Next, this embodiment first performs L2 normalization on both features. This operation converts each feature vector into a unit vector, making the loss function focus more on the similarity of feature directions rather than the magnitude of the features. Then, the mean squared error between the two normalized feature matrices is calculated as the feature alignment loss. Mean squared error penalizes the difference in features across each dimension; its calculation is simple and its gradient is stable. By minimizing... The system mandates that the features extracted by the student network, after linear mapping, should have semantic orientations that are as consistent as possible with the features extracted by the teacher network, which are rich in global prior knowledge. This is the core step in achieving high-quality knowledge distillation.

[0132] Next, calculate the relationship distillation loss. The loss function aims to enable the student network to learn the higher-order knowledge about semantic relationships between different samples implicit in the teacher network. This is based on the student semantic affinity matrix calculated in step S40. semantic affinity matrix with teachers In this embodiment, KL divergence is used to measure the difference between the two probability distribution matrices, and it is defined as the relational distillation loss. The KL divergence accurately measures the information loss of one probability distribution relative to another. Minimizing... This means that student networks are required to mimic the "intra-batch sample relation graph" constructed by the teacher network. For example, the teacher network might consider two images of the same type of defect taken from different angles to be highly similar semantically, but significantly different from a normal image with a complex background. Through relation distillation, student networks not only learn to identify individual samples, but also master this discriminative topological structure between samples, thereby significantly improving their generalization and robustness under complex data distributions.

[0133] Finally, joint optimization is performed. The three loss functions are summed according to preset weight coefficients to obtain the total loss used for training. These weight coefficients... and As a hyperparameter, it is used to balance the importance of classification accuracy, feature imitation fidelity, and relation learning strength. (See figure.) Figure 2 This is an architecture diagram of a heterogeneous feature decoupling distillation defect detection algorithm for power scenarios. During training iterations, the system calculates the gradient of the total loss with respect to all trainable parameters of the student network (including its backbone network and linear projection layers) using the backpropagation algorithm, and updates the parameters using an optimizer based on these gradients. Total Loss The definition is as follows:

[0134]

[0135] in For classification loss, binary cross-entropy loss is adopted to directly supervise the consistency between the model's predicted category and the true label, ensuring basic detection accuracy.

[0136] Student characteristics after projection with mean square error constraint Approximating the characteristics of a teacher in Euclidean space As shown in the formula:

[0137]

[0138] For feature alignment loss, mean squared error (MSE) is used to constrain the projected student features. Approaching the characteristics of teachers ;

[0139] Based on the KL divergence constraint, the consistency of the topological structure among samples is shown in the formula:

[0140] The loss is calculated based on relational distillation, and the consistency of topological structure between samples is constrained by KL divergence; where... These are the weighting coefficients for each loss component.

[0141] In one embodiment, the step of deploying the student network to edge devices only after the student network training is completed, and performing real-time detection of defects in power equipment, includes:

[0142] After training is completed, the model parameters of the student network are saved, and the teacher network is stripped from the heterogeneous knowledge distillation system.

[0143] The trained student network is converted into a format suitable for inference on the edge device and deployed to the edge device;

[0144] During the operation of the edge device, the power inspection images are acquired in real time and preprocessed, then input into the student network for forward inference, and real-time alarms are issued based on the output defect probability.

[0145] Specifically, in this embodiment, after the student network training is completed, the student network is deployed only to edge devices to perform real-time detection of defects in power equipment. The specific implementation process is as follows:

[0146] First, model solidification and decomposition are performed. After the student network completes training and achieves the expected performance within the heterogeneous knowledge distillation framework, the system saves the final model parameters of the student network (usually in .pt or .pth format). A core operation is the complete decomposition of the massive teacher network from the system. This means that in subsequent deployment packages, only the structural definitions and training weights of the student network (i.e., the MobileNetV3 backbone, linear projection layers, and classification head) are included, while the teacher network (CLIP-ViT) is completely excluded from packaging and deployment. This design is the fundamental premise for achieving edge lightweighting in this invention, resulting in a small final deployed model with fewer dependencies, completely eliminating the stringent requirements for the original large model's operating environment.

[0147] Secondly, model conversion and adaptation deployment are performed. Because the computing chips on edge devices (such as NVIDIA Jetson series GPUs, Huawei Ascend NPUs, or other ARM architecture processors) differ from the training server environment, the trained student network model needs to be converted into an efficient inference format specific to the target device. This step involves:

[0148] 1) Format conversion: Using formats such as ONNX (Open Neural Network Exchange) as intermediate representations, PyTorch models can be exported to cross-platform formats;

[0149] 2) Inference Engine Optimization: The model is further optimized using hardware-specific inference acceleration engines (e.g., TensorRT for NVIDIA devices, OpenVINO for Intel processors, and TFLite for mobile and embedded platforms) through graph optimization, operator fusion, and quantization (e.g., INT8 quantization). Quantization significantly reduces model size and computational latency with minimal loss of accuracy. The optimized model is then integrated into applications or algorithm services developed for edge devices for deployment.

[0150] Finally, online real-time detection and alarms are achieved. The system continues to operate during actual use by edge devices (such as drone-borne computers and intelligent monitoring cameras on poles).

[0151] 1) Image Acquisition and Preprocessing: Real-time images of power equipment inspections are acquired via a connected camera. Subsequently, only the high-resolution streams required by the student network are rapidly preprocessed (i.e., scaled to, for example, 448×448 pixels and normalized to the same level as the training time). This preprocessing process is simple and computationally insignificant.

[0152] 2) Forward Inference: The preprocessed image tensor is input into the deployed student network model, and a forward propagation is performed. Thanks to the lightweight design of the student network, this inference process can be completed in milliseconds.

[0153] 3) Decision and Alarm: The network outputs a scalar value representing the probability of a defect's existence. The system compares this probability with a pre-set threshold (e.g., 0.5). If the probability exceeds the threshold, a defect is immediately detected, and an alarm mechanism is triggered. Alarms can manifest as a marker on the device's local interface, an audible alert, or by uploading the alarm information along with a timestamp, device location, and key image clips to a remote monitoring center via the network. Figure 3The accuracy comparison chart shows the method curve (“Student (Ours)”) for different data efficiencies. This curve is consistently above the baseline curve for all data proportions. Especially when data is extremely scarce (e.g., 10%), its mAP value is significantly higher than the baseline method, demonstrating strong few-shot learning capabilities. Its performance steadily improves with increasing data volume, reaching its highest mAP value (approximately 90%) when using 100% data. The baseline method curve (“Baseline (MobileNetV3)”) is consistently below the method curve for this application. Its performance is lower with less data, indicating that traditional lightweight CNN models struggle to learn robust features when lacking large amounts of labeled data. While its performance improves with increasing data volume, the rate of improvement is slower than that of this method.

[0154] Figure 4 This is a comparison chart of the accuracy and efficiency of the method in this application and other mainstream methods in power defect detection tasks. The distribution curve of this application (corresponding scatter points) deviates from the traditional distribution curve of "high accuracy must be high consumption, and low consumption must be low accuracy", and achieves a leap towards the ideal upper left region. Figure 4 Compared to directly using lightweight models or attempting to compress large models, the distillation migration path proposed in this application can more effectively obtain models that combine high accuracy and low latency.

[0155] Other embodiments or specific implementations of this application based on a heterogeneous feature decoupling distillation defect detection device for power scenarios can be referred to the above-described method embodiments, and will not be repeated here.

[0156] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0157] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0158] It should be particularly noted that, through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, or of course, by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting heterogeneous feature decoupling distillation defects in power applications, characterized in that, Includes the following steps: step S10: Construct a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained. The teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network. Step S20: Implement asymmetric dual-stream input decoupling, preprocess the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network; Step S30: At the end of the student network, a linear projection layer without a non-linear activation function is constructed to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network; Step S40: For the power inspection images in the same training batch, calculate the semantic affinity matrix between the corresponding samples of student features and teacher features respectively. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network. Step S50: Construct a multi-task loss function and use the multi-task loss function to jointly optimize the training process of the student network. The multi-task loss function includes a classification loss for supervising prediction results, a feature alignment loss for aligning student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples. Step S60: After the student network training is completed, the student network is deployed only to edge devices to detect defects in power equipment in real time.

2. The method according to claim 1, characterized in that, The step of constructing a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained, wherein the teacher network is a pre-trained large model based on the Transformer architecture and the student network is a convolutional neural network, includes: The CLIP-ViT model was selected as the teacher network, and all parameters of the teacher network were set to a frozen state during the training process of the heterogeneous knowledge distillation system. The lightweight convolutional neural network MobileNetV3 is selected as the student network, and during the training process of the heterogeneous knowledge distillation system, all parameters of the student network are set to a trainable state for optimization through backpropagation. By using the feature comparison path from the feature output of the student network to the feature of the teacher network, a collaborative training framework is constructed in which the teacher network provides knowledge guidance to the student network and knowledge is transferred unidirectionally.

3. The method according to claim 1, characterized in that, The step of implementing asymmetric dual-stream input decoupling, preprocessing the power inspection images to generate two power inspection images with different resolutions, wherein the high-resolution image is input to the student network and the low-resolution image is input to the teacher network, includes: The power inspection images are downsampled to generate low-resolution teacher flow images; The same original power inspection image is cropped and scaled to generate a high-resolution student stream image; The low-resolution teacher stream image is input into the teacher network, and the high-resolution student stream image is input into the student network; wherein the size of the low-resolution teacher stream image includes, but is not limited to, 336×336 pixels, and the size of the high-resolution student stream image includes, but is not limited to, 448×448 pixels.

4. The method according to claim 1, characterized in that, The step of constructing a linear projection layer without a non-linear activation function at the end of the student network to map the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network includes: After the last convolutional or pooling layer of the student network, a linear fully connected layer and a batch normalization layer are connected to construct the linear projection layer without nonlinear activation functions. The original features extracted by the student network are input into the linear projection layer, and linear mapping and normalization are performed to obtain the projected student features, which are represented as follows: ； in, These are the original characteristics of the student network. and The learnable weights and biases of the linear fully connected layer. This indicates a batch normalization operation; The original characteristics of the student backbone network, and For learnable weights and biases, This indicates batch normalization operation. The linear projection layer maps the feature distributions of the student network and the teacher network in a high-dimensional semantic space, thus preserving the semantic information carried by the negative activation values ​​in the teacher network features.

5. The method according to claim 1, characterized in that, The step of calculating the semantic affinity matrix between corresponding samples of student features and teacher features for the power inspection images in the same training batch, and minimizing the difference loss between the two semantic affinity matrices to enable the student network to learn the topological relationships between samples represented by the teacher network, includes: Obtain the student feature matrix from training batch B after mapping by the linear projection layer. and the teacher feature matrix output by the teacher network ; Calculate the student feature matrix separately Teacher Feature Matrix Semantic affinity matrix between corresponding samples and The calculation formula is: ； in, For temperature coefficient, The semantic affinity matrix for students represents the semantic affinity between samples within a batch. Teacher semantic affinity matrix ; Based on the student semantic affinity matrix, The teacher semantic affinity matrix The KL divergence between samples forces the student network to learn the inter-sample topological relationships constructed by the teacher network within a batch.

6. The method according to claim 1, characterized in that, The process of constructing a multi-task loss function and jointly optimizing the training process of the student network using the multi-task loss function includes the steps of classification loss for supervising prediction results, feature alignment loss for aligning student features and teacher features, and relation distillation loss for constraining the topological relationships between samples, including: Based on the predicted results and true labels output by the student network, the binary cross-entropy loss is calculated as the classification loss. ; The student feature matrix and teacher feature matrix are both L2 normalized, and the mean squared error between them is calculated as the feature alignment loss. ; Based on the student semantic affinity matrix and the teacher semantic affinity matrix, the KL divergence between them is calculated as the relation distillation loss. ; The classification loss, feature alignment loss, and relation distillation loss are weighted and summed according to preset weight coefficients to obtain the total loss, and the model parameters of the student network are optimized through the backpropagation algorithm.

7. The method according to claim 1, characterized in that, The step of deploying the student network to edge devices only after the student network training is completed, and performing real-time detection of defects in power equipment, includes: After training is completed, the model parameters of the student network are saved, and the teacher network is stripped from the heterogeneous knowledge distillation system. The trained student network is converted into a format suitable for inference on the edge device and deployed to the edge device; During the operation of the edge device, the power inspection images are acquired in real time and preprocessed, then input into the student network for forward inference, and real-time alarms are issued based on the output defect probability.

8. A heterogeneous feature decoupling distillation defect detection device for power scenarios, characterized in that, include: The network construction module is used to construct a heterogeneous knowledge distillation system consisting of a teacher network with frozen parameters and a student network to be trained. The teacher network is a pre-trained large model based on the Transformer architecture, and the student network is a convolutional neural network. The dual-stream preprocessing module is used to implement asymmetric dual-stream input decoupling, preprocess the power inspection images, and generate two power inspection images with different resolutions. The high-resolution image is input to the student network, and the low-resolution image is input to the teacher network. The feature projection module constructs a linear projection layer without a non-linear activation function at the end of the student network, mapping the features extracted by the student network to a high-dimensional semantic space that matches the features of the teacher network. The heterogeneous feature decoupling distillation module is used to calculate the semantic affinity matrix between corresponding samples of student features and teacher features for the power inspection images in the same training batch. By minimizing the difference loss between the two semantic affinity matrices, the student network learns the topological relationship between samples represented by the teacher network. A multi-task loss construction module is used to construct a multi-task loss function and use the multi-task loss function to jointly optimize the training process of the student network. The multi-task loss function includes a classification loss for supervising prediction results, a feature alignment loss for aligning student features and teacher features, and a relation distillation loss for constraining the topological relationship between samples. The model deployment module is used to deploy the student network to edge devices only after the student network has been trained, so as to detect defects in power equipment in real time.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the heterogeneous feature decoupling distillation defect detection method for power scenarios as described in any one of claims 1-7.

Images