Power transmission inspection image defect identification method and device based on knowledge distillation and feature compression, equipment, medium and product

Abstract

The invention relates to a power transmission inspection image defect identification method and device based on knowledge distillation and feature compression, equipment, a medium and a product. The method comprises the following steps: performing enhancement processing on an original inspection image of a power transmission line to obtain a target inspection image, constructing a first defect identification model and a second defect identification model, and processing the target inspection image through the first defect identification model and the second defect identification model to obtain a target inspection image; and obtaining a corresponding first defect feature and a corresponding second defect feature, optimizing a second defect identification model based on the first defect feature and the second defect feature, and performing defect identification on the current target inspection image through the optimized second defect identification model. By adopting the method, the recognition accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of power operation and maintenance technology, and in particular to a method, apparatus, equipment, medium and product for identifying defects in transmission line inspection images based on knowledge distillation and feature compression. Background Technology

[0002] With the widespread adoption of drone inspection technology, the amount of inspection image data generated by power transmission lines has exploded, posing a significant challenge to the automated identification of defects such as insulator damage and broken conductor strands. Current defect identification methods mainly rely on deep convolutional neural networks with a large number of parameters. Although these methods offer high accuracy, their high computational complexity and slow inference speed make them difficult to deploy on drone-borne devices or edge devices with limited computing power for real-time detection, thus failing to meet the high accuracy requirements of power transmission line inspection. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, apparatus, equipment, medium, and product for identifying defects in power transmission inspection images based on knowledge distillation and feature compression, which can improve the accuracy of identification, in response to the above-mentioned technical problems.

[0004] Firstly, this application provides a method for defect recognition in power transmission inspection images based on knowledge distillation and feature compression, including:

[0005] Enhancement processing is performed on the original inspection images of transmission lines to obtain the target inspection images;

[0006] Construct a first defect identification model and a second defect identification model; the number of parameters and the depth of the network layers in the first defect identification model are greater than those in the second defect identification model.

[0007] The target inspection image is processed by the first defect recognition model and the second defect recognition model respectively to obtain the corresponding first defect feature and second defect feature; the first defect feature includes the feature components obtained by performing singular value decomposition on the intermediate layer feature map of the first defect recognition model and the first probability distribution; the second defect feature includes the second probability distribution;

[0008] Based on the first and second defect features, the second defect recognition model is optimized, and the optimized second defect recognition model is used to identify defects in the current target inspection image.

[0009] In one embodiment, the step of enhancing the original image of the transmission line inspection to obtain the target inspection image includes:

[0010] The Gaussian pyramid algorithm is used to sample the original images of the transmission line inspection at intervals to obtain multi-level images of different resolutions.

[0011] For each level of image, the image is traversed by a sliding window, and a pixel gray variance corresponding to the sliding window is obtained;

[0012] The filter window is adjusted according to the pixel gray variance, the original image is filtered by the adjusted filter, and a target inspection image is obtained.

[0013] In one embodiment, the process of constructing the second defect recognition model includes:

[0014] An initial network model is constructed;

[0015] The absolute value of the weight of each convolution kernel in the initial network model is obtained, and an importance score is obtained according to the absolute value of the weight;

[0016] The convolution kernel and the corresponding connection channel corresponding to the importance score lower than the preset threshold in the initial network model are removed, and an intermediate network model is obtained;

[0017] The intermediate network model is fine-tuned to obtain a sub-network model, and the plurality of sub-network models are fused to obtain the second defect recognition model.

[0018] In one embodiment, the process of obtaining the feature component by singular value decomposition of the intermediate layer feature map of the first defect recognition model includes:

[0019] The intermediate layer feature map of the first defect recognition model is converted into a two-dimensional feature matrix, the two-dimensional feature matrix is singular value decomposed to obtain a singular value sequence and a singular vector;

[0020] A target singular value is obtained from the singular value sequence, and an updated two-dimensional feature matrix is obtained according to the target singular value and the singular vector corresponding to the target singular value;

[0021] The updated two-dimensional feature matrix is converted into the same dimension as the intermediate layer feature map to obtain the feature component.

[0022] In one embodiment, the step of performing defect recognition on the current target inspection image by the optimized second defect recognition model includes:

[0023] A plurality of target inspection images in a current batch are processed in parallel by the optimized second defect recognition model to obtain a defect confidence score corresponding to each target inspection image;

[0024] A defect category corresponding to the defect confidence score is determined.

[0025] In one embodiment, the method further includes:

[0026] The boundary box coordinates corresponding to each target inspection image are obtained through the second optimized defect identification model; the boundary box coordinates are used to represent the defect position in the corresponding target inspection image;

[0027] For each target inspection image, the defect category corresponding to the target inspection image is converted into a natural language description text, and an inspection report is generated according to the natural language description text and the boundary box coordinates; the inspection report includes the defect position, the defect type and the hazard level.

[0028] In a second aspect, the application further provides a power transmission inspection image defect identification device based on knowledge distillation and feature compression, comprising:

[0029] An image enhancement module is configured to perform enhancement processing on the inspection original image of the power transmission line to obtain a target inspection image.

[0030] A model construction module is configured to construct a first defect identification model and a second defect identification model; the parameter quantity and the network layer depth of the first defect identification model are greater than those of the second defect identification model.

[0031] An image processing module is configured to process the target inspection image through the first defect identification model and the second defect identification model respectively to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition on the intermediate layer feature map of the first defect identification model and a first probability distribution; the second defect features include a second probability distribution.

[0032] A defect identification module is configured to optimize the second defect identification model based on the first defect features and the second defect features, and perform defect identification on the current target inspection image through the optimized second defect identification model.

[0033] In a third aspect, the application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method steps of any one of the first aspect when executing the computer program.

[0034] In a fourth aspect, the application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method steps of any one of the first aspect.

[0035] In a fifth aspect, the application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the method steps of any one of the first aspect.

[0036] The aforementioned method, apparatus, equipment, medium, and product for defect identification of power transmission line inspection images based on knowledge distillation and feature compression enhance the original inspection images of power transmission lines to obtain target inspection images. A first defect identification model and a second defect identification model are constructed. The target inspection image is then processed using both models to obtain corresponding first and second defect features. Based on these features, the second defect identification model is optimized. The optimized model is then used to identify defects in the current target inspection image. This improves defect identification accuracy and efficiency, enabling intelligent and rapid inspection of power transmission lines and effectively ensuring their safety. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is an application environment diagram of a power transmission inspection image defect recognition method based on knowledge distillation and feature compression in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a method for identifying defects in power transmission inspection images based on knowledge distillation and feature compression in one embodiment.

[0040] Figure 3 This is a flowchart illustrating a method for identifying defects in power transmission inspection images based on knowledge distillation and feature compression, as described in another embodiment.

[0041] Figure 4 This is a structural block diagram of a power transmission inspection image defect recognition device based on knowledge distillation and feature compression in one embodiment.

[0042] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0044] The method for identifying defects in power transmission inspection images based on knowledge distillation and feature compression provided in this application can be applied to, for example... Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers. Among them, the terminal 102 is used to enhance the inspection original image of the power transmission line, obtain the target inspection image, construct the first defect recognition model and the second defect recognition model, process the target inspection image through the first defect recognition model and the second defect recognition model respectively, obtain the corresponding first defect feature and second defect feature, and optimize the second defect recognition model based on the first defect feature and the second defect feature. The current target inspection image is recognized by the second defect recognition model after optimization. Among them, the terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude aircraft, Internet of Things devices and portable wearable devices. The Internet of Things device can be a smart speaker, a smart TV, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0045] In an exemplary embodiment, as shown in Figure 2 , a power transmission inspection image defect recognition method based on knowledge distillation and feature compression is provided. Taking the terminal 102 in Figure 1 as an example, the method includes the following steps 202 to 208. Among them:

[0046] S202: Enhancing the inspection original image of the power transmission line to obtain the target inspection image.

[0047] Optionally, the power transmission inspection image is usually collected by a high-definition visible light camera, an infrared thermal imager or an ultraviolet camera mounted on an unmanned aerial vehicle. The resolution of the original image can be as high as 4K or higher, covering multiple defect types such as insulator damage, conductor strand breakage, tower corrosion and bird nest foreign matter. By obtaining the inspection original image of the power transmission line, and performing multi-scale transformation and adaptive filtering processing on the inspection original image, an enhanced power transmission inspection image dataset is generated.

[0048] S204: Constructing a first defect recognition model and a second defect recognition model; the parameter quantity and network layer depth of the first defect recognition model are greater than those of the second defect recognition model.

[0049] Optionally, the first defect recognition model is a teacher network model, which has a large number of parameters and a deep network layer (such as a complex convolutional neural network), has strong feature extraction and defect recognition capabilities, and can learn complex and accurate defect rules in the image. The second defect recognition model is a student network model, which is designed to be lightweight, and is a sparse network with a small number of parameters and low computational complexity.

[0050] S206: The target inspection image is processed by the first defect recognition model and the second defect recognition model respectively to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition of the intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect features include a second probability distribution.

[0051] Optionally, after the target image is input into the first defect recognition model, the model extracts the intermediate layer feature map, rearranges it into a two-dimensional matrix and performs singular value decomposition, selects the first K principal singular values and the corresponding vectors, and reconstructs to obtain feature components with redundancy removed and high energy information. On the other hand, the teacher soft label (i.e. the first probability distribution) is output, which contains more delicate class confidence information. Similarly, after the target image is input into the second defect recognition model, the model outputs the class probability distribution (i.e. the second probability distribution) predicted by itself and the intermediate layer feature map as the benchmark feature for subsequent optimization.

[0052] S208: Based on the first defect features and the second defect features, the second defect recognition model is optimized, and the current target inspection image is recognized by the optimized second defect recognition model.

[0053] Optionally, based on the joint objective function, the second defect recognition model simultaneously fits the principal feature components of the first defect recognition model and the soft label probability distribution of the first defect recognition model, realizing efficient transfer of teacher knowledge to the student model. The optimized second defect recognition model adopts a batch parallel architecture, packs multiple real-time images into an inference batch, uses multiple cores to perform convolution and activation operations synchronously, and quickly outputs the defect category and confidence.

[0054] For example, the mathematical expression of the joint objective function is:

[0055]

[0056] wherein, represents the value of the joint optimization objective function; represents a classification loss function for a defect recognition task; represents an intermediate layer feature map of the second defect recognition model; represents a teacher feature approximation component reconstructed by the jth principal singular value component; The value represents the magnitude of the j-th principal singular value, which serves as a weighting coefficient for feature importance; K represents the number of singular values ​​retained for feature reconstruction. β represents the mean squared error calculation operator; β represents the weighting adjustment factor for characteristic distillation loss.

[0057] in, Essentially, it characterizes the energy proportion or information entropy contribution of the j-th feature component in the original feature space. By... An attention mechanism based on spectral energy was constructed by explicitly using it as a weight term in the loss function. During training, the second defect recognition model prioritizes the optimization capability of gradient descent to fit feature components with large singular values, while the gradient contribution of minor features corresponding to small singular values, such as background noise, is minimized. Suppression. This weighted strategy effectively solves the network convergence problem caused by the traditional loss function's equal weighting of all pixels in the feature map, ensuring the directionality and effectiveness of knowledge transfer. This is achieved by introducing weighting coefficients. This allows the second defect identification model to preferentially fit feature components containing high-energy information.

[0058] In the aforementioned method for defect recognition of power transmission line inspection images based on knowledge distillation and feature compression, the original inspection image of the power transmission line is enhanced to obtain the target inspection image. A first defect recognition model and a second defect recognition model are constructed. The target inspection image is then processed by the first and second defect recognition models respectively to obtain the corresponding first and second defect features. Based on the first and second defect features, the second defect recognition model is optimized. The optimized second defect recognition model is then used to identify defects in the current target inspection image. This method can improve the accuracy and efficiency of defect recognition, realize intelligent and rapid inspection of power transmission lines, and effectively ensure the safety of power transmission lines.

[0059] In an exemplary embodiment, the step of enhancing the original inspection image of a transmission line to obtain a target inspection image includes: performing interval sampling on the original inspection image of the transmission line using the Gaussian pyramid algorithm to obtain multi-level images of different resolutions; for each level of image, traversing the image through a sliding window and obtaining the pixel grayscale variance corresponding to the sliding window; adjusting the filter window according to the pixel grayscale variance, and filtering the original inspection image through the adjusted filter to obtain the target inspection image.

[0060] Optionally, the original image is first smoothed by Gaussian filtering, and then the smoothed image is down-sampled at intervals (e.g., taking one sample point every 1 pixel), to generate multi-level images with gradually reduced resolutions. The essence is to simulate the perception of the human eye to targets of different scales, to ensure that small defects (such as wire micro-breaks) and large defects (such as large-area rust on towers) can be captured at the corresponding level, avoiding defect omission at a single resolution. Then, each level image is traversed pixel by pixel through a fixed-size sliding window, and the gray variance of all pixels in the window is calculated, where a large variance represents an edge / defect (i.e., a significant gray difference), and a small variance represents a flat background (i.e., uniform gray). In this way, the effective defect area and the redundant background area in the image are accurately distinguished. Based on the gray variance, the filter window is dynamically matched, a large-size window (e.g., 7x7) Wiener filter is used for flat background with low variance to strengthen noise suppression effect, and a small-size window (e.g., 3x3) is used for defect / edge area with high variance to reduce the blurring of defect details by filtering. Finally, through multi-level collaborative filtering, the target inspection image with complete noise removal and clear defect features is output.

[0061] In the embodiment, the original inspection image of the transmission line is interval sampled by the Gaussian pyramid algorithm to obtain multi-level images with different resolutions. For each level image, the image is traversed through a sliding window, and the pixel gray variance corresponding to the sliding window is obtained. The filter window is adjusted according to the pixel gray variance, the original inspection image is filtered through the adjusted filter, and the target inspection image is obtained. The image noise can be accurately removed, and the subsequent model recognition efficiency is improved.

[0062] In one exemplary embodiment, the construction process of the second defect recognition model includes: constructing an initial network model; obtaining the weight absolute value of each convolution kernel in the initial network model, and obtaining the importance score according to the weight absolute value; removing the convolution kernel and the corresponding connection channel in the initial network model whose importance score is lower than a preset threshold to obtain an intermediate network model; fine-tuning the intermediate network model to obtain a sub-network model, and fusing a plurality of sub-network models to obtain the second defect recognition model.

[0063] Optionally, based on the lightweight convolutional neural network framework, the initial model itself has a small number of parameters and low computational complexity, and is suitable for the power-limited scene of the unmanned aerial vehicle on-board terminal, thereby providing a basic skeleton for subsequent optimization. Based on global sensitivity analysis technology, the absolute value sum of each convolution kernel weight is calculated as an importance score. The greater the absolute value of the weight is, the higher the contribution of the convolution kernel to defect feature extraction and category determination is, which is the core functional module of the network. Conversely, it is a redundant or low-contribution module. By setting a predetermined threshold, the convolution kernel and its corresponding input / output connection channel with an importance score lower than the threshold are directly removed (rather than simply setting the weight to zero), thereby eliminating the useless redundant structure in the network. Without damaging the core feature extraction capability, the model parameter quantity and computational complexity are greatly reduced.

[0064] Further, the network structure is changed after pruning, and part of the feature extraction capability is temporarily lost. Therefore, fine-tuning is performed on the enhanced target inspection image dataset. A smaller learning rate is used for iterative training to allow the remaining convolution kernels to adapt to the feature distribution again, restore the accuracy decline caused by pruning, and ensure that the model still has stable recognition capability. By training multiple sub-networks with different structures (such as adjusting the number of network layers and the size of the convolution kernel), each sub-network focuses on capturing different types of defect features (such as some are good at identifying broken insulators and some are good at capturing wire breaks). Then, according to the recognition accuracy of each sub-network on the validation set, weights are assigned to the output results, and the weighted average fusion is performed, thereby obtaining a fusion model with stronger generalization capability.

[0065] In the embodiment, the weight absolute value of each convolution kernel in the initial network model is obtained by constructing an initial network model, and the importance score is obtained according to the weight absolute value. The convolution kernel and the corresponding connection channel corresponding to the importance score lower than the preset threshold in the initial network model are removed, thereby obtaining an intermediate network model. The intermediate network model is fine-tuned and trained to obtain a sub-network model. The multiple sub-network models are fused to obtain a second defect recognition model. The second defect recognition model can accurately remove redundant convolution kernels, reduce resource occupation, and improve model recognition accuracy.

[0066] In an exemplary embodiment, the process of singular value decomposition of the intermediate layer feature map of the first defect recognition model to obtain a feature component includes: converting the intermediate layer feature map of the first defect recognition model into a two-dimensional feature matrix, performing singular value decomposition on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector; obtaining a target singular value from the singular value sequence, and obtaining an updated two-dimensional feature matrix according to the target singular value and the singular vector corresponding to the target singular value; converting the updated two-dimensional feature matrix into the same dimension as the intermediate layer feature map to obtain the feature component.

[0067] Optionally, the intermediate layer feature map of the first defect recognition model is a high-dimensional tensor, which cannot be directly singular value decomposed. Therefore, the high-dimensional feature map is first flattened and rearranged, the integrity of the feature dimension is maintained, the spatial dimension is combined into a single dimension, and a two-dimensional feature matrix of feature channel number x total spatial pixel number is converted, which provides an adaptive input for singular value decomposition operation. Singular value decomposition is performed on the two-dimensional feature matrix, and the original matrix is decomposed into the product of three matrices. Among them, the numerical value of the singular value directly represents the energy proportion and information importance of the corresponding feature component, the larger the numerical value, the more defect effective information (such as insulator broken edge, conductor broken strand texture) contained in the corresponding feature component; the smaller the numerical value, the closer to noise or redundant information (such as background clutter texture, image acquisition noise).

[0068] Optionally, the target singular value refers to the first K main singular values with larger numerical values in the singular value sequence. By screening the first K high-energy singular values, noise / redundant features corresponding to small singular values are removed, and the K target singular values are used to reconstruct the diagonal matrix. Combined with the corresponding left singular vector and right singular vector, an updated two-dimensional feature matrix is calculated, which only retains the most critical components with high-value information in the original features, and realizes feature extraction. The reconstructed two-dimensional feature matrix is reshaped according to the channel number x height x width dimension of the original intermediate layer feature map, restored to a tensor consistent with the original feature map structure, and finally the core feature component that can be used for subsequent knowledge distillation is obtained.

[0069] In this embodiment, by converting the intermediate layer feature map of the first defect recognition model into a two-dimensional feature matrix, singular value decomposition is performed on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector, the target singular value is obtained from the singular value sequence, and the updated two-dimensional feature matrix is obtained according to the target singular value and the singular vector corresponding to the target singular value. The two-dimensional feature matrix is converted to the same dimension as the intermediate layer feature map to obtain the feature component, which can effectively filter out the acquisition noise and background redundant information in the intermediate layer feature map of the first model, and ensure that the second defect recognition model can accurately learn the core ability of defect recognition.

[0070] In one exemplary embodiment, the steps of performing defect recognition on the current target inspection image by the optimized second defect recognition model include: processing a plurality of target inspection images in the current batch in parallel by the optimized second defect recognition model to obtain a defect confidence score corresponding to each target inspection image; and determining a defect category corresponding to the defect confidence score.

[0071] Optionally, the plurality of target inspection images in the current batch are packaged into an inference batch through a multi-core architecture of a graphics processing unit (GPU), and the optimized second defect recognition model is input in parallel through a plurality of data input channels. The GPU supports multi-core synchronous operation, and can simultaneously perform convolution, activation, pooling and other operations on all images in the batch, avoiding the waiting time of serial processing of a single image and maximizing the use of hardware computing power. The second defect recognition model has learned the core defect recognition ability of the first model after being optimized by weighted distillation and pruning. The model extracts, maps and classifies the features of each image, and outputs probability values for all preset defect categories. The higher the probability value, the more likely the image belongs to the defect category. A confidence threshold (e.g., 0.8) is set to filter out the categories with a confidence score higher than the threshold in each image. If there are multiple scores higher than the threshold, the category corresponding to the maximum value is taken as the final defect category. If all scores are lower than the threshold, it is determined that there is no defect, and the mapping from quantitative score to specific category is completed.

[0072] In this embodiment, the plurality of target inspection images in the current batch are processed in parallel by the optimized second defect recognition model to obtain the defect confidence scores corresponding to each target inspection image and determine the defect categories corresponding to the defect confidence scores, thereby improving the model processing efficiency and the defect recognition accuracy.

[0073] In one exemplary embodiment, the method further includes: obtaining, by the optimized second defect recognition model, boundary box coordinates corresponding to each target inspection image, wherein the boundary box coordinates represent the positions of defects in the corresponding target inspection images; converting, for each target inspection image, the defect category corresponding to the target inspection image into a natural language description text; and generating an inspection report based on the natural language description text and the boundary box coordinates, wherein the inspection report includes the defect position, the defect type and the hazard level.

[0074] Optionally, the optimized second defect recognition model has a built-in classification and regression dual-branch structure, wherein the classification branch outputs the defect confidence and category, and the regression branch simultaneously predicts the boundary box coordinates of the defect region. After training, the model has learned the spatial position features of the defects, accurately locates the specific range of the defects in the image through regression operation, and realizes the dual output of recognizing the defect type and determining the defect position. Through a pre-established defect category label-natural language term mapping table, the abstract categories output by the model are encoded and mapped into professional texts conforming to the power operation specification.

[0075] In this embodiment, the boundary box coordinates corresponding to each target inspection image are obtained through the optimized second defect identification model, the defect category corresponding to the target inspection image is converted into a natural language description text for each target inspection image, and the inspection report is generated according to the natural language description text and the boundary box coordinates, so that the defect area can be accurately located and the operation and maintenance resource allocation is optimized.

[0076] In one exemplary embodiment, as shown in Figure 3 A power transmission inspection image defect identification method based on knowledge distillation and feature compression is provided, and the method comprises the following steps:

[0077] (1) Image enhancement: interval sampling is performed on the inspection original image of the power transmission line through the Gaussian pyramid algorithm to obtain multi-level images with different resolutions; for each level of image, the image is traversed through a sliding window, and the pixel gray variance corresponding to the sliding window is obtained; the filter window is adjusted according to the pixel gray variance, the inspection original image is filtered through the adjusted filter, and the target inspection image is obtained.

[0078] (2) Model construction: a first defect identification model is constructed; an initial network model is constructed; the weight absolute value of each convolution kernel in the initial network model is obtained, and the importance score is obtained according to the weight absolute value; the convolution kernel and the corresponding connection channel with an importance score lower than a preset threshold in the initial network model are removed to obtain an intermediate network model; the intermediate network model is fine-tuned to obtain a sub-network model, and the plurality of sub-network models are fused to obtain a second defect identification model. The parameter quantity and network layer depth of the first defect identification model are greater than the parameter quantity and network layer depth of the second defect identification model.

[0079] (3) Defect identification: the target inspection image is processed through the first defect identification model and the second defect identification model respectively to obtain corresponding first defect features and second defect features; the first defect features include a feature component obtained by singular value decomposition of an intermediate layer feature map of the first defect identification model and a first probability distribution; the second defect features include a second probability distribution. The intermediate layer feature map of the first defect identification model is converted into a two-dimensional feature matrix, the two-dimensional feature matrix is singular value decomposed to obtain a singular value sequence and a singular vector; a target singular value is obtained from the singular value sequence, and an updated two-dimensional feature matrix is obtained according to the target singular value and the singular vector corresponding to the target singular value; the updated two-dimensional feature matrix is converted into the same dimension as the intermediate layer feature map to obtain the feature component.

[0080] (4) Model optimization: based on the first defect feature and the second defect feature, the second defect recognition model is optimized, and the current batch of multiple target inspection images is processed in parallel through the optimized second defect recognition model to obtain a defect confidence score corresponding to each target inspection image; and a defect category corresponding to the defect confidence score is determined.

[0081] (5) Report generation: the boundary box coordinates corresponding to each target inspection image are obtained through the optimized second defect recognition model; the boundary box coordinates are used to represent the defect position in the corresponding target inspection image; for each target inspection image, the defect category corresponding to the target inspection image is converted into a natural language description text, and an inspection report is generated according to the natural language description text and the boundary box coordinates; the inspection report includes the defect position, the defect type and the hazard level.

[0082] In the embodiment, the target inspection image is obtained by enhancing the inspection original image of the power transmission line, the first defect recognition model and the second defect recognition model are constructed, the target inspection image is processed through the first defect recognition model and the second defect recognition model respectively to obtain the corresponding first defect feature and the second defect feature, the second defect recognition model is optimized based on the first defect feature and the second defect feature, and the defect recognition of the current target inspection image is performed through the optimized second defect recognition model, which can improve the defect recognition accuracy and efficiency, realize the intelligent and rapid inspection of the power transmission line, and effectively guarantee the safety of the power transmission line.

[0083] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0084] Based on the same inventive concept, the embodiments of the present application also provide a knowledge distillation and feature compression based power transmission inspection image defect identification device for implementing the knowledge distillation and feature compression based power transmission inspection image defect identification method described above. The implementation scheme for solving problems provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more knowledge distillation and feature compression based power transmission inspection image defect identification device embodiments provided below can be referred to the limitations of the knowledge distillation and feature compression based power transmission inspection image defect identification method described above, which will not be repeated here.

[0085] In one exemplary embodiment, as shown in Figure 4 A knowledge distillation and feature compression based power transmission inspection image defect identification device is provided, comprising: an image enhancement module 10, a model construction module 20, an image processing module 30 and a defect identification module 40, wherein:

[0086] The image enhancement module 10 is configured to perform enhancement processing on the inspection original image of the power transmission line to obtain a target inspection image.

[0087] The model construction module 20 is configured to construct a first defect identification model and a second defect identification model; the parameter quantity and network layer depth of the first defect identification model are greater than those of the second defect identification model.

[0088] The image processing module 30 is configured to process the target inspection image through the first defect identification model and the second defect identification model respectively to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition on intermediate layer feature maps of the first defect identification model and a first probability distribution; the second defect features include a second probability distribution.

[0089] The defect identification module 40 is configured to optimize the second defect identification model based on the first defect features and the second defect features, and perform defect identification on the current target inspection image through the optimized second defect identification model.

[0090] In one exemplary embodiment, the image enhancement module 10 is further configured to perform interval sampling on the inspection original image of the power transmission line through a Gaussian pyramid algorithm to obtain multi-level images with different resolutions; for each level of image, the image is traversed through a sliding window, and a pixel gray variance corresponding to the sliding window is obtained; a filter window is adjusted according to the pixel gray variance, the inspection original image is filtered through the adjusted filter to obtain the target inspection image.

[0091] In an example embodiment, the model construction module 20 is further configured to construct an initial network model; obtain absolute values of weights of each convolution kernel in the initial network model, and obtain an importance score according to the absolute values of the weights; remove the convolution kernel and the corresponding connection channel corresponding to the importance score lower than a preset threshold in the initial network model to obtain an intermediate network model; perform fine-tuning training on the intermediate network model to obtain a sub-network model, and fuse a plurality of sub-network models to obtain a second defect recognition model.

[0092] In an example embodiment, the image processing module 30 is further configured to convert the intermediate layer feature map of the first defect recognition model into a two-dimensional feature matrix, perform singular value decomposition on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector, obtain a target singular value from the singular value sequence, and obtain an updated two-dimensional feature matrix according to the target singular value and the singular vector corresponding to the target singular value; and convert the updated two-dimensional feature matrix into the same dimension as the intermediate layer feature map to obtain a feature component.

[0093] In an example embodiment, the defect recognition module 40 is further configured to process a plurality of target inspection images in a current batch in parallel through the optimized second defect recognition model to obtain a defect confidence score corresponding to each target inspection image; and determine a defect category corresponding to the defect confidence score.

[0094] In an example embodiment, the defect recognition module 40 is further configured to obtain a bounding box coordinate corresponding to each target inspection image through the optimized second defect recognition model; the bounding box coordinate is used to represent a defect position in the corresponding target inspection image; for each target inspection image, convert the defect category corresponding to the target inspection image into a natural language description text, and generate an inspection report according to the natural language description text and the bounding box coordinate; the inspection report includes a defect position, a defect type, and a hazard level.

[0095] The above-mentioned various modules in the power transmission inspection image defect recognition device based on knowledge distillation and feature compression can be all or partially realized by software, hardware, and combinations thereof. The above-mentioned various modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform operations corresponding to the above-mentioned various modules.

[0096] In an example embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in Figure 5The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, Near Field Communication (NFC) or other technologies. The computer program is executed by the processor to realize a power transmission inspection image defect identification method based on knowledge distillation and feature compression. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0097] Those skilled in the art can understand that, Figure 5 The skilled in the art can understand that,

[0098] In an example embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program: performing enhancement processing on a patrol original image of a power transmission line to obtain a target patrol image; constructing a first defect recognition model and a second defect recognition model; the parameter quantity and network layer depth of the first defect recognition model are greater than those of the second defect recognition model; processing the target patrol image through the first defect recognition model and the second defect recognition model respectively to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition on an intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect features include a second probability distribution; based on the first defect features and the second defect features, optimizing the second defect recognition model, and performing defect recognition on the current target patrol image through the optimized second defect recognition model.

[0099] In an example embodiment, the processor, when executing the computer program, is involved in performing enhancement processing on a patrol original image of a power transmission line to obtain a target patrol image, which includes: performing interval sampling on the patrol original image of the power transmission line through a Gaussian pyramid algorithm to obtain multi-level images of different resolutions; for each level of image, traversing the image through a sliding window and obtaining a pixel gray variance corresponding to the sliding window; adjusting a filter window according to the pixel gray variance, filtering the patrol original image through the adjusted filter to obtain the target patrol image.

[0100] In an example embodiment, the processor, when executing the computer program, is involved in a construction process of the second defect recognition model, which includes: constructing an initial network model; obtaining a weight absolute value of each convolution kernel in the initial network model and obtaining an importance score according to the weight absolute value; removing the convolution kernel and the corresponding connection channel in the initial network model whose importance score is lower than a preset threshold to obtain an intermediate network model; fine-tuning the intermediate network model to obtain a sub-network model, and fusing a plurality of sub-network models to obtain the second defect recognition model.

[0101] In an example embodiment, the processor, when executing the computer program, is involved in a process of performing singular value decomposition on an intermediate layer feature map of the first defect recognition model to obtain feature components, which includes: converting the intermediate layer feature map of the first defect recognition model into a two-dimensional feature matrix, performing singular value decomposition on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector; obtaining a target singular value from the singular value sequence and obtaining an updated two-dimensional feature matrix according to the target singular value and the singular vector corresponding to the target singular value; converting the updated two-dimensional feature matrix into the same dimension as the intermediate layer feature map to obtain the feature components.

[0102] In an embodiment, the processor, when executing the computer program, is directed to performing defect recognition on the current target inspection image by the optimized second defect recognition model, including: processing the plurality of target inspection images in the current batch in parallel by the optimized second defect recognition model, to obtain a defect confidence score corresponding to each target inspection image; and determining a defect category corresponding to the defect confidence score.

[0103] In an embodiment, the processor, when executing the computer program, is further directed to performing the following steps: obtaining, by the optimized second defect recognition model, a bounding box coordinate corresponding to each target inspection image, the bounding box coordinate being used to represent a defect position in the corresponding target inspection image; converting, for each target inspection image, a defect category corresponding to the target inspection image into a natural language description text, and generating an inspection report according to the natural language description text and the bounding box coordinate; and the inspection report including a defect position, a defect type, and a hazard level.

[0104] In an embodiment, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, is directed to performing the following steps: performing enhancement processing on an inspection original image of a power transmission line to obtain a target inspection image; constructing a first defect recognition model and a second defect recognition model; a parameter quantity and a network layer depth of the first defect recognition model are greater than a parameter quantity and a network layer depth of the second defect recognition model; processing the target inspection image by the first defect recognition model and the second defect recognition model respectively to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition on an intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect features include a second probability distribution; and based on the first defect features and the second defect features, optimizing the second defect recognition model, and performing defect recognition on a current target inspection image by the optimized second defect recognition model.

[0105] In an embodiment, the computer program, when executed by a processor, is directed to performing enhancement processing on an inspection original image of a power transmission line to obtain a target inspection image, including: performing interval sampling on the inspection original image of the power transmission line by a Gaussian pyramid algorithm to obtain a plurality of hierarchical images with different resolutions; for each hierarchical image, traversing the image by a sliding window, and obtaining a pixel gray variance corresponding to the sliding window; adjusting a filter window according to the pixel gray variance, and filtering the inspection original image by the adjusted filter to obtain the target inspection image.

[0106] In an embodiment, the computer program, when executed by the processor, involves a process of constructing the second defect identification model, including: constructing an initial network model; obtaining a weight absolute value of each convolution kernel in the initial network model, and obtaining an importance score according to the weight absolute value; removing the convolution kernel and the corresponding connection channel corresponding to the importance score lower than a preset threshold in the initial network model to obtain an intermediate network model; fine-tuning the intermediate network model to obtain a sub-network model, and fusing a plurality of sub-network models to obtain the second defect identification model.

[0107] In an embodiment, the computer program, when executed by the processor, involves a process of performing singular value decomposition on the intermediate layer feature map of the first defect identification model to obtain a feature component, including: converting the intermediate layer feature map of the first defect identification model into a two-dimensional feature matrix, performing singular value decomposition on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector; obtaining a target singular value from the singular value sequence, and obtaining an updated two-dimensional feature matrix according to the target singular value and the singular vector corresponding to the target singular value; converting the updated two-dimensional feature matrix into the same dimension as the intermediate layer feature map to obtain the feature component.

[0108] In an embodiment, the computer program, when executed by the processor, involves a process of performing defect identification on the current target inspection image by the optimized second defect identification model, including: processing a plurality of target inspection images in a current batch in parallel by the optimized second defect identification model to obtain a defect confidence score corresponding to each target inspection image; determining a defect category corresponding to the defect confidence score.

[0109] In an embodiment, the computer program, when executed by the processor, further involves the following steps: obtaining a bounding box coordinate corresponding to each target inspection image by the optimized second defect identification model; the bounding box coordinate is used to represent a defect position in the corresponding target inspection image; for each target inspection image, converting the defect category corresponding to the target inspection image into a natural language description text, and generating an inspection report according to the natural language description text and the bounding box coordinate; the inspection report includes a defect position, a defect type, and a hazard level.

[0110] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps: performing enhancement processing on a patrol original image of a power transmission line to obtain a target patrol image; constructing a first defect recognition model and a second defect recognition model; the parameter quantity and network layer depth of the first defect recognition model are greater than the parameter quantity and network layer depth of the second defect recognition model; respectively processing the target patrol image through the first defect recognition model and the second defect recognition model to obtain corresponding first defect features and second defect features; the first defect features include feature components obtained by singular value decomposition on an intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect features include a second probability distribution; based on the first defect features and the second defect features, optimizing the second defect recognition model, and performing defect recognition on the current target patrol image through the optimized second defect recognition model.

[0111] In one embodiment, the computer program, when executed by a processor, involves performing enhancement processing on a patrol original image of a power transmission line to obtain a target patrol image, including: performing interval sampling on the patrol original image of the power transmission line through a Gaussian pyramid algorithm to obtain multi-level images of different resolutions; for each level of image, traversing the image through a sliding window and obtaining a pixel gray variance corresponding to the sliding window; adjusting a filter window according to the pixel gray variance, filtering the patrol original image through the adjusted filter to obtain the target patrol image.

[0112] In one embodiment, the computer program, when executed by a processor, involves a construction process of the second defect recognition model, including: constructing an initial network model; obtaining the weight absolute value of each convolution kernel in the initial network model, and obtaining an importance score according to the weight absolute value; removing the convolution kernel and the corresponding connection channel in the initial network model whose importance score is lower than a preset threshold to obtain an intermediate network model; fine-tuning the intermediate network model to obtain a sub-network model, and fusing a plurality of sub-network models to obtain the second defect recognition model.

[0113] In one embodiment, the computer program, when executed by a processor, involves a process of performing singular value decomposition on an intermediate layer feature map of the first defect recognition model to obtain a feature component, including: converting the intermediate layer feature map of the first defect recognition model into a two-dimensional feature matrix, performing singular value decomposition on the two-dimensional feature matrix to obtain a singular value sequence and a singular vector; obtaining a target singular value from the singular value sequence, and obtaining an updated two-dimensional feature matrix according to the target singular value and the singular vector corresponding to the target singular value; converting the updated two-dimensional feature matrix into the same dimension as the intermediate layer feature map to obtain the feature component.

[0114] In one embodiment, the computer program, when executed by the processor, involves defect identification on the current target inspection image by the optimized second defect identification model, including: processing a plurality of target inspection images in the current batch in parallel by the optimized second defect identification model to obtain a defect confidence score corresponding to each target inspection image; and determining a defect category corresponding to the defect confidence score.

[0115] In one embodiment, the computer program, when executed by the processor, further implements the following steps: obtaining, by the optimized second defect identification model, a bounding box coordinate corresponding to each target inspection image; the bounding box coordinate is used to represent a defect position in the corresponding target inspection image; for each target inspection image, converting the defect category corresponding to the target inspection image into a natural language description text, and generating an inspection report according to the natural language description text and the bounding box coordinate; the inspection report includes a defect position, a defect type, and a hazard level.

[0116] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0117] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0118] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for defect recognition in power transmission inspection images based on knowledge distillation and feature compression, characterized in that, The method includes: Enhancement processing is performed on the original inspection images of transmission lines to obtain the target inspection images; Construct a first defect identification model and a second defect identification model; the number of parameters and the depth of the network layer of the first defect identification model are greater than the number of parameters and the depth of the network layer of the second defect identification model. The target inspection image is processed by the first defect recognition model and the second defect recognition model respectively to obtain corresponding first defect features and second defect features; the first defect feature includes feature components obtained by performing singular value decomposition on the intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect feature includes a second probability distribution. Based on the first defect feature and the second defect feature, the second defect recognition model is optimized, and the optimized second defect recognition model is used to identify defects in the current target inspection image.

2. The method according to claim 1, characterized in that, The enhancement processing of the original inspection images of the transmission lines to obtain the target inspection image includes: The Gaussian pyramid algorithm is used to sample the original images of the transmission line inspection at intervals to obtain multi-level images of different resolutions. For each level of the image, the image is traversed through a sliding window, and the pixel grayscale variance corresponding to the sliding window is obtained; The filter window is adjusted according to the pixel grayscale variance, and the original inspection image is filtered by the adjusted filter to obtain the target inspection image.

3. The method according to claim 1, characterized in that, The construction process of the second defect identification model includes: Construct the initial network model; Obtain the absolute value of the weights of each convolutional kernel in the initial network model, and obtain an importance score based on the absolute value of the weights; Remove the convolutional kernels and corresponding connection channels in the initial network model that have importance scores below a preset threshold to obtain an intermediate network model; The intermediate network model is fine-tuned and trained to obtain sub-network models. Multiple sub-network models are then fused to obtain a second defect recognition model.

4. The method according to claim 1, characterized in that, The process of obtaining feature components by performing singular value decomposition on the intermediate layer feature map of the first defect identification model includes: The intermediate layer feature map of the first defect identification model is converted into a two-dimensional feature matrix, and the two-dimensional feature matrix is ​​subjected to singular value decomposition to obtain a singular value sequence and singular vectors. Obtain the target singular value from the singular value sequence, and obtain the updated two-dimensional feature matrix based on the target singular value and the singular vector corresponding to the target singular value; The updated two-dimensional feature matrix is ​​converted to the same dimension as the intermediate layer feature map to obtain the feature components.

5. The method according to claim 1, characterized in that, The step of identifying defects in the current target inspection image using the optimized second defect identification model includes: The optimized second defect identification model is used to process multiple target inspection images in the current batch in parallel to obtain the defect confidence score corresponding to each target inspection image. Determine the defect category corresponding to the defect confidence score.

6. The method according to claim 5, characterized in that, The method further includes: The bounding box coordinates corresponding to each target inspection image are obtained by the optimized second defect recognition model; the bounding box coordinates are used to characterize the defect location in the corresponding target inspection image. For each target inspection image, the defect category corresponding to the target inspection image is converted into natural language description text. Based on the natural language description text and the bounding box coordinates, an inspection report is generated. The inspection report includes the defect location, defect type, and hazard level.

7. A power transmission inspection image defect recognition device based on knowledge distillation and feature compression, characterized in that, The device includes: The image enhancement module is used to enhance the original images of the transmission line inspection to obtain the target inspection image; The model building module is used to build a first defect identification model and a second defect identification model; the number of parameters and the depth of the network layers of the first defect identification model are greater than those of the second defect identification model. The image processing module is used to process the target inspection image through the first defect recognition model and the second defect recognition model respectively to obtain corresponding first defect features and second defect features; the first defect feature includes feature components obtained by performing singular value decomposition on the intermediate layer feature map of the first defect recognition model and a first probability distribution; the second defect feature includes a second probability distribution; The defect identification module is used to optimize the second defect identification model based on the first defect feature and the second defect feature, and to identify defects in the current target inspection image through the optimized second defect identification model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images