A power transmission line fastener defect detection method and system based on frequency domain distillation

Abstract

The application discloses a kind of power transmission line fastener defect detection method and system based on frequency domain distillation, comprising: image is collected with unmanned aerial vehicle, constructs data set after being labeled, enhances and divides training set and verification set;Residual network is constructed as backbone, and teacher model and light weight self model containing feature pyramid structure and classification and regression branch detection head are included;Teacher model is trained and weight is saved, student model is trained after being loaded, frequency domain transformation is carried out on multi-scale feature by frequency domain perception distillation module, high and low frequency of separation amplitude is calculated and frequency domain distillation loss is calculated, parameter is updated by reverse propagation in combination with student model itself detection loss, and the student model trained is obtained;Verification set is input to obtain detection result.The method enhances the perception of target structure information of student model by frequency domain distillation module, suppresses background noise, improves robustness and precision, and the obtained small model saves storage space, inferences fast, is suitable for edge device deployment, and saves manpower and material resources.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of power transmission line fastener defect detection, in particular to a power transmission line fastener defect detection method and system based on frequency domain distillation. BACKGROUND

[0002] The power transmission line fastener is an important part of the power system transmission and transformation equipment, and mainly plays a role in positioning, connecting and reinforcing various elements in the power transmission line. However, the power transmission line is exposed to complex natural environment for a long time, and is affected by factors such as temperature and humidity, rain, lightning, wind erosion and high temperature. The fastener is prone to corrosion, wear, shedding and even loss, which seriously threatens the operation safety and service life of the power transmission line. Therefore, carrying out fastener defect detection is of great significance to ensure the safety of the power transmission line and improve the reliability of the power grid operation.

[0003] In the face of large-scale power transmission lines, the manual inspection method has the problems of low detection efficiency and high risk coefficient. With the popularization of unmanned aerial vehicle inspection technology in the power industry, collecting power transmission line fastener images through unmanned aerial vehicles and combining image processing algorithms for processing and analysis have become an important means to realize the automatic detection of fastener defects. In recent years, with the rapid development and application of deep learning technology in many fields, power grid enterprises have gradually adopted target detection technology based on deep learning to realize the automatic detection of power transmission line fasteners. Compared with traditional methods, deep learning technology has obvious advantages in detection accuracy and efficiency. However, in complex power transmission line scenes, there are often a lot of interference information in the image background, such as grass, trees, power poles, iron towers and other structures, as well as changes brought by different weather and lighting conditions, which greatly increases the difficulty of target detection. In addition, in actual deployment, edge computing devices are usually relied on for defect detection tasks, and the computing and storage resources of such devices are relatively limited, which is difficult to support the real-time operation of large-scale deep neural networks. As a model compression technology, knowledge distillation is widely used to migrate the ability of large teacher models to lightweight student models, so as to realize the balance between performance and efficiency. However, the existing knowledge distillation method still has the problem of insufficient robustness when facing complex background interference. The student model is easily disturbed by non-target region noise, and it is difficult to fully inherit the discrimination ability of the teacher model to the key target region. This to some extent limits its application effect in actual edge deployment scenes such as power inspection. SUMMARY

[0004] In view of the above technical problems, the present application provides a power transmission line fastener defect detection method based on frequency domain distillation.

[0005] The technical solution adopted by the present application to solve its technical problems is:

[0006] A power line fastener defect detection method based on frequency domain distillation, the method comprising the following steps:

[0007] S100: collecting power line fastener images using a drone, labeling and data enhancing the collected power line fastener images, constructing a defect detection dataset, and dividing it into a training set and a validation set;

[0008] S200: constructing a teacher model and a lightweight student model for power line fastener defect detection, using a residual network as a backbone network, introducing a feature pyramid structure to horizontally fuse low-level high-resolution features and high-level semantic features, and connecting a classification branch and a regression branch at the pyramid level to form a detection head, wherein the classification branch is used to output the category of the target, and the regression branch is used to predict the position of the target;

[0009] S300: training the teacher model using the power line fastener defect detection dataset, saving the trained teacher model weights, loading the saved teacher model weights into the teacher model, training the student model using the defect detection dataset, performing frequency domain transformation on the multi-scale features output by the teacher model and the student model through a frequency domain perception distillation module, further extracting amplitude and phase components, and separating the amplitude part into high and low frequencies, and calculating the frequency domain distillation loss;

[0010] S400: calculating the detection loss of the student model itself, updating the student model parameters based on the frequency domain distillation loss and the detection loss using a backpropagation algorithm, and thus obtaining a trained lightweight student model;

[0011] S500: inputting the validation set into the trained student model to obtain the power line fastener defect detection result.

[0012] Preferably, S100 comprises:

[0013] S110: collecting power line fastener images using a drone, selecting images that meet the requirements from the obtained original power line fastener images in terms of different times, different shooting angles, and different locations, and constructing the most initial unlabeled fault data;

[0014] S120: accurately labeling the defects in the power line fastener images using Label Me software, ensuring that the boundary box of each defect is correctly covered, and thus obtaining a.json format file;

[0015] S130: performing image enhancement operations on the labeled power line fastener images, and thus constructing a power line fastener defect detection dataset;

[0016] S140: dividing the processed power line fastener defect detection dataset into a training set and a validation set.

[0017] Preferably, the backbone network of the teacher model in S200 adopts a residual neural network structure with a depth of 50 layers, and the backbone network of the student model adopts a residual neural network structure with a depth of 18 layers, the residual neural network is composed of a plurality of stacked residual units, and the classification branch and the regression branch are both full convolution network structures connected to each feature pyramid layer.

[0018] Preferably, S500 comprises:

[0019] S510: loading the saved teacher model weight into the corresponding teacher model, and training the student model using the power line fastener defect detection dataset;

[0020] S520: the power line fastener defect image respectively passes through the backbone network and the feature pyramid structure of the teacher model and the student model, and obtains spatial domain feature maps of different scales and , wherein i [0,4], T represents the teacher model, and S represents the student model;

[0021] S530: inputting the spatial domain features of different scales output by the teacher model and the student model and into a frequency domain perception distillation module, mapping the feature maps from the spatial domain to the frequency domain through fast Fourier transform, further extracting the amplitude and phase components thereof, and separating the high and low frequencies of the amplitude part to construct a multi-branch distillation supervision respectively, so as to calculate a frequency domain distillation loss, wherein the frequency domain distillation loss comprises a low-frequency amplitude distillation loss , a high-frequency amplitude distillation loss and a phase distillation loss ;

[0022] S540: calculating the original detection loss of the student model, combining the frequency domain distillation loss and the detection loss to update the parameters of the student model using the back propagation algorithm, and obtaining a trained lightweight student model, wherein the original detection loss of the student model comprises a classification loss and a regression loss .

[0023] Preferably, S530 comprises:

[0024] S531: performing fast Fourier transform on the spatial domain features of different scales output by the teacher model and the student model and to map them from the spatial domain to the frequency domain to obtain frequency domain features and ;

[0025] S532: inputting the frequency domain features and The input is to a frequency domain feature enhancement module, which includes a plurality of convolution operations in series, and the multi-scale frequency domain feature map and is processed through a plurality of convolution layers in sequence to obtain an enhanced frequency domain feature and ;

[0026] S533: The enhanced frequency domain feature and is phase spectrum and amplitude separation, and the amplitude is further separated into low frequency and high frequency parts, and the low frequency amplitude distillation loss and global distillation loss and phase distillation loss .

[0027] Preferably, S531 is specifically:

[0028] ;

[0029] ;

[0030] wherein, , represent the frequency domain feature value of the cth channel of the ith feature map in the teacher model and the student model at the frequency domain coordinate (u, v), H, W represent the height and width of the feature map, , represent the spatial domain feature value of the cth channel of the ith feature map in the teacher model and the student model, represent the frequency domain coordinate;

[0031] S532 is specifically:

[0032] ;

[0033] ;

[0034] wherein, , represent the enhanced frequency domain feature of the cth channel of the ith feature map in the teacher model and the student model, , , represent different convolution operations, represent the activation function.

[0035] Preferably, S533 includes:

[0036] S5331: The enhanced frequency domain feature is phase spectrum and amplitude separation, specifically:

[0037] ;

[0038] ;

[0039] ;

[0040] ;

[0041] wherein, , respectively represent the amplitude value of the cth channel of the ith feature map in the frequency domain coordinates (u, v) in the teacher model and the student model, , respectively represent the phase value of the cth channel of the ith feature map in the frequency domain coordinates (u, v) in the teacher model and the student model, and R and I represent the real part and the imaginary part respectively;

[0042] S5332: High and low frequency separation is performed on the amplitude value, and the implementation is as follows:

[0043] For each channel , a Gaussian convolution kernel is used to perform a two-dimensional convolution operation on the input feature , to obtain a low-frequency feature kernel , , and the expression is as follows:

[0044] ;

[0045] ;

[0046] wherein, represents a two-dimensional convolution operation, is the Gaussian convolution kernel corresponding to the cth channel of the ith feature map;

[0047] The Gaussian convolution kernel is defined as follows:

[0048] ;

[0049] wherein u and v respectively represent the frequency domain coordinates, and respectively represent the center position of the Gaussian distribution in the frequency domain, represents the standard deviation in the cth channel dimension of the ith feature map;

[0050] Subsequently, the input feature , is respectively subtracted by the low-frequency part , Thus, the high-frequency feature is obtained , The expression is:

[0051] ;

[0052] ;

[0053] S5333: Calculate low-frequency amplitude distillation loss and global distillation loss and phase distillation loss , specifically:

[0054] ;

[0055] ;

[0056] ;

[0057] Among them, , and represent low-frequency amplitude distillation loss, high-frequency amplitude distillation loss and phase distillation loss, respectively;

[0058] The final frequency domain distillation loss function is:

[0059] ;

[0060] Among them, are the weight coefficients of different loss terms, used to balance the contribution of different frequency band features to the distillation target.

[0061] Preferably, the detection loss of the student model originally calculated in S540 is specifically:

[0062] ;

[0063] ;

[0064] Among them, denotes the predicted target probability, denotes the true label, is a factor used to control the weight of positive and negative samples, is a weight factor used to control the difficulty of samples, the value is the difference between the predicted value and the true value;

[0065] The final loss function is:

[0066] .

[0067] A power line fastener defect detection system based on frequency domain distillation, comprising:

[0068] A dataset acquisition module is configured to collect power line fastener images using a UAV, label defects in the collected power line fastener images, and perform data enhancement, construct a defect detection dataset, and divide the dataset into a training set and a validation set.

[0069] A teacher and student model construction module is configured to construct a teacher model and a lightweight student model for power line fastener defect detection, use a residual network as a backbone network, introduce a feature pyramid structure to horizontally fuse low-level high-resolution features and high-level semantic features, and connect a classification branch and a regression branch at the pyramid level to form a detection head, wherein the classification branch is configured to output the category of the target, and the regression branch is configured to predict the location of the target.

[0070] A frequency domain distillation loss calculation module is configured to train a teacher model using a power line fastener defect detection dataset, save the trained teacher model weights, load the saved teacher model weights into the teacher model, train a student model using the defect detection dataset, perform frequency domain transformation on the multi-scale features output by the teacher model and the student model, and calculate the frequency domain distillation loss.

[0071] A student model training module is configured to calculate the detection loss of the student model itself, update the student model parameters based on the frequency domain distillation loss and the detection loss using a backpropagation algorithm, and thus obtain a trained lightweight student model.

[0072] A verification and evaluation module is configured to input the validation set into the trained student model and obtain the power line fastener defect detection result.

[0073] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of a power line defect detection method based on frequency domain distillation when executing the computer program.

[0074] The present application provides a power line defect detection method and system based on frequency domain distillation. The scheme designs a frequency domain perception distillation module to enhance the perception ability of the student model to the target structure information and suppress the background noise interference. Specifically, first, the multi-scale feature maps extracted by the teacher model and the student model are converted into frequency domain representation through fast Fourier transform. Considering that the high-frequency region usually contains more edge details and background noise, while the low-frequency region is more concentrated on the main structure information of the target, the module separates the high and low frequency components in the frequency domain and guides the student model to focus on the low-frequency information highly related to the defect target, thereby improving its robustness and detection accuracy. Through the introduction of this module, the student model significantly enhances the perception ability to the key target area while maintaining the lightweight structure, effectively alleviates the performance degradation problem of the knowledge distillation method in complex power scenarios, and improves the practicability on the edge device. The small model obtained by the frequency domain distillation is deployed on the hardware detection device of the power scene, which can save a lot of parameter storage space, while providing good accuracy and inference speed. Moreover, the deep learning method is used for defect detection, which greatly saves manpower and material resources. BRIEF DESCRIPTION OF DRAWINGS

[0075] Figure 1 A flowchart of a power line fastener defect detection method based on frequency domain distillation in an embodiment of the present application;

[0076] Figure 2 A principle framework diagram of a power line fastener defect detection method based on frequency domain distillation in an embodiment of the present application;

[0077] Figure 3 A structure diagram of a frequency domain feature enhancement module in an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to enable personnel in the technical field to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings.

[0079] In one embodiment, as shown in Figure 1 and Figure 2 A power line fastener defect detection method based on frequency domain distillation, the method comprising the following steps:

[0080] S100: Collecting power line fastener images using a UAV, labeling and data enhancing the collected power line fastener images, constructing a defect detection dataset, and dividing it into a training set and a validation set;

[0081] S200: Construct a teacher model and a lightweight student model for transmission line fastener defect detection, adopt a residual network as a backbone network, introduce a feature pyramid structure to laterally fuse low-layer high-resolution features and high-layer semantic features, and connect a classification branch and a regression branch at a pyramid level to form a detection head, wherein the classification branch is used to output the category of the target, and the regression branch is used to predict the position of the target;

[0082] S300: Train the teacher model using the transmission line fastener defect detection dataset, save the trained teacher model weight, load the saved teacher model weight into the teacher model, train the student model using the defect detection dataset, perform frequency domain transformation on the multi-scale features output by the teacher model and the student model through the frequency domain perception distillation module, further extract the amplitude and phase components, separate the high-frequency and low-frequency parts of the amplitude, and calculate the frequency domain distillation loss;

[0083] S400: Calculate the detection loss of the student model itself, update the student model parameters based on the frequency domain distillation loss and the detection loss using the back propagation algorithm, and thus obtain the trained lightweight student model;

[0084] S500: Input the verification set into the trained student model to obtain the transmission line fastener defect detection result.

[0085] Specifically, to solve the problems of complex background interference and limited edge computing resources, the application provides a transmission line defect detection method based on frequency domain distillation. The method designs a frequency domain perception distillation module to enhance the perception ability of the student model to the target structure information and suppress the background noise interference. Specifically, first, the feature maps extracted by the teacher model and the student model are converted into frequency domain representation through fast Fourier transform. Considering that the high-frequency region usually contains more edge details and background noise, and the low-frequency region is more concentrated on the main structure information of the target, the module separates the high-frequency and low-frequency components in the frequency domain, and guides the student model to focus on the low-frequency information highly related to the defect target, thereby improving the robustness and detection accuracy. Through the introduction of the module, the student model significantly enhances the perception ability to the key target area while maintaining the lightweight structure, effectively alleviates the performance degradation problem of the knowledge distillation method in the complex power scene, and improves the practicability on the edge device.

[0086] In one embodiment, S100 includes:

[0087] S110: Collect transmission line fastener images using a UAV, screen the images obtained from the original transmission line fastener images to obtain images meeting the requirements in different times, different shooting angles and different locations, and construct the most initial unlabeled fault data;

[0088] S120: Precise labeling of defects in the power line fastener image is performed using Label Me software to ensure that the bounding box of each defect is correctly covered, and then a.json format file is obtained;

[0089] S130: Image enhancement is performed on the labeled power line fastener image, and then a power line fastener defect detection dataset is constructed;

[0090] S140: The processed power line fastener defect detection dataset is divided into a training set and a validation set.

[0091] Specifically, the image enhancement in S130 includes scaling, cropping, horizontal flipping, contrast enhancement, etc. This operation can increase the diversity and robustness of the data and improve the generalization ability of the model.

[0092] In one embodiment, the backbone network of the teacher model in S200 adopts a residual neural network structure with a depth of 50 layers, and the backbone network of the student model adopts a residual neural network structure with a depth of 18 layers. The residual neural network is composed of multiple stacked residual units. The classification branch and the regression branch are both full convolution network structures and are connected to each feature pyramid layer.

[0093] Specifically, constructing the teacher model includes: building a backbone feature extraction network, designing a feature fusion structure, and constructing a target detection head module. The backbone network preferably adopts a residual neural network structure with a depth of 50 layers. This network is composed of multiple stacked residual units and has strong multi-scale feature extraction capability. To enhance the information fusion between different scale features, a feature pyramid structure is further introduced. The low-layer high-resolution features are fused with the high-layer semantic features through a horizontal connection method to realize fine information transmission and more comprehensive feature expression capability. The detection head is connected after the feature pyramid structure. The detection head includes a classification branch and a regression branch. The classification branch is used to output the class of the target, and the regression branch is used to predict the position of the target. The classification branch is a small full convolution network structure connected to each feature pyramid layer. The regression branch adds another small full convolution network to each pyramid level to restore the anchor box offset to the nearby real object.

[0094] The student model is constructed, including building a backbone feature extraction network, designing a feature fusion structure, and constructing a target detection head module. The backbone network preferably adopts a residual neural network structure with a depth of 18 layers. The network is composed of multiple stacked residual units and has strong multi-scale feature extraction capability. To enhance the information fusion between different scale features, a feature pyramid structure is further introduced. The low-layer high-resolution features are fused with the high-layer semantic features through a horizontal connection method to realize fine information transmission and more comprehensive feature expression capability. The detection head is connected after the feature pyramid structure. The detection head includes two parts, a classification branch and a regression branch. The classification branch is used to output the category of the target, and the regression branch is used to predict the position of the target. The classification branch is a small full convolution network structure connected to each feature pyramid layer. The regression branch adds another small full convolution network to each pyramid level to restore the anchor box offset to the nearby real object.

[0095] In one embodiment, S500 includes:

[0096] S510: Load the saved teacher model weight into the corresponding teacher model, and train the student model using the power line fastener defect detection dataset;

[0097] S520: After the power line fastener defect image passes through the backbone network and the feature pyramid structure of the teacher model and the student model respectively, spatial domain feature maps of different scales are obtained and , wherein i [0,4], T represents the teacher model, and S represents the student model;

[0098] S530: Input the spatial domain features of multiple scales output by the teacher model and the student model and into the frequency domain perception distillation module. The feature maps are mapped from the spatial domain to the frequency domain through fast Fourier transform, and the amplitude and phase components are further extracted. The high and low frequency parts of the amplitude are separated, and multi-branch distillation supervision is constructed respectively, so as to calculate the frequency domain distillation loss, wherein the frequency domain distillation loss includes low frequency amplitude distillation loss , high frequency amplitude distillation loss and phase distillation loss ;

[0099] S540: Calculate the original detection loss of the student model, combine the frequency domain distillation loss and the detection loss, and update the parameters of the student model using the back propagation algorithm to obtain the trained lightweight student model, wherein the original detection loss of the student model includes classification loss and regression loss .

[0100] In one embodiment, S530 comprises:

[0101] S531: inputting the spatial domain features at multiple scales output by the teacher model and the student model and into a fast Fourier transform to map them from the spatial domain to the frequency domain to obtain frequency domain features and ;

[0102] S532: inputting the frequency domain features and into a frequency domain feature enhancement module, the frequency domain feature enhancement module comprising a plurality of convolution operations connected in series, the multi-scale frequency domain feature maps and being processed in turn by several convolution layers to obtain enhanced frequency domain features and ; further, a structural diagram of the frequency domain feature enhancement module is shown in Figure 3 ;

[0103] S533: performing phase spectrum and amplitude separation on the enhanced frequency domain features and , and further separating the amplitudes into low frequency and high frequency parts, and calculating low frequency amplitude distillation loss and global distillation loss and phase distillation loss .

[0104] In one embodiment, S531 specifically comprises:

[0105] ;

[0106] ;

[0107] wherein, , represent the frequency domain feature values of the cth channel of the ith feature map in the teacher model and the student model at the frequency domain coordinates (u, v) respectively, H, W represent the height and width of the feature map, , represent the spatial domain feature values of the cth channel of the ith feature map in the teacher model and the student model respectively, represent the frequency domain coordinates;

[0108] S532 specifically comprises:

[0109] ;

[0110] ;

[0111] wherein, , respectively represent the enhanced frequency domain features of the cth channel of the ith feature map in the teacher model and the student model, , , represent different convolution operations, represent an activation function.

[0112] In one embodiment, S533 comprises:

[0113] S5331: performing phase spectrum and amplitude separation on the enhanced frequency domain features, specifically:

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] wherein, , respectively represent the amplitude values of the cth channel of the ith feature map in the teacher model and the student model at the frequency domain coordinates (u, v), , respectively represent the phase values of the cth channel of the ith feature map in the teacher model and the student model at the frequency domain coordinates (u, v), R and I represent real part and imaginary part respectively;

[0119] S5332: performing high and low frequency separation on the amplitude values, the implementation is as follows:

[0120] For each channel , a Gaussian convolution kernel is adopted to perform a two-dimensional convolution operation on the input feature , to obtain a low-frequency feature kernel , , and the expression is as follows:

[0121] ;

[0122] ;

[0123] wherein, represents a two-dimensional convolution operation, is the Gaussian convolution kernel corresponding to the cth channel of the ith feature map;

[0124] The Gaussian convolution kernel The definitions are as follows:

[0125]

[0126] where u, v represent the frequency domain coordinates respectively, and represent the center positions of the Gaussian distribution in the frequency domain respectively, represent the standard deviation of the c-th channel dimension of the i-th feature map;

[0127] Subsequently, the input features , are subtracted by the low-frequency parts , respectively, so as to obtain the high-frequency features , , and the expression is as follows:

[0128]

[0129]

[0130] S5333: Calculate the low-frequency amplitude distillation loss and the global distillation loss and the phase distillation loss respectively, specifically as follows:

[0131]

[0132]

[0133]

[0134] wherein, , and represent the low-frequency amplitude distillation loss, the high-frequency amplitude distillation loss and the phase distillation loss respectively;

[0135] The final frequency domain distillation loss function is as follows:

[0136]

[0137] wherein, are the weight coefficients of different loss terms, used to balance the contribution of different frequency band features to the distillation target.

[0138] In an embodiment, the detection loss of the student model originally calculated in S540 is specifically as follows:

[0139]

[0140] ;​​​​​​​​

[0141] wherein, represents a predicted target probability, represents a real label, is a factor for controlling the weight of positive and negative samples, is a weight factor for controlling difficult samples, the value is the difference between the predicted value and the real value;

[0142] The final loss function is:

[0143] .

[0144] In this example, the obtained student model weight is loaded into the student model, and the verification set is used to test it. According to the test results, the performance indicators of the model are calculated, such as accuracy, precision, recall, etc.

[0145] The beneficial effects of the above-mentioned power line fastener defect detection method based on frequency domain distillation are as follows:

[0146] (1) The deep learning method is used for defect detection of power line fasteners, which greatly saves manpower and resources.

[0147] (2) A frequency domain perception distillation module is designed. The module converts the feature map to the frequency domain space through fast Fourier transform, and separates and processes the high-frequency and low-frequency components, so that the student model can focus more on the low-frequency structural information highly related to the defect target, while suppressing the background noise interference in the high-frequency region. Effectively alleviate the performance degradation problem of the knowledge distillation method in complex power scenarios.

[0148] (3) The small model obtained by frequency domain distillation can save a lot of parameter storage space when deployed on hardware detection devices in power scenarios, while providing good accuracy and inference speed.

[0149] In one embodiment, a power line fastener defect detection system based on frequency domain distillation is also provided, comprising:

[0150] A data set acquisition module is used to collect power line fastener images using a drone, label and data enhance the collected power line fastener images, construct a defect detection data set, and divide it into a training set and a verification set.

[0151] The teacher and student model construction module is configured to construct a teacher model and a lightweight student model for transmission line fastener defect detection, adopts a residual network as a backbone network, introduces a feature pyramid structure to laterally fuse low-layer high-resolution features and high-layer semantic features, and connects a classification branch and a regression branch at a pyramid level to form a detection head, wherein the classification branch is configured to output a category of a target, and the regression branch is configured to predict a position of the target.

[0152] The frequency domain distillation loss calculation module is configured to train the teacher model using a transmission line fastener defect detection dataset, save a trained teacher model weight, load the saved teacher model weight into the teacher model, train the student model using the defect detection dataset, perform frequency domain transformation on multi-scale features output by the teacher model and the student model, and calculate a frequency domain distillation loss.

[0153] The student model training module is configured to calculate a detection loss of the student model itself, update student model parameters based on the frequency domain distillation loss and the detection loss using a back propagation algorithm, and thus obtain a trained lightweight student model.

[0154] The verification and evaluation module is configured to input a verification set into the trained student model and obtain a transmission line fastener defect detection result.

[0155] Specific limitations of the transmission line fastener defect detection system based on frequency domain distillation can be seen in the limitations of the transmission line fastener defect detection method based on frequency domain distillation described above, and will not be repeated here. Each module in the transmission line fastener defect detection system based on frequency domain distillation can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0156] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the transmission line fastener defect detection method based on frequency domain distillation when executing the computer program.

[0157] A computer readable storage medium stores a computer program, and the computer program implements the steps of the transmission line fastener defect detection method based on frequency domain distillation when executed by a processor.

[0158] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. The volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0159] The above describes in detail the power line fastener defect detection method based on frequency domain distillation provided by the present application. The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the core idea of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A power line fastener defect detection method based on frequency domain distillation, characterized by, The method comprises the following steps: S100: collecting power line fastener images using a UAV, labeling defects in the collected power line fastener images and performing data enhancement, constructing a defect detection dataset, and dividing the dataset into a training set and a validation set; S200: constructing a teacher model and a lightweight student model for power line fastener defect detection, using a residual network as a backbone network, introducing a feature pyramid structure to laterally fuse low-layer high-resolution features and high-layer semantic features, and connecting a classification branch and a regression branch at the pyramid level to form a detection head, wherein the classification branch is used to output the category of the target, and the regression branch is used to predict the position of the target; S300: training the teacher model using the power line fastener defect detection data set, saving the trained teacher model weight, loading the saved teacher model weight into the teacher model, training the student model using the defect detection data set, and obtaining spatial domain feature maps of different scales after the power line fastener defect image passes through the backbone network and the feature pyramid structure of the teacher model and the student model and wherein i [0,4], T represents the teacher model, and S represents the student model; The spatial domain features output by the teacher model and the student model under multiple scales and Performing fast Fourier transform to map it from the spatial domain to the frequency domain to obtain the frequency domain features and ; frequency domain features and The input is fed into the frequency domain feature enhancement module, which includes multiple cascaded convolution operations to generate multi-scale frequency domain feature maps. and The enhanced frequency domain features are obtained by sequentially processing through several convolutional layers. and ; enhanced frequency domain features and performing phase spectrum and amplitude separation, and further separating the amplitude into low and high frequency parts, respectively calculating low frequency amplitude distillation loss and global distillation loss and phase distillation loss ; S400: calculating the detection loss of the student model itself, updating the student model parameters based on the frequency domain distillation loss and the detection loss using a back propagation algorithm, and thus obtaining a trained lightweight student model; S500: inputting the validation set into the trained student model to obtain a power line fastener defect detection result.

2. The method of claim 1, wherein, S100 comprises: S110: collecting power line fastener images using a UAV, screening images that meet the requirements from the obtained original power line fastener images in terms of different times, different shooting angles, and different locations, and constructing the most initial unlabeled fault data; S120: accurately labeling defects in the power line fastener images using Label Me software, ensuring that the boundary box of each defect is correctly covered, and thus obtaining a.json format file; S130: performing image enhancement on the labeled power line fastener images, and thus constructing a power line fastener defect detection dataset; S140: dividing the processed power line fastener defect detection dataset into a training set and a validation set.

3. The method of claim 2, wherein, The backbone network of the teacher model in S200 adopts a residual neural network structure with a depth of 50 layers, and the backbone network of the student model adopts a residual neural network structure with a depth of 18 layers. The residual neural network is composed of multiple stacked residual units. The classification branch and the regression branch are both full convolution network structures and are connected to each feature pyramid layer.

4. The method of claim 3, wherein, The spatial domain features in multiple scales output by the teacher model and the student model in S300 and Performing fast Fourier transform to map the spatial domain features to the frequency domain to obtain frequency domain features and Specifically, ； ； wherein, , respectively denote the frequency domain feature value of the c-th channel of the i-th feature map in the teacher model and the student model at the frequency domain coordinate (u, v), H, W denote the height and width of the feature map, , respectively denote the spatial domain feature value of the c-th channel of the i-th feature map in the teacher model and the student model, denote the frequency domain coordinate; S300 frequency domain characteristics and The input is fed into the frequency domain feature enhancement module, which includes multiple cascaded convolution operations to generate multi-scale frequency domain feature maps. and The enhanced frequency domain features are obtained by sequentially processing through several convolutional layers. and Specifically: ； ； wherein, , respectively represent the enhanced frequency domain features of the cth channel of the ith feature map in the teacher model and the student model, , , represent different convolution operations, represent activation functions.

5. The method of claim 4, wherein, The enhanced frequency domain features in S300 and Phase spectrum and amplitude separation are performed, and the amplitude is further separated into low and high frequency parts, and the low frequency amplitude distillation loss and the global distillation loss and the phase distillation loss , comprising: The enhanced frequency domain features are separated in phase spectrum and amplitude. Specifically, ； ； ； ； wherein, , respectively represent the amplitude value of the cth channel of the ith feature map in the frequency domain coordinate (u, v) in the teacher model and the student model, , respectively represent the phase value of the cth channel of the ith feature map in the frequency domain coordinate (u, v) in the teacher model and the student model, R and I represent the real part and the imaginary part, respectively; The amplitude value is separated into high and low frequencies, and the implementation is as follows: For each channel , a Gaussian convolution kernel is adopted to perform a two-dimensional convolution operation on the input features , to obtain a low-frequency feature kernel , , the expression of which is as follows: ； ； wherein, denotes a two-dimensional convolution operation, is a Gaussian convolution kernel corresponding to the cth channel of the ith feature map; Gaussian convolution kernel is defined as follows: ； where u, v represent the frequency domain coordinates respectively, and represent the center position of the Gaussian distribution in the frequency domain respectively, represents the standard deviation of the cth channel dimension of the ith feature map. Subsequently, the input features , are subtracted by the low frequency parts , respectively, so as to obtain high frequency features , , the expression of which is: ； ； respectively, the low frequency amplitude distillation loss and the global distillation loss and the phase distillation loss are calculated as follows: ； ； ； wherein, , and respectively represent low frequency amplitude distillation loss, high frequency amplitude distillation loss and phase distillation loss; The final frequency domain distillation loss function is: ； wherein, are weight coefficients of different loss terms, respectively, for balancing the contribution of different frequency band features to the distillation target.

6. The method of claim 5, wherein, The detection loss of the student model itself in S400 is calculated as follows: ； ； wherein, denotes the predicted target probability, denotes the true label, is a factor to control the weight of positive and negative samples, is a weight factor to control the difficulty of samples, the value is the difference between the predicted value and the true value; The final loss function is: 。 7. A frequency domain distillation based power line fastener defect detection system performing the method of any one of claims 1 to 6, characterized by, It comprises: A dataset acquisition module for collecting power line fastener images using a UAV, labeling defects in the collected power line fastener images and performing data enhancement, constructing a defect detection dataset, and dividing the dataset into a training set and a validation set; A teacher and student model construction module for constructing a teacher model and a lightweight student model for power line fastener defect detection, using a residual network as a backbone network, introducing a feature pyramid structure to laterally fuse low-layer high-resolution features and high-layer semantic features, and connecting a classification branch and a regression branch at the pyramid level to form a detection head, wherein the classification branch is used to output the category of the target, and the regression branch is used to predict the position of the target; The frequency domain distillation loss calculation module is configured to train a teacher model using a power line fastener defect detection dataset, save the trained teacher model weight, load the saved teacher model weight into the teacher model, train a student model using the defect detection dataset, and calculate a frequency domain distillation loss by performing frequency domain transformation on multi-scale features output by the teacher model and the student model. The student model training module is configured to calculate a detection loss of the student model itself, update the student model parameters based on the frequency domain distillation loss and the detection loss using a back propagation algorithm, and thus obtain a trained lightweight student model. The verification and evaluation module is configured to input a verification set into the trained student model and obtain a power line fastener defect detection result.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6. The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

Images