Electric power inspection fault diagnosis method and system based on multi-spectral image of unmanned aerial vehicle

Abstract

The invention relates to the technical field of power equipment safety inspection. The power inspection fault diagnosis method and system based on the unmanned aerial vehicle multispectral image are provided, and the method comprises the steps: carrying out the multispectral data collection of power equipment, and obtaining a visible light image, multispectral reflectivity data and infrared thermal image data; dynamically correcting emissivity change caused by oxidation or dirt on the surface of the equipment to generate a dynamic emissivity correction coefficient table; according to the dynamic emissivity correction coefficient table, temperature calibration is carried out on the infrared thermal image data, and a calibrated temperature distribution diagram is generated; and fusing the calibrated temperature distribution diagram, the preprocessed visible light image and a multispectral reflectivity distribution diagram obtained by preprocessing the multispectral reflectivity data, performing multimodal fault diagnosis, and outputting fault types and positions, so as to solve the problems that single spectral data is easy to misjudge, emissivity static assumption errors are large, and the accuracy is high in the prior art. And the problem of insufficient confidence caused by multi-modal data splitting analysis is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment safety inspection, and in particular to a power inspection fault diagnosis method and system based on unmanned aerial vehicle multispectral images. Background Art

[0002] With the rapid development of smart grid and drone technology, power equipment inspection has gradually evolved from the traditional manual mode to intelligent and automated. Drones equipped with multi-spectral sensors can efficiently obtain multi-dimensional data such as visible light, infrared and ultraviolet of power equipment, becoming the core means of fault detection in scenarios such as transmission lines and substations.

[0003] However, related technologies have problems such as single spectral data being easily misjudged, large errors in static assumptions about emissivity, and insufficient confidence caused by split analysis of multimodal data. Summary of the invention

[0004] Based on this, it is necessary to provide a power inspection fault diagnosis method and system based on UAV multispectral images to address the above-mentioned technical problems, so as to solve the problems existing in related technologies, such as single spectral data is easy to misjudge, the static assumption error of emissivity is large, and the multimodal data split analysis leads to insufficient confidence.

[0005] In the first aspect, the present application provides a method for diagnosing power inspection faults based on multispectral images of unmanned aerial vehicles, including:

[0006] Collect multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal imaging data;

[0007] Based on visible light images and multi-spectral reflectance data, dynamic correction is performed on the emissivity changes caused by oxidation or contamination on the equipment surface, and a dynamic emissivity correction coefficient table is generated;

[0008] According to the dynamic emissivity correction coefficient table, the infrared thermal imaging data is temperature calibrated to generate a calibrated temperature distribution map;

[0009] The multi-spectral reflectance distribution map obtained by fusing the calibrated temperature distribution map, the pre-processed visible light image and the pre-processed multi-spectral reflectance data is used to perform multi-modal fault diagnosis and output the fault type and location.

[0010] Furthermore, the infrared thermal imaging data is temperature calibrated according to the dynamic emissivity correction coefficient table to generate a calibrated temperature distribution diagram, including:

[0011] Use the following formula and the dynamic emissivity correction coefficient table to match and replace the emissivity values ​​of different areas in the pre-processed infrared thermal imaging data to generate the corrected infrared thermal imaging data:

[0012]

[0013] Among them, ε corr represents the corrected emissivity value, M represents the number of rows of infrared thermal imaging data, N represents the number of columns of infrared thermal imaging data, ε ij Indicates the emissivity value of the corresponding position in the dynamic emissivity correction coefficient table, T ij Indicates the temperature value of the corresponding position in the preprocessed infrared thermal image data;

[0014] Based on the corrected infrared thermal imaging data, the temperature distribution is recalculated to generate a calibrated temperature distribution map.

[0015] Furthermore, according to the dynamic emissivity correction coefficient table, the emissivity values ​​of different regions in the pre-processed infrared thermal imaging data are matched and replaced to generate corrected infrared thermal imaging data, including:

[0016] Use the following formula to spatially align the dynamic emissivity correction coefficient table with the pixel position of the preprocessed infrared thermal imaging data to generate aligned infrared thermal imaging data:

[0017]

[0018] Where A(u,v) represents the spatial alignment transformation function, S(x,y) represents the original signal, W represents the image width, H represents the image height, and u and v represent the frequency domain coordinates;

[0019] The emissivity values ​​of the oxidized areas and the dirty areas in the aligned infrared thermal imaging data are replaced pixel by pixel to generate the corrected infrared thermal imaging data.

[0020] Furthermore, the multi-spectral reflectance distribution map obtained by fusing the calibrated temperature distribution map, the pre-processed visible light image and the pre-processed multi-spectral reflectance data is used to perform multi-modal fault diagnosis and output the fault type and location, including:

[0021] Perform threshold segmentation processing on the abnormal temperature area in the calibrated temperature distribution map to generate candidate fault areas;

[0022] Based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data, multimodal verification processing is performed on the candidate fault area, and the fault type and location are output.

[0023] Furthermore, based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data, multimodal verification processing is performed on the candidate fault area to output the fault type and location, including:

[0024] Perform discharge spot detection processing on the ultraviolet band reflectivity in the multi-spectral reflectivity data to generate a discharge spot distribution map;

[0025] Identify and process the mechanical damage features in the visible light image to generate a mechanical damage area;

[0026] Based on the discharge spot distribution map, mechanical damage area and candidate fault area, spatial logical association verification processing is performed to output the fault type and location.

[0027] Furthermore, based on the discharge spot distribution map, mechanical damage area and candidate fault area, spatial logic association verification processing is performed to output the fault type and location, including:

[0028] Perform threshold judgment processing on the overlap rate between the discharge spot distribution map and the candidate fault area to generate a judgment result, the judgment result includes the overlap rate being greater than or equal to a preset threshold and the overlap rate being less than a preset threshold;

[0029] When the judgment result is that the overlap rate is less than the preset threshold, it is excluded as an interference signal and the fault type and location are output.

[0030] Furthermore, based on the visible light image and multispectral reflectance data, the emissivity change caused by oxidation or contamination on the surface of the equipment is dynamically corrected to generate a dynamic emissivity correction coefficient table, including:

[0031] Surface texture feature extraction and region segmentation are performed on the pre-processed visible light image to generate a mask image of the oxidized or contaminated area on the device surface;

[0032] Based on the multi-spectral reflectance distribution map and mask map, the equipment surface is divided into normal area, oxidation area and dirty area, and the reflectance offset of each area is calculated;

[0033] According to the corresponding relationship between the reflectivity offset and the surface texture characteristics, a dynamic emissivity correction coefficient table is generated.

[0034] In a second aspect, the present application also provides a power inspection fault diagnosis system based on drone multispectral images, the system comprising:

[0035] Multispectral data acquisition and preprocessing module, used to collect multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal imaging data;

[0036] Dynamic emissivity correction module, which is used to dynamically correct the emissivity changes caused by oxidation or contamination on the equipment surface based on visible light images and multi-spectral reflectivity data, and generate a dynamic emissivity correction coefficient table;

[0037] Infrared temperature calibration module, used to perform temperature calibration on infrared thermal imaging data according to the dynamic emissivity correction coefficient table and generate a calibrated temperature distribution diagram;

[0038] The multimodal fault diagnosis module is used to fuse the calibrated temperature distribution map, visible light image and multispectral reflectance data to perform multimodal fault diagnosis and output the fault type and location.

[0039] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of any method in the first aspect of the present application when executing the computer program.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any method in the first aspect of the present application are implemented.

[0041] The technical solution provided by the present application includes the following technical effects: by providing a power inspection fault diagnosis method and system based on unmanned aerial vehicle multispectral images, the method includes: collecting multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal imaging data; based on the visible light image and the multispectral reflectance data, dynamically correcting the emissivity changes caused by oxidation or contamination on the equipment surface, and generating a dynamic emissivity correction coefficient table; according to the dynamic emissivity correction coefficient table, temperature calibrating the infrared thermal imaging data to generate a calibrated temperature distribution map; fusing the calibrated temperature distribution map, the pre-processed visible light image and the pre-processed multispectral reflectance data to obtain a multispectral reflectance distribution map, perform multimodal fault diagnosis, and output the fault type and location, so as to solve the problems existing in the related technology that single spectral data is easy to misjudge, the static assumption error of the emissivity is large, and the multimodal data split analysis leads to insufficient confidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of a method for diagnosing electric power inspection faults based on multispectral images of unmanned aerial vehicles in one embodiment of the present invention;

[0044] Figure 2 The present invention is a structural diagram of a power inspection fault diagnosis system based on multispectral images of unmanned aerial vehicles in one embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and understandable, the specific real-time methods of the present application are described in detail below in conjunction with the accompanying drawings. Many specific details are set forth in the following description to fully understand the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the application, so the present application is not limited by the specific embodiments disclosed below.

[0046] like Figure 1 As shown, the present application provides a power inspection fault diagnosis method based on drone multispectral images, including:

[0047] S101: Collect multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal image data.

[0048] Specifically, a multispectral imaging system that integrates visible light cameras, multispectral sensors, and infrared thermal imagers is used on drones. The drones fly autonomously to the vicinity of power equipment according to the preset inspection route to ensure data collection within a suitable distance and angle range. The visible light camera captures the appearance image of the power equipment with high resolution and records the macroscopic state of the equipment, including the structure, color, and obvious physical damage of the equipment, providing an intuitive visual basis for subsequent fault analysis. At the same time, the multispectral sensor scans within a specific band range to obtain reflectivity data of different bands on the surface of the equipment. The above data can reflect the spectral characteristics of the equipment material and the chemical or physical changes on the surface, such as reflectivity differences caused by oxidation, contamination, etc. The infrared thermal imager generates infrared thermal imaging data by detecting the infrared radiation emitted from the surface of the equipment, presenting the invisible heat distribution in the form of images, and helping to identify abnormal heating areas caused by overload, poor contact, etc. The collected visible light images, multispectral reflectivity data, and infrared thermal imaging data are transmitted to the ground control center or background processing system via wireless transmission or storage media for further analysis and processing.

[0049] S102: Based on the visible light image and the multi-spectral reflectance data, dynamically correct the emissivity change of the equipment surface caused by oxidation or contamination, and generate a dynamic emissivity correction coefficient table.

[0050] Specifically, the collected visible light images and multispectral reflectance data are preprocessed to remove noise and correct geometric distortion. Then, the visible light images are used to extract surface texture features. By analyzing the grayscale, edge and texture information in the image, the areas on the device surface that may be oxidized or contaminated are identified, and the corresponding mask map is generated. At the same time, combined with the multispectral reflectance data, the normal area, oxidation area and contaminated area on the device surface are further divided according to the reflectance difference of each band. By comparing the reflectance values ​​of the above areas, the reflectance offset of each area is calculated, that is, the reflectance change compared with the normal area. Afterwards, according to the corresponding relationship between the reflectance offset and the surface texture characteristics, a dynamic emissivity correction coefficient table is established. The table quantifies the emissivity correction coefficients corresponding to different texture features, so that in the subsequent infrared thermal imaging data analysis, accurate dynamic correction of emissivity can be performed according to the specific situation of the device surface, thereby improving the accuracy of temperature measurement.

[0051] S103: Performing temperature calibration on the infrared thermal imaging data according to the dynamic emissivity correction coefficient table to generate a calibrated temperature distribution diagram.

[0052] Specifically, the collected infrared thermal imaging data is preprocessed to remove noise and perform geometric correction. Then, the previously generated dynamic emissivity correction coefficient table is used to match the corresponding emissivity correction coefficient according to the texture characteristics of different areas on the surface of the device. The temperature value in the original infrared thermal imaging data is adjusted by applying the above correction coefficient to the temperature calculation formula of the infrared thermal imaging data. Among them, the temperature of each pixel is recalculated using the corrected emissivity value to generate a more accurate calibrated temperature distribution map. The above process involves aligning the dynamic emissivity correction coefficient table with the pixel position of the infrared thermal imaging data to ensure that the temperature calibration of each area is based on the correct emissivity value, and to obtain a calibrated temperature distribution map that can truly reflect the temperature distribution on the surface of the equipment, providing a reliable data basis for subsequent fault diagnosis.

[0053] S104: fusing the calibrated temperature distribution map, the preprocessed visible light image, and the multispectral reflectance distribution map obtained by preprocessing the multispectral reflectance data, performing multimodal fault diagnosis, and outputting the fault type and location.

[0054] Specifically, the calibrated temperature distribution map, the preprocessed visible light image, and the multispectral reflectance distribution map are aligned and fused. By analyzing the calibrated temperature distribution map, the abnormal temperature area is identified as the candidate fault area. Then, the above candidate fault area is multimodally verified using the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectance data. Among them, the discharge spot detection is performed on the ultraviolet band reflectivity in the multispectral reflectance data to generate a discharge spot distribution map; at the same time, the mechanical damage features in the visible light image are identified to generate a mechanical damage area. Afterwards, a comprehensive judgment is made based on the spatial logical association between the discharge spot distribution map, the mechanical damage area and the candidate fault area. If the overlap rate between the discharge spot distribution map and the candidate fault area is greater than or equal to the preset threshold, it is confirmed as a fault area, and the fault type and specific location are further determined in combination with the mechanical damage features, and an accurate fault diagnosis result is output.

[0055] The electric power inspection fault diagnosis method based on drone multispectral images provided in the embodiment of the present application obtains visible light images, multispectral reflectance data and infrared thermal imaging data by collecting multispectral data of electric power equipment; based on the visible light image and multispectral reflectance data, dynamically corrects the emissivity change caused by oxidation or contamination on the equipment surface, and generates a dynamic emissivity correction coefficient table; according to the dynamic emissivity correction coefficient table, performs temperature calibration on the infrared thermal imaging data to generate a calibrated temperature distribution map; performs multimodal fault diagnosis by fusing the calibrated temperature distribution map, the pre-processed visible light image and the pre-processed multispectral reflectance data to obtain a multispectral reflectance distribution map, outputs the fault type and location, so as to solve the problems existing in the related technology that single spectral data is easy to misjudge, the static assumption error of emissivity is large, and the multimodal data split analysis leads to insufficient confidence.

[0056] Furthermore, the infrared thermal imaging data is temperature calibrated according to the dynamic emissivity correction coefficient table to generate a calibrated temperature distribution diagram, including:

[0057] Use the following formula and the dynamic emissivity correction coefficient table to match and replace the emissivity values ​​of different areas in the pre-processed infrared thermal imaging data to generate the corrected infrared thermal imaging data:

[0058]

[0059] Among them, ε corr represents the corrected emissivity value, M represents the number of rows of infrared thermal imaging data, N represents the number of columns of infrared thermal imaging data, ε ij Indicates the emissivity value of the corresponding position in the dynamic emissivity correction coefficient table, T ij Indicates the temperature value of the corresponding position in the preprocessed infrared thermal image data;

[0060] Based on the corrected infrared thermal imaging data, the temperature distribution is recalculated to generate a calibrated temperature distribution map.

[0061] Specifically, the collected infrared thermal imaging data is preprocessed to remove noise and perform geometric correction. Then, the previously generated dynamic emissivity correction coefficient table is used to match the corresponding emissivity correction coefficient according to the texture characteristics of different areas on the surface of the equipment. The temperature value in the original infrared thermal imaging data is adjusted by applying the above correction coefficient to the temperature calculation formula of the infrared thermal imaging data. Among them, the temperature of each pixel is recalculated using the corrected emissivity value to generate a more accurate calibrated temperature distribution map. This process involves aligning the dynamic emissivity correction coefficient table with the pixel position of the infrared thermal imaging data to ensure that the temperature calibration of each area is based on the correct emissivity value, and to obtain a calibrated temperature distribution map that can truly reflect the temperature distribution on the surface of the equipment, providing a reliable data basis for subsequent fault diagnosis.

[0062] Furthermore, according to the dynamic emissivity correction coefficient table, the emissivity values ​​of different regions in the pre-processed infrared thermal imaging data are matched and replaced to generate corrected infrared thermal imaging data, including:

[0063] Use the following formula to spatially align the dynamic emissivity correction coefficient table with the pixel position of the preprocessed infrared thermal imaging data to generate aligned infrared thermal imaging data:

[0064]

[0065] Where A(u,v) represents the spatial alignment transformation function, S(x,y) represents the original signal, W represents the image width, H represents the image height, and u and v represent the frequency domain coordinates;

[0066] The emissivity values ​​of the oxidized areas and the dirty areas in the aligned infrared thermal imaging data are replaced pixel by pixel to generate the corrected infrared thermal imaging data.

[0067] Specifically, the dynamic emissivity correction coefficient table is spatially aligned with the preprocessed infrared thermal imaging data. This step is achieved through the spatial alignment transformation function to ensure that the two correspond one-to-one in pixel position. Among them, the dynamic emissivity correction coefficient table is matched with the pixel position of the infrared thermal imaging data using the spatial alignment transformation function in the formula, where W and H represent the width and height of the image, respectively, and u and v are frequency domain coordinates for transformation and alignment operations in the frequency domain. After the spatial alignment process, the aligned infrared thermal imaging data is obtained. Then, the emissivity value is replaced pixel by pixel for the oxidized area and the dirty area in the aligned infrared thermal imaging data. According to the emissivity value in the previously generated dynamic emissivity correction coefficient table, the emissivity value of the corresponding position in the original infrared thermal imaging data is replaced to generate the corrected infrared thermal imaging data. This process ensures that the emissivity value of each pixel point can be accurately corrected according to the actual condition of the equipment surface, providing a more accurate data basis for subsequent temperature calibration and fault diagnosis.

[0068] Furthermore, the multi-spectral reflectance distribution map obtained by fusing the calibrated temperature distribution map, the pre-processed visible light image and the pre-processed multi-spectral reflectance data is used to perform multi-modal fault diagnosis and output the fault type and location, including:

[0069] Perform threshold segmentation processing on the abnormal temperature area in the calibrated temperature distribution map to generate candidate fault areas;

[0070] Based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data, multimodal verification processing is performed on the candidate fault area, and the fault type and location are output.

[0071] Specifically, the calibrated temperature distribution map is subjected to threshold segmentation processing. By setting the threshold of temperature anomaly, the area in the temperature distribution map that exceeds the threshold is identified as an abnormal area temperature. The above area may potentially have a fault, thereby generating a candidate fault area. Then, based on the mechanical damage features in the preprocessed visible light image, such as damage and cracks of equipment components, and the discharge spot features in the multispectral reflectivity data, the above candidate fault area is subjected to multimodal verification processing. Among them, the position, shape and severity of the mechanical damage in the visible light image are analyzed, and the distribution of the discharge spot is extracted from the multispectral reflectivity data. These features are spatially matched and logically associated with the candidate fault area. If the candidate fault area overlaps with the mechanical damage feature or the discharge spot feature in spatial position or conforms to a specific logical relationship, the area is confirmed to be the actual fault area, and further outputs the specific fault type and location information according to the type and combination of features, thereby realizing accurate diagnosis of power equipment faults.

[0072] Furthermore, based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data, multimodal verification processing is performed on the candidate fault area to output the fault type and location, including:

[0073] Perform discharge spot detection processing on the ultraviolet band reflectivity in the multi-spectral reflectivity data to generate a discharge spot distribution map;

[0074] Identify and process the mechanical damage features in the visible light image to generate a mechanical damage area;

[0075] Based on the discharge spot distribution map, mechanical damage area and candidate fault area, spatial logical association verification processing is performed to output the fault type and location.

[0076] Specifically, discharge spot detection processing is performed on the ultraviolet band reflectivity in the multispectral reflectivity data. Through specific image processing algorithms, such as threshold segmentation, morphological operations, etc., the areas with abnormal reflectivity in the ultraviolet band are identified. The above areas may correspond to the discharge phenomenon on the surface of the equipment, thereby generating a discharge spot distribution map. At the same time, the mechanical damage features in the visible light image are identified and processed, and the mechanical damage on the surface of the equipment, such as cracks and breakage, is detected by using the target detection algorithm or image segmentation technology to generate a mechanical damage area. After that, based on the discharge spot distribution map, the mechanical damage area and the candidate fault area, a spatial logical association verification process is performed. By comparing the overlap and logical relationship of these areas in spatial position, such as judging whether the discharge spot area and the mechanical damage area have an overlap or adjacent relationship with the candidate fault area in space, if so, the type and location of the fault are further confirmed, and the final fault diagnosis result is output, including the specific fault type (such as overheating fault, discharge fault, mechanical damage fault, etc.) and its precise position coordinates on the equipment.

[0077] Furthermore, based on the discharge spot distribution map, mechanical damage area and candidate fault area, spatial logic association verification processing is performed to output the fault type and location, including:

[0078] Perform threshold judgment processing on the overlap rate between the discharge spot distribution map and the candidate fault area to generate a judgment result, the judgment result includes the overlap rate being greater than or equal to a preset threshold and the overlap rate being less than a preset threshold;

[0079] When the judgment result is that the overlap rate is less than the preset threshold, it is excluded as an interference signal and the fault type and location are output.

[0080] Specifically, the overlap rate between the discharge spot distribution map and the candidate fault area is calculated. By comparing the pixel positions of the two areas, the ratio of the number of pixels in the overlapping part to the total number of pixels in the candidate fault area is counted to obtain the overlap rate value. Then, the calculated overlap rate is compared with the preset threshold to generate a judgment result. If the overlap rate is greater than or equal to the preset threshold, it means that the discharge spot area and the candidate fault area have a high degree of consistency in spatial position, which further supports the possibility of a fault in the area. Combined with the verification information of the mechanical damage area, the fault type and location are determined. On the contrary, when the judgment result is that the overlap rate is less than the preset threshold, it is confirmed that the candidate fault area may be an interference signal caused by other factors, such as environmental noise, thermal radiation during normal operation of the equipment, etc. At this time, the possibility that the area is the actual fault area is excluded and it is not used as the final fault output.

[0081] Furthermore, based on the visible light image and multispectral reflectance data, the emissivity change caused by oxidation or contamination on the surface of the equipment is dynamically corrected to generate a dynamic emissivity correction coefficient table, including:

[0082] Perform surface texture feature extraction and region segmentation on the pre-processed visible light image to generate a mask image of the oxidized or contaminated area on the device surface;

[0083] Based on the multi-spectral reflectance distribution map and mask map, the equipment surface is divided into normal area, oxidation area and dirty area, and the reflectance offset of each area is calculated;

[0084] According to the corresponding relationship between the reflectivity offset and the surface texture characteristics, a dynamic emissivity correction coefficient table is generated.

[0085] Specifically, the surface texture features of the pre-processed visible light image are extracted and the region is segmented. Image processing techniques, such as gray-level co-occurrence matrix (GLCM) and other methods, are used to analyze the texture features in the visible light image, identify the areas on the device surface that may be oxidized or contaminated, and generate a corresponding mask map, which divides the device surface into different areas for subsequent processing. Then, the multispectral reflectance distribution map and the mask map are combined to further divide the normal area, oxidation area and contaminated area on the device surface. By comparing the reflectance values ​​of the above-mentioned areas in different bands, the reflectance offset of each area is calculated, that is, the reflectance change compared with the normal area. This step involves detailed analysis and processing of the multispectral reflectance data to ensure the accuracy of the reflectance offset. Afterwards, a dynamic emissivity correction coefficient table is generated based on the correspondence between the reflectance offset and the surface texture features. Among them, by establishing a mapping relationship between the reflectivity offset and the texture feature, a correction coefficient is determined for the area corresponding to each texture feature. The correction coefficient can reflect the change in the emissivity of the area. Therefore, in the subsequent infrared thermal imaging data analysis, the emissivity value can be dynamically adjusted according to the specific texture characteristics of the device surface to improve the accuracy of temperature measurement.

[0086] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0087] In one embodiment, Figure 2 As shown, the present application also provides a power inspection fault diagnosis system 200 based on drone multispectral images, the system comprising:

[0088] The multispectral data acquisition and preprocessing module 201 is used to acquire multispectral data of the power equipment to obtain visible light images, multispectral reflectance data and infrared thermal image data;

[0089] A dynamic emissivity correction module 202 is used to dynamically correct the emissivity change caused by oxidation or contamination on the surface of the equipment based on the visible light image and the multi-spectral reflectivity data, and generate a dynamic emissivity correction coefficient table;

[0090] The infrared temperature calibration module 203 is used to perform temperature calibration on the infrared thermal imaging data according to the dynamic emissivity correction coefficient table and generate a calibrated temperature distribution diagram;

[0091] The multi-modal fault diagnosis module 204 is used to fuse the calibrated temperature distribution map, visible light image and multi-spectral reflectance data to perform multi-modal fault diagnosis and output the fault type and location.

[0092] Specifically, the visible light image, multispectral reflectance data and infrared thermal imaging data of the power equipment are obtained through the multispectral data acquisition and preprocessing module 201 as the basis for subsequent analysis. The dynamic emissivity correction module 202 uses the visible light image and multispectral reflectance data to dynamically correct the emissivity changes caused by oxidation or contamination on the equipment surface, and generates a dynamic emissivity correction coefficient table to improve the accuracy of the infrared thermal imaging data. The infrared temperature calibration module 203 performs temperature calibration on the infrared thermal imaging data according to the coefficient table, and generates a calibrated temperature distribution map to reflect the actual temperature of the equipment surface. The multimodal fault diagnosis module 204 integrates the calibrated temperature distribution map, visible light image and multispectral reflectance data to perform multimodal fault diagnosis, integrate information from multiple data sources, improve the accuracy and reliability of fault diagnosis, and finally output the fault type and location to achieve a more comprehensive and accurate detection of power equipment faults.

[0093] The infrared temperature calibration module 203 is also used for:

[0094] Use the following formula and the dynamic emissivity correction coefficient table to match and replace the emissivity values ​​of different areas in the pre-processed infrared thermal imaging data to generate the corrected infrared thermal imaging data:

[0095]

[0096] Among them, ε corr represents the corrected emissivity value, M represents the number of rows of infrared thermal imaging data, N represents the number of columns of infrared thermal imaging data, ε ij Indicates the emissivity value of the corresponding position in the dynamic emissivity correction coefficient table, T ij Indicates the temperature value of the corresponding position in the preprocessed infrared thermal image data;

[0097] Based on the corrected infrared thermal imaging data, the temperature distribution is recalculated to generate a calibrated temperature distribution map.

[0098] The infrared temperature calibration module 203 is also used for:

[0099] Use the following formula to spatially align the dynamic emissivity correction coefficient table with the pixel position of the preprocessed infrared thermal imaging data to generate aligned infrared thermal imaging data:

[0100]

[0101] Where A(u,v) represents the spatial alignment transformation function, S(x,y) represents the original signal, W represents the image width, H represents the image height, and u and v represent the frequency domain coordinates;

[0102] The emissivity values ​​of the oxidized areas and the dirty areas in the aligned infrared thermal imaging data are replaced pixel by pixel to generate the corrected infrared thermal imaging data.

[0103] The multi-modal fault diagnosis module 204 is also used for:

[0104] Perform threshold segmentation processing on the abnormal temperature area in the calibrated temperature distribution map to generate candidate fault areas;

[0105] Based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data, multimodal verification processing is performed on the candidate fault area, and the fault type and location are output.

[0106] The multi-modal fault diagnosis module 204 is also used for:

[0107] Perform discharge spot detection processing on the ultraviolet band reflectivity in the multi-spectral reflectivity data to generate a discharge spot distribution map;

[0108] Identify and process the mechanical damage features in the visible light image to generate a mechanical damage area;

[0109] Based on the discharge spot distribution map, mechanical damage area and candidate fault area, spatial logical association verification processing is performed to output the fault type and location.

[0110] The multi-modal fault diagnosis module 204 is also used for:

[0111] Perform threshold judgment processing on the overlap rate between the discharge spot distribution map and the candidate fault area to generate a judgment result, the judgment result includes the overlap rate being greater than or equal to a preset threshold and the overlap rate being less than a preset threshold;

[0112] When the judgment result is that the overlap rate is less than the preset threshold, it is excluded as an interference signal and the fault type and location are output.

[0113] The dynamic emissivity correction module 202 is also used for:

[0114] Surface texture feature extraction and region segmentation are performed on the pre-processed visible light image to generate a mask image of the oxidized or contaminated area on the device surface;

[0115] Based on the multi-spectral reflectance distribution map and mask map, the equipment surface is divided into normal area, oxidation area and dirty area, and the reflectance offset of each area is calculated;

[0116] According to the corresponding relationship between the reflectivity offset and the surface texture characteristics, a dynamic emissivity correction coefficient table is generated.

[0117] In one embodiment, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0118] In one embodiment, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0119] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the components described as separate parts may or may not be physically separated, and the parts displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0120] The above-mentioned embodiments only express several implementation methods of the embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the embodiments of the present application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the embodiments of the present application, and these all belong to the protection scope of the embodiments of the present application.

Claims

1. A power inspection fault diagnosis method based on UAV multispectral images, characterized in that: The method comprises: Collect multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal imaging data; Based on the visible light image and the multi-spectral reflectance data, dynamically correct the emissivity change caused by oxidation or contamination on the surface of the equipment, and generate a dynamic emissivity correction coefficient table; According to the dynamic emissivity correction coefficient table, the infrared thermal imaging data is temperature calibrated to generate a calibrated temperature distribution diagram; The calibrated temperature distribution map, the preprocessed visible light image and the multispectral reflectance distribution map obtained after preprocessing the multispectral reflectance data are integrated to perform multimodal fault diagnosis and output the fault type and location.

2. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 1 is characterized in that: The step of performing temperature calibration on the infrared thermal imaging data according to the dynamic emissivity correction coefficient table to generate a calibrated temperature distribution diagram includes: Use the following formula to match and replace the emissivity values ​​of different areas in the pre-processed infrared thermal imaging data according to the dynamic emissivity correction coefficient table to generate corrected infrared thermal imaging data: Among them, ε corr represents the corrected emissivity value, M represents the number of rows of infrared thermal imaging data, N represents the number of columns of infrared thermal imaging data, ε ij Indicates the emissivity value of the corresponding position in the dynamic emissivity correction coefficient table, T ij Indicates the temperature value of the corresponding position in the preprocessed infrared thermal image data; Based on the corrected infrared thermal imaging data, the temperature distribution is recalculated to generate the calibrated temperature distribution map.

3. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 2 is characterized in that: The step of performing matching and replacement processing on the emissivity values ​​of different regions in the pre-processed infrared thermal imaging data according to the dynamic emissivity correction coefficient table to generate corrected infrared thermal imaging data includes: The dynamic emissivity correction coefficient table and the pixel positions of the pre-processed infrared thermal imaging data are spatially aligned using the following formula to generate aligned infrared thermal imaging data: Where A(u,v) represents the spatial alignment transformation function, S(x,y) represents the original signal, W represents the image width, H represents the image height, and u and v represent the frequency domain coordinates; The oxidized area and the dirty area in the aligned infrared thermal imaging data are subjected to pixel-by-pixel replacement processing of the emissivity value to generate the corrected infrared thermal imaging data.

4. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 1 is characterized in that: The method of fusing the calibrated temperature distribution map, the preprocessed visible light image, and the multispectral reflectance distribution map obtained by preprocessing the multispectral reflectance data, performing multimodal fault diagnosis, and outputting the fault type and location includes: Performing threshold segmentation processing on the abnormal temperature area in the calibrated temperature distribution map to generate a candidate fault area; Based on the mechanical damage features in the visible light image and the discharge spot features in the multi-spectral reflectivity data, a multi-modal verification process is performed on the candidate fault area to output the fault type and location.

5. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 4 is characterized in that: The method of performing multimodal verification processing on the candidate fault area based on the mechanical damage features in the visible light image and the discharge spot features in the multispectral reflectivity data and outputting the fault type and location includes: Performing discharge light spot detection processing on the ultraviolet band reflectivity in the multi-spectral reflectivity data to generate a discharge light spot distribution map; Identifying and processing the mechanical damage features in the visible light image to generate a mechanical damage area; Based on the discharge spot distribution map, the mechanical damage area and the candidate fault area, a spatial logic association verification process is performed to output the fault type and location.

6. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 5 is characterized in that: The method of performing spatial logic association verification processing based on the discharge spot distribution map, the mechanical damage area and the candidate fault area, and outputting the fault type and location, includes: Performing threshold judgment processing on the overlap rate between the discharge spot distribution map and the candidate fault area to generate a judgment result, wherein the judgment result includes that the overlap rate is greater than or equal to a preset threshold and that the overlap rate is less than a preset threshold; When the judgment result is that the overlap rate is less than the preset threshold, it is excluded as an interference signal, and the fault type and location are output.

7. The power inspection fault diagnosis method based on unmanned aerial vehicle multispectral image according to claim 1 is characterized in that: The method of dynamically correcting the emissivity change of the device surface due to oxidation or contamination based on the visible light image and the multi-spectral reflectivity data, and generating a dynamic emissivity correction coefficient table, includes: Extracting surface texture features and performing region segmentation on the preprocessed visible light image to generate a mask image of oxidized or contaminated areas on the device surface; Based on the multispectral reflectance distribution map and the mask map, the device surface is divided into a normal area, an oxidized area, and a dirty area, and the reflectance offset of each area is calculated; The dynamic emissivity correction coefficient table is generated according to the corresponding relationship between the reflectivity offset and the surface texture feature.

8. The power inspection fault diagnosis system based on UAV multispectral images is characterized by: The system comprises: Multispectral data acquisition and preprocessing module, used to collect multispectral data of power equipment to obtain visible light images, multispectral reflectance data and infrared thermal imaging data; A dynamic emissivity correction module, for dynamically correcting emissivity changes on the surface of the device due to oxidation or contamination based on the visible light image and the multispectral reflectivity data, and generating a dynamic emissivity correction coefficient table; An infrared temperature calibration module, used to perform temperature calibration on the infrared thermal imaging data according to the dynamic emissivity correction coefficient table, and generate a calibrated temperature distribution diagram; The multimodal fault diagnosis module is used to fuse the calibrated temperature distribution map, the visible light image and the multispectral reflectance data to perform multimodal fault diagnosis and output the fault type and location.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the power inspection fault diagnosis method based on unmanned aerial vehicle multispectral imagery described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the power inspection fault diagnosis method based on unmanned aerial vehicle multispectral imagery described in any one of claims 1 to 7 are implemented.

Images