Unmanned aerial vehicle power grid inspection data processing method and system based on edge cloud cooperation

Abstract

The application belongs to the technical field of communication data processing, and specifically discloses a UAV power grid inspection data processing method and system based on edge cloud cooperation. Instant quality evaluation is performed on the visible light image and infrared thermal image collected by each flight point in UAV inspection, real-time discrimination of the effectiveness of the inspection data is realized, and on this basis, the quality evaluation result is combined with the mutation area detection result to perform differentiated transmission, thereby reducing the invalid data transmission amount and reducing the bandwidth consumption on the premise of ensuring that the sudden change can be captured in time. At the same time, after the low-quality data is discriminated, the cloud retrieves the high-quality data of the same coordinate in another channel collected this time according to the spatial grid identifier carried by the quality abnormality notification frame, judges whether there is a defect, and when it is judged that there is a defect, the fixed-point and fixed-channel resampling is performed. After the resampling is successful, the atomicity is replaced to replace the local storage data, and the directional repair of the low-quality data is completed on the premise of not affecting the integrity of the global data.

Description

Technical Field

[0001] This invention belongs to the field of communication data processing technology, and specifically discloses a method and system for processing UAV power grid inspection data based on edge cloud collaboration. Background Technology

[0002] As the power grid continues to expand, traditional manual inspection methods are no longer sufficient to meet the daily operation and maintenance needs of transmission lines. Drones, due to their mobility, flexibility, and lack of terrain limitations, are widely used in power grid inspections. At the same time, the increasing availability of sensors such as visible light cameras and infrared thermal imagers integrated into drone platforms has led to a dramatic increase in the amount of data collected in a single inspection mission. Furthermore, the timely detection of sudden defects in transmission lines places higher demands on real-time data processing.

[0003] To address the challenges of large data transmission volumes and high real-time requirements in drone inspections, existing technologies have proposed edge computing-based drone inspection solutions. For example, Chinese invention patent CN119363820A discloses a drone multi-dimensional perception intelligent inspection and early warning system and method based on edge computing. This system uses a multi-dimensional perception data acquisition device to send collected multi-dimensional perception data to a mobile edge computing server. The mobile edge computing server determines the status data of the inspection area based on the multi-dimensional perception data, generates corresponding early warning information, and sends this information to an intelligent computing monitoring and decision support platform. This solution alleviates the latency problem of centralized cloud processing to some extent by offloading some computing tasks to the edge.

[0004] However, the above-mentioned existing technical solutions still have the following shortcomings:

[0005] First, existing edge computing servers lack a mechanism to verify the validity of multidimensional sensing data. Images collected by drones under complex weather conditions, lighting changes, and flight attitude fluctuations often suffer from quality problems such as blurry images, overexposure, underexposure, and infrared temperature drift. Low-quality data and high-quality data are processed in the same process without being distinguished, which not only consumes computing resources at the edge and in the cloud, but may also lead to false warnings due to unreliable data.

[0006] Second, the existing solution sends all sensing data to the edge computing server for processing. In patrol scenarios where wireless communication link bandwidth is limited, transmitting all data indiscriminately will occupy scarce transmission resources. When a large amount of routine patrol data continuously occupies the channel, emergency alarm data may not be delivered in time due to channel congestion, affecting the rapid response to sudden defects. Summary of the Invention

[0007] To address the aforementioned technical problems, or at least partially address them, this invention provides a method and system for processing UAV power grid inspection data based on edge cloud collaboration. Through the collaboration of edge-end quality assessment and cloud-based cross-modal verification, differentiated transmission and fixed-point re-sampling closed-loop control of inspection data are achieved.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] In a first aspect, the present invention proposes a method for processing UAV power grid inspection data based on edge cloud collaboration, comprising the following steps:

[0010] A spatial grid index is established based on the spatial distribution of the inspection route. During the UAV inspection, the visible light images and infrared thermal images collected at each waypoint are evaluated for quality and a quality confidence vector for each channel is generated.

[0011] Abrupt change region detection is performed using the visible light image of each waypoint relative to the adjacent previous waypoint;

[0012] A transmission disposal code is generated based on the combined judgment result of the quality confidence vector and the mutation region detection result.

[0013] Differential data transmission is performed according to the transmission handling code: regional uploading occurs when each channel is of high quality and there are abrupt changes; normal uploading occurs when each channel is of high quality and there are no abrupt changes; and a quality anomaly notification frame is sent when any channel is of low quality.

[0014] After receiving the quality anomaly notification frame, the cloud retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried by the frame, and determines whether there is a defect. If there is a defect, a fixed-point and fixed-channel re-sampling instruction is generated and sent to the edge node of the drone.

[0015] After receiving the re-sampling command, the edge node of the drone flies to the designated coordinates to perform hovering re-sampling, and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

[0016] Secondly, this invention proposes a UAV power grid inspection data processing system based on edge cloud collaboration, comprising the following modules:

[0017] The spatial grid index module establishes a spatial grid index based on the spatial distribution of the inspection route.

[0018] The quality assessment module, located at the edge node of the UAV, performs quality assessment on the visible light images and infrared thermal images collected at each waypoint and generates quality confidence vectors for each channel.

[0019] The mutation region detection module is set at the edge node of the UAV and performs mutation region detection on the visible light images collected at each waypoint;

[0020] The transmission decision module is set at the edge node of the UAV and generates a transmission handling code based on the combined judgment result of the quality confidence vector and the mutation region detection result.

[0021] The data transmission execution module is located at the edge node of the UAV and performs differentiated data transmission based on the transmission processing code;

[0022] The defect judgment module, located in the cloud, retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried in the quality anomaly notification frame to determine whether a defect exists.

[0023] The resampling instruction generation module is located in the cloud. When a defect exists, it generates a fixed-point and fixed-channel resampling instruction and sends it to the edge node of the drone.

[0024] The re-sampling execution module is set at the edge node of the drone. After receiving the re-sampling command, it flies to the designated coordinates to perform hovering re-sampling and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

[0025] Combining all the above technical solutions, the positive effects of this invention are as follows:

[0026] 1. This invention achieves real-time identification of the validity of inspection data by performing real-time quality assessment on visible light images and infrared thermal images collected at each waypoint during UAV inspections and generating quality confidence vectors for each channel. It can identify and lock low-quality data at the edge, avoiding invalid data from occupying transmission and cloud computing resources. At the same time, it performs differentiated regional transmission based on the detection results of abrupt change areas. When abrupt change areas exist, only the data blocks of the abrupt change areas are uploaded, and when no abrupt change areas exist, normal transmission is performed. Thus, while ensuring that sudden changes can be captured in a timely manner, the amount of invalid data transmission is significantly reduced and bandwidth consumption is lowered.

[0027] 2. After identifying low-quality data, this invention retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried in the quality anomaly notification frame, determines whether there is a defect, and triggers fixed-point resampling of the corresponding channel when a defect is found, thus avoiding misjudgment or omission based on unreliable data. After successful resampling, the locally stored data is replaced with atomic overwrite, completing the targeted repair of low-quality data without affecting the integrity of the global data, further reducing the cost of data correction. Attached Figure Description

[0028] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0029] Figure 1This is a flowchart illustrating the method steps of Embodiment 1 of the present invention;

[0030] Figure 2 This is a schematic diagram illustrating the generation of transmission processing codes based on the combined judgment result of the quality confidence vector and the mutation region detection result in this invention.

[0031] Figure 3 This is a system module connection diagram of Embodiment 2 of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example 1:

[0034] This invention proposes a data processing method for UAV power grid inspection based on edge cloud collaboration, comprising the following steps:

[0035] S1. Establish a spatial grid index based on the spatial distribution of the inspection route, perform quality assessment on the visible light images and infrared thermal images collected at each waypoint during UAV inspection, and generate quality confidence vectors for each channel.

[0036] When using drones for power grid inspections, considering that transmission lines typically extend for tens to hundreds of kilometers, and the field of view of a single drone hovering is limited, it is difficult to cover the entire line in a single data collection operation. Without a unified spatial reference framework, subsequent location correlation between visible light images and infrared thermal images will lack a consistent spatial benchmark. Therefore, this invention establishes a spatial grid index based on the spatial distribution of the inspection route before the formal inspection mission, providing a unified spatial coordinate reference for the spatial attribution of data collected at each waypoint.

[0037] In a specific application embodiment, the process of establishing a spatial grid index is as follows:

[0038] First, the tower coordinates and route of the current inspection line are obtained from the power grid GIS system. The tower coordinates are used to determine the actual geographical location of the line in space and its turning points, while the route direction is used to determine the flight direction of the UAV. Together, they serve as the spatial reference for spatial grid division.

[0039] Secondly, based on the centerline of the route, several patrol waypoints are set along the route extension direction with a fixed spatial step size, so that each waypoint corresponds to a patrol and data collection position. The value of the spatial step size is determined based on the effective coverage width of the visible light camera and infrared thermal imager when the UAV hovers and collects data. The step size range is 60% to 80% of the effective coverage width, ensuring that there is an overlapping area between the coverage of adjacent waypoints, thereby eliminating patrol blind spots caused by discontinuous coverage.

[0040] Subsequently, using the direction of the route extension as the longitudinal reference and the direction perpendicular to the route as the lateral reference, spatial coverage units corresponding to each waypoint are delineated with each patrol point as the center and a fixed distance as the radius, forming a continuous spatial grid coverage zone covering the entire route. The purpose of delineating the spatial coverage units is to discretize the continuous space of the entire route into several interconnected local spatial partitions, so that the visible light images and infrared thermal images collected at each waypoint can be assigned to a uniquely determined spatial coverage unit.

[0041] Finally, a unique identifier is assigned to the spatial coverage unit corresponding to each patrol waypoint. The unique identifier is used to realize bidirectional indexing of data and spatial location in subsequent processing, so that the edge end can classify the data according to the spatial grid identifier during transmission and processing.

[0042] Furthermore, after establishing a spatial grid index, the UAV can sequentially collect data according to waypoints. Considering that the operational status assessment of power grid equipment requires simultaneous acquisition of equipment appearance information and temperature distribution information, the data types collected in this invention are visible light images and infrared thermal images. Visible light images are used to present the external morphological characteristics of the equipment; infrared thermal images are used to present the temperature distribution of the equipment. The two reflect the health status of the equipment from different physical dimensions, forming a complementary relationship. After data collection, if all raw data is directly uploaded, on the one hand, the images may have quality problems such as blurriness, overexposure, underexposure, or temperature drift due to factors such as weather conditions, changes in illumination, and fluctuations in flight attitude during the collection process. Low-quality data entering subsequent analysis will affect the accuracy of defect identification.

[0043] To address the aforementioned issues, this invention performs a quality assessment after data acquisition to verify the validity of data from each channel. The specific process is as follows:

[0044] After the UAV arrives at each patrol point, it simultaneously triggers the visible light camera and infrared thermal imager to collect data, and binds the visible light image and infrared thermal image collected at the same point to the same spatial grid identifier.

[0045] For the current waypoint, quality indicators are extracted for the visible light channel and the infrared channel, respectively. The extraction process for the quality indicators is as follows:

[0046] Visible light channel: The acquired visible light image is converted from the RGB color space to the grayscale space. The purpose of this conversion is to remove redundant chromaticity information from the color image, retaining only luminance information for subsequent edge feature extraction. This reduces computational complexity while maintaining the effectiveness of sharpness evaluation. Edge detection is then performed on the converted grayscale image. The goal of edge detection is to extract pixel locations where grayscale values ​​change drastically; these locations correspond to the boundaries between the device outline and the background in the image. The sharper the image, the more concentrated and higher the edge response; the blurrier the image, the more dispersed and lower the edge response.

[0047] In the edge response amplitude map output by the edge detection operation, the amplitude of each pixel represents the degree of gray-level change at that location. The proportion of pixels with amplitudes higher than the average edge response amplitude of the entire image is counted out of the total number of pixels in the entire image. The higher the proportion, the greater the proportion of clear edges in the image and the higher the overall clarity. Therefore, the proportion is used as the edge clarity.

[0048] Infrared channel: Extracts the temperature reading of a reference blackbody at a fixed position within the infrared field of view from the infrared thermal image. The reference blackbody is a cavity blackbody radiation source with an emissivity of not less than 0.97. It is fixedly installed at the edge of the field of view above the lens of the infrared thermal imager. The nominal temperature value is obtained by comparison with a standard blackbody before leaving the factory and serves as the reference for temperature measurement of the infrared thermal imager.

[0049] By comparing the measured temperature reading of the infrared thermal imager on the reference blackbody with its nominal temperature value, the difference between the two is calculated. The difference reflects the temperature drift caused by factors such as device temperature rise and environmental radiation interference under the current environmental conditions. Therefore, the difference is used as the temperature drift offset.

[0050] To eliminate the impact of instantaneous environmental fluctuations on the quality assessment of single waypoints, this invention uses the average values ​​of the corresponding indicators of each channel for 4 to 6 waypoints, each with 2 to 3 waypoints before and after the current waypoint, to form the reference benchmark for each channel. The physical meaning is that adjacent waypoints in the same patrol voyage are located in similar geographical areas and environmental conditions, and their data quality should have a similar level. Therefore, using the average value of the indicators of adjacent waypoints as the reference benchmark for the current waypoint can reflect the normal quality level that a local segment should achieve under the current environmental conditions.

[0051] The edge sharpness and temperature drift offset of the current waypoint are compared with their respective reference benchmarks. The specific comparison method is as follows: for each quality index, the current index value is calculated by subtracting the reference benchmark value from the current index value and then dividing by the reference benchmark value and taking the absolute value. The resulting ratio is the volatility of the waypoint channel. The volatility characterizes the deviation of the current waypoint data quality from the local normal level.

[0052] When the current volatility of any channel exceeds a fixed multiple of the average volatility of all channels in the completed waypoints, it indicates that the data quality of that channel at that waypoint significantly deviates from the normal volatility level of the collected data, and the channel is marked as low quality; otherwise, it is marked as high quality. The fixed multiple is typically set to 2.

[0053] The confidence markers of the visible light channel and the infrared channel are combined into a quality confidence vector corresponding to the spatial grid identifier of each waypoint. This vector indicates the data quality status of each of the two channels at the current waypoint in the form of a binary marker, providing a basis for judgment in the generation of subsequent transmission processing codes.

[0054] S2. Perform abrupt region detection using the visible light image of each waypoint relative to the adjacent previous waypoint.

[0055] After obtaining the quality confidence vector corresponding to the spatial grid identifier of each waypoint, a quantitative judgment of the validity of the data in each channel is obtained. However, good quality only indicates that the data is usable, and does not mean that the data contains all the key information that needs attention. In power transmission line inspections, sudden defects such as spark discharge and equipment smoke often only occupy a local area in the image. If the entire image is uploaded, not only will the transmission efficiency be low, but the delivery of key information may also be delayed due to the transmission of a large amount of irrelevant background data. Therefore, this invention further introduces a sudden change region detection mechanism to analyze the visible light image acquired at the current waypoint to identify the sudden change region of the image, and directly uses the detection result as the basis for determining the range of transmitted data, so that the transmission strategy is refined from the transmission of the entire image to the on-demand regional transmission.

[0056] As an improved embodiment of the present invention, mutation region detection is performed according to the following process:

[0057] The current frame image is selected from the visible light images acquired at the current waypoint, and the previous frame image is selected from the visible light images acquired at the previous waypoint adjacent to the current waypoint along the inspection direction. The frame image of the adjacent previous waypoint is selected as the comparison benchmark because the images acquired at adjacent waypoints in the same inspection voyage have similar shooting angles and lighting conditions, and the time interval is short with little change in the background environment. If there is a significant difference between the two frames, it is highly likely to be due to a newly appearing moving target or sudden change on the inspection path.

[0058] The grayscale difference distribution is obtained by performing pixel-by-pixel grayscale difference calculations between the current frame and the previous frame. Each pixel value in the grayscale difference distribution reflects the magnitude of brightness change at the corresponding position in the two frames.

[0059] The average grayscale difference of all pixels in the grayscale difference distribution is used as the overall offset benchmark, and the standard deviation of all pixel grayscale differences is used as the fluctuation amplitude benchmark. Considering that the average value itself is easily affected by the overall brightness shift, and the pixels corresponding to the sum of the average value and the standard deviation are pixels that deviate further from the normal change on the basis of the overall offset, it can filter out background differences caused by slow changes in illumination, sensor noise, etc., and retain real abrupt change areas. Therefore, the sum of twice the average value and the standard deviation is used as the difference feature value. The proportion of pixels whose grayscale difference exceeds the difference feature value to the total number of pixels in the whole image is defined as the change area ratio. The change area ratio is used to quantify the proportion of the area that has changed in the current frame relative to the previous frame to the entire field of view. The higher the value, the greater the range of scene changes and the higher the possibility of sudden anomalies.

[0060] When the percentage of changed area exceeds a preset threshold, a sudden change region is determined to exist in the visible light image of the current waypoint; otherwise, no sudden change region is determined to exist. The preset threshold for the percentage of changed area can be configured according to the inspection scenario: 5% for inspection scenarios that focus on quickly capturing sudden defects, 15% for inspection scenarios that focus on filtering out environmental interference, and 10% for regular inspection scenarios.

[0061] It should be noted that the abrupt change region detection in this invention is performed only on visible light images, not on infrared thermal images. This is because inter-frame differential detection requires comparable pixel-level grayscale consistency between two frames. Visible light images reflect the reflection of ambient light by a target. When the shooting angle and lighting conditions are similar at adjacent waypoints, the grayscale performance of the same feature exhibits good stability. However, infrared thermal imaging is affected by a combination of non-equipment factors such as ambient temperature, sunlight, and wind speed. Temperature fluctuations between adjacent waypoints may originate from environmental changes rather than equipment defects. Using infrared thermal images as input for differential detection would introduce a large amount of noise unrelated to equipment defects. Therefore, this invention performs abrupt change region detection only on visible light images.

[0062] S3. Generate a transmission processing code based on the combined judgment result of the quality confidence vector and the mutation region detection result.

[0063] After completing the quality confidence vector and abrupt change region detection results for each channel, the collected data needs to be uploaded to the cloud for defect identification. This invention combines the quality confidence vector and abrupt change region detection results for a combined judgment. The quality confidence vector determines whether data can be uploaded, while the abrupt change region detection results determine the content of the uploaded data, thereby achieving differentiated transmission control. Specifically, this invention generates corresponding transmission handling codes based on the judgment results of the above two dimensions. See [link to relevant documentation]. Figure 2 The specific rules are as follows:

[0064] When all channels are of high quality and there are abrupt changes, it indicates that the current waypoint data is valid but only a local area has changed. There is no need to upload the complete data. Therefore, the first transmission processing code for regional upload is generated, and only the data block corresponding to the abrupt change area is uploaded to save transmission resources and improve the transmission efficiency of key changed areas.

[0065] When all channels are of high quality and there are no abrupt changes, it indicates that the current waypoint data is valid and the scene has not changed significantly compared to the previous waypoint. A second transmission processing code corresponding to the complete upload is generated, and the visible light image and infrared thermal image of the current spatial grid are uploaded to the cloud normally.

[0066] When any channel is of low quality, a third transmission handling code corresponding to the quality anomaly notification is generated, regardless of whether there are abrupt change regions. The priority of judging low-quality data is placed above detecting abrupt change regions because even if low-quality data contains suspected abrupt change regions, its unreliability may lead to misjudgment by the cloud; therefore, the upload of the original data must be blocked first.

[0067] S4. Perform differentiated data transmission according to the transmission processing code: regional upload when each channel is of high quality and there are abrupt change areas, normal upload when each channel is of high quality and there are no abrupt change areas, and send a quality abnormality notification frame when any channel is of low quality.

[0068] After obtaining the transmission processing code in step S3, differentiated data transmission strategies can be executed based on different processing codes. The specific implementation method is as follows:

[0069] When the transmission processing code is the first transmission processing code, it indicates that all channel data of the current waypoint are of high quality and there is a sudden change region. At this time, the pixel coordinate range of the sudden change region in the visible light image is determined according to the sudden change region detection result. Based on the pixel coordinate range, it is mapped to the infrared thermal image corresponding to the same spatial grid identifier. Visible light image data blocks within the pixel coordinate range of the sudden change region are extracted from the visible light image, and infrared thermal image data blocks within the pixel coordinate range of the sudden change region are extracted from the infrared thermal image. The extracted data blocks, together with the spatial grid identifier and the position information of the data blocks in the original image, are uploaded to the cloud.

[0070] By using the aforementioned regional upload method, it is not necessary to transmit the complete visible light image and infrared thermal image of the current waypoint. Only the data blocks of the local area that has changed are transmitted, thereby significantly reducing the amount of data transmitted per time while ensuring timely reporting of change information.

[0071] When the transmission processing code is the second transmission processing code, it indicates that all channel data of the current waypoint are of high quality and there are no abrupt change areas. The visible light image and infrared thermal image of the current spatial grid will be completely uploaded to the cloud.

[0072] When the transmission handling code is the third transmission handling code, it indicates that there is low-quality data at the current waypoint, and its original file cannot be directly used for defect analysis. At this time, the visible light image and infrared thermal image original files of the current spatial grid are locked in the local cache and prohibited from entering the wireless transmission queue to avoid invalid data occupying transmission resources. At the same time, the spatial grid identifier and quality confidence vector are encapsulated into a quality anomaly notification frame and sent to the cloud. The quality anomaly notification frame is used to inform the cloud that the data corresponding to the spatial grid identifier has quality problems.

[0073] S5. After receiving the quality anomaly notification frame, the cloud retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried by the frame, determines whether there is a defect, and if there is a defect, generates a fixed-point and fixed-channel re-sampling instruction and sends it to the edge node of the UAV.

[0074] In step S4, when the transmission processing code is the third transmission processing code, the edge node sends the quality anomaly notification frame to the cloud. The cloud does not discard the notification frame after receiving it. Considering that the low quality may be due to non-equipment factors such as sensor failure and environmental interference, but it may also be due to a real equipment anomaly that is limited by the acquisition conditions and fails to image effectively, in order to clarify the cause, it is necessary to re-acquire. However, if re-acquisition is initiated directly, it may result in invalid flight and waste of resources because the equipment is not actually abnormal.

[0075] Therefore, before triggering re-sampling, this invention first uses high-quality data from another channel at the same coordinates in the current acquisition to determine defects. If no defects are detected in the other channel data, re-sampling is unnecessary. If the other channel data confirms the existence of a real defect, then the original low-quality channel is re-sampled specifically to obtain effective complementary evidence. In this way, the re-sampling operation is triggered based on the defect prediction result, avoiding the waste of resources caused by blind re-sampling, while ensuring the re-sampling is targeted.

[0076] The specific judgment process is as follows:

[0077] Based on the spatial grid identifier carried in the quality anomaly notification frame, the cloud retrieves high-quality data from another channel corresponding to the same spatial grid identifier collected this time.

[0078] When the visible light channel is marked as low quality and the infrared channel as high quality in the quality anomaly notification frame, the high-quality infrared data is mapped to the spatial grid coordinate reference system, and temperature field analysis is performed on the infrared thermal image. The specific analysis process is as follows: extract the temperature values ​​of all pixels within the spatial grid coverage area, use the average temperature value of all pixels within the spatial grid coverage area as the ambient temperature reference value, calculate the temperature rise value of each pixel relative to the reference value, and mark pixels with temperature rise values ​​higher than the average temperature rise value within the spatial grid coverage area as candidate heat-generating pixels; perform connected component analysis on the candidate heat-generating pixels, and merge adjacent candidate heat-generating pixels into candidate heat-generating regions; spatially compare the shape of the candidate heat-generating regions with the tower coordinates and equipment mounting point positions corresponding to the spatial grid identifiers. When the concentrated location of the candidate heat-generating regions is consistent with the equipment mounting point position, it is determined that there is a heat-generating defect, and the visible light channel is re-sampled to obtain clear appearance evidence.

[0079] When the infrared channel is marked as low quality while the visible light channel is marked as high quality in the quality anomaly notification frame, the high-quality visible light data is mapped to the spatial grid coordinate reference system to identify appearance anomalies in the visible light image. The specific identification process is as follows: edge detection is performed on the visible light image, and an edge response amplitude map is output; the average edge response amplitude of the entire image is used as the baseline amplitude, and pixels with edge response amplitudes higher than the baseline amplitude are marked as candidate edge pixels; connected component analysis is performed on the candidate edge pixels, merging adjacent candidate edge pixels into a connected component as the target region; the shape of the target region is spatially compared with the tower coordinates and equipment mounting point positions corresponding to the spatial grid identifier. When the spatial distribution position of the target region matches the equipment mounting point position and the shape of the target region is inconsistent with the normal outline shape of the equipment, a structural defect is determined, triggering a resampling of the infrared channel to obtain temperature distribution evidence.

[0080] In one application example, the criteria for judging whether the spatial distribution location is consistent and whether the shape is consistent with the normal contour shape of the equipment are: the spatial distance between the centroid of the target area and the attachment point of the equipment exceeds 1.5 times the radius of the circumscribed circle of the normal contour of the equipment, and the deviation of the shape factor of the target area, such as the ratio of area to convex hull area, from the shape factor of the normal contour of the equipment exceeds 30%.

[0081] When both the visible light channel and the infrared channel are marked as low quality, the data of both modes are unavailable. The cloud cannot determine whether there is a defect, nor can it confirm whether the low quality is due to equipment defects or problems with the acquisition conditions. Therefore, re-acquisition is not triggered. The cloud discards the quality anomaly notification frame and records the spatial grid as an invalid acquisition point. After the current inspection mission is completed, it will be covered by ground manual inspection or other supplementary means.

[0082] Furthermore, when a defect is identified, the cloud determines the target space grid identifier and sensor channel identifier that need to be re-collected, generates a re-collection command, and sends it to the original UAV edge node via the downlink with priority. The priority sending means that the re-collection command has a higher sending priority than the regular inspection data transmission in the UAV downlink.

[0083] Specifically, when a heat-generating defect is detected, the sensor channel is identified as a visible light channel; when a structural defect is detected, the sensor channel is identified as an infrared channel.

[0084] S6. After receiving the re-sampling command, the edge node of the drone flies to the designated coordinates to perform hovering re-sampling and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

[0085] After the cloud generates a re-sampling command and sends it to the drone edge node, the drone edge node executes the re-sampling operation according to the command:

[0086] After receiving the resampling command, the edge node of the UAV interrupts the current patrol route and flies to the designated coordinates. It then performs hovering resampling according to the sensor channel identifier specified in the resampling command. The reason for using hovering resampling is that the UAV's attitude is most stable when hovering, which can ensure that the optical axis of the camera or thermal imager maintains a fixed spatial geometric relationship with the target being collected, thereby obtaining high-quality, low-motion-blur data.

[0087] After re-collection is completed, an immediate quality assessment is performed on the re-collected data. The immediate quality assessment uses the same quality assessment indicators and judgment methods as the initial collection in step S1. If the re-collected data is of high quality, the data is uploaded to the cloud along with the original spatial grid identifier. If the re-collected data is still of low quality, the re-collection attempt for that coordinate is stopped, the spatial grid identifier is recorded as a collection failure, and subsequent ground manual inspection or other supplementary means are used to cover it, so as to avoid invalid flights and power consumption caused by repeated re-collection.

[0088] After receiving the resampled data, the cloud performs atomic overlay replacement based on the spatial grid identifier. That is, it only replaces the original stored data corresponding to the coordinate with the resampled data, without affecting the data of other coordinates within the spatial grid coverage area.

[0089] After the replacement is complete, the cloud sends a coverage confirmation receipt to the drone edge node. The purpose of the confirmation receipt is to inform the edge node that the cloud has successfully completed the data replacement. Based on this, the edge node knows that the recollected data has been safely stored and can safely release the corresponding low-quality original files in its local cache, avoiding data residue or accidental deletion due to communication uncertainties.

[0090] After receiving the coverage confirmation receipt, the edge node of the UAV releases the low-quality original cache file corresponding to the spatial grid identifier in the local cache, thereby freeing up limited local storage space for subsequent waypoint data collection and caching.

[0091] Example 2

[0092] See Figure 3 This invention proposes a UAV power grid inspection data processing system based on edge cloud collaboration, comprising the following modules:

[0093] The spatial grid index module establishes a spatial grid index based on the spatial distribution of the inspection route.

[0094] The quality assessment module, connected to the spatial grid indexing module, is set at the edge node of the UAV. It performs quality assessment on the visible light images and infrared thermal images collected at each waypoint and generates quality confidence vectors for each channel. At the same time, in response to the call request from the resampling execution module, the quality assessment module performs a second quality assessment on the resampling data and returns the assessment results to the resampling execution module.

[0095] The mutation region detection module is set at the edge node of the UAV and performs mutation region detection on the visible light images collected at each waypoint;

[0096] The transmission decision module is connected to the quality assessment module and the mutation region detection module respectively. It is set at the edge node of the UAV and generates a transmission disposal code based on the combined judgment result of the quality confidence vector and the mutation region detection result.

[0097] The data transmission execution module, connected to the transmission decision module, is located at the edge node of the UAV and performs differentiated data transmission based on the transmission processing code.

[0098] The defect judgment module, connected to the data transmission execution module and located in the cloud, retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried in the quality anomaly notification frame to determine whether a defect exists.

[0099] The resampling instruction generation module is connected to the defect judgment module and is set in the cloud. When a defect exists, it generates a fixed-point and fixed-channel resampling instruction and sends it to the edge node of the drone.

[0100] The re-sampling execution module, connected to the re-sampling instruction generation module, is set at the edge node of the UAV. After receiving the re-sampling instruction, it flies to the designated coordinates to perform hovering re-sampling and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

[0101] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0102] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0103] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0104] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0105] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for processing UAV power grid inspection data based on edge cloud collaboration, characterized in that, Includes the following steps: A spatial grid index is established based on the spatial distribution of the inspection route. During the UAV inspection, the visible light images and infrared thermal images collected at each waypoint are evaluated for quality and a quality confidence vector for each channel is generated. Abrupt change region detection is performed using the visible light image of each waypoint relative to the adjacent previous waypoint; A transmission handling code is generated based on the combined judgment result of the quality confidence vector and the mutation region detection result. Differential data transmission is performed according to the transmission handling code: regional uploading occurs when each channel is of high quality and there are abrupt changes; normal uploading occurs when each channel is of high quality and there are no abrupt changes; and a quality anomaly notification frame is sent when any channel is of low quality. After receiving the quality anomaly notification frame, the cloud retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried by the frame, and determines whether there is a defect. If there is a defect, a fixed-point and fixed-channel re-sampling instruction is generated and sent to the edge node of the drone. After receiving the re-sampling command, the edge node of the drone flies to the designated coordinates to perform hovering re-sampling, and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

2. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 1, characterized in that: The establishment of the spatial grid index includes the following: Obtain the tower coordinates and route of the current inspection line, and set several inspection waypoints along the extension direction with a fixed spatial step length, based on the center line of the line. Using the direction of the route extension as the longitudinal reference and the direction perpendicular to the route as the lateral reference, the spatial coverage unit corresponding to each waypoint is delineated with each patrol waypoint as the center and a fixed distance as the radius; Spatial coverage units of adjacent patrol points are spliced ​​together in a longitudinal sequence to form a continuous spatial grid coverage strip covering the entire route; Assign a unique identifier to the spatial coverage unit corresponding to each patrol waypoint.

3. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 1, characterized in that: The generation of the quality confidence vector for each channel is performed according to the following process: After the drone arrives at each waypoint, it simultaneously triggers the visible light camera and infrared thermal imager to collect data, and binds the visible light image and infrared thermal image of the same waypoint to the same spatial grid identifier. Extract quality indicators for each channel: The percentage of pixels with edge response amplitude values ​​higher than the overall image mean in the visible light image edge response amplitude map is used as edge sharpness. The difference between the reference blackbody temperature reading and the nominal value in the infrared thermal image is calculated as the temperature drift offset. The average values ​​of the corresponding indicators of each channel for several adjacent waypoints are used to form the reference benchmarks for each channel. The edge sharpness and temperature drift offset of the current waypoint are compared with their respective reference benchmarks. When the current indicator of any channel deviates from the reference benchmark, the channel is marked as low quality, and vice versa. The confidence labels of each channel are combined into a quality confidence vector corresponding to the spatial grid code of each waypoint.

4. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 1, characterized in that: The mutation region detection process is as follows: Select the current frame from the visible light image acquired at the current waypoint, and select the previous frame from the visible light image acquired at the previous waypoint adjacent to the inspection direction. Perform gray-level difference calculation on the two frames pixel by pixel to obtain the gray-level difference distribution. The difference feature value is determined based on the statistical distribution characteristics of the gray-level difference values ​​of all pixels in the gray-level difference distribution. The proportion of the number of pixels whose gray-level difference exceeds the difference feature value to the total number of pixels in the whole image is used as the change area ratio. When the percentage of the changed area exceeds a preset threshold, a sudden change region is identified.

5. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 3, characterized in that: The generated transmission processing code includes the following: When all channels are of high quality and there are abrupt regions, the first transmission handling code is generated. When all channels are of high quality and there are no abrupt changes, a second transmission processing code is generated. When any channel is of low quality, a third transmission disposal code is generated.

6. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 5, characterized in that: The step of performing differentiated data transmission based on the transmission processing code includes the following: When it is the first transmission processing code, the pixel coordinate range of the mutation region in the visible light image is determined according to the mutation region detection result, and the pixel coordinate range is mapped to the infrared thermal image corresponding to the same spatial grid identifier. The visible light image data block within the coordinate range is extracted from the visible light image and the infrared thermal image data block within the coordinate range is extracted from the infrared thermal image. The extracted data block, together with the spatial grid identifier and the position information of the data block in the original image, is uploaded to the cloud. When the second transmission processing code is used, the visible light image and infrared thermal image of the current spatial grid will be uploaded to the cloud normally; When the third transmission processing code is used, the visible light image and infrared thermal image of the current spatial grid are locked in the local cache, and only the spatial grid identifier and quality confidence vector are encapsulated into a quality anomaly reporting frame and sent to the cloud.

7. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 1, characterized in that: The determination of whether a defect exists includes the following: Based on the spatial grid identifier carried in the quality anomaly notification frame, the cloud retrieves high-quality data from another channel corresponding to the same spatial grid identifier collected in this instance. When the visible light channel is marked as low quality and the infrared channel is marked as high quality in the quality anomaly notification frame, the high-quality infrared data is mapped to the spatial grid coordinate reference system. If there are local temperature rise features in the infrared thermal image, it is determined that there is a heating defect. When the infrared channel is marked as low quality and the visible light channel is marked as high quality in the quality anomaly notification frame, the high quality visible light data is mapped to the spatial grid coordinate reference system. If there are abnormal appearance features in the visible light image, it is determined that there is a structural defect. When both the visible light channel and the infrared channel are marked as low quality, the data acquisition is deemed invalid.

8. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 1, characterized in that: The method for generating fixed-point, fixed-channel resampling instructions includes: When a defect is detected, the cloud determines the target space grid identifier and sensor channel identifier that need to be re-collected, generates a re-collection command, and sends it to the original UAV edge node via the downlink.

9. The UAV power grid inspection data processing method based on edge cloud collaboration as described in claim 8, characterized in that: The process of completing the local data replacement includes the following: After receiving the re-sampling instruction, the edge node of the UAV interrupts the current patrol route and flies to the designated coordinates, and performs hovering re-sampling according to the sensor channel identifier specified in the re-sampling instruction; After the re-collection is completed, an immediate quality assessment is performed on the re-collected data. Once the high quality is confirmed, the re-collected data, along with the original spatial grid identifier, is uploaded to the cloud. The cloud performs atomic overlay replacement based on the spatial grid identifier, replaces the original stored data corresponding to the coordinate with resampled data, and sends an overlay confirmation receipt to the edge node of the drone. After receiving the acknowledgment, the edge node of the drone releases the low-quality original cache file corresponding to the spatial grid identifier in the local cache.

10. A UAV power grid inspection data processing system based on edge cloud collaboration, characterized in that, Includes the following modules: The spatial grid index module establishes a spatial grid index based on the spatial distribution of the inspection route. The quality assessment module, located at the edge node of the UAV, performs quality assessment on the visible light images and infrared thermal images collected at each waypoint and generates quality confidence vectors for each channel. The mutation region detection module is set at the edge node of the UAV and performs mutation region detection on the visible light images collected at each waypoint; The transmission decision module is set at the edge node of the UAV and generates a transmission handling code based on the combined judgment result of the quality confidence vector and the mutation region detection result. The data transmission execution module is located at the edge node of the UAV and performs differentiated data transmission based on the transmission processing code; The defect judgment module, located in the cloud, retrieves high-quality data from another channel at the same coordinates based on the spatial grid identifier carried in the quality anomaly notification frame to determine whether a defect exists. The resampling instruction generation module is located in the cloud. When a defect exists, it generates a fixed-point and fixed-channel resampling instruction and sends it to the edge node of the drone. The re-sampling execution module is set at the edge node of the drone. After receiving the re-sampling command, it flies to the designated coordinates to perform hovering re-sampling and uploads the successfully re-sampling high-quality data to the cloud to complete the local data replacement.

Images