Method, storage medium and device for overhead line defect detection based on event camera

Abstract

The present application relates to the technical field of image defect detection, and specifically provides a method, storage medium, and device for detecting overhead line defects based on an event camera. The method may include: filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; performing a frame accumulation operation on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects when they exist, and the defect types include: abnormal high frequency and texture frequency loss. Some embodiments of the present application can achieve accurate and rapid detection of whether the overhead line has defects through an event camera.

Description

Technical Field

[0001] The present application relates to the technical field of image defect detection, and in particular to a method, storage medium, and device for overhead line defect detection based on an event camera. Background Art

[0002] With the continuous development of drone technology, drone inspection technology has been widely used in transmission line inspection.

[0003] Currently, when inspecting overhead power lines, drones use their onboard industrial cameras to capture video of the lines. This video is then transmitted back to a ground service station for manual or computer vision-based defect detection. However, the video contains large amounts of data, and due to limited resources, transmission efficiency is low, which in turn affects the efficiency of overhead line defect detection and prevents timely detection of problems.

[0004] Therefore, how to provide an efficient technical solution for overhead line defect detection based on event cameras has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The purpose of some embodiments of the present application is to provide a method, storage medium and device for overhead line defect detection based on an event camera. Through the technical solutions of the embodiments of the present application, high-efficiency detection of overhead line defects can be achieved through event cameras with high accuracy.

[0006] In a first aspect, some embodiments of the present application provide a method for detecting overhead line defects based on an event camera, comprising: filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; performing a frame accumulation operation on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defect when defects exist, and the defect types include: abnormal high frequency and texture frequency loss.

[0007] Some embodiments of the present application filter the collected event stream data of overhead lines, locate it within the overhead line location frame, and then perform a frame accumulation operation on the events in the processed event data within the overhead line location frame to obtain an overhead line image. Finally, the overhead line image is detected and analyzed to obtain an overhead line detection result. Some embodiments of the present application can achieve high-efficiency, real-time detection of overhead line defects using an event camera, reducing the amount of detection data and achieving higher accuracy.

[0008] In some embodiments, the event stream data of the overhead line collected by the event camera is filtered to obtain processed event data, including: obtaining an event occurrence matrix corresponding to the event stream data; performing a convolution operation on the event occurrence matrix to obtain an event convolution matrix; setting the event convolution matrix based on a preset threshold to obtain a binary matrix; and filtering the event stream data using an event filter mask corresponding to the binary matrix to obtain the processed event data.

[0009] Some embodiments of the present application determine a corresponding event filter mask by performing relevant operations on event stream data, and use the mask to filter the event stream data to obtain processed event data, thereby effectively filtering noise in the event stream data and removing interference.

[0010] In some embodiments, the event filter mask is obtained by the following method: obtaining the first vector and the second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; obtaining the values ​​corresponding to the first vector and the second vector as indices in the binary matrix to obtain the event filter mask.

[0011] Some embodiments of the present application can achieve effective filtering of event stream data by determining event filter masks with the same vector length in a binary matrix using the first vector and the second vector corresponding to the horizontal axis coordinates of all events in the event stream data as indices.

[0012] In some embodiments, obtaining the overhead line positioning frame based on the binary matrix corresponding to the event stream data includes: performing a closing operation on the binary matrix, screening out connected domains with an area not less than a threshold; obtaining the circumscribed rotated rectangle of each connected domain in the connected domain; and using the area in the circumscribed rotated rectangle of each connected domain with an aspect ratio greater than the ratio threshold as the overhead line positioning frame.

[0013] Some embodiments of the present application filter the connected domains of the binary matrix by closing the operation, and then filter the corresponding area from the external rotation matrix corresponding to each connected domain to locate the overhead line positioning frame, so as to achieve accurate positioning of the overhead line.

[0014] In some embodiments, the frame accumulation operation is performed on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image, including: polarity accumulation of the events within the overhead line positioning frame to obtain a positive pole image and a negative pole image, wherein the accumulated value of each pixel point in the positive pole image and the negative pole image does not exceed a preset value; after regularizing the positive pole image and the negative pole image, combining with an all-zero matrix, to obtain the overhead line image containing three channels.

[0015] Some embodiments of the present application obtain an overhead line image by accumulating and regularizing the polarity of events in the overhead line positioning frame, thereby achieving effective conversion of event data into images and providing data support for subsequent defect detection.

[0016] In some embodiments, the analyzing the overhead line image to obtain the detection result of the overhead line includes: obtaining a frequency domain image corresponding to the overhead line image; filtering the frequency domain image to obtain a filtered frequency domain image; performing an inverse Fourier transform on the filtered frequency domain image to obtain a restored image; and when it is confirmed that the number of non-zero pixels in the restored image is greater than a pixel threshold, the detection result is that a defect exists, and the defect type is the abnormally high frequency.

[0017] Some embodiments of the present application process and analyze the frequency domain image corresponding to the overhead line image to confirm whether there are abnormally high-frequency defects, thereby achieving accurate detection of overhead line defects.

[0018] In some embodiments, the analyzing the overhead line image to obtain the detection result of the overhead line includes: intercepting the overhead line image according to the sliding step size and the sliding window size to obtain the image to be analyzed; performing Fourier transform on the image to be analyzed to obtain the frequency domain image to be analyzed; intercepting a local image of a specified area in the frequency domain image to be analyzed; when it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency loss.

[0019] Some embodiments of the present application intercept the image to be analyzed in the overhead line image, and then perform frequency domain conversion and analysis on the image to be analyzed to confirm whether there is a defect of missing texture frequency, thereby achieving accurate detection of overhead line defects.

[0020] In some embodiments, confirming that there is no light spot in the local image includes: segmenting the local image using a segmentation threshold to obtain a binary local image; and when confirming that there is no connected domain in the binary local image that matches a preset light spot size, then the light spot does not exist.

[0021] Some embodiments of the present application perform connected domain analysis on the binarized local image after local image segmentation through a segmentation threshold to determine whether there is a light spot, thereby achieving accurate detection of whether there are defects in the overhead line.

[0022] In a second aspect, some embodiments of the present application provide a device for detecting overhead line defects based on an event camera, comprising: a filtering module for filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; a positioning module for obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; a frame accumulation module for performing frame accumulation operations on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; a detection module for analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defect when defects exist, and the defect type includes: abnormal high frequency and texture frequency loss.

[0023] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0024] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0025] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the following is a brief introduction to the drawings required for use in some embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A system diagram of overhead line defect detection based on an event camera provided for some embodiments of the present application;

[0028] Figure 2 One of the flow charts of the method for overhead line defect detection based on an event camera provided in some embodiments of the present application;

[0029] Figure 3 A schematic diagram of two types of defects provided for some embodiments of the present application;

[0030] Figure 4A schematic diagram comparing frequency domain images of overhead lines under normal conditions and abnormal high-frequency conditions provided in some embodiments of the present application;

[0031] Figure 5 A schematic diagram of changes in the overall frequency domain filtering and inverse transformation image restoration provided in some embodiments of the present application;

[0032] Figure 6 A second flow chart of a method for detecting overhead line defects based on an event camera provided in some embodiments of the present application;

[0033] Figure 7 A block diagram of an apparatus for overhead line defect detection based on an event camera provided in some embodiments of the present application;

[0034] Figure 8 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0036] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0037] With the development of drone technology, the stability, flexibility, payload capacity, and range of drone flights have greatly improved. The task of inspecting overhead transmission lines has gradually shifted from manual inspection to drones equipped with cameras. The general steps of current drone inspection schemes are as follows: drones equipped with industrial cameras fly along a route to capture visible light images of overhead lines. Large amounts of video data are transmitted back to ground service stations for defect detection using manual or computer vision. However, the current method of inspecting overhead lines using visible light images has the following problems:

[0038] 1) Traditional industrial cameras have difficulty capturing images at high speeds, resulting in motion blur. Therefore, high-speed cameras are generally required. However, these cameras increase costs and fail to achieve the high speeds required for line inspection, creating a bottleneck in line inspection efficiency. 2) Visible light image data is enormous, making transmission difficult. Currently, to meet inspection accuracy requirements, a single image from an area array industrial camera generates data volumes ranging from several to tens of megabytes, and even image compression technology struggles to reduce this to less than 1 megabyte. The massive amount of data generated at high speeds is difficult to transmit in real time. In some scenarios, transmission is even abandoned, requiring the use of storage media such as SSDs to store the data and then review it after the drone lands. 3) Visible light image processing algorithms are complex and time-consuming. Due to the large amount of image data and the complexity of the image processing algorithms, image processing takes a long time, making real-time inspection difficult. Typically, raw video data must be captured and then processed by a ground service station. Discovery of defects often requires re-flight to the defect site for confirmation, significantly increasing the workload.

[0039] It can be seen from the above-mentioned related technologies that the existing method of detecting whether there are defects in overhead lines requires a large amount of data transmission and has low efficiency.

[0040] In view of this, some embodiments of the present application provide a method for detecting overhead line defects using an event camera. This method uses an event camera to collect event stream data from the overhead line, then processes and analyzes the event stream data to determine whether the overhead line is defective. Some embodiments of the present application use event cameras for detection, which can reduce data transmission volume and improve detection efficiency.

[0041] It should be noted that unlike traditional area array cameras, event cameras do not capture area array images through a fixed exposure time. Instead, they detect changes in the brightness of each pixel. Specifically, if the pixel brightness remains unchanged, no data is output. If the pixel brightness changes, an event is output as a four-tuple (x, y, p, t), where x and y represent the horizontal and vertical coordinates of the event, p is the polarity (1 or 0), indicating whether the pixel has brightened or dimmed, and t is a microsecond timestamp. Using event camera data output for defect detection significantly reduces data volume, laying the foundation for edge computing and real-time detection.

[0042] The following is combined with Figure 1 The overall composition structure of the system for overhead line defect detection based on event cameras provided by some embodiments of the present application is exemplified.

[0043] like Figure 1As shown, some embodiments of the present application provide a system for detecting overhead power line defects based on an event camera. The system includes an event camera 100 mounted on a drone and an edge device 200. The edge device 200 can be a Raspberry Pi or other similar Linux development board. Its function is to perform edge computing on the event stream data output by the event camera 100, thereby achieving real-time defect detection of overhead power lines.

[0044] The following is combined with Figure 2 The implementation process of overhead line defect detection based on an event camera performed by the edge device 200 provided in some embodiments of the present application is exemplified.

[0045] Please see the attached Figure 2 , Figure 2 A flow chart of a method for detecting overhead power line defects based on an event camera is provided for some embodiments of the present application. The method for detecting overhead power line defects based on an event camera may include:

[0046] S210 , filtering the event stream data of the overhead line collected by the event camera to obtain processed event data.

[0047] For example, in some embodiments of the present application, in order to reduce the interference of noise in event stream data on subsequent detection and analysis, it is necessary to filter the event stream data to remove the noise. Noise generally occurs independently, and generally no other events occur in the vicinity of the noise point.

[0048] In some embodiments of the present application, S210 may include:

[0049] S211, obtaining an event occurrence matrix corresponding to the event stream data.

[0050] For example, in some embodiments of the present application, after the edge device 200 reads a packet of event stream data from the event camera 100, it creates a new all-zero matrix whose size is the camera resolution. The pixel positions where the event occurs in the all-zero matrix are set to 1, and the matrix M representing the occurrence of the event is obtained. events (As a specific example of an event occurrence matrix). events It only contains 0 and 1 to indicate whether an event occurs or not. The number of events is not accumulated, which reduces the amount of calculation.

[0051] S212: Perform a convolution operation on the event occurrence matrix to obtain an event convolution matrix.

[0052] For example, in some embodiments of the present application, events Perform convolution operation to obtain the event convolution matrix M that characterizes the occurrence of events in the neighborhood of each pixelnear Among them, the convolution kernel size can be selected as a matrix of odd sizes such as 3×3, 5×5, etc. as shown below:

[0053]

[0054] The matrix increases in weight from the edge to the center, indicating that events closer to the pixel being measured have greater weights, and events occurring at this location are more likely to prove that the pixel being measured is not a noise point. A zero value at the center eliminates interference from the pixel being measured itself.

[0055] S213: Setting the event convolution matrix based on a preset threshold to obtain a binary matrix.

[0056] For example, in some embodiments of the present application, near Perform threshold segmentation, set the position greater than the preset threshold to True, and the position less than the preset threshold to False, and obtain the binary matrix M mask The preset threshold is related to the size of the convolution kernel. If the initial threshold is T, then for a convolution kernel of size 2n+1, the preset threshold is calculated as T*n+b, where b is an adjustable parameter that can be adjusted according to the specific imaging conditions of the event camera.

[0057] S214 , filtering the event stream data using the event filter mask corresponding to the binary matrix to obtain the processed event data.

[0058] For example, in some embodiments of the present application, by mask Convert to event filter mask E mask , use it to filter the event stream data to obtain the processed event data.

[0059] In some embodiments of the present application, S214 may include: obtaining the first vector and the second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; obtaining the values ​​corresponding to the first vector and the second vector as indexes in the binary matrix to obtain the event filtering mask.

[0060] For example, in some embodiments of the present application, the event horizontal coordinates x and event vertical coordinates y of all events in the event stream data are respectively taken out to obtain a first vector V x and the second vector V y . With V x and V y As the index, take M mask The value in V x and V y Vectors E of the same length mask Finally use E maskFilter the entire package of event stream data and only retain the events with the value True, so that the event data can be processed.

[0061] S220: Obtain an overhead line positioning frame based on a binary matrix corresponding to the event stream data.

[0062] For example, in some embodiments of the present application, using the M obtained above mask The linear portion where the events are concentrated is located to obtain the overhead line positioning frame.

[0063] In some embodiments of the present application, S220 may include:

[0064] S221 , performing a closing operation on the binary matrix, and screening out connected domains whose area is not less than a threshold.

[0065] For example, in some embodiments of the present application, M mask After performing the morphological closing operation, connected domains whose areas are smaller than the area threshold are deleted, and connected domains for subsequent analysis are selected. The area threshold can be set according to the actual application scenario and is not specifically limited in this embodiment of the application.

[0066] S222: Obtain a circumscribed rotated rectangle of each connected domain in the connected domain.

[0067] For example, in some embodiments of the present application, the minimum circumscribed rotated rectangle of each connected domain is calculated.

[0068] S223 , taking the area of ​​the circumscribed rotated rectangle of each connected domain, where the aspect ratio is greater than a ratio threshold, as the overhead line positioning frame.

[0069] For example, in some embodiments of the present application, a rectangle whose aspect ratio is greater than a ratio threshold (e.g., a ratio threshold of 12) within the minimum circumscribed rotated rectangle of each connected domain is used as the overhead line positioning frame. The ratio threshold can be set according to the actual application scenario and is not specifically limited in the embodiments of the present application.

[0070] S230, performing a frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image.

[0071] For example, in some embodiments of the present application, since the event camera outputs only pixel points, it is necessary to perform a frame accumulation operation on the processed event data to obtain an overhead line image.

[0072] In some embodiments of the present application, S230 may include: performing polarity accumulation on events within the overhead line positioning frame to obtain a positive pole image and a negative pole image, wherein the accumulated value of each pixel point in the positive pole image and the negative pole image does not exceed a preset value; after regularizing the positive pole image and the negative pole image, combining with an all-zero matrix, obtaining the overhead line image containing three channels.

[0073] For example, in some embodiments of the present application, the events occurring in the overhead line positioning frame area in the processed event data obtained above are accumulated, and the events are accumulated into a statistical graph according to polarity to obtain a positive polarity graph H on and negative electrode diagram H off When the accumulated value of the pixel points at a certain position exceeds the preset value (for example, the preset value is 5), the accumulation will no longer be performed on this position to ensure that the maximum value in the image is the preset value. on and H off After normalization to 0~255, H on As the R channel, H off As the B channel, the all-zero matrix as the G channel, forming an RGB image (as a specific example of an overhead line image). By only accumulating the events within the overhead line positioning frame, the amount of data required for subsequent processing can be greatly reduced.

[0074] S240, analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects if defects exist, and the defect types include: abnormal high frequency and texture frequency loss.

[0075] For example, in some embodiments of the present application, a method based on Fast Fourier Transform (FFT) is used to convert the RGB image into the frequency domain for analysis. In the frequency domain image, overhead line defects can be divided into two categories: abnormal high frequency and texture frequency loss. The types of defects covered by these two categories are as follows: Figure 3 Abnormal high frequencies are usually caused by lightning strikes, flashovers, loose or broken strands, and linear foreign matter on overhead lines. Texture frequency loss is usually caused by foreign matter covering or contaminating the overhead lines. The types of defects can be expanded based on actual application scenarios, and the embodiments of this application are not limited thereto.

[0076] In some embodiments of the present application, for defects of the type of abnormally high frequency, S240 may include: obtaining a frequency domain image corresponding to the overhead line image; filtering the frequency domain image to obtain a filtered frequency domain image; performing an inverse Fourier transform on the filtered frequency domain image to obtain a restored image; and when it is confirmed that the number of non-zero pixels in the restored image is greater than a pixel threshold, the detection result is that a defect exists, and the defect type is the abnormally high frequency.

[0077] For example, in some embodiments of the present application, the step of determining abnormal high-frequency defects includes:

[0078] 1) Grayscale the transmission line image (that is, the overhead line image) and then perform Fourier transform on it as a whole to obtain the frequency domain image F (x，y) ;

[0079] 2) Set the frequency domain filter pair F (x，y) Perform filtering, where the filter setting method is as follows: Create a new filter with F (x，y) The same size of the full zero matrix, and then set the two regions x∈[0,0.49*w], y∈[0,0.55*h] and x∈[0.51*w,w], y∈[0.45*h,h] to 1. Where h, w are the height and width of the frequency domain image. After this filter, all the contents of the first and third quadrants and most of the low frequencies are filtered out, and the filtered frequency domain image G is obtained. (x，y) (as a specific example of a filtered frequency domain image);

[0080] 3) Against G (x，y) Perform inverse Fourier transform to obtain the restored image;

[0081] 4) Calculate the number of non-zero pixels N in the restored image restore ;

[0082] 5) Set pixel threshold T restore , to determine whether there is an abnormal high frequency defect. If N restore >T restore , then there is a defect, otherwise there is no defect.

[0083] For ease of understanding, the present application provides the following examples: Figure 4 Schematic diagram of the comparison of frequency domain images of overhead lines under normal conditions and abnormal high frequency conditions shown in FIG; and Figure 5 The schematic diagram of the overall frequency domain filtering and inverse transformation restoration image shown in the figure is as follows. Figure 5 The corresponding images from top to bottom are grayscale image, frequency domain image F (x，y) , filter, filtered frequency domain image G (x，y) and restore the image.

[0084] In some embodiments of the present application, for defects such as texture frequency missing, S240 may include: intercepting the overhead line image according to the sliding step size and the sliding window size to obtain the image to be analyzed; performing Fourier transform on the image to be analyzed to obtain the frequency domain image to be analyzed; intercepting a local image of a specified area in the frequency domain image to be analyzed; when it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency missing.

[0085] For example, in some embodiments of the present application, the step of determining texture frequency loss includes:

[0086] 1) Set the sliding window and sliding step size to capture the image to be analyzed.

[0087] Let the height of the grayscale image be h gray , width is w gray The height of the sliding window is h gray , with a width of w gray *0.04. This window has a sliding step size of w gray *0.02 Slide from left to right to intercept the overhead line image to obtain the image to be analyzed f win .

[0088] 2) For f win Perform Fourier transform to obtain the frequency domain image to be analyzed;

[0089] 3) Intercept the first quadrant image of the frequency domain image to be analyzed, and delete the left and lower edge parts of the first quadrant image where the low frequency is located to obtain the local image F of the specified area win1 The left and lower edge portions where the low frequency is located can be obtained by comparing and analyzing the coordinate values ​​of the region with the threshold.

[0090] 4) Determine F win1 The detection result is obtained by checking whether there is a light spot in the image.

[0091] In some embodiments of the present application, confirming that there is no light spot in the local image includes: segmenting the local image using a segmentation threshold to obtain a binary local image; and confirming that there is no connected domain in the binary local image that matches a preset light spot size, then the light spot does not exist.

[0092] For example: Select a suitable segmentation threshold T win1 F win1 Perform threshold segmentation to obtain a binary local image B win1 ; for B win1 Perform connected domain analysis, if B win1 If there is no connected domain that meets the preset spot size, it means there is no spot, which indicates that there is a defect in the overhead line image corresponding to the sliding window.

[0093] Ultimately, the detection results of the two defect types are combined to return the overhead line defect detection results (hereinafter referred to as the detection results) in real time. For example, the defect detection results may include: normal; or in abnormal cases: only abnormal high frequency, only missing texture frequency, or both abnormal high frequency and missing texture frequency. The data containing the detection results, defects, and operation logs are saved and transmitted in real time to the ground service station for verification and traceability.

[0094] The following is combined with Figure 6 The specific process of overhead line defect detection based on event cameras provided by some embodiments of the present application is exemplified.

[0095] Please see the attached Figure 6 , Figure 6 A flow chart of a method for overhead line defect detection based on an event camera is provided for some embodiments of the present application.

[0096] The above process is explained below as an example.

[0097] S601, the drone uses an event camera to shoot overhead lines during flight and outputs event stream data.

[0098] S602: The edge device filters the event stream data to obtain processed event data.

[0099] S603: The edge device performs a closing operation and screening on the binary matrix corresponding to the event stream data to obtain the overhead line positioning frame.

[0100] S604: The edge device performs a frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image.

[0101] S605 , the edge device processes the overhead line image using Fourier transform to obtain a frequency domain image.

[0102] S606, by processing and analyzing the frequency domain image, determine whether there is abnormal high frequency, if so, execute S609, otherwise execute S610.

[0103] S607 , the edge device uses Fourier transform to process the image to be analyzed that is intercepted according to the sliding step size and the sliding window size to obtain a sliding window frequency domain image (that is, the frequency domain image to be analyzed).

[0104] S608 , by processing and analyzing the sliding window frequency domain image, determine whether there is a texture frequency loss, if so, execute S609 , otherwise execute S610 .

[0105] S609: Confirm whether the overhead line has defects and output the defect type.

[0106] S610, the output overhead line is normal.

[0107] It can be understood that the specific implementation process of S601 to S610 can refer to the method embodiment provided above, and in order to avoid repetition, the detailed description is appropriately omitted here.

[0108] Through some of the embodiments of the present application described above, it can be seen that the present application uses an event camera instead of a visible light camera, and the source data is in the form of an event stream, which can achieve true high-speed shooting; the event stream only records the changes in pixels, rather than saving the entire image, reducing data redundancy. Moreover, the relevant algorithms for event stream data processing in the present application are simple and run fast, and can achieve real-time detection. Only the data of the defective part is saved, which reduces the amount of data and can be used for real-time transmission. In addition, the flight detection process does not require human participation. Personnel only need to set the route, and the drone can complete the automatic inspection task, which is highly practical.

[0109] Please refer to Figure 7 , Figure 7 A block diagram of the components of an event camera-based overhead power line defect detection device provided in some embodiments of the present application is shown. It should be understood that the event camera-based overhead power line defect detection device corresponds to the aforementioned method embodiment and is capable of performing each step involved in the aforementioned method embodiment. The specific functions of the event camera-based overhead power line defect detection device can be found in the description above, and a detailed description is omitted here to avoid repetition.

[0110] Figure 7 The device for detecting defects of overhead lines based on event cameras includes at least one software functional module that can be stored in a memory in the form of software or firmware or solidified in the device for detecting defects of overhead lines based on event cameras. The device for detecting defects of overhead lines based on event cameras includes: a filtering module 710, used to filter the event stream data of the overhead line collected by the event camera to obtain processed event data; a positioning module 720, used to obtain an overhead line positioning frame based on a binary matrix corresponding to the event stream data; a frame accumulation module 730, used to perform frame accumulation operations on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; a detection module 740, used to analyze the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects when they exist, and the defect types include: abnormal high frequency and texture frequency loss.

[0111] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0112] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0113] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0114] like Figure 8 As shown, some embodiments of the present application provide an electronic device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820, wherein the processor 820 can implement a method as described in any of the above embodiments when reading the program from the memory 810 through the bus 830 and executing the program.

[0115] The processor 820 can process digital signals and can include various computing architectures, such as a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements a combination of multiple instruction sets. In some examples, the processor 820 can be a microprocessor.

[0116] The memory 810 can be used to store instructions executed by the processor 820 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all functions of one or more modules described in the embodiments of this application. The processor 820 of the embodiment of the present disclosure can be used to execute the instructions in the memory 810 to implement the method shown above. The memory 810 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memory known to those skilled in the art.

[0117] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0118] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0119] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for detecting overhead line defects based on an event camera, characterized in that: include: Filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; Obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; performing a frame accumulation operation on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; Analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects if defects exist, and the defect types include: abnormal high frequency and texture frequency loss; The filtering process of the event stream data of the overhead line collected by the event camera to obtain the processed event data includes: obtaining an event occurrence matrix corresponding to the event stream data; performing a convolution operation on the event occurrence matrix to obtain an event convolution matrix; setting the event convolution matrix based on a preset threshold to obtain a binary matrix; filtering the event stream data using an event filter mask corresponding to the binary matrix to obtain the processed event data; the event filter mask is obtained by the following method: obtaining a first vector and a second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; obtaining values ​​corresponding to the first vector and the second vector as indices in the binary matrix to obtain the event filter mask; The method of obtaining the overhead line positioning frame based on the binary matrix corresponding to the event stream data includes: performing a closing operation on the binary matrix, screening out connected domains with an area not less than a threshold; obtaining the circumscribed rotated rectangle of each connected domain in the connected domain; and using the area in the circumscribed rotated rectangle of each connected domain with an aspect ratio greater than the ratio threshold as the overhead line positioning frame.

2. The method according to claim 1, wherein The performing a frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image includes: Accumulating polarity of events within the overhead line positioning frame to obtain a positive polarity graph and a negative polarity graph, wherein the accumulated value of each pixel point in the positive polarity graph and the negative polarity graph does not exceed a preset value; After regularizing the positive pole image and the negative pole image, combined with an all-zero matrix, the overhead line image containing three channels is obtained.

3. The method according to claim 1, wherein The analyzing the overhead line image to obtain the detection result of the overhead line includes: Acquiring a frequency domain image corresponding to the overhead line image; Filtering the frequency domain image to obtain a filtered frequency domain image; Performing inverse Fourier transform on the filtered frequency domain image to obtain a restored image; When it is confirmed that the number of non-zero pixels in the restored image is greater than a pixel threshold, the detection result is that a defect exists, and the defect type is the abnormal high frequency.

4. The method according to claim 1, wherein The analyzing the overhead line image to obtain the detection result of the overhead line includes: intercepting the overhead line image according to the sliding step size and the sliding window size to obtain an image to be analyzed; Performing Fourier transform on the image to be analyzed to obtain a frequency domain image to be analyzed; intercepting a local image of a designated area in the frequency domain image to be analyzed; When it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency loss.

5. The method according to claim 4, wherein The confirming that no light spot exists in the local image includes: Segmenting the local image using a segmentation threshold to obtain a binary local image; When it is confirmed that there is no connected domain that matches the preset light spot size in the binarized local image, the light spot does not exist.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to perform the method according to any one of claims 1 to 5.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 5 when run by the processor.

Images