Event camera-based overhead line defect detection method, storage medium and equipment

Abstract

The invention relates to the technical field of image defect detection, and particularly provides an event camera-based overhead line defect detection method, a storage medium and equipment, and the method can comprise the steps: carrying out the filtering processing of the event flow data of an overhead line collected by an event camera, and obtaining the processed event data; obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; performing frame accumulation operation on events in the overhead line positioning frame in the processed event data to obtain an overhead line image; the overhead line image is analyzed, a detection result of the overhead line is obtained, the detection result comprises whether the overhead line has defects or not and defect types when the overhead line has the defects, and the defect types comprise abnormal high frequency and texture frequency deficiency. According to some embodiments of the invention, whether the overhead line has defects can be accurately and rapidly detected through the event camera.

Description

Technical Field

[0001] The present application relates to the technical field of image defect detection, and in particular to a method, storage medium and device for overhead line defect detection based on an event camera. Background Art

[0002] With the continuous development of drone technology, drone inspection technology has been widely used in transmission line inspection.

[0003] At present, when drones perform overhead power line inspection tasks, they use their own industrial cameras to collect video of the overhead lines, and then transmit the video back to the ground service station for manual or computer vision technology to detect defects. However, the amount of data contained in the video is large, and due to limited resources, the transmission efficiency is low, which in turn affects the efficiency of overhead line defect detection, resulting in the inability to detect problems with overhead lines in a timely manner.

[0004] Therefore, how to provide an efficient technical solution for overhead line defect detection method based on event camera has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] Some embodiments of the present application aim to provide a method, storage medium and device for detecting overhead line defects based on an event camera. Through the technical solutions of the embodiments of the present application, efficient detection of overhead line defects can be achieved through an event camera with high accuracy.

[0006] In a first aspect, some embodiments of the present application provide a method for detecting overhead line defects based on an event camera, comprising: filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data; performing frame accumulation operations on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; analyzing the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the defect type when defects exist, and the defect types include: abnormal high frequency and texture frequency loss.

[0007] Some embodiments of the present application filter the collected event stream data of the overhead line, locate it in the overhead line positioning frame, and then perform frame accumulation operation on the events in the processed event data in the overhead line positioning frame to obtain an overhead line image, and finally perform detection and analysis on the overhead line image to obtain the detection result of the overhead line. Some embodiments of the present application can realize high-efficiency real-time detection of whether there are defects in the overhead line through the event camera, reduce the amount of detection data, and have high accuracy.

[0008] In some embodiments, the event stream data of the overhead line collected by the event camera is filtered to obtain processed event data, including: obtaining an event occurrence matrix corresponding to the event stream data; performing a convolution operation on the event occurrence matrix to obtain an event convolution matrix; setting the event convolution matrix based on a preset threshold to obtain a binary matrix; and filtering the event stream data using an event filter mask corresponding to the binary matrix to obtain the processed event data.

[0009] Some embodiments of the present application determine a corresponding event filter mask by performing relevant operations on event stream data, and use the mask to filter the event stream data to obtain processed event data, thereby effectively filtering noise in the event stream data and removing interference.

[0010] In some embodiments, the event filter mask is obtained by the following method: obtaining the first vector and the second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; obtaining the values ​​corresponding to the first vector and the second vector as indices in the binary matrix to obtain the event filter mask.

[0011] Some embodiments of the present application can achieve effective filtering of event stream data by determining event filter masks with the same vector length in a binary matrix using the first vector and the second vector corresponding to the horizontal axis coordinates of all events in the event stream data as indices.

[0012] In some embodiments, obtaining the overhead line locating frame based on the binary matrix corresponding to the event stream data includes: performing a closing operation on the binary matrix, screening out connected domains whose area is not less than a threshold value; obtaining the circumscribed rotated rectangle of each connected domain in the connected domain; and using the area in the circumscribed rotated rectangle of each connected domain whose aspect ratio is greater than the ratio threshold value as the overhead line locating frame.

[0013] Some embodiments of the present application select the connected domains of the binary matrix by closing the operation, and then select the corresponding area from the external rotation matrix corresponding to each connected domain to locate the overhead line positioning frame, so as to achieve accurate positioning of the overhead line.

[0014] In some embodiments, the frame accumulation operation is performed on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image, including: polarity accumulation of the events within the overhead line positioning frame to obtain a positive pole image and a negative pole image, wherein the accumulated value of each pixel in the positive pole image and the negative pole image does not exceed a preset value; after regularizing the positive pole image and the negative pole image, combining with an all-zero matrix, the overhead line image containing three channels is obtained.

[0015] Some embodiments of the present application obtain an overhead line image by accumulating and regularizing the polarity of events in an overhead line positioning frame, thereby achieving effective conversion of event data into an image and providing data support for subsequent defect detection.

[0016] In some embodiments, analyzing the overhead line image to obtain the detection result of the overhead line includes: obtaining a frequency domain image corresponding to the overhead line image; filtering the frequency domain image to obtain a filtered frequency domain image; performing an inverse Fourier transform on the filtered frequency domain image to obtain a restored image; and when it is confirmed that the number of non-zero pixels in the restored image is greater than a pixel threshold, the detection result is that a defect exists, and the defect type is the abnormally high frequency.

[0017] Some embodiments of the present application process and analyze the frequency domain image corresponding to the overhead line image to confirm whether there are abnormally high-frequency defects, thereby achieving accurate detection of overhead line defects.

[0018] In some embodiments, the overhead line image is analyzed to obtain the detection result of the overhead line, including: intercepting the overhead line image according to the sliding step size and the sliding window size to obtain the image to be analyzed; performing Fourier transform on the image to be analyzed to obtain the frequency domain image to be analyzed; intercepting a local image of a specified area in the frequency domain image to be analyzed; when it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency missing.

[0019] Some embodiments of the present application intercept the image to be analyzed in the overhead line image, and then perform frequency domain conversion and analysis on the image to be analyzed to confirm whether there is a defect of missing texture frequency, so as to achieve accurate detection of overhead line defects.

[0020] In some embodiments, confirming that there is no light spot in the local image includes: segmenting the local image using a segmentation threshold to obtain a binary local image; and when confirming that there is no connected domain in the binary local image that matches a preset light spot size, then the light spot does not exist.

[0021] Some embodiments of the present application perform connected domain analysis on the binary local image after local image segmentation through a segmentation threshold to determine whether there is a light spot, thereby achieving accurate detection of whether there are defects in the overhead line.

[0022] In a second aspect, some embodiments of the present application provide an overhead line defect detection device based on an event camera, comprising: a filtering module, used to filter the event stream data of the overhead line collected by the event camera to obtain processed event data; a positioning module, used to obtain an overhead line positioning frame based on a binary matrix corresponding to the event stream data; an accumulation frame module, used to perform an accumulation operation on events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; a detection module, used to analyze the overhead line image to obtain a detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the defect type when defects exist, and the defect type includes: abnormal high frequency and texture frequency loss.

[0023] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0024] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0025] In a fifth aspect, some embodiments of the present application provide a computer program product, wherein the computer program product comprises a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A system diagram of overhead line defect detection based on an event camera provided for some embodiments of the present application;

[0028] Figure 2 One of the flow charts of the method for overhead line defect detection based on event camera provided in some embodiments of the present application;

[0029] Figure 3 A schematic diagram of two types of defects provided for some embodiments of the present application;

[0030] Figure 4A schematic diagram of comparison of frequency domain images of overhead lines under normal conditions and abnormal high frequency conditions provided for some embodiments of the present application;

[0031] Figure 5 A schematic diagram of changes in the overall frequency domain filtering and inverse transformation image restoration provided in some embodiments of the present application;

[0032] Figure 6 A second flowchart of a method for overhead line defect detection based on an event camera provided in some embodiments of the present application;

[0033] Figure 7 A block diagram of an apparatus for overhead line defect detection based on an event camera provided in some embodiments of the present application;

[0034] Figure 8 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0036] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0037] With the development of drone technology, the stability, flexibility, mounting capacity and range of drone flights have been greatly improved. The task of patrolling overhead power lines has gradually changed from manual patrol to patrolling by drones equipped with cameras. The general steps of the current drone patrol scheme are as follows: drones equipped with industrial cameras fly along the route to collect visible light images of overhead lines; a large amount of video data is transmitted back to the ground service station for defect detection by manual or computer vision. However, the current method of image detection of overhead lines based on visible light images has the following problems:

[0038] 1) It is difficult for traditional industrial cameras to collect images under high-speed flight conditions, which will produce motion blur. Therefore, high-speed cameras are generally required, which not only increase the cost, but also cannot achieve the real high speed required for line patrol flight, becoming a bottleneck for line patrol efficiency; 2) The amount of visible light image data is huge and difficult to transmit. At present, in order to meet the requirements of detection accuracy, the amount of data of a single photo of an array industrial camera is about several megabytes to tens of megabytes, and even with the use of image compression technology, it is difficult to reduce it to less than 1 megabyte. The massive amount of data generated under high-speed flight conditions is difficult to transmit in real time. In some scenarios, it is even necessary to give up transmission and use storage media such as SSD to store data, and then check it after the drone lands. 3) The visible light image processing algorithm is complex and time-consuming. Due to the large amount of image data and the complex image processing algorithm, the image processing time is long, making it difficult to achieve real-time detection. It is usually necessary to shoot the original video data first, and then process the video data through the ground service station. After the defect is found, it is often necessary to fly to the defect point again for confirmation, which undoubtedly increases the workload of the task.

[0039] It can be seen from the above-mentioned related technologies that the conventional method of detecting whether there are defects in overhead lines requires a large amount of data transmission and has low efficiency.

[0040] In view of this, some embodiments of the present application provide a method for detecting overhead line defects based on an event camera, which collects event stream data of the overhead line through an event camera, and then processes and analyzes the event stream data to obtain a detection result of whether the overhead line has defects. Some embodiments of the present application use event cameras for detection, which can reduce the amount of data transmission and improve detection efficiency.

[0041] It should be noted that event cameras are different from traditional area array cameras. They do not obtain area array images through a certain exposure time, but detect the brightness change of each pixel. That is, if the brightness of the pixel does not change, no data is output; if the brightness of the pixel changes, an event in the format of a four-tuple (x, y, p, t) is output, where x and y represent the horizontal and vertical coordinates of the event, p is the polarity, which takes a value of 1 or 0, indicating whether the pixel becomes brighter or darker, and t is a microsecond timestamp. The method of outputting data from the event camera for defect detection can greatly reduce the amount of data, laying the foundation for edge computing and real-time detection.

[0042] The following is combined with Figure 1 The overall composition structure of a system for overhead line defect detection based on an event camera provided by some embodiments of the present application is exemplified.

[0043] like Figure 1As shown, some embodiments of the present application provide a system for overhead line defect detection based on an event camera, and the system for overhead line defect detection based on an event camera includes an event camera 100 mounted on a drone and an edge device 200. Among them, the edge device 200 can be a development board of a Raspberry Pi or other similar Linux system, and its function is to perform edge computing on the event stream data output by the event camera 100 to achieve real-time defect detection of overhead lines.

[0044] The following is combined with Figure 2 The implementation process of overhead line defect detection based on an event camera performed by the edge device 200 provided in some embodiments of the present application is exemplified.

[0045] Please see attached Figure 2 , Figure 2 A flow chart of a method for detecting overhead line defects based on an event camera is provided for some embodiments of the present application. The method for detecting overhead line defects based on an event camera may include:

[0046] S210, filtering the event stream data of the overhead line collected by the event camera to obtain processed event data.

[0047] For example, in some embodiments of the present application, in order to reduce the interference of noise in the event stream data on subsequent detection and analysis, the event stream data needs to be filtered to remove the noise. The noise generally occurs independently, and generally no other events occur in the neighborhood of the noise point.

[0048] In some embodiments of the present application, S210 may include:

[0049] S211, obtaining an event occurrence matrix corresponding to the event stream data.

[0050] For example, in some embodiments of the present application, after the edge device 200 reads a packet of event stream data from the event camera 100, it creates a new all-zero matrix whose size is the camera resolution. The pixel positions where the event occurs in the all-zero matrix are set to 1, and the matrix M representing the occurrence of the event is obtained. events (As a specific example of an event occurrence matrix). events It only contains 0 and 1 to indicate whether an event occurs or not. The number of events is not accumulated, which reduces the amount of calculation.

[0051] S212, performing a convolution operation on the event occurrence matrix to obtain an event convolution matrix.

[0052] For example, in some embodiments of the present application, events Perform a convolution operation to obtain the event convolution matrix M that characterizes the occurrence of events in the neighborhood of each pixel.near Among them, the convolution kernel size can be selected as a matrix of odd sizes such as 3×3, 5×5, etc. as shown below:

[0053]

[0054] The matrix increases from the edge to the center, indicating that the closer the event is to the point to be tested, the greater the weight of the event. The event occurring at this location is more likely to prove that the pixel to be tested is not a noise point. The center is 0, which eliminates the interference of the pixel to be tested itself.

[0055] S213, setting the event convolution matrix based on a preset threshold to obtain a binary matrix.

[0056] For example, in some embodiments of the present application, near Perform threshold segmentation, set the position greater than the preset threshold to True, and the position less than the preset threshold to False, and obtain the binary matrix M mask The setting of the preset threshold is related to the size of the convolution kernel. If the initial threshold is T, then for a convolution kernel of size 2n+1, the calculation formula of the preset threshold is T*n+b, where b is an adjustable parameter that can be adjusted according to the specific imaging conditions of the event camera.

[0057] S214, filtering the event stream data using the event filter mask corresponding to the binary matrix to obtain the processed event data.

[0058] For example, in some embodiments of the present application, by mask Convert to event filter mask E mask , use it to filter the event stream data to obtain the processed event data.

[0059] In some embodiments of the present application, S214 may include: obtaining the first vector and the second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; obtaining the values ​​corresponding to the first vector and the second vector as indexes in the binary matrix to obtain the event filtering mask.

[0060] For example, in some embodiments of the present application, the event horizontal coordinates x and event vertical coordinates y of all events in the event stream data are respectively taken out to obtain a first vector V x and the second vector V y . With V x and V y As the index, take M mask The value in V x and V y Vectors E of the same length mask Finally, use E maskFilter the entire package of event stream data and only retain the events with the value True, so that the event data can be processed.

[0061] S220, obtaining an overhead line positioning frame based on the binary matrix corresponding to the event stream data.

[0062] For example, in some embodiments of the present application, using the M obtained above mask The linear part where the events are concentrated is located to obtain the overhead line positioning frame.

[0063] In some embodiments of the present application, S220 may include:

[0064] S221, after performing a closing operation on the binary matrix, a connected domain having an area not less than a threshold is screened out.

[0065] For example, in some embodiments of the present application, M mask After the morphological closing operation, the connected domains whose areas are smaller than the area threshold are deleted, and the connected domains for subsequent analysis are selected. The area threshold can be set according to the actual application scenario, and the embodiment of the present application does not specifically limit it.

[0066] S222: Obtain a circumscribed rotated rectangle of each connected domain in the connected domain.

[0067] For example, in some embodiments of the present application, the minimum circumscribed rotation rectangle of each connected domain is calculated.

[0068] S223, taking the area of ​​the circumscribed rotated rectangle of each connected domain whose length-to-width ratio is greater than the ratio threshold as the overhead line positioning frame.

[0069] For example, in some embodiments of the present application, a rectangle whose length-to-width ratio in the smallest circumscribed rotated rectangle of each connected domain is greater than a ratio threshold (for example, the ratio threshold is 12) is used as an overhead line positioning frame. The ratio threshold can be set according to the actual application scenario, and the embodiments of the present application are not specifically limited here.

[0070] S230, performing frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image.

[0071] For example, in some embodiments of the present application, since the event camera outputs pixels, it is necessary to perform frame accumulation operations on the processed event data to obtain an overhead line image.

[0072] In some embodiments of the present application, S230 may include: performing polarity accumulation on events within the overhead line positioning frame to obtain a positive pole image and a negative pole image, wherein the accumulated value of each pixel point in the positive pole image and the negative pole image does not exceed a preset value; after regularizing the positive pole image and the negative pole image, combining with an all-zero matrix, obtaining the overhead line image containing three channels.

[0073] For example, in some embodiments of the present application, the events occurring in the overhead line positioning frame area in the above-obtained processed event data are accumulated, and the events are accumulated into a statistical graph according to polarity to obtain a positive polarity graph H. on and negative pole diagram H off When the accumulated value of the pixel points at a certain position exceeds the preset value (for example, the preset value is 5), no more accumulation is performed at this position to ensure that the maximum value in the graph is the preset value. on and H off After normalization to 0-255, H on As the R channel, H off As the B channel, the all-zero matrix as the G channel, constitutes an RGB image (as a specific example of an overhead line image). By accumulating only the events within the overhead line positioning frame, the amount of data that needs to be processed later can be greatly reduced.

[0074] S240, analyzing the overhead line image to obtain the detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects when defects exist, and the defect types include: abnormal high frequency and missing texture frequency.

[0075] For example, in some embodiments of the present application, a method based on fast Fourier transform (FFT) is used to convert the RGB image to the frequency domain for analysis. In the frequency domain image, overhead line defects can be divided into two categories: abnormal high frequency and texture frequency loss. The types of these two types of defects are as follows: Figure 3 As shown. Among them, abnormal high frequency is usually caused by lightning strikes, flashovers, loose strands, broken strands and linear foreign objects on the overhead line; texture frequency loss is usually caused by foreign objects covering and contamination on the overhead line. The types of defects can be expanded according to actual application scenarios, and the embodiments of the present application are not limited to this.

[0076] In some embodiments of the present application, for defects such as abnormally high frequency, S240 may include: acquiring a frequency domain image corresponding to the overhead line image; filtering the frequency domain image to obtain a filtered frequency domain image; performing an inverse Fourier transform on the filtered frequency domain image to obtain a restored image; and when it is confirmed that the number of non-zero pixels in the restored image is greater than a pixel threshold, the detection result is that a defect exists, and the defect type is the abnormally high frequency.

[0077] For example, in some embodiments of the present application, the step of determining abnormal high frequency defects includes:

[0078] 1) The transmission line image (that is, the overhead line image) is grayed out, and then Fourier transformed as a whole to obtain the frequency domain image F (x，y) ;

[0079] 2) Set the frequency domain filter pair F (x，y) Filtering is performed, where the filter setting method is as follows: Create a new (x，y) The same size of all-zero matrices, and then set the two regions x∈[0,0.49*w], y∈[0,0.55*h] and x∈[0.51*w,w], y∈[0.45*h,h] to 1. Where h, w are the height and width of the frequency domain image. After this filter, all the contents of the first and third quadrants and most of the low frequencies are filtered out, and the filtered frequency domain image G is obtained. (x，y) (as a specific example of a filtered frequency domain image);

[0080] 3) For G (x，y) Perform inverse Fourier transform to obtain the restored image;

[0081] 4) Calculate the number of non-zero pixels N of the restored image restore ;

[0082] 5) Set pixel threshold T restore , determine whether there is an abnormal high frequency defect. restore >T restore , then there is a defect, otherwise there is no defect.

[0083] For ease of understanding, the present application provides the following examples: Figure 4 Schematic diagram of the comparison of frequency domain images of overhead lines under normal conditions and abnormal high frequency conditions shown in FIG; and Figure 5 The schematic diagram of the overall frequency domain filtering and inverse transformation restoration image shown in the figure is as follows. Figure 5 The corresponding images from top to bottom are grayscale image, frequency domain image F (x，y) , filter, filtered frequency domain image G (x，y) and restore the image.

[0084] In some embodiments of the present application, for defects such as texture frequency missing, S240 may include: intercepting the overhead line image according to the sliding step size and the sliding window size to obtain the image to be analyzed; performing Fourier transform on the image to be analyzed to obtain the frequency domain image to be analyzed; intercepting a local image of a specified area in the frequency domain image to be analyzed; when it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency missing.

[0085] For example, in some embodiments of the present application, the step of determining texture frequency loss includes:

[0086] 1) Set the sliding window and sliding step size to capture the image to be analyzed.

[0087] Let the height of the grayscale image be h gray , width is w gray The height of the sliding window is h gray , with a width of w gray *0.04. This window has a sliding step size of w gray *0.02 Slide from left to right to intercept the overhead line image to obtain the image to be analyzed f win .

[0088] 2) For f win Perform Fourier transform to obtain the frequency domain image to be analyzed;

[0089] 3) Intercept the first quadrant image of the frequency domain image to be analyzed, and delete the left and lower edge parts where the low frequency is located in the first quadrant image to obtain the local image F of the specified area win1 The left and lower edge parts where the low frequency is located can be obtained by comparing and analyzing the coordinate values ​​of the area and the threshold.

[0090] 4) Determine F win1 Whether there is a light spot in the image can be detected and the detection result can be obtained.

[0091] In some embodiments of the present application, confirming that there is no light spot in the local image includes: segmenting the local image using a segmentation threshold to obtain a binary local image; and when confirming that there is no connected domain in the binary local image that matches a preset light spot size, then the light spot does not exist.

[0092] For example: Select a suitable segmentation threshold T win1 F win1 Perform threshold segmentation to obtain the binary local image B win1 ; for B win1 Connected domain analysis is performed. If B win1 If there is no connected domain that meets the preset spot size, it means there is no spot, which indicates that there is a defect in the overhead line image corresponding to the sliding window.

[0093] Finally, the detection results of the above two types of defects are combined, and the defect detection results of the overhead line (hereinafter referred to as the detection results) are returned in real time. For example, the detection results of defects may include: normal, or in abnormal cases: only abnormal high frequency exists, only texture frequency is missing, or abnormal high frequency and texture frequency are missing. The data including the detection results, defects and operation logs are saved and transmitted to the ground service station in real time for verification and backtracking.

[0094] The following is combined with Figure 6 The specific process of overhead line defect detection based on an event camera provided by some embodiments of the present application is exemplified.

[0095] Please see attached Figure 6 , Figure 6 A flow chart of a method for overhead line defect detection based on an event camera is provided for some embodiments of the present application.

[0096] The above process is explained below as an example.

[0097] S601, the drone uses an event camera to photograph the overhead lines during flight and outputs event stream data.

[0098] S602: The edge device performs filtering processing on the event stream data to obtain processed event data.

[0099] S603, the edge device performs a closing operation and screening on the binary matrix corresponding to the event stream data to obtain the overhead line positioning frame.

[0100] S604, the edge device performs a frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image.

[0101] S605, the edge device processes the overhead line image using Fourier transform to obtain a frequency domain image.

[0102] S606, by processing and analyzing the frequency domain image, determine whether there is abnormal high frequency, if so, execute S609, otherwise execute S610.

[0103] S607, the edge device uses Fourier transform to process the image to be analyzed that is intercepted according to the sliding step size and the sliding window size to obtain a sliding window frequency domain image (that is, the frequency domain image to be analyzed).

[0104] S608, by processing and analyzing the sliding window frequency domain image, determine whether there is a texture frequency missing, if so, execute S609, otherwise execute S610.

[0105] S609, confirming that there are defects in the overhead line and outputting the defect type.

[0106] S610, output overhead line is normal.

[0107] It can be understood that the specific implementation process of S601 to S610 can refer to the method embodiment provided above, and in order to avoid repetition, the detailed description is appropriately omitted here.

[0108] Through some of the embodiments of the present application described above, it can be known that the present application uses an event camera instead of a visible light camera, and the source data event stream is in the form of a real high-speed shooting; the event stream only records the changes in pixels, rather than saving the entire image, reducing data redundancy. Moreover, the relevant algorithms for event stream data processing in the present application are simple, run fast, and can achieve real-time detection. Only the data of the defective part is saved, which reduces the amount of data and can be used for real-time transmission. In addition, the flight detection process does not require human participation. Personnel only need to set the route, and the drone can complete the automatic inspection task, which is highly practical.

[0109] Please refer to Figure 7 , Figure 7 The block diagram of the composition of the device for detecting overhead line defects based on an event camera provided by some embodiments of the present application is shown. It should be understood that the device for detecting overhead line defects based on an event camera corresponds to the above method embodiment and can perform the various steps involved in the above method embodiment. The specific functions of the device for detecting overhead line defects based on an event camera can be found in the description above. To avoid repetition, the detailed description is appropriately omitted here.

[0110] Figure 7 The device for detecting defects of overhead lines based on event cameras includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the device for detecting defects of overhead lines based on event cameras. The device for detecting defects of overhead lines based on event cameras includes: a filtering module 710, which is used to filter the event stream data of the overhead lines collected by the event camera to obtain processed event data; a positioning module 720, which is used to obtain an overhead line positioning frame based on a binary matrix corresponding to the event stream data; an accumulation frame module 730, which is used to accumulate frames for events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; a detection module 740, which is used to analyze the overhead line image to obtain the detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the defect type when it exists, and the defect type includes: abnormal high frequency and texture frequency loss.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0112] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations of the method corresponding to any of the above methods provided in the above embodiments.

[0113] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0114] like Figure 8 As shown, some embodiments of the present application provide an electronic device 800, which includes: a memory 810, a processor 820, and a computer program stored in the memory 810 and executable on the processor 820, wherein the processor 820 can implement a method as described in any of the above embodiments when reading the program from the memory 810 through a bus 830 and executing the program.

[0115] Processor 820 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 820 can be a microprocessor.

[0116] The memory 810 may be used to store instructions executed by the processor 820 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some or all functions of one or more modules described in the embodiments of the present application. The processor 820 of the disclosed embodiment may be used to execute instructions in the memory 810 to implement the method shown above. The memory 810 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0117] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0118] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0119] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for detecting overhead line defects based on an event camera, characterized in that: include: Filtering the event stream data of the overhead line collected by the event camera to obtain processed event data; Based on the binary matrix corresponding to the event stream data, obtaining an overhead line positioning frame; Performing frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image; The overhead line image is analyzed to obtain the detection result of the overhead line, wherein the detection result includes: whether the overhead line has defects and the type of defects when defects exist, and the defect types include: abnormal high frequency and missing texture frequency.

2. The method according to claim 1, characterized in that The filtering process of the event stream data of the overhead line collected by the event camera to obtain the processed event data includes: Obtaining an event occurrence matrix corresponding to the event stream data; Performing a convolution operation on the event occurrence matrix to obtain an event convolution matrix; The event convolution matrix is ​​set based on a preset threshold to obtain a binary matrix; The event stream data is filtered using the event filter mask corresponding to the binary matrix to obtain the processed event data.

3. The method according to claim 2, characterized in that The event filter mask is obtained by: Obtain the first vector and the second vector corresponding to all event horizontal coordinates and all event vertical coordinates in the event stream data; The values ​​corresponding to the first vector and the second vector as indexes are obtained in the binary matrix to obtain the event filtering mask.

4. The method according to claim 2 or 3, characterized in that The step of obtaining an overhead line positioning frame based on a binary matrix corresponding to the event stream data includes: After performing a closing operation on the binary matrix, a connected domain having an area not less than a threshold is screened out; Obtain a circumscribed rotated rectangle of each connected domain in the connected domain; The area in the circumscribed rotated rectangle of each connected domain whose length-to-width ratio is greater than the ratio threshold is used as the overhead line positioning frame.

5. The method according to any one of claims 1 to 3, characterized in that The step of performing a frame accumulation operation on the events in the processed event data that are within the overhead line positioning frame to obtain an overhead line image includes: Accumulating the polarity of events in the overhead line positioning frame to obtain a positive polarity graph and a negative polarity graph, wherein the accumulated value of each pixel point in the positive polarity graph and the negative polarity graph does not exceed a preset value; After the positive pole image and the negative pole image are processed in a regularized manner and combined with an all-zero matrix, the overhead line image containing three channels is obtained.

6. The method according to any one of claims 1 to 3, characterized in that The analyzing the overhead line image to obtain the detection result of the overhead line includes: Acquire a frequency domain image corresponding to the overhead line image; Filtering the frequency domain image to obtain a filtered frequency domain image; Performing inverse Fourier transform on the filtered frequency domain image to obtain a restored image; When it is confirmed that the number of non-zero pixels in the restored image is greater than the pixel threshold, the detection result is that a defect exists, and the defect type is the abnormal high frequency.

7. The method according to any one of claims 1 to 3, characterized in that The analyzing the overhead line image to obtain the detection result of the overhead line includes: According to the sliding step length and the sliding window size, the overhead line image is intercepted to obtain an image to be analyzed; Performing Fourier transform on the image to be analyzed to obtain a frequency domain image to be analyzed; intercepting a local image of a designated area in the frequency domain image to be analyzed; When it is confirmed that there is no light spot in the local image, the detection result is that there is a defect, and the defect type is the texture frequency missing.

8. The method according to claim 7, characterized in that The confirming that there is no light spot in the local image includes: Segmenting the local image using a segmentation threshold to obtain a binary local image; When it is confirmed that there is no connected domain that matches the preset light spot size in the binary local image, the light spot does not exist.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program executes the method according to any one of claims 1 to 8 when executed by a processor.

10. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 8 when being run by the processor.

Images