Inspection result data processing system and method for unmanned aerial vehicle inspection

Abstract

The invention provides an inspection result data processing system and method for unmanned aerial vehicle inspection, and belongs to the technical field of data processing, and the method comprises the steps: carrying out the frame extraction operation of video data in an inspection result, and obtaining a plurality of video frames; performing target detection on each video frame to obtain a first target area in each video frame; performing feature extraction on the first target area in each video frame to obtain feature points in each video frame; determining the motion information of each video frame based on the position change of the same feature points in every two adjacent video frames; and affine transformation is performed on the video frames based on the motion information of the video frames, so that the video frames are aligned, and stable processing of the video data is realized. According to the inspection result data processing system and method for unmanned aerial vehicle inspection provided by the invention, the accuracy of an electric power inspection result can be improved.

Description

Technical Field

[0001] The present application belongs to the field of drone technology, and more specifically, relates to a system and method for processing inspection result data for drone inspections. Background Art

[0002] With the continuous advancement of technology, drones are gaining widespread application in fields such as power line inspection, security monitoring, agriculture, and surveying and mapping. For example, drones equipped with various sensors, such as visible light cameras, infrared thermal imagers, and lidar, can collect images, videos, and point cloud data of power lines and equipment. By integrating and analyzing this data from multiple sensors, the operating status of power equipment can be monitored in real time. However, during the video data collection process, changes in the drone's flight attitude can cause video jitter. This makes it difficult to accurately identify and locate targets when analyzing jittery videos, thus affecting the accuracy of power line inspection results. Summary of the Invention

[0003] The purpose of this application is to provide a system and method for processing inspection result data for drone inspections to improve the accuracy of power inspection results.

[0004] A first aspect of an embodiment of the present application provides a method for processing inspection result data for drone inspections, comprising: Performing frame extraction operation on the video data in the inspection result to obtain multiple video frames; Performing target detection on each video frame respectively to obtain a first target area in each video frame; Extracting features of the first target area in each video frame to obtain feature points in each video frame; Determine the motion information of each video frame based on the position change of the same feature point in every two adjacent video frames; Affine transformation is performed on each video frame based on motion information of each video frame to align the video frames and achieve stable processing of video data.

[0005] A second aspect of an embodiment of the present application provides an inspection result data processing system for drone inspections, comprising: The video frame extraction module is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames; The target detection module is used to perform target detection on each video frame to obtain a first target area in each video frame; A feature extraction module is used to extract features of the first target area in each video frame to obtain feature points in each video frame; A first processing module determines motion information of each video frame based on position changes of the same feature points in every two adjacent video frames; The second processing module is used to perform affine transformation on each video frame based on motion information of each video frame to align the video frames and achieve stable processing of video data.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above-mentioned inspection result data processing method for drone inspection are implemented.

[0007] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned inspection result data processing method for drone inspection are implemented.

[0008] The inspection result data processing system and method for drone inspection provided by the embodiments of the present application have the following beneficial effects: The embodiment of the present application focuses on the key area (the first target area) for feature point extraction, which can greatly reduce the amount of calculation and improve the operation speed; based on the position movement of the feature points in adjacent frames, the relative motion relationship between the video frames can be captured, and the motion information such as translation, rotation and scaling of the video frames can be calculated. By performing affine transformation on each video frame according to the motion information, stable alignment of the video frames can be achieved, making the video picture more stable, which is conducive to more accurate subsequent analysis of the inspection target. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0010] Figure 1 A flowchart of a method for processing inspection result data for drone inspections provided in one embodiment of the present application; Figure 2 This is a structural block diagram of an inspection result data processing system for drone inspections provided in one embodiment of the present application; Figure 3 A schematic block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0011] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0012] In order to make the purpose, technical solutions and advantages of this application clearer, specific embodiments will be described below with reference to the accompanying drawings.

[0013] Please refer to Figure 1 , Figure 1 A flowchart of a method for processing inspection result data for drone inspections provided in one embodiment of the present application is provided, the method comprising: S101: performing a frame extraction operation on the video data in the inspection result to obtain multiple video frames.

[0014] In this embodiment, the video data is essentially composed of a series of continuous video frames (or image frames). By extracting multiple video frames from the video data, the video frame image at each moment can be processed and analyzed. S102: Performing target detection on each video frame to obtain a first target region in each video frame.

[0015] In this embodiment, existing deep learning-based fast convolutional neural network (Faster Region-based Convolutional Neural Network, Faster R-CNN), YOLO and other target detection algorithms can be used to perform target detection on each video frame.

[0016] By performing target detection on each video frame, we can identify targets such as power equipment or power lines within the video frame. The area containing the power equipment or power lines is the first target area. By focusing on the first target area for subsequent feature extraction, we can reduce unnecessary background interference and improve the efficiency of subsequent feature extraction and analysis.

[0017] S103: Extract features of the first target area in each video frame to obtain feature points in each video frame.

[0018] In this embodiment, feature points are representative and stable points in each video frame, such as corners and edges. These points can be accurately detected in different video frames with high repeatability. For example, when the same object is shot from different angles or under different lighting conditions, certain points on its edge can still maintain distinct characteristics and be easily identified by the feature detection algorithm.

[0019] Specifically, existing feature extraction algorithms such as Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) may be used to extract feature points in each video frame.

[0020] S104: Determine motion information of each video frame based on position changes of the same feature points in every two adjacent video frames.

[0021] In this embodiment, by tracking the positional movement of the same feature points in adjacent frames, the relative motion relationship between video frames can be captured, and motion information such as translation, rotation, and scaling of the video frames can be calculated. Each two adjacent video frames can be the current video frame and the next one, or the current video frame and the previous one. By calculating the positional change of the same feature points in each of the two adjacent video frames, the motion information of the current video frame can be obtained.

[0022] Specifically, the motion information of each video frame may be determined by using an existing optical flow method (such as the Lucas-Kanade optical flow method) or a feature matching-based method.

[0023] S105: performing affine transformation on each video frame based on motion information of each video frame to align the video frames and achieve stable processing of video data.

[0024] In this embodiment, affine transformation is performed on each video frame according to motion information, so that operations such as translation, rotation and scaling of the video frame can be realized, and the first target area in different video frames is adjusted to the same position and angle to achieve alignment of the video frames, thereby eliminating jitter and making the video picture more stable, which is convenient for subsequent analysis and judgment of the inspection results.

[0025] Specifically, based on the motion information of each video frame, parameters such as translation, rotation angle, and scaling factor can be calculated to construct an affine transformation matrix. For example, the overall translation parameter can be determined by calculating the average translation of the first motion vectors of all feature points. The rotation angle can be obtained by calculating the average direction of all first motion vectors, comparing it to the horizontal direction, and then using the inverse tangent function. For the scaling factor, the lengths of each first motion vector are first calculated and averaged, then compared to a preset reference length. The ratio of the two is the scaling factor.

[0026] On the basis of obtaining the affine transformation matrix, according to the mathematical formula of affine transformation, the coordinates of each pixel point in the current video frame are transformed through matrix multiplication to obtain its coordinate value at the new position.

[0027] From the above, it can be concluded that this embodiment focuses on the key area (the first target area) for feature point extraction, which can greatly reduce the amount of calculation and improve the operation speed; based on the position movement of the feature points in adjacent frames, the relative motion relationship between the video frames can be captured, and the motion information such as translation, rotation and scaling of the video frames can be calculated. By performing affine transformation on each video frame according to the motion information, stable alignment of the video frames can be achieved, making the video picture more stable, which is conducive to more accurate subsequent analysis of the inspection target.

[0028] In one embodiment of the present application, target detection is performed on each video frame to obtain a first target area in each video frame, including: Performing edge detection on each video frame to obtain a second target area in each video frame; Classifying each video frame based on contour information of the second target area in each video frame to obtain classification information of each video frame; If the classification information of the video frame shows that the video frame is a line video frame, detecting the line area in the video frame based on the Hough transform method; If the classification information of the video frame shows that the video frame is a power equipment video frame, detecting the power equipment area in the video frame based on the target detection algorithm; A line area or a power equipment area is determined as a first target area.

[0029] In this embodiment, considering the obvious grayscale difference between the target (power equipment or power lines, etc.) and the background, existing edge detection algorithms such as Sobel operator and Canny operator can be used to roughly outline the target to obtain a preliminary second target area.

[0030] Different targets have different contour features. For example, power lines typically appear as long, thin lines, while power equipment typically has a more regular or unique geometric shape. By analyzing the contour information (such as shape, perimeter, and area) of the second target area, a video frame can be classified as a line video frame or a power equipment video frame. For example, the contour of power equipment typically has a large area and a relatively regular shape, resulting in a high degree of circularity; whereas the contour of a line has a smaller area, a longer perimeter, and a lower degree of circularity. Therefore, a second threshold can be set. When the circularity of the second target area is greater than the second threshold, the video frame containing the second target area is a power equipment video frame. When the circularity of the second target area is less than or equal to the second threshold, the video frame containing the second target area is a line video frame.

[0031] For line video frames, the Hough transform can accurately locate the position and direction of the line. Its calculation speed is relatively fast, and it can quickly locate the line area. The Hough transform is a classic algorithm for detecting straight lines in images. It effectively detects the parameters of the line by mapping points in the image to a parameter space.

[0032] For power equipment video frames, a dedicated target detection algorithm is needed to accurately detect the power equipment and then determine the power equipment area. Both the line area and the power equipment area can be used as the first target area.

[0033] From the above, it can be concluded that this embodiment first uses edge detection to obtain a preliminary target area (second target area), then classifies the video frames according to the contour information of the second target area, and finally adopts different detection methods for different video frames, thereby avoiding the use of complex target detection algorithms for all video frames, reducing the amount of calculation, and improving the efficiency of target detection.

[0034] In one embodiment of the present application, feature extraction is performed on the first target area in each video frame to obtain feature points in each video frame, including: The first target area in each video frame is subjected to feature extraction based on a scale-invariant feature transformation algorithm to obtain feature points in each video frame.

[0035] In this embodiment, the scale-invariant feature transform (SIFT) algorithm is an algorithm used to detect and describe local features of an image. Based on the scale-invariant feature transform algorithm, rich feature points can be extracted from each video frame. These feature points have good stability and uniqueness and can maintain good recognizability under different lighting conditions and viewing angle changes.

[0036] In one embodiment of the present application, determining motion information of each video frame based on a position change of the same feature point in each of two adjacent video frames includes: Determining the motion information of each video frame based on the position change of the same feature point in each two adjacent video frames includes: Matching the feature points in two adjacent video frames with the reference feature points in the feature point template respectively to obtain first target feature points and second target feature points that match the reference feature points; the number of reference feature points is less than the number of feature points in each video frame; Determining the first target feature point and the second target feature point that match the same reference feature point as a feature point pair corresponding to the same feature point; The motion information of each video frame is determined based on the position changes of feature point pairs corresponding to the same feature point.

[0037] In this embodiment, inspection images of various power equipment can be collected in advance, and the key points of the power equipment in the inspection images can be used as reference feature points to obtain corresponding feature point templates. For example, the top, bottom, and connection points of the tower have obvious structural features. These points are relatively stable under different viewing angles and lighting conditions and can be used as reference feature points. For example, the four corners of the transformer, the edges of the heat sink, the terminal blocks and other parts are structural key points. These points can reflect the overall shape and position information of the transformer and can be matched as feature points in the video frame.

[0038] In existing technology, matching feature points in two adjacent video frames typically involves calculating the descriptor distance (such as Euclidean distance or Hamming distance) between the feature points to determine their correspondence and find the common feature points in the two frames. Specifically, all feature points in the current video frame are traversed to find the feature point with the smallest descriptor distance to a feature point in the previous video frame. These feature points are used as the initial matching point pair. The initial matching result may contain some incorrect matching points, requiring multiple iterations to remove these incorrect matching points and ultimately obtain a set of relatively accurate feature point pairs. This process is computationally intensive.

[0039] To solve the above problems, this embodiment matches the feature points in two adjacent video frames with the reference feature points in the feature point template respectively to obtain the first target feature point and the second target feature point, and then determines the first target feature point and the second target feature point that match the same reference feature point as the feature point pair corresponding to the same feature point. The above method can avoid the erroneous matching that may occur during direct matching, and improves the accuracy of matching the same feature points; at the same time, since the number of reference feature points is less than the number of feature points in each video frame, the method of this embodiment can reduce the amount of calculation and improve the operation efficiency compared with the existing method of directly matching between all feature points of two video frames.

[0040] In one embodiment of the present application, determining motion information of each video frame based on position changes of feature point pairs corresponding to the same feature point includes: For any video frame, Calculating a first motion vector corresponding to each identical feature point based on a position change of a feature point pair corresponding to each identical feature point in any video frame; The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of the any video frame.

[0041] In this embodiment, the first motion vector can be obtained by calculating the position change of the feature point pair corresponding to the same feature point in adjacent video frames. The first motion vector can quantify the movement of each identical feature point from one frame to the next (including the direction and distance of movement). For example, if a feature point moves horizontally from coordinates (x1, y1) to (x2, y2), then its horizontal motion vector component is x2-x1, and the same applies to the vertical direction. The set of first motion vectors corresponding to the identical feature points in any video frame constitutes the motion information of the corresponding video frame.

[0042] In one embodiment of the present application, calculating the first motion vector corresponding to each identical feature point based on the position change of the feature point pair corresponding to each identical feature point in any video frame includes: Convert any video frame and adjacent video frames to HSV color space; Determine a second motion vector of each identical feature point in two adjacent video frames based on hue information and saturation information in an HSV color space; Determine a third motion vector of each identical feature point in two adjacent video frames based on brightness information in an HSV color space; The second motion vector and the third motion vector are weightedly summed to obtain the first motion vector corresponding to each identical feature point.

[0043] In this embodiment, the HSV color space is composed of hue (H), saturation (S), and value (V). Hue and saturation are relatively insensitive to changes in lighting and can reflect the essential color characteristics of an object; value, on the other hand, is directly related to light intensity. In power inspection scenarios, even if lighting conditions change, such as when sunlight is blocked by clouds, the color characteristics (hue and saturation) of the device can remain relatively stable. Therefore, by determining the second motion vector based on hue and saturation information, this embodiment can focus on the changes in the color characteristics of the object itself, avoiding interference from lighting changes on motion detection.

[0044] Furthermore, considering that object motion can cause changes in the distribution of brightness within the image, determining the third motion vector based on luminance information can capture brightness changes caused by both illumination changes and object motion. Therefore, this embodiment performs a weighted summation of the second and third motion vectors, comprehensively considering the object's color characteristics and motion information caused by brightness changes. This allows the resulting first motion vector to more comprehensively and accurately reflect the actual motion of the pixel.

[0045] From the above, it can be concluded that this embodiment takes into account that different channels of the HSV color space have different stabilities and distinctions under different lighting conditions and scenes. By integrating the information of the three channels, it can better adapt to various complex lighting conditions and scene changes, thereby improving the robustness and versatility of the algorithm, so that it can more accurately calculate the first motion vector of the feature point in different environments.

[0046] In one embodiment of the present application, the inspection result data processing method for drone inspection further includes: If the image gradient value of each video frame is greater than a first threshold, setting the weight of the second motion vector to the first value; If the image gradient value of each video frame is less than or equal to the first threshold, the weight of the second motion vector is set to a second value; and the first value is greater than the second value.

[0047] In this embodiment, the image gradient value reflects the rate of change of pixel brightness within a video frame. In areas of stable lighting, the brightness differences between pixels do not change suddenly, and the image gradient value is relatively small and changes smoothly. In areas with large lighting variations, the image gradient value increases significantly and may fluctuate significantly. Therefore, a first threshold can be pre-set. When the image gradient value of a video frame is greater than the first threshold, it indicates unstable lighting. Conversely, when the image gradient value of a video frame is less than or equal to the first threshold, it indicates stable lighting.

[0048] In areas with large changes in lighting, color component information can provide more stable features. At this time, setting the weight of the second motion vector to a larger first value helps to accurately track the correspondence between feature points in different frames, thereby more accurately calculating the first motion vector. In areas with stable lighting, the brightness information of the target in the video frame is relatively stable and can truly reflect the surface characteristics and structure of the target. At this time, setting the weight of the second motion vector to a smaller second value can more directly track the motion trajectory of the object, thereby more accurately calculating the first motion vector.

[0049] From the above, it can be concluded that this embodiment dynamically adjusts the weight of the second motion vector based on the image gradient value of the video frame, which can reasonably allocate information usage according to the actual situation of the image and reduce the impact of noise and interference on motion estimation.

[0050] Corresponding to the inspection result data processing method for drone inspection in the above embodiment, Figure 2 This is a structural block diagram of an inspection result data processing system for drone inspections provided in one embodiment of the present application. For ease of illustration, only the parts related to the embodiment of the present application are shown. Figure 2The inspection result data processing system 20 for drone inspection includes: a video frame extraction module 21, a target detection module 22, a feature extraction module 23, a first processing module 24 and a second processing module 25. The video frame extraction module 21 is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames; The target detection module 22 is used to perform target detection on each video frame to obtain a first target area in each video frame; A feature extraction module 23 is used to extract features of the first target area in each video frame to obtain feature points in each video frame; A first processing module 24 is configured to determine motion information of each video frame based on position changes of the same feature points in every two adjacent video frames; The second processing module 25 is configured to perform affine transformation on each video frame based on motion information of each video frame, so as to align the video frames and achieve stable processing of the video data.

[0051] In one embodiment of the present application, the target detection module 22 is specifically configured to: Performing edge detection on each video frame to obtain a second target area in each video frame; Classifying each video frame based on contour information of the second target area in each video frame to obtain classification information of each video frame; If the classification information of the video frame shows that the video frame is a line video frame, detecting the line area in the video frame based on the Hough transform method; If the classification information of the video frame shows that the video frame is a power equipment video frame, detecting the power equipment area in the video frame based on the target detection algorithm; A line area or a power equipment area is determined as a first target area.

[0052] In one embodiment of the present application, the feature extraction module 23 is specifically configured to: perform feature extraction on the first target area in each video frame based on a scale-invariant feature transformation algorithm to obtain feature points in each video frame.

[0053] In one embodiment of the present application, the first processing module 24 is specifically configured to: Matching the feature points in two adjacent video frames with the reference feature points in the feature point template respectively to obtain first target feature points and second target feature points that match the reference feature points; the number of reference feature points is less than the number of feature points in each video frame; Determining the first target feature point and the second target feature point that match the same reference feature point as a feature point pair corresponding to the same feature point; The motion information of each video frame is determined based on the position changes of feature point pairs corresponding to the same feature point.

[0054] In one embodiment of the present application, the first processing module 24 is further configured to: For any video frame, Calculating a first motion vector corresponding to each identical feature point based on a position change of a feature point pair corresponding to each identical feature point in any video frame; The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of the any video frame.

[0055] In one embodiment of the present application, the first processing module 24 is further configured to: Convert any video frame and adjacent video frames to HSV color space; Determine a second motion vector of each identical feature point in two adjacent video frames based on hue information and saturation information in an HSV color space; Determine a third motion vector of each identical feature point in two adjacent video frames based on brightness information in an HSV color space; The second motion vector and the third motion vector are weightedly summed to obtain the first motion vector corresponding to each identical feature point.

[0056] In one embodiment of the present application, the first processing module 24 is further configured to: If the image gradient value of each video frame is greater than a first threshold, setting the weight of the second motion vector to the first value; If the image gradient value of each video frame is less than or equal to the first threshold, the weight of the second motion vector is set to a second value; and the first value is greater than the second value.

[0057] See also Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided in one embodiment of the present application. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules / units in the above-mentioned device embodiments, such as Figure 2 The functions of the video frame extraction module 21, the target detection module 22, the feature extraction module 23, the first processing module 24 and the second processing module 25 are shown.

[0058] It should be understood that in the embodiment of the present application, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0059] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0060] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store device type information.

[0061] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present application can execute the implementation methods described in the first and second embodiments of the inspection result data processing method for drone inspection provided in the embodiments of the present application, and can also execute the implementation methods of the electronic device described in the embodiments of the present application, which will not be repeated here.

[0062] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0063] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0064] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0065] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0066] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0067] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0068] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0069] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for processing inspection result data for drone inspection, characterized in that: include: Performing frame extraction operation on the video data in the inspection result to obtain multiple video frames; Performing target detection on each video frame respectively to obtain a first target area in each video frame; Extracting features of the first target area in each video frame to obtain feature points in each video frame; Determine the motion information of each video frame based on the position change of the same feature point in every two adjacent video frames; Affine transformation is performed on each video frame based on motion information of each video frame to align the video frames and achieve stable processing of video data.

2. The inspection result data processing method for drone inspection according to claim 1, characterized in that: The performing target detection on each video frame to obtain a first target area in each video frame includes: Performing edge detection on each video frame to obtain a second target area in each video frame; Classifying each video frame based on contour information of the second target area in each video frame to obtain classification information of each video frame; If the classification information of the video frame shows that the video frame is a line video frame, detecting a line area in the video frame based on a Hough transform method; If the classification information of the video frame indicates that the video frame is a power equipment video frame, detecting a power equipment area in the video frame based on a target detection algorithm; The line area or the power equipment area is determined as the first target area.

3. The inspection result data processing method for drone inspection according to claim 1, characterized in that: The extracting features of the first target area in each video frame to obtain feature points in each video frame includes: The first target area in each video frame is subjected to feature extraction based on a scale-invariant feature transformation algorithm to obtain feature points in each video frame.

4. The inspection result data processing method for drone inspection according to claim 1, characterized in that: The determining of motion information of each video frame based on the position change of the same feature point in every two adjacent video frames includes: Matching feature points in two adjacent video frames with reference feature points in a feature point template respectively to obtain first target feature points and second target feature points that match the reference feature points; the number of the reference feature points is less than the number of feature points in each video frame; Determining the first target feature point and the second target feature point that match the same reference feature point as a feature point pair corresponding to the same feature point; The motion information of each video frame is determined based on the position changes of feature point pairs corresponding to the same feature point.

5. The inspection result data processing method for drone inspection according to claim 4, characterized in that: The determining of motion information of each video frame based on position changes of feature point pairs corresponding to the same feature point includes: For any video frame, Calculating a first motion vector corresponding to each identical feature point based on a position change of a feature point pair corresponding to each identical feature point in any video frame; The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of the any video frame.

6. The inspection result data processing method for drone inspection according to claim 5, characterized in that: The calculating the first motion vector corresponding to each identical feature point based on the position change of the feature point pair corresponding to each identical feature point in any video frame includes: Convert any video frame and adjacent video frames to HSV color space; Determine a second motion vector of each identical feature point in two adjacent video frames based on hue information and saturation information in an HSV color space; Determine a third motion vector of each identical feature point in two adjacent video frames based on brightness information in an HSV color space; A weighted sum is performed on the second motion vector and the third motion vector to obtain a first motion vector corresponding to each identical feature point.

7. The inspection result data processing method for drone inspection according to claim 6, characterized in that: Also includes: If the image gradient value of each video frame is greater than a first threshold, setting the weight of the second motion vector to a first value; If the image gradient value of each video frame is less than or equal to a first threshold, the weight of the second motion vector is set to a second value; and the first value is greater than the second value.

8. A data processing system for inspection results of drone inspection, characterized in that: include: The video frame extraction module is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames; The target detection module is used to perform target detection on each video frame to obtain a first target area in each video frame; A feature extraction module is used to extract features of the first target area in each video frame to obtain feature points in each video frame; A first processing module determines motion information of each video frame based on position changes of the same feature points in every two adjacent video frames; The second processing module performs affine transformation on each video frame based on motion information of each video frame to align the video frames and achieve stable processing of video data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images