Inspection result data processing system and method for unmanned aerial vehicle inspection

Abstract

The application provides an inspection result data processing system and method for unmanned aerial vehicle inspection, and belongs to the technical field of data processing. The method comprises the following steps: performing a frame extraction operation on video data in an inspection result to obtain a plurality of video frames; performing target detection on each video frame to obtain a first target region in each video frame; performing feature extraction on the first target region in each video frame to obtain feature points in each video frame; determining motion information of each video frame based on position changes of the same feature points in each two adjacent video frames; and performing affine transformation on each video frame based on the motion information of each video frame to align each video frame and realize stable processing of the video data. The inspection result data processing system and method for unmanned aerial vehicle inspection can improve the accuracy of power inspection results.

Description

Technical Field

[0001] This application belongs to the field of unmanned aerial vehicle (UAV) technology, and more specifically, relates to a data processing system and method for UAV inspection results. Background Technology

[0002] With the continuous development of technology, drones are widely used in fields such as power line inspection, security monitoring, agriculture, and surveying. Taking power line inspection as an example, drones equipped with various sensors, such as visible light cameras, infrared thermal imagers, and lidar, can collect images, videos, point cloud data, and other information about power lines and equipment. By fusing and analyzing data from multiple sensors, the operating status of power equipment can be monitored in real time. However, during the video data collection process, changes in the drone's flight attitude can cause video jitter. Analyzing jittery videos makes it difficult to accurately identify and locate targets, thus affecting the accuracy of power line inspection results. Summary of the Invention

[0003] The purpose of this application is to provide a data processing system and method for power line inspection results using unmanned aerial vehicle (UAV) inspections, so as to improve the accuracy of power line inspection results.

[0004] A first aspect of this application provides a method for processing inspection result data for unmanned aerial vehicle (UAV) inspections, including:

[0005] Frame extraction is performed on the video data in the inspection results to obtain multiple video frames;

[0006] Target detection is performed on each video frame to obtain the first target region in each video frame;

[0007] Feature extraction is performed on the first target region in each video frame to obtain feature points in each video frame;

[0008] Motion information of each video frame is determined based on the positional changes of the same feature points in every two adjacent video frames.

[0009] Affine transformations are performed on each video frame based on its motion information to align the frames and achieve stable processing of the video data.

[0010] A second aspect of this application provides a data processing system for inspection results of unmanned aerial vehicle (UAV) inspections, comprising:

[0011] The video frame extraction module is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames.

[0012] The target detection module is used to perform target detection on each video frame to obtain the first target region in each video frame.

[0013] The feature extraction module is used to extract features from the first target region in each video frame to obtain feature points in each video frame.

[0014] The first processing module determines the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames.

[0015] The second processing module is used to perform affine transformations on each video frame based on the motion information of each video frame in order to align the video frames and achieve stable processing of video data.

[0016] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for processing inspection result data for unmanned aerial vehicle (UAV) inspection.

[0017] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for processing inspection result data for unmanned aerial vehicle (UAV) inspections.

[0018] The beneficial effects of the inspection result data processing system and method for UAV inspection provided in this application are as follows:

[0019] This application focuses on key areas (first target areas) for feature point extraction, which can significantly reduce the amount of computation and improve the processing speed. Based on the positional movement of feature points in adjacent frames, the relative motion relationship between video frames can be captured, and motion information such as translation, rotation and scaling of video frames can be calculated. Affine transformation of each video frame can be performed according to the motion information to achieve stable alignment of video frames, making the video picture more stable and facilitating more accurate analysis of the inspection target in the future. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a method for processing inspection result data for unmanned aerial vehicle (UAV) inspections according to an embodiment of this application;

[0022] Figure 2This is a structural block diagram of an inspection result data processing system for unmanned aerial vehicle (UAV) inspection, provided in an embodiment of this application.

[0023] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0026] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for processing inspection result data for unmanned aerial vehicle (UAV) inspections according to an embodiment of this application. The method includes:

[0027] S101: Perform frame extraction on the video data in the inspection results to obtain multiple video frames.

[0028] In this embodiment, the video data is essentially composed of a series of consecutive video frames (or image frames). By extracting multiple video frames from the video data, the video frame image at each moment can be processed and analyzed.

[0029] S102: Perform target detection on each video frame to obtain the first target region in each video frame.

[0030] In this embodiment, existing object detection algorithms based on deep learning, such as Faster Region-based Convolutional Neural Network (Faster R-CNN) and YOLO, can be used to perform object detection on each video frame.

[0031] By performing target detection on each video frame, targets such as power equipment or power lines can be identified. The area containing the power equipment or power lines is designated as the first target region. Focusing on the first target region for subsequent feature extraction reduces unnecessary background interference and improves the efficiency of subsequent feature extraction and analysis.

[0032] S103: Extract features from the first target region in each video frame to obtain feature points in each video frame.

[0033] In this embodiment, feature points are representative and stable points in each video frame, such as corners and edges. These points can be accurately detected in different video frames and have high repeatability. For example, for the same object under different shooting angles or lighting conditions, some points on its edges can still maintain obvious features and are easily identified by feature detection algorithms.

[0034] Specifically, existing feature extraction algorithms such as Scale-Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF) can be used to extract feature points from each video frame.

[0035] S104: Determine the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames.

[0036] In this embodiment, by tracking the positional movement of the same feature points in adjacent frames, the relative motion relationship between video frames can be captured, and motion information such as translation, rotation, and scaling of the video frames can be calculated. Each pair of adjacent video frames can be the current video frame and the next video frame, or the current video frame and the previous video frame. The motion information of the current video frame can be obtained by calculating the positional changes of the same feature points in each pair of adjacent video frames.

[0037] Specifically, existing optical flow methods (such as the Lucas-Kanade optical flow method) or feature matching-based methods can be used to determine the motion information of each video frame.

[0038] S105: Perform affine transformation on each video frame based on the motion information of each video frame to align the video frames and achieve stable processing of video data.

[0039] In this embodiment, affine transformation is performed on each video frame based on motion information, which can realize operations such as translation, rotation and scaling of video frames. The first target area in different video frames is adjusted to the same position and angle, realizing the alignment of video frames, thereby eliminating jitter, making the video picture more stable, and facilitating subsequent analysis and judgment of inspection results.

[0040] Specifically, based on the motion information of each video frame, parameters such as translation, rotation angle, and scaling factor can be calculated, and then an affine transformation matrix can be constructed. For example, the overall translation parameters can be determined by calculating the average translation of the first motion vectors of all feature points; the rotation angle can be obtained by calculating the average direction of all first motion vectors, comparing it with the horizontal direction, and then using the arctangent function. For the scaling factor, the length of each first motion vector is first calculated and its average value is taken, then compared with a preset reference length; the ratio of the two is the scaling factor.

[0041] Based on the affine transformation matrix, the coordinates of each pixel in the current video frame are transformed by matrix multiplication according to the mathematical formula of affine transformation, and the coordinates of the pixel at the new position can be obtained.

[0042] As can be seen from the above, this embodiment focuses on key areas (first target areas) for feature point extraction, which can significantly reduce the amount of computation and improve the processing speed. Based on the positional movement of feature points in adjacent frames, the relative motion relationship between video frames can be captured, and motion information such as translation, rotation and scaling of video frames can be calculated. By performing affine transformation on each video frame according to the motion information, stable alignment of video frames can be achieved, making the video picture more stable and facilitating more accurate subsequent analysis of inspection targets.

[0043] In one embodiment of this application, target detection is performed on each video frame to obtain a first target region in each video frame, including:

[0044] Edge detection is performed on each video frame to obtain the second target region in each video frame;

[0045] Each video frame is classified based on the contour information of the second target region in each video frame to obtain the classification information of each video frame.

[0046] If the classification information of the video frame indicates that the video frame is a line video frame, the line region in the video frame is detected based on the Hough transform method.

[0047] If the classification information of the video frame indicates that the video frame is a power equipment video frame, the power equipment area in the video frame is detected based on the target detection algorithm;

[0048] The line area or power equipment area is identified as the first target area.

[0049] In this embodiment, considering the obvious grayscale difference between the target (electrical equipment or power lines, etc.) and the background, existing edge detection algorithms such as the Sobel operator and the Canny operator can be used to roughly outline the contour of the target and obtain a preliminary second target region.

[0050] Different targets have different contour features. For example, power lines typically appear as long, thin lines, while power equipment usually has more regular or unique geometric shapes. By analyzing the contour information (such as shape, perimeter, and area) of the second target region, video frames can be classified as either power line video frames or power equipment video frames. For instance, the contour of power equipment usually has a larger area and a relatively regular shape, with higher roundness; while the contour of a power line has a smaller area, a longer perimeter, and lower roundness. Therefore, a second threshold can be set: when the roundness of the second target region is greater than the second threshold, it indicates that the video frame containing the second target region is a power equipment video frame; when the roundness of the second target region is less than or equal to the second threshold, it indicates that the video frame containing the second target region is a power line video frame.

[0051] For video frames of a line, the Hough transform can accurately locate the position and direction of the line. Its calculation speed is relatively fast, enabling rapid localization of the line area. The Hough transform is a classic algorithm for detecting straight lines in images; it effectively detects the parameters of a line by mapping points in the image to a parameter space.

[0052] For video frames containing power equipment, specialized target detection algorithms are needed to accurately detect the power equipment and thus determine the power equipment region. Both the power line region and the power equipment region can be used as the primary target region.

[0053] As can be seen from the above, this embodiment first uses edge detection to obtain the preliminary target region (second target region), then classifies the video frames according to the contour information of the second target region, and finally uses different detection methods for different video frames, thus avoiding the use of complex target detection algorithms for all video frames, reducing the amount of computation, and improving the efficiency of target detection.

[0054] In one embodiment of this application, feature extraction is performed on a first target region in each video frame to obtain feature points in each video frame, including:

[0055] The first target region in each video frame is extracted using the scale-invariant feature transform algorithm to obtain the feature points in each video frame.

[0056] In this embodiment, the Scale Invariant Feature Transform (SIFT) algorithm is an algorithm used to detect and describe local features of an image. Based on the Scale Invariant Feature Transform algorithm, rich feature points can be extracted in each video frame. These feature points have good stability and uniqueness and can maintain good recognizability under different lighting conditions and viewing angle changes.

[0057] In one embodiment of this application, determining the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames includes:

[0058] Motion information for each video frame is determined based on the positional changes of identical feature points in every two adjacent video frames, including:

[0059] The feature points in two adjacent video frames are matched with the reference feature points in the feature point template to obtain the first target feature point and the second target feature point that match each reference feature point; the number of reference feature points is less than the number of feature points in each video frame;

[0060] The first target feature point and the second target feature point that match the same reference feature point are determined as the feature point pair corresponding to the same feature point;

[0061] Motion information for each video frame is determined based on the positional changes of feature point pairs corresponding to the same feature points.

[0062] In this embodiment, inspection images of various power equipment can be collected in advance. Key points of the power equipment in the inspection images can be used as reference feature points to obtain corresponding feature point templates. For example, the top and bottom of the tower, the connection point between the horizontal and vertical poles, etc. have obvious structural features. These points are relatively stable under different viewing angles and lighting conditions and can be used as reference feature points. Similarly, the four corners of the transformer, the edge of the heat sink, the terminal block, etc. are key structural points. These points can reflect the overall shape and position information of the transformer and can be used as feature points for matching in the video frame.

[0063] In existing technologies, the method for matching feature points in two adjacent video frames typically involves determining their correspondence by calculating the descriptor distance (such as Euclidean distance, Hamming distance, etc.) between feature points, thus finding the same feature points in the two frames. Specifically, all feature points in the current video frame are traversed, and the feature point with the smallest descriptor distance to a certain feature point in the previous video frame is found. These are used as initial matching point pairs. The initial matching results may contain some incorrect matching points, requiring multiple iterations to remove these incorrect matching points, ultimately resulting in a more accurate set of feature point pairs. This process is computationally intensive.

[0064] To address the aforementioned issues, this embodiment matches feature points in two adjacent video frames with reference feature points in a feature point template to obtain a first target feature point and a second target feature point. The first and second target feature points that match the same reference feature point are then identified as feature point pairs corresponding to the same feature point. This method avoids potential erroneous matching that may occur during direct matching, thus improving the accuracy of matching the same feature points. Furthermore, since the number of reference feature points is less than the number of feature points in each video frame, the method in this embodiment reduces computational load and improves operational efficiency compared to existing methods that directly match all feature points in two video frames.

[0065] In one embodiment of this application, determining the motion information of each video frame based on the positional changes of feature point pairs corresponding to the same feature points includes:

[0066] For any video frame

[0067] Calculate the first motion vector corresponding to each identical feature point based on the positional change of each identical feature point in any video frame;

[0068] The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of that video frame.

[0069] In this embodiment, the first motion vector can be obtained by calculating the positional changes of feature point pairs corresponding to the same feature point in adjacent video frames. The first motion vector can quantify the motion of each identical feature point from one frame to the next (including the direction and distance of the motion). For example, if a feature point moves from coordinates (x1, y1) to (x2, y2) in the horizontal direction, then its horizontal motion vector component is x2-x1, and the same applies to the vertical direction. The set of first motion vectors corresponding to the identical feature points in any video frame constitutes the motion information of the corresponding video frame.

[0070] In one embodiment of this application, calculating the first motion vector corresponding to each identical feature point based on the positional change of feature point pairs corresponding to each identical feature point in any video frame includes:

[0071] Convert any given video frame and its adjacent video frames to the HSV color space;

[0072] The second motion vector of each identical feature point in two adjacent video frames is determined based on the hue and saturation information in the HSV color space.

[0073] The third motion vector of each identical feature point in two adjacent video frames is determined based on the luminance information in the HSV color space.

[0074] The second and third motion vectors are weighted and summed to obtain the first motion vector corresponding to each identical feature point.

[0075] In this embodiment, the HSV color space consists of hue (H), saturation (S), and lightness (V). Hue and saturation are relatively insensitive to changes in illumination and can reflect the essential color characteristics of an object; while lightness is directly related to light intensity. In power line inspection scenarios, even if lighting conditions change, such as sunlight being blocked by clouds, the color characteristics (hue and saturation) of the equipment can remain relatively stable. Therefore, this embodiment, by determining the second motion vector based on hue and saturation information, can focus on the changes in the object's own color characteristics and avoid interference from changes in illumination on motion detection.

[0076] Meanwhile, considering that the movement of an object will cause changes in its brightness distribution in the image, determining the third motion vector based on brightness information can capture the brightness changes caused by the combined effects of illumination changes and object movement. Therefore, this embodiment weights and sums the second and third motion vectors, which can comprehensively consider the motion information brought about by the object's color characteristics and brightness changes, so that the final first motion vector more comprehensively and accurately reflects the actual movement of the pixels.

[0077] As can be seen from the above, this embodiment takes into account that different channels of the HSV color space have different stability and discrimination under different lighting conditions and scenes. By integrating the information of the three channels, it can better adapt to various complex lighting conditions and scene changes, improve the robustness and versatility of the algorithm, and enable it to calculate the first motion vector of the feature point more accurately in different environments.

[0078] In one embodiment of this application, the method for processing inspection result data for UAV inspection further includes:

[0079] If the image gradient value of each video frame is greater than the first threshold, the weight of the second motion vector is set to the first value;

[0080] If the image gradient value of each video frame is less than or equal to the first threshold, the weight of the second motion vector is set to the second value; the first value is greater than the second value.

[0081] In this embodiment, the image gradient value reflects the rate of change of pixel brightness in a video frame. In areas with stable lighting, the brightness difference between pixels does not change suddenly, and the image gradient value is relatively small and changes smoothly. However, in areas with large lighting changes, the image gradient value will increase significantly and may fluctuate greatly. Therefore, a first threshold can be preset. When the image gradient value of a video frame is greater than the first threshold, it indicates that the lighting is unstable; conversely, when the image gradient value of a video frame is less than or equal to the first threshold, it indicates that the lighting is stable.

[0082] In areas with significant lighting variations, color component information provides more stable features. In this case, setting the weight of the second motion vector to a larger first value helps to accurately track the correspondence between feature points in different frames, thereby calculating the first motion vector more accurately. In areas with stable lighting, the brightness information of the target in the video frame is relatively stable, which can realistically reflect the surface characteristics and structure of the target. In this case, setting the weight of the second motion vector to a smaller second value can more directly track the motion trajectory of the object, thereby calculating the first motion vector more accurately.

[0083] As can be seen from the above, this embodiment dynamically adjusts the weight of the second motion vector based on the image gradient value of the video frame, which can reasonably allocate the use of information according to the actual situation of the image and reduce the impact of noise and interference on motion estimation.

[0084] Corresponding to the inspection result data processing method for UAV inspection in the above embodiment, Figure 2 This is a structural block diagram of an inspection result data processing system for unmanned aerial vehicle (UAV) inspections, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The inspection result data processing system 20 for UAV inspection includes: a video frame extraction module 21, a target detection module 22, a feature extraction module 23, a first processing module 24, and a second processing module 25.

[0085] Among them, the video frame extraction module 21 is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames;

[0086] The target detection module 22 is used to perform target detection on each video frame to obtain the first target region in each video frame;

[0087] Feature extraction module 23 is used to extract features from the first target region in each video frame to obtain feature points in each video frame;

[0088] The first processing module 24 is used to determine the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames.

[0089] The second processing module 25 is used to perform affine transformation on each video frame based on the motion information of each video frame in order to align the video frames and achieve stable processing of video data.

[0090] In one embodiment of this application, the target detection module 22 is specifically used for:

[0091] Edge detection is performed on each video frame to obtain the second target region in each video frame;

[0092] Each video frame is classified based on the contour information of the second target region in each video frame to obtain the classification information of each video frame.

[0093] If the classification information of the video frame indicates that the video frame is a line video frame, the line region in the video frame is detected based on the Hough transform method.

[0094] If the classification information of the video frame indicates that the video frame is a power equipment video frame, the power equipment area in the video frame is detected based on the target detection algorithm;

[0095] The line area or power equipment area is identified as the first target area.

[0096] In one embodiment of this application, the feature extraction module 23 is specifically used to: extract features from the first target region in each video frame based on the scale-invariant feature transformation algorithm to obtain feature points in each video frame.

[0097] In one embodiment of this application, the first processing module 24 is specifically used for:

[0098] The feature points in two adjacent video frames are matched with the reference feature points in the feature point template to obtain the first target feature point and the second target feature point that match each reference feature point; the number of reference feature points is less than the number of feature points in each video frame;

[0099] The first target feature point and the second target feature point that match the same reference feature point are determined as the feature point pair corresponding to the same feature point;

[0100] Motion information for each video frame is determined based on the positional changes of feature point pairs corresponding to the same feature points.

[0101] In one embodiment of this application, the first processing module 24 is further configured to:

[0102] For any video frame

[0103] Calculate the first motion vector corresponding to each identical feature point based on the positional change of each identical feature point in any video frame;

[0104] The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of that video frame.

[0105] In one embodiment of this application, the first processing module 24 is further configured to:

[0106] Convert any given video frame and its adjacent video frames to the HSV color space;

[0107] The second motion vector of each identical feature point in two adjacent video frames is determined based on the hue and saturation information in the HSV color space.

[0108] The third motion vector of each identical feature point in two adjacent video frames is determined based on the luminance information in the HSV color space.

[0109] The second and third motion vectors are weighted and summed to obtain the first motion vector corresponding to each identical feature point.

[0110] In one embodiment of this application, the first processing module 24 is further configured to:

[0111] If the image gradient value of each video frame is greater than the first threshold, the weight of the second motion vector is set to the first value;

[0112] If the image gradient value of each video frame is less than or equal to the first threshold, the weight of the second motion vector is set to the second value; the first value is greater than the second value.

[0113] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the video frame extraction module 21, target detection module 22, feature extraction module 23, first processing module 24, and second processing module 25 are shown.

[0114] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0115] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0116] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0117] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the first and second embodiments of the inspection result data processing method for UAV inspection provided in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0118] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0119] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing inspection result data for unmanned aerial vehicle (UAV) inspections, characterized in that, include: Frame extraction is performed on the video data in the inspection results to obtain multiple video frames; Target detection is performed on each video frame to obtain the first target region in each video frame; Feature extraction is performed on the first target region in each video frame to obtain feature points in each video frame; Motion information of each video frame is determined based on the positional changes of the same feature points in every two adjacent video frames. Affine transformation is performed on each video frame based on the motion information of each video frame to align the video frames and achieve stable processing of video data. The determination of motion information for each video frame based on the positional changes of identical feature points in every two adjacent video frames includes: The feature points in two adjacent video frames are matched with reference feature points in the feature point template to obtain the first target feature point and the second target feature point that match each reference feature point; the number of reference feature points is less than the number of feature points in each video frame. The first target feature point and the second target feature point that match the same reference feature point are determined as the feature point pair corresponding to the same feature point; Motion information of each video frame is determined based on the positional changes of feature point pairs corresponding to the same feature points. The method of determining the motion information of each video frame based on the positional changes of feature point pairs corresponding to the same feature points includes: For any video frame Calculate the first motion vector corresponding to each identical feature point based on the positional change of each identical feature point in any video frame; The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of that video frame. The calculation of the first motion vector corresponding to each identical feature point based on the positional changes of feature point pairs corresponding to each identical feature point in any video frame includes: Convert any given video frame and its adjacent video frames to the HSV color space; The second motion vector of each identical feature point in two adjacent video frames is determined based on the hue and saturation information in the HSV color space. The third motion vector of each identical feature point in two adjacent video frames is determined based on the luminance information in the HSV color space. The second motion vector and the third motion vector are weighted and summed to obtain the first motion vector corresponding to each identical feature point.

2. The method for processing inspection result data for UAV inspection as described in claim 1, characterized in that, The step of performing target detection on each video frame to obtain the first target region in each video frame includes: Edge detection is performed on each video frame to obtain the second target region in each video frame; Each video frame is classified based on the contour information of the second target region in each video frame to obtain the classification information of each video frame. If the classification information of the video frame indicates that the video frame is a line video frame, the line region in the video frame is detected based on the Hough transform method; If the classification information of the video frame indicates that the video frame is a power equipment video frame, the power equipment region in the video frame is detected based on the target detection algorithm; The line area or the power equipment area is identified as the first target area.

3. The method for processing inspection result data for UAV inspection as described in claim 1, characterized in that, The step of extracting features from the first target region in each video frame to obtain feature points in each video frame includes: The first target region in each video frame is extracted using the scale-invariant feature transform algorithm to obtain the feature points in each video frame.

4. The method for processing inspection result data for UAV inspection as described in claim 1, characterized in that, Also includes: If the image gradient value of each video frame is greater than the first threshold, the weight of the second motion vector is set to the first value; If the image gradient value of each video frame is less than or equal to the first threshold, the weight of the second motion vector is set to the second value; the first value is greater than the second value.

5. A data processing system for inspection results of unmanned aerial vehicle (UAV) inspections, characterized in that, include: The video frame extraction module is used to perform frame extraction operations on the video data in the inspection results to obtain multiple video frames. The target detection module is used to perform target detection on each video frame to obtain the first target region in each video frame. The feature extraction module is used to extract features from the first target region in each video frame to obtain feature points in each video frame. The first processing module determines the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames. The second processing module performs affine transformations on each video frame based on the motion information of each video frame in order to align the video frames and achieve stable processing of video data. Specifically, when determining the motion information of each video frame based on the positional changes of the same feature points in every two adjacent video frames, the first processing module is used to: The feature points in two adjacent video frames are matched with reference feature points in the feature point template to obtain the first target feature point and the second target feature point that match each reference feature point; the number of reference feature points is less than the number of feature points in each video frame. The first target feature point and the second target feature point that match the same reference feature point are determined as the feature point pair corresponding to the same feature point; Motion information of each video frame is determined based on the positional changes of feature point pairs corresponding to the same feature points. The method of determining the motion information of each video frame based on the positional changes of feature point pairs corresponding to the same feature points includes: For any video frame Calculate the first motion vector corresponding to each identical feature point based on the positional change of each identical feature point in any video frame; The first motion vector corresponding to each identical feature point in any video frame is determined as the motion information of that video frame. The calculation of the first motion vector corresponding to each identical feature point based on the positional changes of feature point pairs corresponding to each identical feature point in any video frame includes: Convert any given video frame and its adjacent video frames to the HSV color space; The second motion vector of each identical feature point in two adjacent video frames is determined based on the hue and saturation information in the HSV color space. The third motion vector of each identical feature point in two adjacent video frames is determined based on the luminance information in the HSV color space. The second motion vector and the third motion vector are weighted and summed to obtain the first motion vector corresponding to each identical feature point.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 4.

Images