Power transmission line hardware defect identification method and system based on cooperative work of unmanned aerial vehicle and robot

Abstract

The invention discloses a power transmission line hardware defect recognition method and system based on cooperative work of an unmanned aerial vehicle and a robot, and belongs to the technical field of image recognition, and the method comprises the steps that the robot receives first image data and flight path data of the unmanned aerial vehicle, and carries out timestamp synchronization; analyzing the flight path data, and outputting a first search signal; obtaining image data under the corresponding timestamp according to the first search signal, and optimizing the obtained image data; and in combination with a deep learning algorithm, track deviation is predicted and early warned. In the implementation process of the technical scheme of the invention, the to-be-analyzed image is analyzed and screened out from the transmission line of the unmanned aerial vehicle, and the accuracy of the flight path of the unmanned aerial vehicle and the quality of image data are improved according to the image analysis result in combination with a deep learning algorithm, so that accurate identification and early warning of the hardware defect of the power transmission line are realized, and the accuracy of the hardware defect is improved. And a powerful guarantee is provided for safe operation of the power transmission line.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and specifically to a method and system for identifying defects in power transmission line fittings based on the collaborative work of drones and robots. Background Art

[0002] With the rapid development of the power industry, the total length of my country's transmission lines has reached a new high. The stable development of the power industry has ensured the reliable operation of the power grid.

[0003] The transmission of electric power relies on a large number of transmission lines. In these transmission lines, hardware is a key component to ensure the safety of power transmission. Hardware is an iron or aluminum metal accessory widely used in transmission lines, which usually needs to withstand a large tensile force. There are many types of hardware, such as various wire clamps for installing conductors, various hanging rings for insulator strings, various crimping tubes and repair tubes for connecting conductors, various types of spacers on split conductors, etc. Defect detection of hardware is crucial to ensure the stability and safety of transmission lines.

[0004] In the existing technology, drones are usually used for preliminary image acquisition, and robots are used to accurately review suspected defective areas. This method combines the advantages of the efficient coverage capability of drones and the precise detection of robots, greatly improving the accuracy and efficiency of recognition. However, in actual application, due to factors such as weather and drone flight routes, the images collected by drones will have defects and deviations, which will in turn affect the subsequent analysis process, making it impossible to accurately determine the defects of hardware, increasing the risk of transmission lines.

[0005] Therefore, it is necessary to provide a method and system for identifying defects in power transmission line fittings based on the collaborative work of drones and robots to solve the above problems.

[0006] It should be noted that the above information disclosed in this background technology section is only for understanding the background technology of the present application concept, and therefore, it may contain information that does not constitute the prior art. Summary of the invention

[0007] Based on the above-mentioned problems existing in the prior art, the problem to be solved by this application is: to provide a method and system for identifying defects in power transmission line fittings based on the collaborative work of drones and robots, by analyzing the transmission lines from drones, screening out images to be analyzed, saving processing resources and improving the efficiency of hardware analysis.

[0008] The technical solution adopted by the present application to solve the technical problem is: a method for identifying defects in power transmission line hardware based on the collaborative work of drones and robots, which is characterized by:

[0009] The robot receives the first image data collected by the drone and the flight trajectory data of the drone, and performs time stamp synchronization on the first image data and the flight trajectory data;

[0010] Analyze the flight trajectory data to determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, where the first search signal is used to search for image data at a corresponding timestamp;

[0011] According to the first search signal, image data corresponding to the timestamp is acquired, and the acquired image data is optimized;

[0012] Combined with deep learning algorithms, the image data collected by the drone is analyzed, and combined with real-time flight data, trajectory deviations are predicted and warned.

[0013] During the implementation of the technical solution of the present application, the transmission lines of the UAV are analyzed, the images to be analyzed are screened out, and the image analysis results are combined with a deep learning algorithm to improve the accuracy of the UAV's flight trajectory and the quality of the image data, thereby achieving accurate identification and early warning of transmission line hardware defects, providing strong protection for the safe operation of the transmission lines.

[0014] Furthermore, there is a data exchange channel between the robot and the drone, and the channel is suitable for realizing the data exchange function, and the channel is a wireless transmission channel.

[0015] Furthermore, analyzing the flight trajectory data and outputting the first search signal further includes: obtaining the flight parameters of the UAV, which include the positioning data and inertial measurement data of the UAV; drawing the flight trajectory of the UAV's flight parameters through a visualization tool, and extracting the key points in the flight trajectory through a key point extraction algorithm; performing trajectory deviation analysis on the extracted key points, establishing optimization standards in combination with preset route requirements, and outputting the first search signal.

[0016] Furthermore, the visualization tool is a geographic information system, and the key point extraction algorithm is an RDP algorithm.

[0017] Furthermore, the trajectory deviation analysis of the extracted key points is implemented using the threshold method. A deviation threshold is set, and it is determined whether the difference between the flight parameters of the key points and the theoretical parameters under the preset trajectory exceeds the threshold. If it exceeds, the first search signal under the timestamp is output. If it does not exceed, the analysis continues.

[0018] Furthermore, the optimization of the image data further includes the following steps: pre-processing the image data to eliminate noise and abnormal points in the image; improving the clarity of details through image sharpening to make the edges of the image data more obvious, thereby facilitating identification and analysis; judging whether the acquired image content is abnormal, if so, correcting it through an image restoration algorithm, and proceeding to the next step of processing if no abnormality occurs; after completing the image correction, performing a secondary analysis on the image to confirm the specific cause of the trajectory deviation and record it.

[0019] Furthermore, the repair algorithm is a pixel filling algorithm, which fills in the missing or damaged parts of the image to ensure the overall integrity and authenticity of the image.

[0020] The power transmission line fittings defect identification system based on the collaborative work of drones and robots includes:

[0021] A data receiving and timestamp synchronization module is used for the robot to receive the first image data collected by the drone and the flight trajectory data of the drone, and to synchronize the timestamps of the first image data and the flight trajectory data;

[0022] A flight trajectory data analysis module is used to analyze the flight trajectory data, determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, wherein the first search signal is used to search for image data at a corresponding timestamp;

[0023] An image search and optimization module, used to obtain image data at a corresponding timestamp according to the first search signal, and optimize the obtained image data;

[0024] The trajectory deviation warning module is used to analyze the image data collected by the drone in combination with deep learning algorithms, and to predict and warn of trajectory deviations in combination with real-time flight data.

[0025] The beneficial effects of the present application are as follows: the method and system for identifying defects in hardware fittings of power transmission lines based on the collaborative work of drones and robots provided in the present application analyzes the transmission lines of the drones, screens out images to be analyzed, and combines the image analysis results with a deep learning algorithm to improve the accuracy of the drone's flight trajectory and the quality of image data, thereby achieving accurate identification and early warning of defects in hardware fittings of power transmission lines, and providing strong guarantees for the safe operation of transmission lines.

[0026] In addition to the above-described purposes, features and advantages, the present application also has other purposes, features and advantages. The present application will be further described in detail with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0028] In the attached picture:

[0029] Figure 1 This is a schematic diagram of the overall process of the power transmission line fittings defect identification method based on the collaborative work of drones and robots in this application;

[0030] Figure 2 This is a schematic diagram of the module composition of the power transmission line fittings defect identification system based on the collaborative work of drones and robots in this application. DETAILED DESCRIPTION

[0031] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0032] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0033] Embodiment 1: Figure 1 As shown, the present application provides a method for identifying defects in hardware of power transmission lines based on the collaborative work of drones and robots. The method is applied to the process of identifying defects in hardware of power transmission lines. The drone is used to collect images of the hardware according to a preset cruise route, and then the collected image data is sent to the robot. The robot analyzes and outputs the results, thereby completing the collaborative work of the drone and the robot to identify defects in the hardware of the power transmission lines. The defect identification method includes the following steps:

[0034] Step 10: The robot receives the first image data collected by the drone and the flight trajectory data of the drone, and performs time stamp synchronization on the first image data and the flight trajectory data;

[0035] In the process of collecting images of hardware of power transmission lines, drones are usually used for collection, because drones can preset fixed routes and correspond to power transmission lines, so as to achieve efficient and large-scale detection. At the same time, the flight trajectory of the drone will be recorded in chronological order, which is convenient for subsequent trajectory analysis, and it can be judged whether the drone has a route deviation during the collection process, and the first image data that needs to be optimized can be screened out according to the judgment results, without traversing all the data during the analysis process, thus improving the analysis efficiency;

[0036] In addition, in order to match the collected first image data with the flight trajectory data, it is also necessary to synchronize the timestamps of the two, and match the first image data with the flight trajectory data of the drone at a fixed time interval, so that rapid positioning can be achieved in the subsequent analysis process;

[0037] In this embodiment, the robot is used as a processor to implement the data analysis and processing function. In other embodiments, other hardware modules with data analysis and processing functions may also be used, such as computers, industrial computers, etc. In this embodiment, a robot is used as an example;

[0038] It should be noted that in order to realize the transmission of various types of data, there is a data exchange channel between the robot and the drone, which is suitable for realizing the data exchange function. In the present embodiment, the channel is a wireless transmission channel. The wireless transmission channel has convenient and efficient data transmission capabilities, ensuring the high efficiency and accuracy of image data and flight trajectory data in real-time processing.

[0039] Step 20: Analyze the flight trajectory data to determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, where the first search signal is used to search for image data at a corresponding timestamp;

[0040] In actual application, the actual route of the drone may not necessarily be strictly in accordance with the preset route, because the drone will be disturbed by many factors during operation, such as wind speed, electromagnetic interference, wind direction change, etc. Under the interference of these factors, the actual route of the drone will deviate from the preset route. Since the preset route is generally set by comprehensively considering the acquisition capability of the drone, after deviating from the route, the quality of the collected hardware image data will be unstable. Although all images can be identified and optimized, this will result in a high processing load on the robot and affect the image processing efficiency, resulting in the inability to identify the defects of the hardware in time, affecting the stable operation of the transmission line. Therefore, in this embodiment, the method of directly identifying and optimizing the collected image data in the prior art is not adopted, but the flight trajectory data of the drone is first analyzed, and a first search signal is output. According to the first search signal, it is determined which image data under which timestamp needs to be optimized. If necessary, the drone needs to repeatedly collect the corresponding defective area. Specifically, the flight trajectory data is analyzed and the first search signal is output, which further includes:

[0041] Step 201: Acquire the flight parameters of the UAV, which include positioning data and inertial measurement data of the UAV;

[0042] In order to analyze the flight trajectory of the drone, we first need to obtain the flight parameters of the drone, which include positioning data and inertial measurement data. The positioning data includes the starting position and the position at each time stamp, and the inertial measurement data includes the speed and direction of the drone during flight.

[0043] Step 202: Draw the flight trajectory of the UAV's flight parameters using a visualization tool, and extract key points in the flight trajectory using a key point extraction algorithm;

[0044] After obtaining the flight parameters of the drone, since these flight parameters are isolated data, they need to be drawn as a flight trajectory. Drawing the flight trajectory can be achieved by using a visualization tool, such as a geographic information system (GIS), and the key points in the flight trajectory are extracted by a key point extraction algorithm. The extraction of these key points is crucial for subsequent analysis. They will serve as the basis for the deviation of the flight trajectory from the preset route, and then guide the optimization processing of the image data. The key point extraction algorithm can use the RDP algorithm (Ramer-Douglas-Peucker). The RDP algorithm is an algorithm for simplifying curves or paths, which is widely used in fields such as computer graphics and GIS. Its main function is to reduce the number of data points while retaining the shape and characteristics of the curve as much as possible. The specific method of use can refer to the prior art and is not described in detail in this embodiment.

[0045] Step 203: performing trajectory deviation analysis on the extracted key points, establishing optimization criteria in combination with preset route requirements, and outputting a first search signal;

[0046] Based on the trajectory deviation analysis, the optimization criteria are established for the preset route requirements, and the first search signal is output to accurately identify the image data to be optimized. The above process combines the fit between the actual route and the flight trajectory, and conducts a detailed analysis of the key points to ensure that the drone has a trajectory deviation at which timestamps, and the corresponding timestamps are converted into the first search signal output;

[0047] The trajectory deviation analysis of the extracted key points can be implemented by using a threshold method, setting a deviation threshold, and judging whether the difference between the flight parameters of the key points and the theoretical parameters under the preset trajectory exceeds the threshold. If it exceeds, the first search signal under the timestamp is output, and if it does not exceed, the analysis continues;

[0048] Step 30: acquiring image data at a corresponding timestamp according to the first search signal, and optimizing the acquired image data;

[0049] After obtaining the first search signal, the image that may have acquisition problems can be located according to the flight process of the drone. Because the first search signal corresponds to a timestamp, the corresponding image data can be quickly and accurately found based on the timestamp, and then the acquired image data is optimized. Specifically, the optimization of the image data further includes the following steps:

[0050] Step 301: preprocess the image data to eliminate noise and abnormal points in the image;

[0051] In the case of abnormal drone trajectories, image data may generate noise and abnormal points. Therefore, it is necessary to pre-process the image data first to eliminate noise and abnormal points in the image, and improve the image quality by enhancing contrast and brightness to ensure more accurate subsequent processing.

[0052] Step 302: improving detail clarity by image sharpening, making the edges of image data more distinct, thereby facilitating identification and analysis;

[0053] After preprocessing the image, it is necessary to further enhance the recognizability of key information in the image through image sharpening, especially in the trajectory deviation area, it is necessary to make the key landmarks and terrain features in the drone aerial photography more prominent, so that the subsequent processing can accurately capture the specific situation of the deviation trajectory;

[0054] Step 303: determine whether the acquired image content is abnormal, if so, correct it using an image restoration algorithm, if not, proceed to the next step;

[0055] After the image is preprocessed, sharpened, and other steps are performed, it is also necessary to determine whether the image content is abnormal, conduct a detailed analysis of the abnormal situation, and adopt a corresponding repair algorithm to correct the image. Among them, the repair algorithm can select a pixel filling algorithm to fill in the missing or damaged parts of the image to ensure the overall integrity and authenticity of the image. The pixel filling algorithm is also called an area filling algorithm, which is a method of filling the pixels in a specified area with the same color. The specific method can refer to the prior art and is not described in detail in this embodiment;

[0056] Step 304: After the image correction is completed, the image is analyzed again to confirm the specific cause of the trajectory deviation and record it;

[0057] After image correction is completed, the image needs to be analyzed again to determine the specific reasons for the trajectory deviation of the drone, and to conduct detailed analysis in combination with the flight parameters of the drone, so as to conduct effective fault diagnosis and flight trajectory optimization, and to conduct in-depth analysis of the reasons for trajectory deviation, including but not limited to weather factors, drone performance, external electromagnetic interference, etc. While recording these data, the drone flight database needs to be updated synchronously so that it can respond quickly and take measures when encountering similar situations in the future;

[0058] In addition, deep learning algorithms can be used to further improve the drone's ability to adapt to complex environments, reduce trajectory deviations caused by environmental factors, and ensure the stability and efficiency of the drone system.

[0059] Step 40: Combined with deep learning algorithms, the image data collected by the drone is analyzed, and combined with real-time flight data, trajectory deviations are predicted and warned.

[0060] With the assistance of deep learning algorithms, drones can analyze image data in real time, combine real-time flight data, and accurately predict potential trajectory deviations, thereby issuing early warnings. On this basis, the drone flight database is continuously improved so that drones can autonomously adjust their flight strategies when facing complex environments, reducing interference with collected images.

[0061] Embodiment 2: Figure 2 As shown, the present application also proposes a power transmission line fitting defect recognition system based on the collaborative work of a drone and a robot. The defect recognition system runs the defect recognition method in the first embodiment, and uses the drone to collect images of the fittings according to a preset cruise route, and then sends the collected image data to the robot, which analyzes and outputs the results, thereby completing the collaborative work of the drone and the robot, and identifying defects in the power transmission line fittings. The defect recognition system includes:

[0062] A data receiving and timestamp synchronization module is used for the robot to receive the first image data collected by the drone and the flight trajectory data of the drone, and to synchronize the timestamps of the first image data and the flight trajectory data;

[0063] A flight trajectory data analysis module is used to analyze the flight trajectory data, determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, wherein the first search signal is used to search for image data at a corresponding timestamp;

[0064] An image search and optimization module, used to obtain image data at a corresponding timestamp according to the first search signal, and optimize the obtained image data;

[0065] The trajectory deviation warning module is used to analyze the image data collected by the drone in combination with deep learning algorithms, and to predict and warn of trajectory deviations in combination with real-time flight data.

[0066] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for identifying defects in power transmission line fittings based on the collaborative work of drones and robots, characterized by: include: The robot receives the first image data collected by the drone and the flight trajectory data of the drone, and performs time stamp synchronization on the first image data and the flight trajectory data; Analyze the flight trajectory data to determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, where the first search signal is used to search for image data at a corresponding timestamp; According to the first search signal, image data corresponding to the timestamp is acquired, and the acquired image data is optimized; Combined with deep learning algorithms, the image data collected by the drone is analyzed, and combined with real-time flight data, trajectory deviations are predicted and warned.

2. The method for identifying defects in power transmission line fittings based on the collaborative work of drones and robots according to claim 1 is characterized in that: There is a data exchange channel between the robot and the drone, and the channel is suitable for realizing the data exchange function, and the channel is a wireless transmission channel.

3. The method for identifying defects in power transmission line fittings based on the collaborative work of drones and robots according to claim 1 is characterized in that: Analyzing the flight trajectory data and outputting the first search signal further includes: obtaining the flight parameters of the UAV, which include the positioning data and inertial measurement data of the UAV; drawing the flight trajectory of the UAV's flight parameters through a visualization tool, and extracting the key points in the flight trajectory through a key point extraction algorithm; performing trajectory deviation analysis on the extracted key points, combining with the preset route requirements, establishing optimization standards, and outputting the first search signal.

4. The power transmission line fitting defect identification method based on the collaborative work of drones and robots according to claim 3 is characterized by: The visualization tool is a geographic information system, and the key point extraction algorithm is an RDP algorithm.

5. The method for identifying defects in power transmission line fittings based on the collaborative work of drones and robots according to claim 3 is characterized in that: The trajectory deviation analysis of the extracted key points is implemented using the threshold method. The deviation threshold is set, and it is determined whether the difference between the flight parameters of the key points and the theoretical parameters under the preset trajectory exceeds the threshold. If it exceeds, the first search signal under the timestamp is output. If it does not exceed, the analysis continues.

6. The power transmission line fitting defect identification method based on the collaborative work of drones and robots according to claim 1 is characterized in that: Optimizing the image data further includes the following steps: preprocessing the image data to eliminate noise and abnormal points in the image; improving detail clarity through image sharpening to make the edges of the image data more obvious, thereby facilitating identification and analysis; determining whether the acquired image content is abnormal, and if so, correcting it through an image restoration algorithm, and if not, proceeding to the next step of processing; after completing the image correction, performing a secondary analysis of the image to confirm the specific cause of the trajectory deviation and record it.

7. The method for identifying defects in power transmission line fittings based on the collaborative work of drones and robots according to claim 6 is characterized in that: The repair algorithm is a pixel filling algorithm that fills in the missing or damaged parts of the image to ensure the overall integrity and authenticity of the image.

8. A power transmission line fittings defect identification system based on the collaborative work of drones and robots, characterized by: include: A data receiving and timestamp synchronization module is used for the robot to receive the first image data collected by the drone and the flight trajectory data of the drone, and to synchronize the timestamps of the first image data and the flight trajectory data; A flight trajectory data analysis module is used to analyze the flight trajectory data, determine whether the actual route of the drone is consistent with the preset route, and output a first search signal according to the determination result, wherein the first search signal is used to search for image data at a corresponding timestamp; An image search and optimization module, used to obtain image data at a corresponding timestamp according to the first search signal, and optimize the obtained image data; The trajectory deviation warning module is used to analyze the image data collected by the drone in combination with deep learning algorithms, and to predict and warn of trajectory deviations in combination with real-time flight data.

9. The power transmission line fitting defect identification system based on the collaborative work of drones and robots according to claim 8 is characterized by: Used to implement the power transmission line fittings defect identification method based on the collaborative work of drones and robots as described in any one of claims 1 to 7.

Images