Power transmission line field data acquisition system based on unmanned aerial vehicle vision

Abstract

The invention discloses a power transmission line field data acquisition system based on unmanned aerial vehicle vision. According to the invention, the CNN is used to extract the features of the visual image and the infrared image, and the 3D-CNN is used to process the laser radar data, so that the system can capture the multi-aspect information of the power transmission line. Information about the state of the power transmission line can be provided from different angles through data of different modes, fusion of the multi-mode data is beneficial for improving the accuracy and robustness of fault detection, the LSTM network is used for processing time series data, the system can better understand the change of the data along with time, the LSTM can process the series data, and the fault detection accuracy and robustness of the power transmission line can be improved. And the change trend of the fault along with time is captured, so that the fault is predicted and diagnosed more accurately. The weights of different modal data are determined through the single-layer sensor, and the system can automatically learn the importance of each modal data to final fault detection so as to reduce prediction errors, thereby improving the accuracy and reliability of fault detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle data collection, and in particular relates to a transmission line field data collection system based on unmanned aerial vehicle vision. Background Art

[0002] Transmission line inspection refers to regular or irregular inspections of transmission lines to ensure the safe and stable operation of transmission lines. Transmission lines are the key link in the power system that connects power plants and power loads, and their operating status directly affects the reliability and safety of power supply. With the continuous growth of electricity demand in modern society, the safety and reliability of transmission lines have become key factors in the development of the power industry. With the rapid development of the power industry, the safety and reliability of transmission lines have become increasingly important. The traditional transmission line inspection method relies on manual inspections, which is not only inefficient but also poses safety risks. In order to improve the operation and maintenance efficiency and safety of transmission lines, in recent years, transmission line field data acquisition systems based on drone vision have been widely used.

[0003] However, the existing transmission line field data acquisition system based on drone vision mainly adopts single-mode data acquisition, such as using only high-definition cameras or infrared thermal imagers, which is difficult to fully reflect the status of the transmission line. Summary of the invention

[0004] The purpose of the present invention is to provide a transmission line field data acquisition system based on UAV vision in order to solve the above-mentioned problems.

[0005] The technical solution adopted by the present invention is as follows: a transmission line field data acquisition system based on UAV vision, the system comprising: a UAV platform module, a visual acquisition module, a data processing and analysis module, a ground control and display module and a UAV auxiliary equipment module;

[0006] The data processing and analysis module is internally provided with an image preprocessing module, a target detection and recognition module, a multimodal deep learning fusion module and a report generation module;

[0007] The drone platform module is connected to the visual acquisition module, carrying and controlling cameras and other sensors for data acquisition;

[0008] The UAV platform module is connected to the ground control and display module to receive instructions from the ground and transmit flight status and collected data;

[0009] The visual acquisition module is integrated on the UAV platform module, and the UAV provides power and support; the collected data is transmitted to the data processing and analysis module for further processing;

[0010] The data processing and analysis module receives the raw data from the visual acquisition module; the analyzed and processed data is transmitted to the ground control and display module for display and storage;

[0011] The ground control and display module is connected to the UAV platform module to send control instructions and receive UAV status information; it is connected to the data processing and analysis module to receive processed data, display and store it, and provide an interface for analysis results;

[0012] The UAV auxiliary equipment module is directly connected to the UAV platform module to provide it with necessary maintenance and support to ensure that the UAV can operate continuously and stably.

[0013] In a preferred embodiment, the UAV platform module is responsible for carrying and controlling various sensors to perform inspection tasks; specifically including:

[0014] The drone itself;

[0015] Flight control system: realizes the flight control of the UAV, including take-off, landing, cruising, and hovering functions, ensuring that the UAV can perform inspection tasks stably and accurately;

[0016] Communication system: ensure stable and reliable data transmission between the UAV and the ground station. Wireless data transmission or satellite communication can be used to ensure real-time transmission of inspection data.

[0017] Navigation system: provides navigation and positioning functions for drones during inspections, ensuring that drones can conduct inspections according to the scheduled routes;

[0018] Mission planning system: formulate UAV flight routes and inspection tasks according to inspection requirements to ensure that UAVs can complete inspection tasks efficiently and accurately.

[0019] In a preferred embodiment, the visual acquisition is responsible for acquiring image and video data of the power transmission line; specifically, it includes:

[0020] HD camera: Equipped with a HD camera, responsible for collecting image and video data of the transmission line;

[0021] Infrared thermal imager: Equipped with an infrared thermal imager to detect temperature anomalies of power transmission lines and auxiliary equipment and find potential fault points;

[0022] LiDAR: Equipped with LiDAR, it can obtain three-dimensional spatial information of the transmission line and generate a topographic model of the line corridor;

[0023] Data transmission: The collected images, videos and three-dimensional spatial information are transmitted to the ground control and display module in real time.

[0024] In a preferred embodiment, the data preprocessing submodule scales the image pixel values ​​to between 0 and 1;

[0025] The data preprocessing submodule converts the pixel values ​​in the infrared image into actual temperature values; this step makes the infrared image data more accurate, which is helpful for fault detection and analysis;

[0026] The data preprocessing submodule scales the temperature value to between 0 and 1.

[0027] In a preferred embodiment, the multimodal deep learning fusion module uses CNN to extract features of visual images and infrared images respectively, then uses LSTM network to process time series data, and finally fuses the features of different modalities through a fusion layer for final fault detection;

[0028] The image features and infrared image features are extracted using multiple convolutional layers and pooling layers, and the extraction formula is:

[0029] F modal =CNN(I modal_norm );

[0030] Among them, F modal is the eigenvector, I modal_norm is the preprocessed image;

[0031] The lidar data uses a 3D convolution layer to process the point cloud data. The formula is:

[0032] F LIDAR =3D-CNN(P LIDAR_norm ) where FLIDAR is the feature vector extracted from the point cloud data, P LIDAR_norm It is the preprocessed point cloud data;

[0033] In the weighted sum fusion process of the multimodal deep learning fusion module, the weights are first determined by a single-layer perceptron, specifically including:

[0034] Forward propagation: Calculate the weighted input and output of the network:

[0035] net=wvisual·Fvisual+wthermal·Fthermal+wLIDAR·FLIDAR, where net is the weighted input of the perceptron;

[0036] Activation function: Use the step function as the activation function to convert the net into output F fusion :

[0037]

[0038] Weight update: adjust the weights according to the error; the error is the difference between the actual output and the expected output; for a single-layer perceptron, the weight update rule is as follows: wmodal(t+1)=wmodal(t)+α·(d-Ffusion)·Fmodal;

[0039] Where wmodal(t) is the weight at time step t, wmodal(t+1) is the updated weight, α is the learning rate, d is the expected output, and Fmodal is the eigenvector of the corresponding mode;

[0040] The calculation formula of the fusion feature vector of the multimodal deep learning fusion module is:

[0041] F fusion =∑ modal wmodal·Fmodal.

[0042] In a preferred embodiment, the multimodal deep learning fusion module uses an LSTM unit to process the time series feature vector during fault detection, and the processing formula is:

[0043] (ht,ct)=LSTM(Ffusion,t,(ht-1,ct-1)); where ht is the hidden state, ct is the cell state, and F fusion,t is the fused feature vector at time step t;

[0044] The multimodal deep learning fusion module maps the last hidden state of LSTM to the fault category after processing the time series feature vector. The formula is:

[0045] y^=softmax(W·hT+b); where W and b are the weights and biases of the fully connected layer, and y^ is the predicted fault category;

[0046] During the training process, the multimodal deep learning fusion module uses the cross entropy loss function to measure the difference between the predicted label and the true label; for multi-classification problems, the cross entropy loss function is defined as follows:

[0047]

[0048] in:

[0049] L is the loss function;

[0050] N is the number of samples in the batch;

[0051] C is the number of categories;

[0052] yic is the true probability that the i-th sample belongs to category c;

[0053] y^ic is the predicted probability that the i-th sample belongs to category c.

[0054] In a preferred embodiment, the target detection and recognition module collects a large amount of transmission line image data, including images in normal and abnormal states, for training the target detection model; the model is trained using the labeled data set, and the detection accuracy of the model for the transmission line target is improved by adjusting the model parameters and optimizing the algorithm;

[0055] The target detection and recognition module improves the positioning accuracy of target detection and reduces missed detection and false detection by adjusting the size and position of the detection frame; it extracts attributes of the detected targets, including the size of the insulator and the material of the hardware, to provide more information for subsequent fault diagnosis.

[0056] In a preferred embodiment, the report generation module first collects the results from the target detection and identification module and the fault detection module, which include the location, category, attributes of the target and the type, location, and severity of the fault; then, the report generation module writes a detailed inspection report based on this information, which includes not only the basic information of the fault, but also the impact that the fault may have on the operation of the transmission line, as well as the recommended maintenance plan and preventive measures;

[0057] To make the report more intuitive and easy to understand, the report generation module will also generate a series of visual charts based on the collected results, including heat maps and fault distribution maps.

[0058] In a preferred embodiment, the ground control and display module is responsible for receiving data from the drone and the visual acquisition module, processing and displaying the data; specifically including:

[0059] Data reception: real-time reception of images, videos and three-dimensional space information transmitted by drones and visual acquisition modules;

[0060] Data processing and display: pre-process and analyze the received data, and display the processed results in the form of graphics and tables on the interface;

[0061] Mission planning and control: According to inspection requirements, the UAV and visual acquisition module are planned and controlled;

[0062] Report generation and storage: Generate inspection reports based on the processing and analysis results and store the reports in the database;

[0063] Through the operation of the ground control and display module, real-time monitoring and control of the UAV and visual acquisition module can be achieved.

[0064] In a preferred embodiment, the drone auxiliary equipment module provides necessary support and guarantee for the drone platform module and the visual acquisition module, including:

[0065] Charging equipment: provide charging services for drones;

[0066] Maintenance tools: used for daily maintenance and troubleshooting of drones;

[0067] Protective device: protect the safe operation of drones in harsh environments.

[0068] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0069] 1. In the present invention, by using CNN to extract features of visual images and infrared images respectively, and using 3D-CNN to process lidar data, the system can capture various information about the transmission line. Data of different modes can provide information about the state of the transmission line from different angles. The fusion of such multimodal data helps to improve the accuracy and robustness of fault detection. By using LSTM network to process time series data, the system can better understand the changes of data over time. LSTM can process sequence data and capture the changing trend of faults over time, so as to more accurately predict and diagnose faults.

[0070] 2. In the present invention, the weights of different modal data are determined by a single-layer perceptron, and the system can automatically learn the importance of each modal data to the final fault detection. This ability to automatically adjust weights helps to improve the adaptability and flexibility of the system, enabling it to adjust its behavior according to different data quality and input conditions. After processing the time series feature vector, the system maps the last hidden state of the LSTM to the fault category and uses the cross-entropy loss function to measure the difference between the predicted label and the true label. This method helps the system to continuously adjust its weights during the learning process to reduce prediction errors, thereby improving the accuracy and reliability of fault detection.

[0071] 3. In the present invention, the drone platform module can cover a wide range of power transmission line areas in a short time by carrying sensors such as high-definition cameras, infrared thermal imagers and lidars. The rapid movement capability of the drone enables the inspection work to be completed efficiently, saving a lot of manpower and time resources. The scalability of the drone platform module allows the system to add or replace different sensors as needed to adapt to different inspection tasks and environmental conditions. In addition, the flexibility of drones also enables them to perform tasks in hard-to-reach areas, such as high-altitude lines or areas with complex terrain, improving the comprehensiveness and coverage of inspections.

[0072] The ground control and display module provides the ability to monitor drone and sensor data in real time, allowing maintenance personnel to understand the inspection progress and line status in real time. This real-time monitoring helps to respond quickly to emergencies and take immediate action when problems arise.

[0073] Data analysis and report generation: The ground control and display module can process and analyze the data collected by the drone and generate detailed inspection reports. These reports not only include the location and type of faults, but also potential causes of faults and repair suggestions, providing important decision support for operation and maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is the overall system block diagram of the present invention;

[0075] Figure 2 This is a system block diagram of the data processing and analysis module in the present invention. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0077] Reference Figure 1-2 ,

[0078] Example:

[0079] A transmission line field data acquisition system based on UAV vision, the system includes: a UAV platform module, a visual acquisition module, a data processing and analysis module, a ground control and display module and a UAV auxiliary equipment module;

[0080] The data processing and analysis module is internally configured with an image preprocessing module, a target detection and recognition module, a multimodal deep learning fusion module, and a report generation module;

[0081] The UAV platform module is connected to the visual acquisition module, carrying and controlling cameras and other sensors for data collection.

[0082] The UAV platform module is connected to the ground control and display module, receiving instructions from the ground and transmitting flight status and collected data.

[0083] The visual acquisition module is integrated on the UAV platform module, which is powered and supported by the UAV. The collected data is transmitted to the data processing and analysis module for further processing.

[0084] The data processing and analysis module receives the raw data from the visual acquisition module, and transmits the analyzed and processed data to the ground control and display module for display and storage.

[0085] Ground control and display module: connected to the UAV platform module, sends control instructions and receives UAV status information. Connected to the data processing and analysis module, receives processed data, displays and stores it, and provides an interface for analysis results.

[0086] The drone auxiliary equipment module is directly connected to the drone platform module to provide it with necessary maintenance and support to ensure the continuous and stable operation of the drone.

[0087] The UAV platform module is responsible for carrying and controlling various sensors and performing inspection tasks. Specifically, it includes:

[0088] The drone itself;

[0089] Flight control system: realizes the flight control of the UAV, including take-off, landing, cruising, hovering and other functions, ensuring that the UAV can perform inspection tasks stably and accurately.

[0090] Communication system: Ensure that the data transmission between the UAV and the ground station is stable and reliable. Wireless data transmission or satellite communication can be used to ensure real-time transmission of inspection data.

[0091] Navigation system: provides navigation and positioning functions for drones during the inspection process, ensuring that the drones can inspect according to the predetermined route.

[0092] Mission planning system: formulate UAV flight routes and inspection tasks according to inspection requirements to ensure that UAVs can complete inspection tasks efficiently and accurately.

[0093] Visual acquisition is responsible for collecting image and video data of power transmission lines. Specifically, it includes:

[0094] High-definition camera: Equipped with a high-definition camera, it is responsible for collecting image and video data of the transmission line, and is required to have characteristics such as high resolution, high frame rate, and low illumination.

[0095] Infrared thermal imager: Equipped with an infrared thermal imager to detect temperature anomalies in power transmission lines and ancillary equipment and identify potential fault points.

[0096] LiDAR: Equipped with LiDAR, it can obtain three-dimensional spatial information of transmission lines and be used to generate topographic models of line corridors.

[0097] Data transmission: The collected images, videos and three-dimensional spatial information are transmitted to the ground control and display module in real time for subsequent analysis.

[0098] The data preprocessing submodule scales the image pixel values ​​to between 0 and 1. This step can effectively avoid the impact of differences in brightness, contrast, etc. between different images, making subsequent feature extraction and analysis more accurate;

[0099] The data preprocessing submodule converts the pixel values ​​in the infrared image into actual temperature values. This step can make the infrared image data more accurate and help fault detection and analysis;

[0100] The data preprocessing submodule scales the temperature value to between 0 and 1. This step can make the infrared image data more standardized, which is helpful for subsequent feature extraction and analysis;

[0101] The multimodal deep learning fusion module uses CNN to extract features of visual images and infrared images respectively, then uses LSTM network to process time series data, and finally fuses the features of different modalities through a fusion layer for final fault detection;

[0102] The image features and infrared image features are extracted using multiple convolutional layers and pooling layers, and the extraction formula is:

[0103] F modal =CNN(I modal_norm );

[0104] Among them, F modal is the eigenvector, I modal_norm is the preprocessed image;

[0105] The lidar data uses a 3D convolution layer to process the point cloud data. The formula is:

[0106] F LIDAR =3D-CNN(P LIDAR_norm ) where FLIDAR is the feature vector extracted from the point cloud data, P LIDAR_norm It is the preprocessed point cloud data.

[0107] In the weighted sum fusion process of the multimodal deep learning fusion module, the weights are first determined by a single-layer perceptron, including:

[0108] Forward propagation: Calculate the weighted input and output of the network:

[0109] net=wvisual·Fvisual+wthermal·Fthermal+wLIDAR·FLIDAR where net is the weighted input of the perceptron.

[0110] Activation function: Use the step function as the activation function to convert the net into output F fusion :

[0111]

[0112] Weight update: adjust the weights according to the error. The error is the difference between the actual output and the expected output. For a single-layer perceptron, the weight update rule is as follows: wmodal(t+1)=wmodal(t)+α·(d-Ffusion)·Fmodal;

[0113] Where wmodal(t) is the weight at time step t, wmodal(t+1) is the updated weight, α is the learning rate, d is the expected output, and Fmodal is the eigenvector of the corresponding mode;

[0114] The calculation formula of the fusion feature vector of the multimodal deep learning fusion module is:

[0115] F fusion＝ ∑ modal wmodal·Fmodal

[0116] The multimodal deep learning fusion module uses LSTM units to process time series feature vectors during fault detection. The processing formula is:

[0117] (ht,ct)=LSTM(Ffusion,t,(ht-1,ct-1)); where ht is the hidden state, ct is the cell state, and F fusion,t is the fused feature vector at time step t;

[0118] The multimodal deep learning fusion module maps the last hidden state of LSTM to the fault category after processing the time series feature vector. The formula is:

[0119] y^=softmax(W·hT+b); where W and b are the weights and biases of the fully connected layer, and y^ is the predicted fault category;

[0120] During the training process, the multimodal deep learning fusion module uses the cross entropy loss function to measure the difference between the predicted label and the true label. For multi-classification problems, the cross entropy loss function is defined as follows:

[0121]

[0122] in:

[0123] L is the loss function.

[0124] N is the number of samples in the batch.

[0125] C is the number of categories.

[0126] yic is the true probability that the i-th sample belongs to category c (usually one-hot encoded).

[0127] y^ic is the predicted probability that the i-th sample belongs to category c.

[0128] The target detection and recognition module collects a large amount of transmission line image data, including images in normal and abnormal states, for training the target detection model. The model is trained using the labeled data set, and the detection accuracy of the model for transmission line targets is improved by adjusting the model parameters and optimizing the algorithm;

[0129] The target detection and recognition module improves the positioning accuracy of target detection and reduces missed detection and false detection by adjusting the size and position of the detection frame. It extracts attributes of the detected targets, such as the size of the insulator and the material of the hardware, to provide more information for subsequent fault diagnosis.

[0130] The report generation module first collects the results from the target detection and recognition module and the fault detection module, which include the location, category, attributes of the target and the type, location, severity of the fault, etc. Then, based on this information, the report generation module writes a detailed inspection report, which not only contains the basic information of the fault, but also includes the impact that the fault may have on the operation of the transmission line, as well as the recommended maintenance plan and preventive measures.

[0131] In order to make the report more intuitive and easy to understand, the report generation module will also generate a series of visual charts based on the collected results, such as heat maps, fault distribution maps, etc. These charts can help operation and maintenance personnel understand the fault situation more intuitively, quickly locate the fault location, and assess the severity of the fault, so as to make more accurate decisions.

[0132] Through the processing of the report generation module, the fault situation can be displayed intuitively, providing a basis for decision-making for operation and maintenance personnel, and improving the efficiency of transmission line operation and maintenance. At the same time, the generation of reports also facilitates operation and maintenance personnel to conduct long-term monitoring and analysis of the operating status of the transmission line, providing strong support for the maintenance and management of the transmission line.

[0133] The ground control and display module is responsible for receiving, processing and displaying data from the drone and the visual acquisition module. Specifically, it includes:

[0134] Data reception: Real-time reception of images, videos and three-dimensional spatial information transmitted by drones and visual acquisition modules.

[0135] Data processing and display: Preprocess and analyze the received data, and display the processed results on the interface in the form of graphics, tables, etc. for easy viewing by operation and maintenance personnel.

[0136] Mission planning and control: According to inspection requirements, the UAV and visual acquisition module are planned and controlled to ensure efficient completion of inspection tasks.

[0137] Report generation and storage: Generate inspection reports based on the results of processing and analysis, and store the reports in the database for subsequent query and analysis.

[0138] Through the operation of the ground control and display module, real-time monitoring and control of the drone and visual acquisition module can be achieved to improve inspection efficiency.

[0139] The UAV auxiliary equipment module provides necessary support and guarantee for the UAV platform module and the visual acquisition module, including:

[0140] Charging equipment: Provide charging services for drones to ensure the endurance of drones and the continuity of inspection tasks.

[0141] Maintenance tools: used for daily maintenance and troubleshooting of drones to ensure the good operating condition of drone equipment.

[0142] Protective device: protects the safe operation of the drone in harsh environments and prevents the drone from being damaged by external factors.

[0143] Through the operation of the UAV auxiliary equipment module, the reliability and safety of the UAV platform module and the visual acquisition module can be improved to ensure the smooth completion of the inspection task.

[0144] From the above, we know that:

[0145] In the present invention, by using CNN to extract features of visual images and infrared images respectively, and using 3D-CNN to process lidar data, the system can capture various information about the transmission line. Data of different modes can provide information about the state of the transmission line from different angles. The fusion of such multimodal data helps to improve the accuracy and robustness of fault detection. By using LSTM network to process time series data, the system can better understand the changes of data over time. LSTM can process sequence data and capture the trend of fault changes over time, so as to more accurately predict and diagnose faults.

[0146] In the present invention, the weights of different modal data are determined by a single-layer perceptron, and the system can automatically learn the importance of each modal data to the final fault detection. This ability to automatically adjust weights helps to improve the adaptability and flexibility of the system, enabling it to adjust its behavior according to different data quality and input conditions. After processing the time series feature vector, the system maps the last hidden state of the LSTM to the fault category and uses a cross-entropy loss function to measure the difference between the predicted label and the true label. This method helps the system to continuously adjust its weights during the learning process to reduce prediction errors, thereby improving the accuracy and reliability of fault detection.

[0147] In the present invention, the drone platform module can cover a wide range of power transmission line areas in a short time by carrying sensors such as high-definition cameras, infrared thermal imagers and laser radars. The rapid movement capability of the drone enables the inspection work to be completed efficiently, saving a lot of manpower and time resources. The scalability of the drone platform module allows the system to add or replace different sensors as needed to adapt to different inspection tasks and environmental conditions. In addition, the flexibility of drones also enables them to perform tasks in difficult-to-reach areas, such as high-altitude lines or areas with complex terrain, improving the comprehensiveness and coverage of inspections.

[0148] The ground control and display module provides the ability to monitor drone and sensor data in real time, allowing maintenance personnel to understand the inspection progress and line status in real time. This real-time monitoring helps to respond quickly to emergencies and take immediate action when problems arise.

[0149] Data analysis and report generation: The ground control and display module can process and analyze the data collected by the drone and generate detailed inspection reports. These reports not only include the location and type of faults, but also potential causes of faults and repair suggestions, providing important decision support for operation and maintenance personnel.

[0150] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or still includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0151] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A transmission line field data acquisition system based on drone vision, characterized by: The system includes: a UAV platform module, a visual acquisition module, a data processing and analysis module, a ground control and display module and a UAV auxiliary equipment module; The data processing and analysis module is internally provided with an image preprocessing module, a target detection and recognition module, a multimodal deep learning fusion module and a report generation module; The drone platform module is connected to the visual acquisition module, carrying and controlling cameras and other sensors for data acquisition; The UAV platform module is connected to the ground control and display module to receive instructions from the ground and transmit flight status and collected data; The visual acquisition module is integrated on the UAV platform module, and the UAV provides power and support; the collected data is transmitted to the data processing and analysis module for further processing; The data processing and analysis module receives the raw data from the visual acquisition module; the analyzed and processed data is transmitted to the ground control and display module for display and storage; The ground control and display module is connected to the UAV platform module to send control instructions and receive UAV status information; it is connected to the data processing and analysis module to receive processed data, display and store it, and provide an interface for analysis results; The UAV auxiliary equipment module is directly connected to the UAV platform module to provide it with necessary maintenance and support to ensure that the UAV can operate continuously and stably.

2. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The UAV platform module is responsible for carrying and controlling various sensors and performing inspection tasks; Specifically include: The drone itself; Flight control system: realizes the flight control of the UAV, including take-off, landing, cruising, and hovering functions, ensuring that the UAV can perform inspection tasks stably and accurately; Communication system: ensure stable and reliable data transmission between the UAV and the ground station. Wireless data transmission or satellite communication can be used to ensure real-time transmission of inspection data. Navigation system: provides navigation and positioning functions for drones during inspections, ensuring that drones can conduct inspections according to the scheduled routes; Mission planning system: formulate UAV flight routes and inspection tasks according to inspection requirements to ensure that UAVs can complete inspection tasks efficiently and accurately.

3. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The visual acquisition is responsible for acquiring the image and video data of the transmission line; specifically, it includes: HD camera: Equipped with a HD camera, responsible for collecting image and video data of the transmission line; Infrared thermal imager: Equipped with an infrared thermal imager to detect temperature anomalies of power transmission lines and auxiliary equipment and find potential fault points; LiDAR: Equipped with LiDAR, it can obtain three-dimensional spatial information of the transmission line and generate a topographic model of the line corridor; Data transmission: The collected images, videos and three-dimensional spatial information are transmitted to the ground control and display module in real time.

4. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The data preprocessing submodule scales the image pixel values ​​to between 0 and 1; The data preprocessing submodule converts the pixel values ​​in the infrared image into actual temperature values; This step makes the infrared image data more accurate and helps with fault detection and analysis; The data preprocessing submodule scales the temperature value to between 0 and 1.

5. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The multimodal deep learning fusion module uses CNN to extract features of visual images and infrared images respectively, then uses LSTM network to process time series data, and finally fuses the features of different modes through a fusion layer for final fault detection; The image features and infrared image features are extracted using multiple convolutional layers and pooling layers, and the extraction formula is: F modal =CNN(I modal_norm ); Among them, F modal is the eigenvector, I modal_norm is the preprocessed image; The lidar data uses a 3D convolution layer to process the point cloud data. The formula is: F LIDAR =3D-CNN(P LIDAR_norm ) where FLIDAR is the feature vector extracted from the point cloud data, P LIDAR_norm It is the preprocessed point cloud data; In the weighted sum fusion process of the multimodal deep learning fusion module, the weights are first determined by a single-layer perceptron, specifically including: Forward propagation: Calculate the weighted input and output of the network: net=wvisual·Fvisual+wthermal·Fthermal+wLIDAR·FLIDAR, where net is the weighted input of the perceptron; Activation function: Use the step function as the activation function to convert the net into output F fusion : Weight update: adjust the weights according to the error; the error is the difference between the actual output and the expected output; for a single-layer perceptron, the weight update rule is as follows: wmodal(t+1)=wmodal(t)+α·(d-Ffusion)·Fmodal; Where wmodal(t) is the weight at time step t, wmodal(t+1) is the updated weight, α is the learning rate, d is the expected output, and Fmodal is the eigenvector of the corresponding mode; The calculation formula of the fusion feature vector of the multimodal deep learning fusion module is: F fusion＝ ∑ modal wmodal·Fmodal。 6. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The multimodal deep learning fusion module uses LSTM units to process time series feature vectors during fault detection. The processing formula is: (ht,ct)=LSTM(Ffusion,t,(ht-1,ct-1)); where ht is the hidden state, ct is the cell state, and F fusion,t is the fused feature vector at time step t; The multimodal deep learning fusion module maps the last hidden state of LSTM to the fault category after processing the time series feature vector. The formula is: y^=softmax(W·hT+b); where W and b are the weights and biases of the fully connected layer, and y^ is the predicted fault category; During the training process, the multimodal deep learning fusion module uses the cross entropy loss function to measure the difference between the predicted label and the true label; for multi-classification problems, the cross entropy loss function is defined as follows: in: L is the loss function; N is the number of samples in the batch; C is the number of categories; yic is the true probability that the i-th sample belongs to category c; y^ic is the predicted probability that the i-th sample belongs to category c.

7. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The target detection and recognition module collects a large amount of transmission line image data, including images in normal and abnormal states, for training the target detection model; the model is trained using the labeled data set, and the detection accuracy of the model for transmission line targets is improved by adjusting the model parameters and optimizing the algorithm; The target detection and recognition module improves the positioning accuracy of target detection and reduces missed detection and false detection by adjusting the size and position of the detection frame; it extracts attributes of the detected targets, including the size of the insulator and the material of the hardware, to provide more information for subsequent fault diagnosis.

8. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The report generation module first collects the results from the target detection and recognition module and the fault detection module, which include the location, category, attributes of the target and the type, location, and severity of the fault; then, the report generation module writes a detailed inspection report based on this information, which includes not only the basic information of the fault, but also the impact that the fault may have on the operation of the transmission line, as well as the recommended maintenance plan and preventive measures; To make the report more intuitive and easy to understand, the report generation module will also generate a series of visual charts based on the collected results, including heat maps and fault distribution maps.

9. The transmission line field data acquisition system based on drone vision as claimed in claim 1, characterized in that: The ground control and display module is responsible for receiving data from the UAV and the visual acquisition module, processing and displaying it; specifically, it includes: Data reception: real-time reception of images, videos and three-dimensional space information transmitted by drones and visual acquisition modules; Data processing and display: pre-process and analyze the received data, and display the processed results in the form of graphics and tables on the interface; Mission planning and control: According to inspection requirements, the UAV and visual acquisition module are planned and controlled; Report generation and storage: Generate inspection reports based on the processing and analysis results and store the reports in the database; Through the operation of the ground control and display module, real-time monitoring and control of the UAV and visual acquisition module can be achieved.

10. The transmission line field data acquisition system based on drone vision according to claim 1, characterized in that: The UAV auxiliary equipment module provides necessary support and guarantee for the UAV platform module and the visual acquisition module, including: Charging equipment: provide charging services for drones; Maintenance tools: used for daily maintenance and troubleshooting of drones; Protective device: protect the safe operation of drones in harsh environments.

Images