A thunder channel three-dimensional reconstruction method based on multiple cameras and a drone

Abstract

The application discloses a thunder channel three-dimensional reconstruction method based on multiple cameras and a drone, relates to the technical field of thunder channel three-dimensional reconstruction, and solves the technical problem that the three-dimensional reconstruction of a thunder channel needs to depend on a calibration object, and does not depend on traditional calibration objects for calibration, but realizes the direct reconstruction of a thunder channel by combining image recognition and path planning algorithms through the observation of multiple cameras and the path reproduction process of the drone. The method can not only overcome the limitations of traditional stereo vision three-dimensional reconstruction technology, but also provide direct and high-precision thunder three-dimensional channel data, and has the advantages of low cost, simple operation, strong adaptability and the like.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction of lightning channels, and in particular to a method for three-dimensional reconstruction of lightning channels based on multiple cameras and drones. Background Technology

[0002] Lightning is a complex natural phenomenon involving high energy and intricate electromagnetic effects, posing a potential threat to human activities and infrastructure. Accurately pinpointing the location and path of lightning channels is crucial for effective lightning protection and in-depth research into lightning mechanisms. Three-dimensional reconstruction of lightning channels not only provides precise information on lightning discharge paths but also offers fundamental data for lightning research, early warning systems, and lightning protection engineering.

[0003] Currently, the main methods for three-dimensional reconstruction of lightning channels include the following:

[0004] (1) Optical observation method: High-speed cameras or high-speed video cameras are used to capture the lightning discharge process, and the lightning channel is reconstructed by capturing images of the lightning discharge moment. Its advantage is that it can intuitively record the shape and path of the lightning channel; its disadvantage is that it is greatly affected by weather and ambient light, and it is difficult to obtain clear images at night or in bad weather conditions; at the same time, a lot of manual analysis of image data is required.

[0005] (2) Electromagnetic field measurement method: By deploying electric and magnetic field detectors in the lightning area, the changes in the electromagnetic field generated by lightning are measured. The location and path of the lightning channel are determined by inversion calculation using multi-point measurement data. Its advantages are that it can work in all weather conditions and is less affected by sunlight and weather conditions; its disadvantages are that the electromagnetic field measurement equipment is complex to set up, the cost is high, the data processing and inversion calculation process is complex, and the accuracy is affected by the density of detectors and the calculation method.

[0006] (3) Radar observation method: The lightning discharge process is observed using meteorological radar or dedicated lightning radar. The radar signal reflection can provide three-dimensional structural information of the lightning channel. Its advantages are that it can provide three-dimensional information of the lightning channel and has a wide coverage area; its disadvantages are that the radar equipment is expensive, is greatly affected by the environment and terrain, the radar data processing is complicated, and the real-time performance is poor.

[0007] (4) Radio direction finding method: This method uses multiple receiving stations to find the direction of the radio waves generated by lightning discharge and uses triangulation to determine the location of the lightning channel. Its advantages are that it can cover a large area and has high positioning accuracy; its disadvantages are that it requires multiple receiving stations to work in coordination, making the system complex; and the radio wave propagation path may be deviated due to factors such as the ionosphere.

[0008] (5) Lightning Positioning System (LPS): This system uses multiple positioning stations to locate lightning channels via Time Difference of Arrival (TDOA) technology. Examples include the United States Precision Lightning Network (USPLN) and the World Wide Lightning Location Network (WWLLN). Its advantages include high positioning accuracy and the ability to provide real-time lightning activity information; its disadvantages include high system construction and maintenance costs, and positioning accuracy being affected by the density of the positioning stations and the accuracy of time synchronization.

[0009] While the aforementioned methods have achieved some success in the three-dimensional reconstruction of lightning channels, they still have some limitations. Optical observations are greatly affected by weather and light conditions, electromagnetic field measurements and radar observation equipment are expensive and data processing is complex, and radio direction finding and lightning location systems require complex multi-point collaborative work, resulting in high construction and maintenance costs. How to achieve high-precision, automated three-dimensional reconstruction of lightning channels without relying on calibration objects is a problem that this application aims to solve. Summary of the Invention

[0010] This application provides a method for three-dimensional reconstruction of lightning channels based on multiple cameras and drones. Its technical purpose is to achieve high-precision and automated three-dimensional reconstruction of lightning channels without relying on calibration objects. Instead, it uses all-weather shooting by multiple cameras and path reproduction by drones, combined with image recognition and path planning algorithms, to achieve this.

[0011] The above-mentioned technical objective of this application is achieved through the following technical solution:

[0012] A method for 3D reconstruction of lightning channels based on multiple cameras and drones includes:

[0013] Step S1: Continuously monitor and photograph the target area using a camera array to capture lightning phenomena;

[0014] Step S2: Identify the lightning channel image using the lightning recognition algorithm. When any camera detects a lightning phenomenon, all cameras in the camera group start shooting simultaneously to record the lightning channel from different angles.

[0015] Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the drone's flight, the camera group continues to capture the drone's position. By comparing the photos of the drone's position with the photos of the lightning channel, the drone control algorithm is used to adjust the drone's flight path so that the drone can reproduce the lightning channel and obtain the drone's flight trajectory data.

[0016] Step S4: Use UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.

[0017] Furthermore, in step S1, the camera group includes four high-resolution cameras, which are distributed at the four corners of the target area, forming a square observation network.

[0018] Further, in step S2, the lightning recognition algorithm includes image preprocessing, feature extraction, feature matching and classification, dynamic detection and temporal domain analysis, multi-camera data fusion, alarm and data storage; the image preprocessing includes image denoising, brightness adjustment, and color space conversion; the feature extraction includes brightness threshold segmentation, edge detection, and morphological processing; the feature matching and classification includes feature matching and classification; the dynamic detection and temporal domain analysis includes inter-frame difference method and time series analysis; the multi-camera data fusion includes viewpoint correction, feature fusion, and consistency verification; and the alarm and data storage includes alarm triggering and data storage.

[0019] Further, in step S3, the UAV control algorithm includes UAV initial position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis; the UAV initial position setting includes starting point setting and flight parameter initialization; the real-time image acquisition and processing includes image acquisition and image processing; the feature comparison and matching includes feature extraction and position matching; the control signal calculation includes position error calculation and control signal generation; the flight path adjustment and correction includes real-time adjustment and path prediction and planning; the data recording and feedback includes flight data recording and real-time feedback; the flight mission completion and return includes mission completion detection and return path planning; and the data storage and analysis includes data storage and data analysis.

[0020] Furthermore, in step S4, the steps of 3D reconstruction include acquiring the UAV's position and attitude, feature point extraction and matching technology, 3D point cloud generation and 3D model reconstruction, error analysis and correction, and output and display.

[0021] The beneficial effects of this application are as follows: The 3D reconstruction method for lightning channels based on multiple cameras and UAVs described in this application uses four camera groups to continuously monitor the target area from different angles 24 hours a day, and captures and acquires multi-view image data of the lightning channel using a lightning recognition algorithm. Image processing technology is used to extract feature points of the lightning channel and perform 3D reconstruction to generate a preliminary lightning channel model. Subsequently, the UAV plans its flight path based on the generated 3D model data and flies along this path. During flight, the UAV's position information and attitude data are recorded in real time and compared and adjusted with the lightning channel images captured by the cameras to correct path deviations. Finally, the flight data and multi-view image data are fused to form an accurate 3D model of the lightning channel. This method does not rely on external calibration objects. Through stereo observation by multiple cameras and flexible flight of the UAV, combined with advanced image processing and path planning algorithms, it achieves high-precision, automated 3D reconstruction of lightning channels. This method not only provides accurate lightning channel path data but also has low cost, simple operation process, and strong adaptability, overcoming the limitations of traditional methods. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the camera arrangement in an embodiment of this application;

[0023] Figure 2 This is a flowchart of lightning identification in an embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating the drone control process in an embodiment of this application.

[0025] Figure 4 This is a flowchart of the three-dimensional reconstruction process of the UAV in an embodiment of this application. Detailed Implementation

[0026] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0027] The lightning channel 3D reconstruction method based on multiple cameras and drones described in this application includes:

[0028] Step S1: Continuously monitor and photograph the target area using a camera array to capture lightning phenomena.

[0029] Specifically, the placement and installation of the cameras include:

[0030] like Figure 1As shown, four high-resolution cameras (C1, C2, C3, and C4) are positioned at the four corners of the target area, forming a square observation network. The specific positions and heights of the cameras are adjusted according to the actual site conditions to ensure comprehensive coverage of the target area. The cameras must have all-weather shooting capabilities, including waterproof, dustproof, and wind-resistant features, to ensure normal operation even in adverse weather conditions. Simultaneously, the cameras must have infrared night vision capabilities to ensure clear image capture even at night or in low-light conditions. The cameras are mounted at a high position using fixed brackets, ensuring a good shooting angle and avoiding obstruction of the view. The four cameras continuously monitor the sky over the observation area around the clock.

[0031] Step S2: Identify the lightning channel image using a lightning recognition algorithm. When any camera detects a lightning phenomenon, all cameras in the camera group start shooting simultaneously to record the lightning channel from different angles.

[0032] Specifically, the lightning recognition algorithm includes image preprocessing, feature extraction, feature matching and classification, dynamic detection and time domain analysis, multi-camera data fusion, alarm and data storage.

[0033] After the camera captures the image, it undergoes image preprocessing, namely image denoising: using methods such as Gaussian filtering, mean filtering, or median filtering to remove noise from the image; brightness adjustment: using histogram equalization to enhance image contrast and highlight lightning features; color space conversion: converting the image from RGB color space to HSV color space or grayscale image to facilitate subsequent feature extraction. Then, lightning feature extraction is performed, namely brightness thresholding: setting a brightness threshold to extract areas in the image whose brightness exceeds the threshold. These areas may contain lightning channels; edge detection: using edge detection algorithms such as Sobel and Canny operators to detect significant edges in the image to help determine the boundaries of lightning channels; morphological processing: removing noise points and small regions through morphological operations (such as dilation, erosion, opening, and closing operations) while retaining the main lightning structures. Finally, feature matching and classification are performed, namely feature matching: using template matching or shape-based feature matching techniques to compare the extracted features with predefined lightning morphological features. The matching degree can be evaluated using metrics such as Structural Similarity Indices (SSIM) and Normalized Cross-Correlation (NCC). Classifier application: A machine learning classifier (convolutional neural network) is used to classify the matching results, further confirming the existence of lightning phenomena. Then, dynamic detection and temporal analysis are performed, i.e., inter-frame difference method: Rapidly changing regions are extracted from the difference images of consecutive frames to identify dynamic lightning phenomena. Time series analysis: Time series analysis is performed on the detected lightning features to ensure that these features are consistent over time, eliminating interference from transient light spots that are not lightning. Next, multi-camera data fusion is performed, i.e., viewpoint correction: Geometric correction is performed on the images from each viewpoint based on the relative position and angle of the cameras. Feature fusion: The lightning features detected by each camera are fused to form complete lightning channel information. Consistency verification: The consistency of the data from each camera is checked to confirm the actual occurrence and location of the lightning phenomenon. Finally, after identifying a lightning event, the system will generate an alarm and save the relevant data. Alarm triggering: When multiple cameras simultaneously detect lightning features and the feature matching degree exceeds a preset threshold, the system automatically triggers an alarm; Data storage: All raw data and processing results from the identification process are stored in a data center for subsequent analysis and 3D reconstruction. The lightning identification algorithm flowchart is as follows: Figure 2 As shown.

[0034] Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the drone's flight, the camera group continues to capture the drone's position. By comparing the photos of the drone's position with the photos of the lightning channel, the drone control algorithm is used to adjust the drone's flight path so that the drone can reproduce the lightning channel and obtain the drone's flight trajectory data.

[0035] Specifically, the UAV control algorithm includes initial UAV position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis.

[0036] First, the initial position of the drone, i.e., the starting point setting, is performed: the drone's starting point is usually set at the beginning of the lightning channel or a known reference point. The selection of the starting point needs to consider the panoramic coverage of the lightning channel; flight parameter initialization: initial flight parameters are set, including flight altitude, speed, and direction. These parameters can be adjusted according to the expected height and length of the lightning channel. Then, real-time image acquisition and processing, i.e., image acquisition: four fixed cameras acquire images of the drone's current position in real time and compare them with previously captured images of the lightning channel; image processing: the acquired images are preprocessed to remove noise and enhance key features. This step is similar to the preprocessing in the lightning recognition algorithm, but the focus is on the comparison between the drone's position and the lightning channel position. Next, feature comparison and matching are performed, i.e., feature extraction: features are extracted from real-time acquired images and previous lightning channel images, mainly including the drone's position, the shape of the lightning channel, and brightness features; position matching: image registration techniques (such as template matching, feature point matching) are used to compare the drone's current position with the position of the lightning channel to determine the drone's displacement relative to the channel; then, control signal calculation, i.e., position error calculation: based on the position matching results, the error between the drone's current position and the lightning channel reference path is calculated, including lateral error and longitudinal error; control signal generation: based on the error value, control signals for the drone are generated (adjusting flight speed, direction, and altitude). A proportional-integral-derivative (PID) control algorithm is initially adopted for control, specifically divided into lateral control: adjusting the drone's left and right displacement to bring it closer to the centerline of the lightning channel; longitudinal control: adjusting the flight speed to keep the drone in the forward direction of the lightning channel; and altitude control: adjusting the drone's flight altitude according to the altitude changes of the lightning channel. Next, flight path adjustment and correction are performed, i.e., real-time adjustment: the UAV continuously adjusts its flight path based on real-time updated control signals to correct deviations from the lightning channel; path prediction and planning: machine learning or optimization algorithms are used to predict the future path of the lightning channel, allowing for advance planning of the UAV's flight to improve the stability and accuracy of path tracking. Then, data recording and feedback are performed, i.e., flight data recording: the UAV's flight data, including position, speed, altitude, and control signals, is recorded. The recorded data is used for subsequent analysis and system optimization; real-time feedback: any abnormal situations during flight (such as deviation from the path or signal loss) are fed back to the ground control center, and emergency measures are taken when necessary. Finally, flight mission completion and return are performed, i.e., mission completion detection: when the UAV reaches the end of the lightning channel or meets the preset mission completion conditions, the system automatically terminates the flight mission; return path planning: the return path of the UAV is planned to ensure its safe return to the starting point or designated landing point.Finally, data storage and analysis are performed. Data storage involves storing all data collected during flight in a database for subsequent analysis and optimization of the UAV control algorithm. Data analysis involves analyzing flight data to evaluate the performance of the control algorithm, including path tracking accuracy, control stability, and response speed. Based on the analysis results, the algorithm is further optimized.

[0037] Step S4: Use UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.

[0038] Specifically, the process includes: position and attitude acquisition: acquiring real-time position and attitude data of the UAV through sensors; feature point extraction and matching: extracting key feature points in the flight path and matching them with lightning channel features; 3D point cloud generation: generating a 3D point cloud of the lightning channel using the feature point position data; 3D model reconstruction: constructing a 3D model of the lightning channel based on the point cloud data; error analysis and correction: analyzing the error between the model and the actual path and correcting it to improve the model's accuracy; output and display: the final 3D model of the lightning channel is displayed through visualization tools and used for lightning research, lightning protection engineering design, and the development of lightning early warning systems, providing scientific data support for related fields.

Claims

1. A method for three-dimensional reconstruction of lightning channels based on multiple cameras and drones, characterized in that, include: Step S1: Continuously monitor and photograph the target area using a camera group to capture lightning phenomena; wherein, the camera group includes four high-resolution cameras, which are distributed in the four corners of the target area, forming a square observation network. Step S2: Identify the lightning channel image using the lightning recognition algorithm. When any camera detects a lightning phenomenon, all cameras in the camera group start shooting simultaneously to record the lightning channel from different angles. Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the drone's flight, the camera group continues to capture images of the drone's position. By comparing the images of the drone's position with those of the lightning channel, the drone's flight path is adjusted using the drone control algorithm to reproduce the lightning channel and acquire the drone's flight trajectory data. This specifically includes: Image acquisition: Four fixed cameras acquire images of the drone's current position in real time and compare them with previously captured images of the lightning channel; Feature extraction: Features are extracted from real-time acquired images and previous lightning channel images, mainly including the location of the UAV, the shape of the lightning channel, and brightness features; Position matching: Image registration technology is used to compare the current position of the drone with the position of the lightning channel to determine the displacement of the drone relative to the channel; Next, control signal calculation, i.e., position error calculation, is performed: based on the position matching results, the error between the current position of the UAV and the reference path of the lightning channel is calculated, including lateral error and longitudinal error; Control signal generation: Generate control signals for the UAV based on the error value; Then, the flight path is adjusted and corrected, that is, adjusted in real time: the UAV continuously adjusts its flight path according to the real-time updated control signals to correct the deviation from the lightning channel; Step S4: Use UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.

2. The three-dimensional reconstruction method for lightning channels as described in claim 1, characterized in that, In step S2, the lightning recognition algorithm includes image preprocessing, feature extraction, feature matching and classification, dynamic detection and temporal domain analysis, multi-camera data fusion, alarm and data storage; the image preprocessing includes image denoising, brightness adjustment and color space conversion; the feature extraction includes brightness threshold segmentation, edge detection and morphological processing. The feature matching and classification includes feature matching and classification; the dynamic detection and time domain analysis includes inter-frame difference method and time series analysis; The multi-camera data fusion includes viewpoint correction, feature fusion, and consistency verification; The alarms and data storage include alarm triggering and data storage.

3. The three-dimensional reconstruction method for lightning channels as described in claim 2, characterized in that, In step S3, the UAV control algorithm includes UAV initial position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis; the UAV initial position setting includes starting point setting and flight parameter initialization; the real-time image acquisition and processing includes image acquisition and image processing. The feature comparison and matching includes feature extraction and position matching; the control signal calculation includes position error calculation and control signal generation; the flight path adjustment and correction includes real-time adjustment and path prediction and planning; the data recording and feedback includes flight data recording and real-time feedback; the flight mission completion and return includes mission completion detection and return path planning; and the data storage and analysis includes data storage and data analysis.

4. The three-dimensional reconstruction method for lightning channels as described in claim 3, characterized in that, In step S4, the steps of 3D reconstruction include UAV position information and attitude acquisition, feature point extraction and matching technology, 3D point cloud generation and 3D model reconstruction, error analysis and correction, and output and display.

Images