Lightning channel three-dimensional reconstruction method based on multiple cameras and unmanned aerial vehicle
Through the combination of multi-camera and drone, image recognition and path planning algorithms are used to realize high-precision and automated three-dimensional reconstruction of lightning channels, solving the problems of high weather and equipment costs in the existing technology, and providing accurate Thunder path data and reduces operational complexity.
Patent Information
- Application Number
- CN202411077382.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-07
AI Technical Summary
The existing three-dimensional reconstruction method of lightning channels has problems such as high weather and light impact, expensive equipment, complex data processing, and high system construction and maintenance costs. In particular, it is difficult to achieve high-precision and automated three-dimensional reconstruction without relying on calibration materials.
The combination of multi-camera and drone is used to identify lightning channels through all-weather shooting and image recognition algorithms, and the drone replicates the lightning channels and obtains flight trajectory data, combining the path planning algorithm to achieve high-precision three-dimensional reconstruction.
It realizes high-precision and automated three-dimensional reconstruction of lightning channels, reduces costs, simplifies operational processes, overcomes the limitations of traditional methods, and provides accurate lightning path data.
Smart Images

Figure CN119048669A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of three-dimensional reconstruction of lightning channels, and in particular to a three-dimensional reconstruction method of lightning channels based on multiple cameras and unmanned aerial vehicles. Background Art
[0002] Lightning is a complex natural phenomenon. Its discharge process involves high energy and complex electromagnetic effects, which poses a potential threat to human activities and infrastructure. In order to effectively protect against lightning disasters and conduct in-depth research on lightning mechanisms, it is particularly important to accurately calibrate the location and path of lightning channels. Three-dimensional reconstruction of lightning channels can not only provide accurate information on lightning discharge paths, but also provide basic data for lightning research, lightning warning, and lightning protection engineering.
[0003] At present, the three-dimensional reconstruction methods of lightning channels mainly include the following:
[0004] (1) Optical observation method: Use a high-speed camera or high-speed video camera to record the lightning discharge process and reconstruct the lightning channel by capturing the image of the lightning discharge moment. Its advantage is that it can intuitively record the shape and path of the lightning channel; its disadvantage is that it is greatly affected by weather and ambient light, and it is difficult to obtain clear images at night or in bad weather conditions; at the same time, a lot of manual analysis of image data is required.
[0005] (2) Electromagnetic field measurement method: By arranging electric field and magnetic field detectors in the lightning area, measuring the changes in the electromagnetic field generated by lightning, and using multi-point measurement data for inversion calculation, the location and path of the lightning channel are determined. Its advantages are that it can work all day and is less affected by light and weather conditions; its disadvantages are that the layout of electromagnetic field measurement equipment is complex, the cost is high, the data processing and inversion calculation process are complex, and the accuracy is affected by the density of detector layout and the calculation method.
[0006] (3) Radar observation method: Use weather radar or special lightning radar to observe the lightning discharge process. The radar signal reflection can provide three-dimensional structural information of the lightning channel. Its advantages are that it can provide three-dimensional information of the lightning channel and has a wide coverage area; its disadvantages are that the radar equipment is expensive, greatly affected by the environment and terrain, and the radar data processing is complex and has poor real-time performance.
[0007] (4) Radio direction finding method: Use multiple receiving stations to find the direction of the radio waves generated by lightning discharge, and determine the location of the lightning channel through triangulation positioning technology. Its advantages are that it can cover a large area and has high positioning accuracy; its disadvantages are that it requires the coordination of multiple receiving stations and the system is complex; affected by factors such as the ionosphere, the radio wave propagation path may deviate.
[0008] (5) Lightning positioning system (LPS): A lightning positioning system consisting of multiple positioning stations is used to locate the lightning channel through the time difference of arrival (TDOA) technology. For example, USPLN (United States Precision Lightning Network) and WWLLN (World Wide Lightning Location Network). Its advantages are high positioning accuracy and the ability to provide real-time lightning activity information; its disadvantages are high system construction and maintenance costs, and the positioning accuracy is affected by the density of positioning station layout and time synchronization accuracy.
[0009] Although the above methods have achieved certain results in the three-dimensional reconstruction of lightning channels, there are still some limitations. Optical observation is greatly affected by weather and light, electromagnetic field measurement and radar observation equipment are expensive and data processing is complex, and radio direction finding and lightning location systems require complex multi-point collaborative work, with high construction and maintenance costs. How to achieve high-precision, automated three-dimensional reconstruction of lightning channels without relying on calibration objects needs to be solved in this application. Summary of the invention
[0010] The present application provides a method for three-dimensional reconstruction of lightning channels based on multiple cameras and drones. Its technical purpose is to achieve high-precision, automated three-dimensional reconstruction of lightning channels through all-weather shooting of multiple cameras and path reproduction of drones, combined with image recognition and path planning algorithms, without relying on calibration objects.
[0011] The above technical objectives of this application are achieved through the following technical solutions:
[0012] A three-dimensional reconstruction method of lightning channels based on multiple cameras and unmanned aerial vehicles, comprising:
[0013] Step S1: Continuously monitor and shoot the target area through a camera group to capture lightning phenomena;
[0014] Step S2: Identify the lightning channel image through the lightning recognition algorithm. When any camera recognizes the lightning phenomenon, all cameras in the camera group start shooting at the same time to record the lightning channel from different angles.
[0015] Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the flight of the drone, the camera group continues to take pictures of the drone's position. By comparing the pictures of the drone's position with the pictures of the lightning channel, the drone control algorithm is used to adjust the drone's flight route, so that the drone reproduces the lightning channel and obtains the drone's flight trajectory data.
[0016] Step S4: Use the UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.
[0017] Furthermore, in step S1, the camera group includes four high-resolution cameras, and the four high-resolution cameras are respectively distributed in four corners of the target area, and the four high-resolution cameras form a square observation network.
[0018] Furthermore, in step S2, the steps of the lightning recognition algorithm include image preprocessing, feature extraction, feature matching and classification, dynamic detection and time domain analysis, multi-camera data fusion, alarm and data storage; the image preprocessing includes image denoising, brightness adjustment and color space conversion; the feature extraction includes brightness threshold segmentation, edge detection and morphological processing; the feature matching and classification include feature matching and classification; the dynamic detection and time domain analysis include frame difference method and time series analysis; the multi-camera data fusion includes perspective correction, feature fusion and consistency verification; the alarm and data storage include alarm triggering and data storage.
[0019] Furthermore, in step S3, the UAV control algorithm includes UAV initial position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis; the UAV initial position setting includes starting point setting and flight parameter initialization; the real-time image acquisition and processing includes image acquisition and image processing; the feature comparison and matching includes feature extraction and position matching; the control signal calculation includes position error calculation and control signal generation; the flight path adjustment and correction includes real-time adjustment and path prediction and planning; the data recording and feedback includes flight data recording and real-time feedback; the flight mission completion and return includes mission completion detection and return path planning; the data storage and analysis includes data storage and data analysis.
[0020] Furthermore, in step S4, the three-dimensional reconstruction step includes obtaining the position information and posture of the UAV, feature point extraction and matching technology, three-dimensional point cloud generation and three-dimensional model reconstruction, error analysis and correction, and output and display.
[0021] The beneficial effects of the present application are as follows: the three-dimensional reconstruction method of lightning channels based on multiple cameras and unmanned aerial vehicles described in the present application monitors the target area 24 hours a day from different angles through four camera groups, and shoots and obtains multi-view image data of lightning channels through lightning recognition algorithms. Through image processing technology, the characteristic points of the lightning channel are extracted and three-dimensionally reconstructed to generate a preliminary lightning channel model. Subsequently, the drone formulates a flight path based on the generated three-dimensional model data and flies along this path. During the flight, the position information and attitude data of the drone are recorded in real time, and compared and adjusted with the lightning channel image captured by the camera to correct the path deviation. Finally, the flight data and multi-view image data are fused to form an accurate three-dimensional model of the lightning channel. This method does not need to rely on external calibration objects. Through the stereoscopic observation of multiple cameras and the flexible flight of the drone, combined with advanced image processing and path planning algorithms, high-precision and automated three-dimensional reconstruction of the lightning channel is achieved. This method not only provides accurate lightning channel path data, but also has low cost, simple operation process and strong adaptability, overcoming the limitations of traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 This is a schematic diagram of the camera arrangement in the embodiment of the present application;
[0023] Figure 2 This is a flow chart of lightning identification in an embodiment of the present application;
[0024] Figure 3 This is a flow chart of drone control in the embodiment of this application;
[0025] Figure 4 This is a flowchart of the three-dimensional reconstruction of the drone in the embodiment of this application. DETAILED DESCRIPTION
[0026] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.
[0027] The three-dimensional reconstruction method of lightning channels based on multiple cameras and drones described in this application includes:
[0028] Step S1: The target area is continuously monitored and photographed by a camera group to capture lightning phenomena.
[0029] Specifically, the arrangement and installation of the camera includes:
[0030] like Figure 1As shown in the figure, four high-resolution cameras (C1, C2, C3, C4) are arranged at the four corners of the target area to form a square observation network. The specific position and height of the camera are adjusted according to the actual site conditions to ensure that the target area can be fully covered. The camera must have all-weather shooting capabilities, including waterproof, dustproof, windproof and other characteristics to ensure normal operation under adverse weather conditions. At the same time, the camera must have infrared night vision function to ensure that clear images can be captured at night or in low light conditions. The camera is installed at a high place through a fixed bracket, and ensure that the shooting angle is good to avoid obstruction of the field of view. The four cameras monitor the sky of the observation area uninterruptedly around the clock.
[0031] Step S2: Identify the lightning channel image through the lightning recognition algorithm. When any camera recognizes the lightning phenomenon, all cameras in the camera group start shooting at the same time to record the lightning channel from different angles.
[0032] Specifically, the lightning recognition algorithm includes image preprocessing, feature extraction, feature matching and classification, dynamic detection and time domain analysis, multi-camera data fusion, alarm and data storage.
[0033] After the camera is finished shooting, the captured image is preprocessed, namely image denoising: using Gaussian filtering, mean filtering or median filtering to remove noise in the image; brightness adjustment: using histogram equalization technology to enhance the contrast of the image and highlight the lightning features; color space conversion: converting the image from RGB color space to HSV color space or grayscale image to facilitate subsequent feature extraction. Then lightning feature extraction is performed, namely brightness threshold segmentation: setting the brightness threshold, extracting the area in the image where the brightness exceeds the threshold. These areas may contain lightning channels; edge detection: using edge detection algorithms such as Sobel operator and Canny operator to detect significant edges in the image, and assist in determining the boundaries of lightning channels; morphological processing: removing noise points and small areas through morphological operations (such as expansion, corrosion, opening operation, closing operation), retaining the main lightning structure. Then feature matching classification is performed, namely feature matching: using template matching or shape-based feature matching technology, the extracted features are compared with predefined lightning morphological features. The matching degree can be evaluated by indicators such as structural similarity (SSIM) and normalized cross correlation (NCC); classifier application: use machine learning classifier (convolutional neural network) to classify the matching results to further confirm the existence of lightning phenomena. Then dynamic detection and time domain analysis, namely inter-frame difference method, are carried out: the fast-changing areas are extracted through the differential images of consecutive frames to identify dynamic lightning phenomena; time series analysis: time series analysis is performed on the detected lightning features to ensure that these features are consistent in time to exclude the interference of instantaneous light spots other than lightning. After that, multi-camera data is fused, namely perspective correction: geometric correction is performed on the images of each perspective according to the relative position and angle of the camera; feature fusion: the lightning features detected by each camera are fused to form complete lightning channel information; consistency verification: the consistency of the data of each camera is checked to confirm the actual occurrence and location of the lightning phenomenon. Finally, after identifying the lightning event, the system will generate an alarm and save the relevant data. That is, alarm triggering: when multiple cameras recognize lightning features at the same time and the feature matching degree exceeds the preset threshold, the system automatically triggers an alarm; data storage: all raw data and processing results in the recognition process are stored in the data center for subsequent analysis and 3D reconstruction. The lightning recognition algorithm flow chart is as follows Figure 2 shown.
[0034] Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the flight of the drone, the camera group continues to take pictures of the drone's position. By comparing the pictures of the drone's position with the pictures of the lightning channel, the drone control algorithm is used to adjust the drone's flight route, so that the drone reproduces the lightning channel and obtains the drone's flight trajectory data.
[0035] Specifically, the UAV control algorithm includes UAV initial position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis.
[0036] First, the initial position of the drone is set, that is, the starting point is set: the starting point of the drone is usually set at the starting position of the lightning channel or a known reference point. The selection of the starting point needs to consider the panoramic coverage of the lightning channel; flight parameter initialization: set the initial flight parameters, including flight altitude, speed and direction. These parameters can be adjusted according to the expected height and length of the lightning channel. Then real-time image acquisition and processing, that is, image acquisition: four fixed cameras collect the current position image of the drone in real time and compare it with the previously taken lightning channel image; image processing: pre-process the collected image to remove noise and enhance key features. This step is similar to the preprocessing in the lightning recognition algorithm, but the focus is on the comparison between the position of the drone and the lightning channel. After that, feature comparison and matching are performed, namely feature extraction: extracting features from the real-time collected images and the previous lightning channel images, mainly including the position of the UAV, the shape and brightness characteristics of the lightning channel; position matching: using image registration technology (such as template matching, feature point matching) to compare the current position of the UAV with the position of the lightning channel to determine the displacement of the UAV relative to the channel; then the control signal calculation is performed, namely position error calculation: based on the position matching result, the error between the current position of the UAV and the reference path of the lightning channel is calculated, including lateral error and longitudinal error; control signal generation: according to the error value, the control signal of the UAV is generated (adjusting the flight speed, direction and altitude). The proportional-integral-differential (PID) control algorithm is initially used for control, which is specifically divided into lateral control: adjusting the left and right displacement of the UAV to make it close to the center line of the lightning channel. Longitudinal control: adjusting the flight speed to keep the UAV in the forward direction of the lightning channel. Altitude control: adjusting the flight altitude of the UAV according to the height change of the lightning channel. Then the flight path is adjusted and corrected, that is, real-time adjustment: the UAV continuously adjusts the flight path according to the real-time updated control signal to correct the deviation from the lightning channel; path prediction and planning: use machine learning or optimization algorithms to predict the future path of the lightning channel, and plan the flight of the UAV in advance to improve the stability and accuracy of path tracking. Then data recording and feedback are carried out, that is, flight data recording: record the flight data of the UAV, including information such as position, speed, altitude and control signals. The recorded data is used for subsequent analysis and system optimization; real-time feedback: abnormal conditions during the flight (such as deviation from the path, signal loss) are fed back to the ground control center, and emergency measures are taken when necessary. Then the flight mission is completed and returned, that is, mission completion detection: when the UAV flies to the end of the lightning channel or reaches the preset mission completion conditions, the system automatically ends the flight mission; return path planning: plan the return path of the UAV to ensure its safe return to the starting point or designated landing point.Finally, data storage and analysis are performed, namely data storage: all data collected during the flight are stored in the database for subsequent analysis and optimization of the drone control algorithm; data analysis: flight data is analyzed to evaluate the performance of the control algorithm, including path tracking accuracy, control stability, and response speed. Based on the analysis results, the algorithm is further optimized.
[0037] Step S4: Use the UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.
[0038] Specifically, it includes position information and attitude acquisition: obtaining the real-time position and attitude data of the drone through sensors. Feature point extraction and matching: extracting key feature points in the flight path and matching them with the characteristics of the lightning channel. Three-dimensional point cloud generation: using the position data of the feature points to generate a three-dimensional point cloud of the lightning channel. Three-dimensional model reconstruction: constructing a three-dimensional model of the lightning channel based on the point cloud data. Error analysis and correction: analyzing the error between the model and the actual path, and correcting it to improve the accuracy of the model. Output and display: the final three-dimensional model of the lightning channel is displayed through visualization tools and used for lightning research, lightning protection engineering design, and the development of lightning warning systems, providing scientific data support for related fields.
Claims
1. A three-dimensional reconstruction method of lightning channels based on multiple cameras and drones, characterized in that: include: Step S1: Continuously monitor and shoot the target area through a camera group to capture lightning phenomena; Step S2: Identify the lightning channel image through the lightning recognition algorithm. When any camera recognizes the lightning phenomenon, all cameras in the camera group start shooting at the same time to record the lightning channel from different angles. Step S3: After the lightning event ends, the drone takes off from the predetermined starting point. During the flight of the drone, the camera group continues to take pictures of the drone's position. By comparing the pictures of the drone's position with the pictures of the lightning channel, the drone control algorithm is used to adjust the drone's flight route, so that the drone reproduces the lightning channel and obtains the drone's flight trajectory data. Step S4: Use the UAV flight trajectory data to perform three-dimensional reconstruction of the lightning channel.
2. The lightning channel three-dimensional reconstruction method according to claim 1, characterized in that: In step S1, the camera group includes four high-resolution cameras, and the four high-resolution cameras are respectively distributed in four corners of the target area. The four high-resolution cameras form a square observation network.
3. The lightning channel three-dimensional reconstruction method according to claim 2, characterized in that: In step S2, the steps of the lightning recognition algorithm include image preprocessing, feature extraction, feature matching and classification, dynamic detection and time domain analysis, multi-camera data fusion, alarm and data storage; the image preprocessing includes image denoising, brightness adjustment and color space conversion; the feature extraction includes brightness threshold segmentation, edge detection and morphological processing; The feature matching and classification include feature matching and classification; the dynamic detection and time domain analysis include frame difference method and time series analysis; The multi-camera data fusion includes perspective correction, feature fusion and consistency verification; The alarm and data storage include alarm triggering and data storage.
4. The lightning channel three-dimensional reconstruction method according to claim 3, characterized in that: In step S3, the UAV control algorithm includes UAV initial position setting, real-time image acquisition and processing, feature comparison and matching, control signal calculation, flight path adjustment and correction, data recording and feedback, flight mission completion and return, and data storage and analysis; the UAV initial position setting includes starting point setting and flight parameter initialization; the real-time image acquisition and processing includes image acquisition and image processing; The feature comparison and matching includes feature extraction and position matching; the control signal calculation includes position error calculation and control signal generation; the flight path adjustment and correction includes real-time adjustment and path prediction and planning; the data recording and feedback includes flight data recording and real-time feedback; the flight mission completion and return includes mission completion detection and return path planning; the data storage and analysis includes data storage and data analysis.
5. The lightning channel three-dimensional reconstruction method according to claim 4, characterized in that: In step S4, the three-dimensional reconstruction step includes obtaining the position information and posture of the UAV, feature point extraction and matching technology, three-dimensional point cloud generation and three-dimensional model reconstruction, error analysis and correction, and output and display.
Citation Information
Patent Citations
Rockfall motion parameter digital reduction method based on unmanned aerial vehicle vision
CN113804166A
Space observation and reconstruction method and system based on unmanned aerial vehicle
CN116957360A
Multi-unmanned aerial vehicle path planning method and system for three-dimensional reconstruction
CN117193364A
Unmanned aerial vehicle platform monocular trajectory reconstruction method
CN117519256A
Cited By
Dynamic discharge channel three-dimensional adaptive reconstruction method based on multiple high-speed cameras and related device
CN119579792A
