A method and system for quantitative estimation of precipitation based on drone video
Through drone video sensors and deep learning technology, combined with MobileNetV2 and BI-LSTM models, the problem of obtaining precipitation information in urban areas is solved, real-time and accurate perception of precipitation information by drones is realized, and flight safety and path planning efficiency are improved. .
Patent Information
- Application Number
- CN202411288264.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-09-14
AI Technical Summary
The number of rainfall stations in urban areas is sparse and the maintenance costs are high, making it difficult to provide high spatial resolution meteorological information. Remote sensing observations such as ground-based radar and satellite are affected by satellite revisit/radar scanning cycle, making it difficult to quickly change weather in the drone flight area. Accurate perception.
Using the drone's own vision sensor, combined with computer vision and deep learning technology, a quantitative precipitation estimation method based on drone video is established, and precipitation features are extracted and quantified using MobileNetV2 and BI-LSTM models, and real-time use of 4G/5G communication module Feedback precipitation information.
It has achieved self-sufficiency of drones for precipitation information in the flight area, and has the advantages of real-time, sensitivity and efficiency, which has improved the accuracy of safe flight and flight path planning of drones, and reduced the demand for high-quality precipitation data.
Smart Images

Figure CN118799767B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to but is not limited to the field of meteorological technology, and in particular relates to a precipitation quantitative estimation method and system based on unmanned aerial vehicle video. Background Art
[0002] In recent years, drones have played an increasingly important role in urban supervision, express delivery, military operations and other fields. Emerging applications such as drone takeout delivery and express delivery have become an important part of the development of urban low-altitude economy. Drone flight has extremely high requirements for weather conditions, especially the occurrence of precipitation and crosswinds, which can easily cause drones to shake, deviate, or even fall, causing unpredictable accidental damage. In particular, affected by human activities and micro-topography in urban areas, "heat island" and "rain island" phenomena occur frequently, and detailed and real-time weather information is increasingly important for drone flight, scheduling and planning. At present, the meteorological data used by drones mainly comes from the results released by conventional means (including ground meteorological stations, radar / satellite remote sensing observations). However, these means are difficult to meet the needs of drones for high-resolution precipitation data in urban areas.
[0003] Although observations based on surface rain gauges can provide real-time and accurate point precipitation information, the number of rain gauges in urban areas is sparse and the maintenance cost is high, making it difficult to provide meteorological information with high spatial resolution. At the same time, remote sensing observations such as ground-based radars and satellites, as the main means of meteorological observation, are affected by the satellite revisit / radar scanning cycle, and the acquisition and interpretation of observation data have a long delay, making it difficult to quickly provide weather change information.
[0004] In view of the above analysis, the technical problems that urgently need to be solved in the existing technology are: the number of rain gauges in urban areas is sparse and the maintenance cost is high, making it difficult to provide meteorological information with high spatial resolution; at the same time, ground-based radar, satellite and other remote sensing observations are the main means of meteorological observation. However, they are affected by the satellite revisit / radar scanning cycle, and the acquisition and interpretation of observation data have a long delay, making it difficult to quickly and accurately perceive changes in the area where the drone is located. Summary of the invention
[0005] In view of the problems existing in the prior art, the present invention provides a method and system for quantitative estimation of precipitation based on drone video. Taking the visual sensor of the drone as a breakthrough, computer vision, deep learning and other methods are used to establish a method for quantitative estimation of precipitation based on drone video. This method makes full use of the perception and computing resources carried by the drone itself, and realizes the perception of precipitation events in the surrounding area at the drone terminal. The present invention realizes the self-sufficiency of the drone in the precipitation information of the flight area, has the advantages of real-time, sensitivity and high efficiency, effectively alleviates the urgent demand for high-quality precipitation data, and provides technical support for improving the safety of drone flight, flight path planning, task management, etc.
[0006] The present invention is implemented as follows: a method for quantitatively estimating precipitation based on drone video, comprising:
[0007] S1, initialization of drone status; obtaining drone coordinates (including longitude, latitude, flight altitude) and video stream of drone camera;
[0008] S2, UAV video preprocessing performs downsampling on the UAV video. After sampling, the image resolution of the UAV video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation;
[0009] S3, precipitation image spatial feature extraction model: To prevent the UAV from excessive computation and energy consumption due to the large model, the present invention adopts the MobileNetV2 deep learning model as the basic framework for image spatial feature extraction;
[0010] S4, precipitation temporal variation feature extraction model: using the obtained image spatial features as input, the precipitation temporal variation features are mined through the BI-LSTM layer; BI-LSTM processes the forward and backward sequential data by using two independent LSTM networks;
[0011] S5, UAV video precipitation quantification model: four fully connected layers are used to integrate the precipitation spatial features with the precipitation temporal features, and linear regression is used to achieve the regression of precipitation intensity; after obtaining the precipitation intensity, the cumulative precipitation is obtained by multiplying the calculated rainfall intensity with the duration;
[0012] S6, model training dataset preparation: construct a set of precipitation drone video datasets for training and testing the constructed deep learning model;
[0013] S7, precipitation information release and management: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation situation.
[0014] Furthermore, S3 specifically includes:
[0015] (1) Use the imagenet public dataset to pre-train MobileNetV2;
[0016] (2) Fine-tune the last 40 layers of the MobileNetV2 network so that the model can learn more about the characteristics of precipitation in drone images;
[0017] (3) In order to ensure the real-time processing of the model, a series of tensor operations are performed using time series, that is, the MobileNet model is passed as a parameter to the TimeDistributed layer. The feature maps of different layers share weights;
[0018] Furthermore, each LSTM unit in S4 includes: input gate, output gate and forget gate to control the information flow; the forward LSTM processes the sequence from beginning to end, while the backward LSTM processes the sequence from beginning to end. The outputs of the two networks are then connected to produce the final prediction; at the same time, the overfitting problem that occurs during model training is solved by adding the Dropout layer and the Batch Normalized layer.
[0019] Another object of the present invention is to provide a system for quantitatively estimating precipitation based on drone video for implementing the method for quantitatively estimating precipitation based on drone video, comprising:
[0020] The drone status initialization module obtains the drone coordinates (including longitude, latitude, flight altitude) and the video stream of the drone camera;
[0021] The drone video preprocessing module performs downsampling on the drone video. After sampling, the image resolution of the drone video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation.
[0022] Image spatial feature extraction model building module: To prevent the UAV from excessive computing and energy consumption due to a large model, the MobileNetV2 deep learning model is used as the basic framework for image spatial feature extraction;
[0023] Temporal variation feature extraction model: Taking the obtained image spatial features as input, the temporal variation features of precipitation are mined through the BI-LSTM layer; BI-LSTM processes the forward and backward sequential data by using two independent LSTM networks;
[0024] The module for establishing the precipitation quantification model of drone video is as follows: four fully connected layers are used to integrate the spatial characteristics of precipitation with the temporal characteristics of precipitation, and linear regression is used to achieve the regression of precipitation intensity. After obtaining the precipitation intensity, the cumulative precipitation is obtained by multiplying the calculated rainfall intensity with the duration.
[0025] Model training data set production module: Build a set of precipitation drone video data sets for training and testing the constructed deep learning model;
[0026] Precipitation information release and management module: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation conditions.
[0027] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for quantitatively estimating precipitation based on drone video.
[0028] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method for quantitatively estimating precipitation based on drone video.
[0029] Another object of the present invention is to provide an information data processing terminal, which includes the above-mentioned precipitation quantitative estimation system based on drone video.
[0030] First, the video sensor carried by the drone of the present invention realizes the perception of precipitation, without the need for additional hardware equipment investment, and has a low cost; the precipitation information is interpreted in real time through video, so as to achieve self-sufficiency of precipitation information, omit the link of receiving the measurement results of conventional meteorological observation means, greatly improve the real-time and accuracy of precipitation information acquisition, thereby improving the efficiency and accuracy of drones in tasks scheduling, risk assessment, route planning and other operations; a new deep learning model is proposed, which combines CNN and BI-LSTM to effectively mine the spatiotemporal information of precipitation in drone videos, realize accurate calculation of rainfall intensity, is suitable for complex and diverse shooting backgrounds, and has strong stability and reliability.
[0031] In view of the limited computing and battery life resources of drones, the lightweight deep learning precipitation quantization proposed has small memory usage, low computing consumption, and is suitable for mobile use.
[0032] Second, the expected benefits and commercial value of the technical solution of the present invention after transformation are:
[0033] The present invention realizes the self-sufficiency of the drone in the surrounding precipitation information. The implementation of the invention does not require additional hardware installation and has the characteristics of low cost, simple implementation, and plug-and-play. Especially now that drones are playing an increasingly important role in daily life and economic development, the present invention plays an important and positive role in improving the safety of drone travel and promoting the development of the low-altitude economy.
[0034] This invention innovatively develops a quantitative estimation method for precipitation based on the video taken by the drone itself, providing support for the flight, planning, and decision-making of the drone. Since no additional hardware facilities are required, it has the advantages of low cost and real-time dynamics. Similar work has not yet appeared at home and abroad.
[0035] Precipitation has a huge impact on the flight safety of drones. The present invention solves the pain point of how to perceive precipitation information in the drone flight area in real time, dynamically and with high resolution.
[0036] Third, the technical solution of the present invention solves multiple technical problems existing in existing precipitation quantitative estimation methods and has achieved significant technical progress.
[0037] First, traditional precipitation measurement methods often rely on ground-based meteorological stations or radar equipment, but these methods are limited in terms of spatial coverage and real-time performance. This invention uses drone video to quantitatively estimate precipitation, effectively expanding the scope of precipitation monitoring and improving the real-time performance of data acquisition. Drones can flexibly fly to specific areas for observation, thereby providing more comprehensive and timely precipitation information.
[0038] Secondly, the present invention solves the problem of excessive computation and energy consumption of drones caused by overly large models by using the MobileNetV2 deep learning model as the basic framework for image spatial feature extraction. The MobileNetV2 model is lightweight and can reduce computational complexity and energy consumption while maintaining high performance, making it very suitable for running on resource-constrained drone platforms.
[0039] In addition, the present invention further improves the accuracy of quantitative estimation of precipitation by combining the BI-LSTM layer to mine the temporal variation characteristics of precipitation. BI-LSTM can process forward and backward sequential data, effectively capture the time dependency of the precipitation process, and enable the model to better understand and predict the dynamic changes of precipitation.
[0040] Finally, the present invention publishes and manages precipitation information in real time through the 4G / 5G communication module, providing timely and accurate precipitation data support for the decision-making center. This helps the decision-making center take targeted measures according to the precipitation situation, such as flood control and drainage, agricultural irrigation, etc., to ensure social security and agricultural production.
[0041] In summary, the technical solution of the present invention realizes accurate quantitative estimation of precipitation and real-time information release by combining drone video, deep learning model and communication technology, bringing significant technological progress to the field of meteorological monitoring and emergency management. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a flow chart of a method for quantitatively estimating precipitation based on drone video provided by an embodiment of the present invention;
[0043] Figure 2 is a structural diagram of a precipitation quantitative estimation system based on drone video provided by an embodiment of the present invention;
[0044] Figure 3This is a schematic diagram of some precipitation scenes in the data set provided by the embodiment of the present invention.
[0045] Figure 4 is a structural diagram of a deep learning model provided by an embodiment of the present invention;
[0046] Figure 5 It is a test effect diagram provided by an embodiment of the present invention;
[0047] Figure 6 Schematic diagram of the Parsivel raindrop spectrometer structure for comparison of precipitation measurement accuracy by drones;
[0048] Figure 7 is a scatter plot of rainfall;
[0049] Figure 8 This is the prediction effect curve of a heavy rainfall event occurring in a certain area on May 17, 2024.
[0050] Fig. 9 This is the prediction effect curve of a heavy rainfall event occurring in a certain area on June 8, 2024.
[0051] Fig.10 This is the prediction effect curve of a heavy rainfall event occurring in a certain area on June 28, 2024. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0053] The following are two specific embodiments for illustrating the application scenarios of the method for quantitatively estimating precipitation based on drone video proposed in the present invention:
[0054] Example 1: Flood warning and monitoring
[0055] In a certain area, due to continuous heavy rains, the river water level rises rapidly, and there is a risk of flooding. In order to monitor and warn of floods in real time, the local emergency management department decides to adopt the quantitative precipitation estimation method based on drone video of the present invention.
[0056] 1. The drone was sent to the flood risk area for aerial video monitoring. After the drone status was initialized, the precise latitude and longitude coordinates and flight altitude were obtained, and the video stream from the drone camera was received.
[0057] 2. The UAV video preprocessing module downsamples the received video and reduces the image resolution to 300×300 to reduce the energy consumption of subsequent image processing and calculation, while retaining sufficient image information for precipitation estimation.
[0058] 3. The image spatial feature extraction model uses the MobileNetV2 deep learning model to extract the characteristics of precipitation in drone images. The temporal change feature extraction model further mines the temporal change characteristics of precipitation through the BI-LSTM layer.
[0059] 4. The precipitation quantification model combines the spatial characteristics of precipitation with the temporal characteristics of precipitation, and uses linear regression to achieve the regression of precipitation intensity. The cumulative precipitation is obtained by calculating the rainfall intensity and duration.
[0060] 5. The drone uses the 4G / 5G communication module to feed back real-time information such as location, precipitation intensity, and precipitation to the emergency management department. Based on this information, the emergency management department can accurately judge the flood risk and take timely warning and response measures, such as evacuating residents and strengthening embankments.
[0061] Example 2: Agricultural irrigation management
[0062] In a certain farmland area, in order to reasonably arrange irrigation plans and improve water resource utilization efficiency, farmers decided to adopt the precipitation quantitative estimation method based on drone video of the present invention.
[0063] 1. The drone was flying over the farmland for monitoring. After initialization, the precise coordinates and video stream of the drone were obtained.
[0064] 2. The drone video is downsampled through the preprocessing module to meet the subsequent image processing and computing requirements.
[0065] 3. The MobileNetV2 deep learning model is used to extract the spatial features of precipitation in the image, and the temporal variation characteristics of precipitation are mined through the BI-LSTM layer.
[0066] 4. The precipitation intensity and cumulative precipitation in the farmland area are calculated through the precipitation quantification model.
[0067] 5. Farmers formulate reasonable irrigation plans based on the precipitation information fed back by drones, combined with factors such as soil moisture in farmland and crop water requirements. This can not only avoid the waste of water resources, but also ensure that crops get the right amount of water supply and improve agricultural production efficiency.
[0068] Through the above two embodiments, it can be seen that the precipitation quantitative estimation method based on drone video of the present invention has broad application prospects and practical value in the fields of flood warning and monitoring, agricultural irrigation management, etc.
[0069] like Figure 1 As shown, the precipitation quantitative estimation method based on drone video provided by the embodiment of the present invention includes:
[0070] S1, initialization of drone status; obtaining drone coordinates (including longitude, latitude, flight altitude) and video stream of drone camera;
[0071] S2, UAV video preprocessing performs downsampling on the UAV video. After sampling, the image resolution of the UAV video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation;
[0072] S3, precipitation image spatial feature extraction model: To prevent the UAV from excessive computation and energy consumption due to the large model, the present invention adopts the MobileNetV2 deep learning model as the basic framework for image spatial feature extraction;
[0073] S4, precipitation temporal variation feature extraction model: using the obtained image spatial features as input, the precipitation temporal variation features are mined through the BI-LSTM layer; BI-LSTM processes the forward and backward sequential data by using two independent LSTM networks;
[0074] S5, UAV video precipitation quantification model: four fully connected layers are used to integrate the precipitation spatial features with the precipitation temporal features, and linear regression is used to achieve the regression of precipitation intensity; after obtaining the precipitation intensity, the cumulative precipitation is obtained by multiplying the calculated rainfall intensity with the duration;
[0075] S6, model training dataset preparation: construct a set of precipitation drone video datasets for training and testing the constructed deep learning model;
[0076] S7, precipitation information release and management: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation situation.
[0077] like Figure 2 As shown, the precipitation quantitative estimation system based on drone video provided by the embodiment of the present invention includes:
[0078] The drone status initialization module obtains the drone coordinates (including longitude, latitude, flight altitude) and the video stream of the drone camera;
[0079] The drone video preprocessing module performs downsampling on the drone video. After sampling, the image resolution of the drone video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation.
[0080] Image spatial feature extraction model building module: To prevent the UAV from excessive computing and energy consumption due to a large model, the MobileNetV2 deep learning model is used as the basic framework for image spatial feature extraction;
[0081] Temporal variation feature extraction model: Taking the obtained image spatial features as input, the temporal variation features of precipitation are mined through the BI-LSTM layer; BI-LSTM processes the forward and backward sequential data by using two independent LSTM networks;
[0082] The module for establishing the precipitation quantification model of drone video is as follows: four fully connected layers are used to integrate the spatial characteristics of precipitation with the temporal characteristics of precipitation, and linear regression is used to achieve the regression of precipitation intensity. After obtaining the precipitation intensity, the cumulative precipitation is obtained by multiplying the calculated rainfall intensity with the duration.
[0083] Model training data set production module: Build a set of precipitation drone video data sets for training and testing the constructed deep learning model;
[0084] Precipitation information release and management module: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation conditions.
[0085] The method for quantitatively estimating precipitation based on drone video provided by an embodiment of the present invention includes:
[0086] Step 1. Initialize the drone status;
[0087] Get the coordinates of the drone (including longitude, latitude, and flight altitude) and the video stream of the drone camera;
[0088] Step 2. UAV video precipitation quantification model;
[0089] Due to the limited computing resources of the drone itself, if the precipitation quantification model is too large, it will take up a large storage space and consume too much energy to run, causing the drone to be overloaded, affecting the drone's endurance and the execution and analysis of other tasks. In response to the above problems, the lightweight design and implementation of the quantification model has become a key consideration, aiming to make it suitable for mobile terminal (drone) use and deployment.
[0090] Step 2.1 Drone video preprocessing
[0091] The drone video is downsampled. After sampling, the image resolution of the drone video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation.
[0092] Step 2.2 Precipitation image spatial feature extraction model
[0093] To prevent the UAV from excessive computing and energy consumption due to the large model, the present invention adopts the MobileNetV2 deep learning model as the basic framework for image spatial feature extraction. Specifically,
[0094] (1) Use the imagenet public dataset to pre-train MobileNetV2;
[0095] (2) Fine-tune the last 40 layers of the MobileNetV2 network so that the model can learn more about the characteristics of precipitation in drone images;
[0096] (3) In order to ensure the real-time processing of the model, a series of tensor operations are performed using time series, that is, the MobileNet model is passed as a parameter to the TimeDistributed layer. The feature maps of different layers share weights;
[0097] Step 2.3 Precipitation temporal variation feature extraction model
[0098] Taking the image spatial features obtained in step 2.2 as input, the temporal variation characteristics of precipitation are mined through the BI-LSTM layer. BI-LSTM processes the forward and backward sequential data by using two independent LSTM networks. Each LSTM unit includes: input gate, output gate and forget gate to control the flow of information. The forward LSTM processes the sequence from beginning to end, while the backward LSTM processes the sequence from beginning to end. The outputs of the two networks are then connected to produce the final prediction. At the same time, the overfitting problem that occurs during model training is solved by adding the Dropout layer and the Batch Normalized layer.
[0099] Step 2.4 UAV video precipitation quantification model
[0100] (1) Use four fully connected layers to merge the precipitation spatial features obtained in step 2.2 with the precipitation temporal features obtained in step 2.3;
[0101] (2) Use linear regression to achieve the regression of precipitation intensity;
[0102] (3) After obtaining the precipitation intensity, multiply the rainfall intensity calculated in step (2) by the duration to obtain the cumulative precipitation.
[0103] Step 2.5: Create training dataset for the model
[0104] In order to train and test the deep learning models constructed in step 2.2, step 2.3, and step 2.4, the present invention constructs a set of precipitation drone video data sets. The data set includes a total of 6,000 precipitation videos shot by drones in different rainfall scenes (rainfall intensity range: 0 mm / h -130 mm / h). The video resolution is: 1280×720, the frame rate is 15 frames / second, the video is saved in .avi format, each video is 5 seconds long, and 25 frames are selected for model training (5 frames are randomly selected per second). This not only ensures the accuracy of the results, but also reduces the computing pressure. Some precipitation scenes in the data set are as follows: Figure 3 shown.
[0105] Step 3. Release and management of precipitation information
[0106] Through its own 4G / 5G communication module, it feeds back information such as location, precipitation intensity, and precipitation amount to the decision-making center, which then takes the next step based on the precipitation situation.
[0107] Preferably, the method for quantitatively estimating precipitation based on drone video provided in an embodiment of the present invention includes:
[0108] (1) Configure the geographic coordinates corresponding to the survey starting point in the drone;
[0109] (2) The UAV flies to the measurement starting point;
[0110] (3) Use drones to detect precipitation intensity.
[0111] (4) Drones will release precipitation information.
[0112] In steps (1) and (2), the configured geographic coordinates can be geographic coordinate system, ECEF coordinate system, local NED coordinate system, etc.
[0113] In step (3), the drone uses the video precipitation quantification model to measure the rainfall intensity and rainfall amount at the current location.
[0114] In step (4), the drone publishes the precipitation information calculated from the video and the corresponding location information. Repeat the above steps to realize the perception of precipitation meteorological conditions along the flight path of the drone, such as Figure 4 Shown is a diagram of the deep learning model structure.
[0115] In actual application, the precipitation intensity is calculated every 30 seconds and the accumulated rainfall is calculated every 2 minutes. The test results are as follows: Figure 5 shown.
[0116] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of a method for quantitatively estimating precipitation based on drone videos.
[0117] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a method for quantitatively estimating precipitation based on drone videos.
[0118] An application embodiment of the present invention provides an information data processing terminal, which includes a precipitation quantitative estimation system based on drone video.
[0119] The present invention measured precipitation at Nanjing University (Xianlin Campus), Nanjing Jiangning National Climate Reference Station, Nanjing Institute of Technology and other places. During the measurement process, the drone flew at an altitude of <100m. At the same time, meteorological professional precipitation measurement equipment such as Parsivel raindrop spectrometer was used (such as Figure 6 Synchronous observations were performed to compare the accuracy of the present invention.
[0120] Take the heavy rainfall events that occurred in a certain area on May 17, June 8, and June 28, 2024 as an example. During the above three rainfall events, the maximum rainfall intensity was 110 mm / hor. Figure 8 , Fig. 9 , Fig.10 As shown, by comparing the precipitation measurement method based on drone video of the present invention with Parsivel on the ground, it is found that the cumulative rainfall estimation accuracy of the present invention is greater than 90%.
[0121] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.
[0122] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. A method for quantitatively estimating precipitation based on drone video, characterized in that: include: S1: Initialize the drone status; obtain the drone coordinates and the video stream of the drone camera; S2: UAV video preprocessing, downsampling the UAV video. After sampling, the image resolution of the UAV video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation. S3: Precipitation image spatial feature extraction model: The MobileNetV2 deep learning model is used as the basic framework for image spatial feature extraction; S4: Temporal variation feature extraction model of precipitation: Taking the obtained image spatial features as input, the temporal variation features of precipitation are mined through the BI-LSTM layer; BI-LSTM processes forward and backward sequential data by using two independent LSTM networks; S5: UAV video precipitation quantification model: Four fully connected layers are used to integrate the precipitation spatial features with the precipitation temporal features, and linear regression is used to achieve the regression of precipitation intensity. After obtaining the precipitation intensity, the calculated precipitation intensity is multiplied by the duration to obtain the cumulative precipitation. S6: Model training dataset preparation: A precipitation drone video dataset is constructed for training and testing the constructed deep learning model; S7: Release and management of precipitation information: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation situation.
2. The method for quantitatively estimating precipitation based on drone video according to claim 1, characterized in that: S3 specifically includes: (1) Use the Imagenet public dataset to pre-train MobileNetV2; (2) Fine-tune the last 40 layers of the MobileNetV2 network so that the model can learn more about the characteristics of precipitation in drone images; (3) In order to ensure the real-time processing of the model, a series of tensor operations are performed using time series. That is, the MobileNetV2 model is passed as a parameter to the TimeDistributed layer so that the feature maps of different layers share weights.
3. The method for quantitatively estimating precipitation based on drone video according to claim 1, characterized in that: Each LSTM unit in S4 consists of: Input gate, output gate and forget gate to control the flow of information; The forward LSTM processes the sequence from beginning to end, while the backward LSTM processes the sequence from end to beginning, and then concatenates the outputs of the two networks to produce the final prediction; The overfitting problem that occurs during model training is solved by adding Dropout layer and Batch Normalization layer.
4. The method for quantitatively estimating precipitation based on drone video according to claim 1, characterized in that: The fully connected layers used in S5 include: The first fully connected layer: receives the spatial and temporal features of precipitation as input and performs preliminary fusion; The second fully connected layer: further fuses features and extracts higher-level features; The third fully connected layer: reduces the dimension and compactly represents the fused features; The fourth fully connected layer: finally outputs the regression result of precipitation intensity.
5. The method for quantitatively estimating precipitation based on drone video according to claim 1, characterized in that: The release and management of precipitation information in S7 includes: Through the drone’s 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center in real time; The decision-making center conducts data analysis and processing based on the precipitation information received, and takes corresponding response measures, including launching emergency plans and dispatching resources.
6. A system for quantitatively estimating precipitation based on drone video that implements the method for quantitatively estimating precipitation based on drone video as claimed in any one of claims 1 to 3, characterized in that: include: The drone status initialization module obtains the drone coordinates and the video stream of the drone camera; The drone video preprocessing module performs downsampling on the drone video. After sampling, the image resolution of the drone video is reduced to 300×300. Downsampling reduces the energy consumption of subsequent image processing and calculation. Image spatial feature extraction model building module: To prevent the UAV from excessive computing and energy consumption due to a large model, the MobileNetV2 deep learning model is used as the basic framework for image spatial feature extraction; Temporal variation feature extraction model building module: using the obtained image spatial features as input, the temporal variation features of precipitation are mined through the BI-LSTM layer; BI-LSTM processes forward and backward sequential data by using two independent LSTM networks; The module for establishing the precipitation quantification model of drone videos: four fully connected layers are used to integrate the spatial characteristics of precipitation with the temporal characteristics of precipitation, and linear regression is used to achieve the regression of precipitation intensity. After obtaining the precipitation intensity, the cumulative precipitation is obtained by multiplying the calculated precipitation intensity with the duration. Model training data set production module: Build a set of precipitation drone video data sets for training and testing the constructed deep learning model; Precipitation information release and management module: Through its own 4G / 5G communication module, the location, precipitation intensity, and precipitation amount information are fed back to the decision-making center, and the decision-making center takes the next step based on the precipitation conditions.
7. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method for quantitatively estimating precipitation based on drone video as described in any one of claims 1 to 3.
8. A computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the method for quantitatively estimating precipitation based on drone video as described in any one of claims 1 to 3.
9. An information data processing terminal, comprising the precipitation quantitative estimation system based on drone video according to claim 6.
Citation Information
Patent Citations
Short-time near rainfall forecasting method and system and computer readable storage medium
CN111145245A
Radar quantitative rainfall estimation method based on deep learning
CN113791415A
Satellite quantitative rainfall estimation method
CN115542430A
Multi-modal data fusion and uncertainty estimation water level prediction method and system
CN116702090A
