Forest and grass fire slash recovery monitoring method based on unmanned aerial vehicle and aircraft vision
By using unmanned aerial vehicles equipped with multispectral cameras and image processing technology, the problems of low efficiency and poor accuracy in monitoring the restoration of forest and grass fire scars in traditional methods have been solved, and intelligent monitoring and prediction of the restoration of forest and grass fire scars has been achieved.
Patent Information
- Application Number
- CN202510527036.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-25
- Publication Date
- 2025-09-05
AI Technical Summary
Traditional methods for monitoring ecosystem recovery after forest and grassland fires rely on ground surveys and satellite remote sensing, which have problems such as low efficiency, poor accuracy, and significant influence from weather, making it difficult to achieve real-time and efficient monitoring and evaluation.
By using unmanned aerial vehicles equipped with multispectral cameras, combined with image preprocessing, image segmentation, feature enhancement and forest and grass fire site recovery time prediction models, intelligent monitoring and prediction of forest and grass fire sites are achieved through computer vision and generative adversarial networks.
The monitoring and prediction accuracy and efficiency of forest and grassland fire site restoration areas have been improved, and intelligent management and scientific guidance of forest and grassland fire site restoration have been achieved.
Smart Images
Figure CN120599459A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing information technology, and in particular to a forest and grass fire scar restoration monitoring method based on unmanned aerial vehicle (UAV) vision. Background Art
[0002] With global climate change and intensified human activities, frequent forest and grassland fires have become a common natural disaster. These fires not only severely damage the ecological environment but also cause enormous losses to human life and property. To assess and restore damaged ecosystems after fires, Chinese scientists and engineers have been exploring more efficient and intelligent methods.
[0003] Disadvantages of existing technologies: Traditional methods rely primarily on ground surveys and satellite remote sensing. However, ground surveys are typically labor-intensive and time-consuming, making it difficult to cover large areas of affected areas. While satellite remote sensing can provide monitoring data over a wide range, its low spatial and temporal resolution makes it difficult to obtain detailed ground information in real time. Furthermore, satellite data is significantly affected by weather and does not function properly in cloudy and rainy conditions. These limitations of traditional methods significantly reduce the efficiency and accuracy of post-fire ecosystem recovery monitoring. Summary of the Invention
[0004] The present invention provides a forest and grass fire scar restoration monitoring method based on UAV aircraft vision, which realizes intelligent monitoring and prediction of forest and grass fire scar restoration areas.
[0005] To achieve the above-mentioned purpose, the present invention provides a method for monitoring forest and grass fire scar restoration based on UAV aircraft vision, the key of which is to include the following steps:
[0006] Step 1: Construct a forest and grass fire scar restoration monitoring system, which includes an unmanned aerial vehicle (UAV) equipped with a multispectral camera. The multispectral camera is sequentially connected to an image preprocessing module, an image segmentation module, a feature enhancement module, and a forest and grass fire scar restoration time prediction model FRTime.
[0007] Step 2: The UAV aircraft carries a multispectral camera to shoot video stream remote sensing data a of the forest area in real time, and converts the video stream remote sensing data a into image frame data b through a computer vision calculation method, and then transmits it to the image preprocessing module;
[0008] Step 3: The image preprocessing module performs a preprocessing operation on the image frame data b to obtain a standard image c, and passes it to the image segmentation module;
[0009] Step 4: The image segmentation module uses the object segmentation algorithm to segment the forest and grass restoration area in the standard image c to obtain the forest and grass restoration area image d, and passes it to the feature enhancement module;
[0010] Step 5: The feature enhancement module uses the generative adversarial network (GAN) to perform feature enhancement on the forest and grassland restoration area image d to obtain an enhanced image e, and passes it to the forest and grassland fire scar recovery time prediction model FRTime;
[0011] Step 6: The forest and grass fire site recovery time prediction model FRTime performs forest and grass recovery prediction on the enhanced image e, and predicts the normalized forest and grass recovery time index T of the enhanced image e.
[0012] Through the above design, the present invention utilizes the high maneuverability, high-resolution imaging capability of UAV aircraft and the light wave capture capability of multispectral cameras to collect image data of forest and grass areas in real time, and improves image quality through image preprocessing operations, and then uses the object segmentation algorithm to obtain more accurate and specific forest and grass restoration areas, and uses the generative adversarial network method to enhance the forest and grass features of the segmented forest and grass restoration areas, thereby achieving the purpose of enhancing forest and grass image features; finally, the trained forest and grass fire site recovery time prediction model FRTime is used to predict the normalized forest and grass recovery time index of the corresponding image, thereby realizing intelligent monitoring and prediction of the recovery status of the forest and grass fire site recovery area.
[0013] This method effectively improves the prediction accuracy and efficiency of the restoration status of forest and grassland fire scar restoration areas.
[0014] Preferably, in step 2, the UAV captures aerial video of the forest and grassland area in real time in a vertical shooting manner, and uses the VedioCapture video capture method in the cross-platform computer vision library OpenCV to extract static image frame data b from the video stream remote sensing data a at time intervals. The extraction function of the image frame sequence is defined as follows:
[0015] a={a1,a2,...,a n}=f(b)
[0016] Where a is the video frame sequence; b is the extracted image frame sequence; and f represents the function for extracting images from the video.
[0017] The video stream captured by the drone in real time is converted into image frames through computer vision calculation methods to ensure the smooth subsequent use of the data.
[0018] Preferably, in step 3, the preprocessing operation includes removing noise in the image frame data b by using Gaussian filtering and adjusting the image size. The preprocessing formula is as follows:
[0019] I denoised =b*G
[0020] I resized =resize(I denoised ,224,224)
[0021] Where G is a Gaussian filter; I denoised is the denoised image; I resized is the resized image, that is, the standard image c.
[0022] The collected original images were preprocessed, including using Gaussian filtering to remove noise in the images and adjusting the image size to 224×224 pixels for subsequent processing.
[0023] Preferably, in step 4, the image segmentation module uses a multi-object segmentation algorithm to segment the forest and grass restoration area in the standard image c. The function of the multi-object segmentation algorithm is defined as:
[0024] d=SegMent(c)
[0025] Among them, SegMent is the method for segmenting everything.
[0026] Preferably, in step 5, the feature enhancement module uses an improved generative adversarial network Real-RSRGAN to perform feature enhancement on the forest and grass restoration area image d, and the feature enhancement function is defined as follows:
[0027] e=Real_RSRGAN(d)
[0028] Among them, Real_RSRGAN is an image feature enhancement method.
[0029] The improved generative adversarial network Real-RSRGAN based on the generative adversarial network is used to enhance the features of the more accurate segmented fire scar restoration area image, thereby enhancing the forest and grass features of the forest and grass restoration area image.
[0030] Preferably, in step 6, the forest and grass fire site recovery time prediction model FRTime performs a deep learning artificial intelligence model operation on the enhanced image e and the normalized vegetation index NDVI corresponding to the image to predict the normalized forest and grass recovery time index T of the enhanced image e. The prediction formula is as follows:
[0031] T=FRTime(e,NDVI)
[0032]
[0033] Among them, FRTime is the normalized forest and grassland recovery time index prediction method, NIR is the near-infrared band reflectance of the enhanced image e, and Red is the red band reflectance of the enhanced image e.
[0034] Preferably, the forest and grass fire site recovery time prediction model FRTime is trained by the following steps:
[0035] Step A1: The UAV aircraft carries a multispectral camera to obtain a change image of the forest fire scar restoration area, obtains an original image set, and transmits it to the preprocessing module;
[0036] Step A2: the preprocessing module performs a preprocessing operation on the original image set to obtain a standard image set, and passes it to the image segmentation module;
[0037] Step A3: The image segmentation module uses the object segmentation algorithm to accurately segment the standard image set and extract fine images of the forest and grassland restoration area; combining the image's near-infrared band reflectance NIR, red band reflectance Red and time record, calculates the normalized vegetation index NDVI and normalized forest and grassland restoration time index T of the corresponding image, and organizes these data into the data set TimeData;
[0038] Step A4: Construct an initial model for predicting the recovery time of forest and grass fire scars;
[0039] Step A5: Use the data set TimeData to train the initial model for predicting the recovery time of forest and grass fire scars. When the training cycle is reached, save the model parameters, end the training, and obtain the trained forest and grass fire scar recovery time prediction model FRTime.
[0040] Unmanned aerial vehicles (UAVs) were used to collect images, near-infrared band reflectance, and red band reflectance of the forest and grassland restoration area from the time point of the fire site to the time point of complete recovery. The normalized vegetation index NDVI and the normalized forest and grassland recovery time index T were calculated based on the near-infrared band reflectance, red band reflectance, and image time records, and a forest and grassland recovery time dataset TimeData was constructed for model training.
[0041] Based on the NDVI, T and segmented images in the dataset TimeData as the training data of the model, the forest and grass fire scar recovery time prediction model FRTime is trained to input the forest and grass recovery area image to predict the forest and grass recovery time index T, so that the UAV can capture the UAV video in real time and automatically monitor the recovery of the forest and grass fire scar.
[0042] Preferably, in step A3, based on the purpose of forest and grass fire site recovery monitoring, a custom normalized forest and grass time index T is used to represent the normalized recovery time at a certain moment from the occurrence of the fire to complete recovery; the calculation formula of the normalized recovery time is as follows:
[0043]
[0044] Among them, t i Indicates the recovery time (days) corresponding to the current image, t total The number of days from the time a fire occurs to full recovery in that area is represented by the number of days. T is the normalized forest and grassland recovery time index, ranging from 0 to 1, indicating the proportion of the recovery time corresponding to the current image to the total recovery time. This metric is used to monitor and assess the recovery progress of burned areas.
[0045] Preferably, in step A5, the initial model for predicting the recovery time of forest and grass fire scars uses the VGG16 network model as a feature extractor, extracts the image data in the dataset TimeData as feature vectors, concatenates the extracted image features with the features mapped by the NDVI value, and performs prediction at the fully connected layer; the calculation expression is as follows:
[0046] F=VGG16(I resized )
[0047] F concat =concat(F,NDVI)
[0048] T predicted =FC(F concat )
[0049] Among them, F is the feature vector extracted by the VGG16 network. VGG16 is a convolutional neural network model for feature extraction. Concat is the concatenation operation that connects two vectors. concat is the new vector after concatenation of F and NDVI, FC is the fully connected layer, T predicted is the predicted normalized forest and grassland recovery time index T;
[0050] During the model training phase, the optimizer was set to Adam, the loss function used binary cross entropy, the training epoch was 300, and the validation data ratio was 0.2.
[0051] The beneficial effects of the present invention are as follows: the present invention can perceive the forest and grass fire site restoration area using only real-time videos taken by UAV aircraft, and predict the normalized forest and grass recovery time index of the forest and grass restoration area, laying the foundation for more in-depth artificial intelligence target detection, time reasoning and intelligent perception of forest and grass; it realizes the intelligent management of forest and grass fire site restoration, and provides scientific guidance and basis for manual assistance in post-disaster site restoration. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 It is a schematic diagram of the process of the present invention;
[0053] Figure 2 This is a structural block diagram of the forest and grass fire site restoration monitoring system in the embodiment;
[0054] Figure 3 This is a network structure diagram of the forest and grass fire site recovery time prediction model in the embodiment. DETAILED DESCRIPTION
[0055] The present invention will be further described in detail below with reference to the accompanying drawings and specific examples. The following examples or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0056] like Figure 1 A method for monitoring forest and grass fire scar restoration based on UAV aircraft vision is shown, comprising the following steps:
[0057] Step 1: Construct a forest and grass fire scar restoration monitoring system, which includes an unmanned aerial vehicle (UAV) equipped with a multispectral camera. The multispectral camera is sequentially connected to an image preprocessing module, an image segmentation module, a feature enhancement module, and a forest and grass fire scar restoration time prediction model FRTime, as shown in the following example: Figure 2 As shown;
[0058] Step 2: The UAV aircraft carries a multispectral camera to shoot video stream remote sensing data a of the forest area in real time, and converts the video stream remote sensing data a into image frame data b through a computer vision calculation method, and then transmits it to the image preprocessing module;
[0059] Step 3: The image preprocessing module performs a preprocessing operation on the image frame data b to obtain a standard image c, and passes it to the image segmentation module;
[0060] Step 4: The image segmentation module uses the object segmentation algorithm to segment the forest and grass restoration area in the standard image c to obtain the forest and grass restoration area image d, and passes it to the feature enhancement module;
[0061] Step 5: The feature enhancement module uses the generative adversarial network (GAN) to perform feature enhancement on the forest and grassland restoration area image d to obtain an enhanced image e, and passes it to the forest and grassland fire scar recovery time prediction model FRTime;
[0062] Step 6: The forest and grass fire site recovery time prediction model FRTime performs forest and grass recovery prediction on the enhanced image e, and predicts the normalized forest and grass recovery time index T of the enhanced image e.
[0063] In step 2, the UAV captures aerial video of the forest and grassland area in real time in a vertical shooting manner, and uses the VedioCapture video capture method in the cross-platform computer vision library OpenCV to extract static image frame data b from the video stream remote sensing data a at time intervals. The extraction function of the image frame sequence is defined as follows:
[0064] a={a1,a2,...,a n}=f(b)
[0065] Where a is the video frame sequence; b is the extracted image frame sequence; and f represents the function for extracting images from the video.
[0066] In step 3, the preprocessing operation includes removing noise from the image frame data b by using Gaussian filtering and adjusting the image size. The preprocessing formula is as follows:
[0067] I denoised =b*G
[0068] I resized =resize(I denoised ,224,224)
[0069] Where G is a Gaussian filter; I denoised is the denoised image; I resized is the resized image, that is, the standard image c.
[0070] In step 4, the image segmentation module uses the all-things segmentation algorithm to segment the forest and grass restoration area in the standard image c. The function of the all-things segmentation algorithm is defined as:
[0071] d=SegMent(c)
[0072] Among them, SegMent is the method for segmenting everything.
[0073] In step 5, the feature enhancement module uses the improved generative adversarial network Real-RSRGAN to perform feature enhancement on the forest and grass restoration area image d. The feature enhancement function is defined as follows:
[0074] e=Real_RSRGAN(d)
[0075] Among them, Real_RSRGAN is an image feature enhancement method.
[0076] In step 6, the forest and grass fire scar recovery time prediction model FRTime performs a deep learning artificial intelligence model operation on the enhanced image e and the normalized vegetation index NDVI corresponding to the image to predict the normalized forest and grass recovery time index T of the enhanced image e. The prediction formula is as follows:
[0077] T=FRTime(e,NDVI)
[0078]
[0079] Among them, FRTime is the normalized forest and grassland recovery time index prediction method, NIR is the near-infrared band reflectance of the enhanced image e, and Red is the red band reflectance of the enhanced image e.
[0080] The forest and grass fire site recovery time prediction model FRTime is trained by the following steps:
[0081] Step A1: The UAV aircraft is equipped with a multispectral camera to obtain images of changes in the forest fire scar restoration area. The UAV is set to an altitude of 80 meters during the flight mission and the shooting angle is set to "vertical to the ground". Every day, images of a single forest and grass restoration area from scar to complete restoration are recorded to obtain the original image set and pass it to the preprocessing module;
[0082] Step A2: the preprocessing module performs a preprocessing operation on the original image set to obtain a standard image set, and passes it to the image segmentation module;
[0083] Step A3: The image segmentation module uses the object segmentation algorithm to accurately segment the standard image set and extract fine images of the forest and grassland restoration area; combining the image's near-infrared band reflectance NIR, red band reflectance Red and time record, calculates the normalized vegetation index NDVI and normalized forest and grassland restoration time index T of the corresponding image, and organizes these data into the data set TimeData;
[0084] Step A4: Construct an initial model for predicting the recovery time of forest and grass fire scars;
[0085] Step A5: Use the data set TimeData to train the initial model for predicting the recovery time of forest and grass fire scars. When the training cycle is reached, save the model parameters, end the training, and obtain the trained forest and grass fire scar recovery time prediction model FRTime.
[0086] In step A3, based on the purpose of forest and grassland fire site recovery monitoring, a custom normalized forest and grassland time index T is used to represent the normalized recovery time at a certain moment from the occurrence of the fire to complete recovery. The calculation formula of the normalized recovery time is as follows:
[0087]
[0088] Among them, t i Indicates the recovery time (days) corresponding to the current image, t total The number of days from the time a fire occurs to full recovery in that area is represented by the number of days. T is the normalized forest and grassland recovery time index, ranging from 0 to 1, indicating the proportion of the recovery time corresponding to the current image to the total recovery time. This metric is used to monitor and assess the recovery progress of burned areas.
[0089] In step A5, Figure 3 As shown in the figure, the initial model for predicting the recovery time of forest and grass fire scars uses the VGG16 network model as a feature extractor, extracts the image data in the dataset TimeData as feature vectors, concatenates the extracted image features with the features mapped by the NDVI value, and performs prediction in the fully connected layer; the calculation expression is as follows:
[0090] F=VGG16(I resized )
[0091] F concat =concat(F,NDVI)
[0092] T predicted =FC(F concat )
[0093] Among them, F is the feature vector extracted by the VGG16 network. VGG16 is a convolutional neural network model for feature extraction. Concat is the concatenation operation that connects two vectors. concat is the new vector after concatenation of F and NDVI, FC is the fully connected layer, T predicted is the predicted normalized forest and grassland recovery time index T;
[0094] During the model training phase, the optimizer was set to Adam, the loss function used binary cross entropy, the training epoch was 300, and the validation data ratio was 0.2.
[0095] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A forest and grass fire scar restoration monitoring method based on UAV aircraft vision, characterized by: The following steps are involved: Step 1: Construct a forest and grass fire scar restoration monitoring system, which includes an unmanned aerial vehicle (UAV) equipped with a multispectral camera. The multispectral camera is sequentially connected to an image preprocessing module, an image segmentation module, a feature enhancement module, and a forest and grass fire scar restoration time prediction model FRTime. Step 2: The UAV aircraft carries a multispectral camera to shoot video stream remote sensing data a of the forest area in real time, and converts the video stream remote sensing data a into image frame data b through a computer vision calculation method, and then transmits it to the image preprocessing module; Step 3: The image preprocessing module performs a preprocessing operation on the image frame data b to obtain a standard image c, and passes it to the image segmentation module; Step 4: The image segmentation module uses the object segmentation algorithm to segment the forest and grass restoration area in the standard image c to obtain the forest and grass restoration area image d, and passes it to the feature enhancement module; Step 5: The feature enhancement module uses the generative adversarial network (GAN) to perform feature enhancement on the forest and grassland restoration area image d to obtain an enhanced image e, and passes it to the forest and grassland fire scar recovery time prediction model FRTime; Step 6: The forest and grass fire site recovery time prediction model FRTime performs forest and grass recovery prediction on the enhanced image e, and predicts the normalized forest and grass recovery time index T of the enhanced image e.
2. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 is characterized by: In step 2, the UAV captures aerial video of the forest and grassland area in real time in a vertical shooting manner, and uses the VedioCapture video capture method in the cross-platform computer vision library OpenCV to extract static image frame data b from the video stream remote sensing data a at time intervals. The extraction function of the image frame sequence is defined as follows: a={a1,a2,...,a n }=f(b) Where a is the video frame sequence; b is the extracted image frame sequence; and f represents the function for extracting images from the video.
3. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 is characterized by: In step 3, the preprocessing operation includes removing noise from the image frame data b by using Gaussian filtering and adjusting the image size. The preprocessing formula is as follows: I denoised =b*G I resized =resize(I denoised ,224,224) Where G is a Gaussian filter; I denoised is the denoised image; I resized is the resized image, that is, the standard image c.
4. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 is characterized by: In step 4, the image segmentation module uses the all-things segmentation algorithm to segment the forest and grass restoration area in the standard image c. The function of the all-things segmentation algorithm is defined as: d=SegMent(c) Among them, SegMent is the method for segmenting everything.
5. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 is characterized by: In step 5, the feature enhancement module uses the improved generative adversarial network Real-RSRGAN to perform feature enhancement on the forest and grass restoration area image d. The feature enhancement function is defined as follows: e=Real_RSRGAN(d) Among them, Real_RSRGAN is an image feature enhancement method.
6. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 is characterized by: In step 6, the forest and grass fire scar recovery time prediction model FRTime performs a deep learning artificial intelligence model operation on the enhanced image e and the normalized vegetation index NDVI corresponding to the image to predict the normalized forest and grass recovery time index T of the enhanced image e. The prediction formula is as follows: T=FRTime(e,NDVI) Among them, FRTime is the normalized forest and grassland recovery time index prediction method, NIR is the near-infrared band reflectance of the enhanced image e, and Red is the red band reflectance of the enhanced image e.
7. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 1 or 6, characterized in that: The forest and grass fire site recovery time prediction model FRTime is trained by the following steps: Step A1: The UAV aircraft carries a multispectral camera to obtain a change image of the forest fire scar restoration area, obtains an original image set, and transmits it to the preprocessing module; Step A2: the preprocessing module performs a preprocessing operation on the original image set to obtain a standard image set, and passes it to the image segmentation module; Step A3: The image segmentation module uses the object segmentation algorithm to accurately segment the standard image set and extract fine images of the forest and grassland restoration area; combining the image's near-infrared band reflectance NIR, red band reflectance Red and time record, calculates the normalized vegetation index NDVI and normalized forest and grassland restoration time index T of the corresponding image, and organizes these data into the data set TimeData; Step A4: Construct an initial model for predicting the recovery time of forest and grass fire scars; Step A5: Use the data set TimeData to train the initial model for predicting the recovery time of forest and grass fire scars. When the training cycle is reached, save the model parameters, end the training, and obtain the trained forest and grass fire scar recovery time prediction model FRTime.
8. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 7 is characterized by: In step A3, based on the purpose of forest and grassland fire site recovery monitoring, a custom normalized forest and grassland time index T is used to represent the normalized recovery time at a certain moment from the occurrence of the fire to complete recovery. The calculation formula of the normalized recovery time is as follows: Among them, t i Indicates the recovery time (days) corresponding to the current image, t total It represents the total number of days from the occurrence of fire to complete recovery in the area. T is the normalized forest and grassland recovery time index, ranging from 0 to 1, which represents the proportion of the recovery time corresponding to the current image to the total recovery time.
9. The method for monitoring forest and grass fire scar restoration based on UAV aircraft vision according to claim 7 is characterized by: In step A5, the initial model for predicting the recovery time of forest and grass fire scars uses the VGG16 network model as a feature extractor, extracts the image data in the dataset TimeData as feature vectors, concatenates the extracted image features with the features mapped by the NDVI value, and performs prediction in the fully connected layer; the calculation expression is as follows: F=VGG16(I resized ) F concat =concat(F,NDVI) T predicted =FC(F concat ) Among them, F is the feature vector extracted by the VGG16 network. VGG16 is a convolutional neural network model for feature extraction. Concat is the concatenation operation that connects two vectors. concat is the new vector after concatenation of F and NDVI, FC is the fully connected layer, T predicted is the predicted normalized forest and grassland recovery time index T; During the model training phase, the optimizer was set to Adam, the loss function used binary cross entropy, the training epoch was 300, and the validation data ratio was 0.2.