A high-altitude monitoring and early warning method and device for forest fire resurgence based on multispectral images
By carrying a multi-spectral camera with a drone and combining physical models and deep learning algorithms, real-time monitoring and early warning of forest fire rekinds has been achieved, solving the defects in the existing technology that it is difficult to monitor the rekindling of dead objects, and improving the Monitoring accuracy and efficiency.
Patent Information
- Application Number
- CN202410401714.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-03
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-04-03
AI Technical Summary
The prior art is difficult to accurately monitor and early warning of the rekindling of forest fires, especially when it is covered with dead objects or ashes, and there is a lack of real-time monitoring and early warning methods.
The multi-spectral camera equipped with a drone collects spectral data, establishes a physical model and deep learning model of radiation brightness correlation function, and predicts the probability of forest fire rekind through high-temperature anomaly detection and surface-deep temperature depth control relationship data set. .
It has achieved rapid and accurate monitoring of the rekindling of forest fires, with the characteristics of unattended service, improved the accuracy and efficiency of monitoring, and supported emergency responses for forest fire monitoring and fire rescue.
Smart Images

Figure CN118521920B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of forest fire rekindling detection technology in the field of forest fire safety, and in particular to a forest fire rekindling high-altitude monitoring and early warning method and device based on multispectral images. Background Art
[0002] Forest fire resurgence refers to the smoldering of dead branches and leaves at the bottom of vegetation such as forests and grasslands, which often occurs in the firefighting area during forest fires. Due to the humid and hypoxic environment, the smoldering of dead branches and leaves shows the characteristics of slow and persistent spread, hidden combustion, large temperature difference span and release of large amounts of smoke, which is difficult to accurately monitor compared with surface fires and crown fires. There is currently no dedicated quantifiable means to monitor and warn of forest fire resurgence. Most of them are based on the measurement and observation of thermal radiation characteristics. For example, internationally, most of them are based on remote sensing images and multi-spectral sensors carried by satellites or drones, and monitor forest fire conditions by analyzing infrared radiation and visible light data. There are few studies on forest fire resurgence detection by remote sensing in China. Most of them use portable infrared temperature measuring equipment carried by humans to explore the fire boundary of underground fires, or install sensors for plugging detection and gas monitoring to determine whether resurgence occurs. However, at present, domestic and foreign studies are mostly for remote sensing real-time monitoring of forest and grassland fires that have occurred or even spread, and have not yet involved real-time monitoring and early warning of resurgence under the cover of litter or ash. Summary of the invention
[0003] In view of the above problems, the present invention proposes a method and device for high-altitude monitoring and early warning of forest fire re-ignition based on multispectral images, which overcomes the above problems or at least partially solves the above problems.
[0004] The present invention provides a high-altitude monitoring and early warning method for forest fire resurgence based on multispectral images, the method comprising:
[0005] Step S1, using a multispectral camera carried by the UAV to monitor the scene data of the target area;
[0006] Step S2, building a multi-band forest fire resurgence detection physical model based on the radiation brightness correlation function;
[0007] Step S3, based on the scene data, marking the high temperature abnormal area for the area photographed by the drone;
[0008] Step S4, establishing a surface-deep temperature depth comparison relationship data set;
[0009] Step S5, building a multi-band forest fire rekindling detection deep learning model;
[0010] Step S6, combining the results of the multi-band forest fire resurgence detection physical model and the multi-band forest fire resurgence detection deep learning model to predict the probability of forest fire resurgence in the target monitoring area.
[0011] Optionally, the step S2, building a multi-band forest fire rekindling detection physical model according to the radiation brightness correlation function, comprises:
[0012] Establish a functional relationship between the ratio of the output voltage of the temperature measurement system at two bands and the ratio of the radiation brightness of the two bands;
[0013] A multi-band forest fire re-ignition detection physical model is constructed based on the functional relationship.
[0014] Optionally, the step S3, based on the scene data, marking the abnormal high temperature area for the aerial photography area of the drone includes:
[0015] The temperature of each position in the aerial photography area of the drone in the scene data is obtained in a traversal manner in units of pixels to form a temperature matrix;
[0016] The temperature information in the temperature matrix is processed by clustering, and the areas with temperature values higher than the set value in the matrix are marked as high temperature areas.
[0017] Optionally, the step S4, establishing a surface-deep temperature depth comparison relationship data set includes:
[0018] Step S4-1, building a forest fire simulation test box, in which sawdust is laid to simulate forest litter, and a heater and a thermocouple are inserted into the center of each layer of sawdust on both sides of the test box;
[0019] Step S4-2, control the drone equipped with the multispectral camera to rise to the designated area, use the multispectral camera to collect data from the experimental box area below, and establish a wireless connection with the data collection terminal;
[0020] Step S4-3, adjusting the heater in the test box to a specified temperature to simulate the occurrence of forest fire resurgence at different degrees;
[0021] Step S4-4, recording the data of thermocouples at five positions;
[0022] Step S4-5, recording images of different bands collected by the multispectral camera on the drone;
[0023] Step S4-6, recording the distances between different high temperature points and the surface and the heat source;
[0024] Step S4-7, adjust the heater in the experimental box to different temperatures, repeat steps S4-1 to S4-6, organize each experimental record, and form a surface-deep temperature depth comparison relationship data set.
[0025] Optionally, the step S5 of building a multi-band forest fire rekindling detection deep learning model includes:
[0026] Step S5-1, performing a convolution operation on the multispectral images of different mid- and long-wave bands and the multispectral images of adjacent bands transmitted back by the UAV;
[0027] Step S5-2, performing an average pooling operation on the result in step S5-1;
[0028] Step S5-3, cross-adding the results in step S5-2, sharing the features between the two bands, and performing the convolution operation again to obtain the corresponding result;
[0029] Step S5-4, performing an average pooling operation on the result in step S5-3;
[0030] Step S5-5, cross-adding the results in step S5-4, sharing the features between the two bands, performing a full connection operation, and obtaining the corresponding result;
[0031] Step S5-6, repeating the operations of step S5-1 to step S5-5 on all bands acquired in the multispectral camera;
[0032] Step S5-7, add the results of all band pairs in step S5-6, integrate the features between the two bands, and perform three full connection operations again to reduce the feature dimension.
[0033] The present invention also provides a high-altitude monitoring and early warning device for forest fire resurgence based on multispectral images, comprising one or more processors and a non-temporary computer-readable storage medium storing program instructions. When the one or more processors execute the program instructions, the one or more processors are used to implement the method described in any one of the above items.
[0034] The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium is used to store program code, and the program code is used to execute any of the methods described above.
[0035] The present invention further provides a computing device, characterized in that the computing device comprises a processor and a memory:
[0036] The memory is used to store program code and transmit the program code to the processor;
[0037] The processor is configured to execute any of the above methods according to instructions in the program code.
[0038] The present invention proposes a high-altitude monitoring and early warning method and device for forest fire resurgence based on multispectral images. This technology uses a multispectral camera carried by an unmanned aerial vehicle to collect spectral data in spectral remote sensing images. On the one hand, a physical model is established: the heat radiation emitted by the high-temperature zone formed by the fire source or heat accumulation is collected and calculated, and the intensity change of the spectral image of a specific band formed on the surface after being absorbed by the soil, fallen objects, and ash in the covering layer; on the other hand, a deep learning model is established: a two-stream convolution classification network is built to learn the image features of different bands transmitted back by the unmanned aerial vehicle. Finally, a machine learning algorithm is used to judge the probability of the resurgence of the fire under the table by combining the surface temperature calculated by the physical model and the feature vector containing the surface temperature and the depth of the fire source obtained by the deep learning model.
[0039] The solution of the present invention can effectively monitor the resurgence of forest fires, and has the characteristics of being fast, accurate, and unattended, and has higher accuracy and efficiency than traditional methods. The research results can provide support and guidance for forest fire monitoring and emergency response of forest fire rescue personnel, and can also provide reference and reference for remote sensing image processing and classification in other fields.
[0040] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below.
[0041] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:
[0043] Figure 1 A schematic diagram of the process flow of a high-altitude monitoring and early warning method for forest fire resurgence based on multispectral images according to an embodiment of the present invention is shown;
[0044] Figure 2 A schematic diagram of a forest fire rekindling simulation experiment box according to an embodiment of the present invention is shown;
[0045] Figure 3 A schematic diagram of the experimental process of the surface-deep temperature depth comparison data set of an embodiment of the present invention is shown;
[0046] Figure 4A schematic diagram of the algorithm training and detection process of an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0047] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.
[0048] The embodiment of the present invention provides a high-altitude monitoring and early warning method for forest fire resurgence based on multispectral images, such as Figure 1 As shown, the high-altitude monitoring and early warning method for forest fire resurgence based on multispectral images in an embodiment of the present invention includes:
[0049] Step S1, using a multispectral camera carried by the UAV to monitor the scene data of the target area;
[0050] Step S2, building a multi-band forest fire resurgence detection physical model based on the radiation brightness correlation function;
[0051] Step S3, based on the scene data, marking the high temperature abnormal area for the area photographed by the drone;
[0052] Step S4, establishing a surface-deep temperature depth comparison relationship data set;
[0053] Step S5, building a multi-band forest fire rekindling detection deep learning model;
[0054] Step S6, combining the results of the multi-band forest fire resurgence detection physical model and the multi-band forest fire resurgence detection deep learning model to predict the probability of forest fire resurgence in the target monitoring area.
[0055] The present invention proposes a high-altitude monitoring and early warning method and device for forest fire resurgence based on multispectral images. This technology collects spectral data in spectral remote sensing images through a multispectral camera carried by an unmanned aerial vehicle. On the one hand, a physical model is established: the heat radiation emitted by the high-temperature zone formed by the fire source or heat accumulation is collected and calculated, and the intensity change of the spectral image in a specific band formed on the surface after being absorbed by the soil, fallen leaves, and ash in the covering layer; on the other hand, a deep learning model is established: a two-stream convolution classification network is built to learn the image features of different bands transmitted back by the unmanned aerial vehicle. Finally, a machine learning algorithm is used to judge the probability of the resurgence of the fire under the table by combining the surface temperature calculated by the physical model and the characteristic vector containing the surface temperature and the depth of the fire source obtained by the deep learning model. Each step is described in detail below.
[0056] Step S1, using the multispectral camera carried by the UAV to monitor the scene data of the target area.
[0057] Start the drone and collect scene data of the target monitoring area according to the instructions of the route planning. The scene data can be hyperspectral data. The multispectral camera carried by the gimbal is used to collect data of the scene below at an interval of 10 meters. At the same time, the output voltage of the temperature measurement system at different bands is recorded. Select the long wave λ1 and medium wave λ2, respectively denoted as V(λ1,T) and V(λ2,T), record the returned medium wave image and long wave image, respectively as frame i and frame i+1 , in order to realize the collection of spectral data.
[0058] Step S2, building a multi-band forest fire re-ignition detection physical model based on the radiation brightness correlation function.
[0059] Step S2-1, establishing a functional relationship between the ratio of the output voltage of the temperature measurement system at two bands and the ratio of the radiation brightness of the two bands, such as formula (1);
[0060]
[0061] Where Δλ=λ1-λ2, f(λ) is the spectral transmittance of the filter, D(λ) is the spectral response function of the photoelectric converter, F(λ) is the spectral transmittance of the optical system, and ε(λ,T) is the emissivity of the measured object at a certain temperature and wavelength.
[0062] Step S2-2, simplify the formula in step S2-1, and let
[0063]
[0064]
[0065]
[0066] Then the formula (1) in step S2-1 can be simplified to:
[0067]
[0068] Step S2-3, assuming that the surface characteristics of the object being measured are similar to a gray body, and the object radiates two bands per unit surface area in all directions of its upper hemisphere space per unit time The radiation energy is and The emissivity is εg1 and εg2, and the ratio between them is K. Then the following ratio relationship is listed:
[0069]
[0070] According to Wien's formula, formula (6) can be finally integrated into:
[0071]
[0072] Step S2-4, let step 2.3) Derived formula (8) right side of the numerator and denominator expansion, we have:
[0073]
[0074]
[0075] Then formula (7) can be integrated as:
[0076]
[0077] in
[0078] Step S2-5, performing cubic spline interpolation fitting according to the emissivity measured by the multispectral camera on the UAV and the experimental results to obtain the surface temperature T.
[0079] Step S3: Based on the scene data, the area photographed by the drone is marked as an abnormally high temperature area.
[0080] S3-1, using a traversal method, obtaining the temperature of each position in the drone's aerial photography area in the scene data in units of pixels, and forming a temperature matrix TMatrix;
[0081] S3-2, using a clustering method, the temperature information in the temperature matrix TMatrix is processed, and the area with a temperature value higher than the set value in the matrix is marked as a high temperature area. Optionally, the area with a temperature value higher than the average temperature in the matrix by more than 30% can be marked as a high temperature area.
[0082] Step S4, establishing a surface-deep temperature depth comparison relationship data set.
[0083] Step S4-1, build a forest fire simulation test box, the outer shell of the test box is a 100cm×100cm×50cm high transparent box, and a layer of sawdust is laid every 10cm in the box to simulate forest litter. A heater and a thermocouple are inserted into the center of each layer of sawdust on both sides of the test box. Figure 2 shown.
[0084] Step S4-2, control the drone equipped with the multispectral camera to rise to the designated area, use the multispectral camera to collect data from the experimental box area below, and establish a wireless connection with the data collection terminal; Figure 3 shown.
[0085] Step S4-3, adjusting the heater in the test box to a specified temperature to simulate the occurrence of forest fire resurgence at different degrees;
[0086] Step S4-4, recording the data of thermocouples at five positions;
[0087] Step S4-5, recording images of different bands collected by the multispectral camera on the drone;
[0088] Step S4-6, recording the distances between different high temperature points and the surface and the heat source;
[0089] Step S4-7, adjust the heater in the experimental box to different temperatures, repeat steps S4-1 to S4-6, organize each experimental record, and form a surface-deep temperature depth comparison relationship data set.
[0090] Step S5: Build a multi-band forest fire re-ignition detection deep learning model. Figure 4 As shown, the learning process of the multi-band forest fire rekindling detection deep learning model in this embodiment can be as follows:
[0091] Step S5-1, performing convolution operation on the multispectral images of different mid-bands and long-bands and multispectral images of adjacent bands transmitted by the drone; specifically, for the multispectral images of different mid-bands and long-bands transmitted by the drone, i And adjacent band multispectral image frame i+1 Perform convolution operation.
[0092] convFrame i =Conv(frame i )
[0093] convFrame i+1 =Conv(frame i+1 )
[0094] Where Conv() represents the convolution operation.
[0095] Step S5-2, performing an average pooling operation on the result in step S5-1;
[0096] avgFrame i =AvgPool(convFrame i )
[0097] avgFrame i+1 =AvgPool(convFrame i+1 )
[0098] Where AvgPool() represents the average pooling operation.
[0099] Step S5-3, cross-adding the results in step S5-2, sharing the features between the two bands, and performing the convolution operation again to obtain the corresponding result;
[0100] con2frame i =Conv(avgFrame i +avgFrame i+1 )
[0101] con2frame i+1 =Conv(avgFrame i +avgFrame i+1 )
[0102] Step S5-4, performing an average pooling operation on the result in step S5-3;
[0103] poolframe i =AvgPool(con2frame i )
[0104] poolframe i+1 =AvgPool(con2frame i+1 )
[0105] Step S5-5, cross-adding the results in step S5-4, sharing the features between the two bands, performing a full connection operation, and obtaining the corresponding result;
[0106] addframe i =FC(poolframe i +poolframe i+1 )
[0107] addframe i+1 =FC(poolframe i +poolframe i+1 )
[0108] where represents the average pooling operation.
[0109] Step S5-6, repeating the operations of step S5-1 to step S5-5 on all bands acquired in the multispectral camera;
[0110] Step S5-7, add the results of all band pairs in step S5-6, integrate the features between the two bands, and perform three full connection operations again to reduce the feature dimension.
[0111]
[0112] Where concat() represents the concatenation operation, n represents the total number of different bands of the multispectral camera, and T d represents the temperature predicted by the deep learning model, and d represents the depth of the predicted point from the surface. Based on the above process, the image features of different bands sent back by the drone can be learned, and the image features can contain feature vectors of surface temperature and fire source depth information.
[0113] Step S6, combining the results of the multi-band forest fire resurgence detection physical model and the multi-band forest fire resurgence detection deep learning model to predict the probability of forest fire resurgence in the target monitoring area.
[0114] The surface-deep temperature depth comparison relationship data set collected in step S4 is divided into a training set and a test set in a ratio of 7:3 and sent to the random forest model. The resurgence probability is predicted based on the training of the labeled sample data.
[0115] P = RandomForest(d,T,T d )
[0116] Among them, RandomForest() represents the random forest model, T d and d are the temperature and depth obtained by the forest fire rekindling detection deep learning model in step S5-7, T is the surface temperature obtained in step S2-5, and P is the probability of rekindling occurring at a depth of d cm from the surface predicted by the random forest model.
[0117] An embodiment of the present invention also provides a high-altitude monitoring and early warning device for forest fire re-ignition based on multi-spectral images, including one or more processors and a non-temporary computer-readable storage medium storing program instructions. When the one or more processors execute the program instructions, the one or more processors are used to implement the method described in the above embodiment.
[0118] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute the methods described in the above embodiments.
[0119] An embodiment of the present invention further provides a computing device, which includes a processor and a memory: the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the method described in the above embodiment according to the instructions in the program code.
[0120] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.
[0121] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.
[0122] Those skilled in the art can understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.
[0123] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the methods described in the embodiments of the present invention.
[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate from the protection scope of the present invention.
Claims
1. A high-altitude monitoring and early warning method for forest fire resurgence based on multispectral images, characterized in that: The method comprises: Step S1, using a multispectral camera carried by the UAV to monitor the scene data of the target area; Step S2, building a multi-band forest fire rekindling detection physical model based on the radiation brightness correlation function; Step S3, based on the scene data, marks the high temperature abnormal area for the drone overhead shooting area; including: using a traversal method to obtain the temperature of each position in the drone overhead shooting area in the scene data in units of pixels to form a temperature matrix; using a clustering method to process the temperature information in the temperature matrix, and marking the area with a temperature value higher than the set value in the matrix as a high temperature area; Step S4, establishing a surface-deep temperature depth comparison relationship data set; Step S5, building a multi-band forest fire rekindling detection deep learning model; Step S6, combining the results of the multi-band forest fire resurgence detection physical model and the multi-band forest fire resurgence detection deep learning model to predict the probability of forest fire resurgence in the target monitoring area; The step S4, establishing a surface-deep temperature depth comparison relationship data set, comprises: Step S4-1, building a forest fire simulation test box, in which sawdust is laid to simulate forest litter, and a heater and a thermocouple are inserted into the center of each layer of sawdust on both sides of the test box; Step S4-2, control the drone equipped with the multispectral camera to rise to the designated area, use the multispectral camera to collect data from the experimental box area below, and establish a wireless connection with the data collection terminal; Step S4-3, adjusting the heater in the test box to a specified temperature to simulate the occurrence of forest fire resurgence at different degrees; Step S4-4, recording the data of thermocouples at five positions; Step S4-5, recording images of different bands collected by the multispectral camera on the drone; Step S4-6, recording the distances between different high temperature points and the surface and the heat source; Step S4-7, adjusting the heater in the experimental box to different temperatures, repeating steps S4-1 to S4-6, sorting out each experimental record, and forming a surface-deep temperature depth comparison relationship data set; The step S5 of building a multi-band forest fire rekindling detection deep learning model includes: Step S5-1, performing a convolution operation on the multispectral images of different mid- and long-wave bands and the multispectral images of adjacent bands transmitted back by the UAV; Step S5-2, performing an average pooling operation on the result in step S5-1; Step S5-3, cross-adding the results in step S5-2, sharing the features between the two bands, and performing the convolution operation again to obtain the corresponding result; Step S5-4, performing an average pooling operation on the result in step S5-3; Step S5-5, cross-adding the results in step S5-4, sharing the features between the two bands, performing a full connection operation, and obtaining the corresponding result; Step S5-6, repeating the operations of step S5-1 to step S5-5 on all bands acquired in the multispectral camera; Step S5-7, add the results of all band pairs in step S5-6, integrate the features between the two bands, and perform three full connection operations again to reduce the feature dimension.
2. The method according to claim 1, characterized in that The step S2, building a multi-band forest fire rekindling detection physical model according to the radiation brightness correlation function, comprises: Establish a functional relationship between the ratio of the output voltage of the temperature measurement system at two bands and the ratio of the radiation brightness of the two bands; A multi-band forest fire re-ignition detection physical model is constructed based on the functional relationship.
3. A high-altitude monitoring and early warning device for forest fire resurgence based on multispectral images, characterized in that: The method comprises one or more processors and a non-transitory computer-readable storage medium storing program instructions. When the one or more processors execute the program instructions, the one or more processors are configured to implement the method according to any one of claims 1 to 2.
4. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the method according to any one of claims 1 to 2.
5. A computing device, characterized in that: The computing device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 2 according to the instructions in the program code.
