Fire wire dynamic monitoring method for forest and grass fire hazard by unmanned aerial vehicle
The forest and grass fire fire line monitoring system built through unmanned aircraft and flame detectors uses image processing technology to realize accurate calculation of the fire line length and real-time dynamic monitoring, solving problems such as high labor costs, low efficiency, and high safety hazards in traditional methods, significantly improving monitoring accuracy and response speed.
Patent Information
- Application Number
- CN202510056380.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-06-20
AI Technical Summary
The traditional forest and grass fire line evaluation method relies on manual exploration, which has problems such as high labor costs, low operating efficiency, safety hazards, poor identification accuracy and lagging information updates.
The forest and grass fire fire line monitoring system is constructed by unmanned aircraft and flame detectors. The unmanned aircraft collects remote sensing image data and flame detectors in real time to collect surface image data and environmental parameter information, and combines preprocessing, feature extraction, live line identification and depth estimation technologies to achieve accurate calculation of the fire line length and real-time dynamic monitoring.
It realizes accurate calculation of the length of the fire line and real-time dynamic monitoring, significantly improves the accuracy and response speed of forest and grass fire monitoring, avoids the danger of people entering the fire directly, and improves the safety and intelligence level of fire management.
Smart Images
Figure CN120182903A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing information technology, and particularly to a method for dynamically monitoring the fire line of forest and grassland fires by unmanned aerial vehicles. Background Art
[0002] In the field of forest and grassland fire monitoring and fire line assessment, traditional methods rely on manual on-foot or vehicle patrols to complete on-site surveys. This approach has obvious limitations in terms of resource allocation, operation efficiency, and safety. Facing the suddenness and vastness of forest or grassland fires, it is difficult for manpower to quickly and comprehensively grasp the dynamic of the fire line. Especially under extreme conditions such as fierce fire and vast fire area, traditional manual survey methods seem inadequate and cannot meet the urgent needs of rapid response and efficient prevention and control.
[0003] Manual fire line assessment is not only inefficient but also poses a significant safety threat to the staff involved in monitoring. Forest and grassland fire sites are usually accompanied by high temperatures, thick smoke, toxic gases, and complex terrains, which greatly increase the life risks of frontline personnel.
[0004] In the complex environment of forest and grassland fire sites, traditional fire line identification methods relying on naked-eye observation are difficult to ensure high precision. The combined effects of limited visibility, narrow field of view, and individual judgment differences lead to uncertainties in the determination of the fire line position and length. In addition, the real-time nature of information is crucial for disaster management and decision-making. There is a time lag in the process of collecting data from the field to the command center receiving and analyzing the information, which reduces the timeliness and accuracy of decision-making, and thus affects the fire control and rescue efficiency.
[0005] The above analysis reveals the inherent defects of traditional forest and grassland fire line assessment technologies: high labor costs, low operation efficiency, safety hazards, poor identification accuracy, and lagging information updates. These limitations hinder the practical application of traditional methods in forest and grassland fire management. Summary of the Invention
[0006] A method for dynamically monitoring the fire line of forest and grassland fires provided by the present invention can achieve precise calculation and real-time dynamic monitoring of the fire line length.
[0007] To achieve the above object, the key to a method for dynamically monitoring the fire line of forest and grassland fires provided by the present invention is to include the following steps:
[0008] Step 1: Construct a forest and grassland fire line monitoring system. The forest and grassland fire line monitoring system is provided with a collection module, which is sequentially connected to a preprocessing module, a feature extraction module, a fire line identification and calculation module, and a fire line length calculation module; the collection module includes an unmanned aerial vehicle in the air and a flame detector fixed on the ground surface.
[0009] Step 2: The unmanned aerial vehicle (UAV) collects the remote sensing image data a1 of the forest and grassland site in real time and sends it to the preprocessing module, and the flame detector collects the surface image data a2 of the forest and grassland site and the surface environment parameter information in real time and sends them to the preprocessing module;
[0010] Step 3: After the preprocessing module performs preprocessing operations on the remote sensing image data a1, the surface image data a2, and the surface environment parameter information, it transmits them to the feature extraction module;
[0011] Step 4: The feature extraction module performs feature fusion and extraction on the remote sensing image data a1, the surface image data a2, and the surface environment parameter information to obtain multi-light fusion image data and corresponding environmental parameter features, and transmits them to the fire line recognition calculation module;
[0012] Step 5: The fire line recognition module detects and recognizes the fire line boundary based on the multi-light fusion image data and the corresponding environmental parameter features to obtain fire line data, and transmits them to the fire line length calculation module;
[0013] Step 6: The fire line length calculation module uses a neural network to perform depth estimation on the fire line data to obtain the depth information Z of each fire line pixel point, and combines the sensor data in the UAV log to calculate the fire line length data.
[0014] Through the above design, the UAV and the flame detector are used to remotely control and collect and transmit the image data of the forest and grassland site in real time. After a series of preprocessing steps, the fire line data of the forest and grassland fire is recognized, and key information such as the fire line length is calculated in real time, realizing the dynamic monitoring of the fire line of the forest and grassland fire. This method can timely feedback the fire line dynamics, greatly improving the monitoring efficiency and response speed. More importantly, the present invention avoids the danger of personnel directly entering the fire site, realizes true remote real-time monitoring, and greatly improves the safety and intelligent level of fire situation management.
[0015] Preferably: In step 2, the UAV is equipped with a dual-light imaging device, and the dual-light imaging device includes an ultraviolet light imaging device and a visible light imaging device;
[0016] The ultraviolet light imaging device is used to capture the ultraviolet spectrum pictures in the forest and grassland site, and the visible light imaging device is used to capture the remote sensing visible light pictures in the forest and grassland site;
[0017] The ultraviolet spectrum pictures and the remote sensing visible light pictures constitute the remote sensing image data a1;
[0018] The flame detector is provided with m image sensor modules, and each image sensor module includes a visible light image sensor, an infrared image sensor, and a temperature and humidity sensor; the m image sensor modules are distributed in an array in the forest and grassland site;
[0019] The visible light image sensor is used to capture visible light pictures of the surface in the forest and grassland areas, and the infrared image sensor is used to capture infrared thermal imaging pictures of the forest and grassland areas and the temperature values of each pixel in the pictures.
[0020] The visible light pictures of the surface and the infrared thermal imaging pictures constitute the surface image data a2.
[0021] The temperature and humidity sensor is used to collect the surface temperature and humidity data of the forest and grassland areas, that is, the surface environmental parameter information.
[0022] Through the above design, the remote sensing image data a1 and the surface image data a2 of the forest and grassland areas are collected simultaneously by the unmanned aerial vehicle and the flame detector, effectively improving the accuracy and reliability of the data source. At the same time, various image acquisition devices are combined to effectively collect various types of image data, improving the accurate capture of fire information, facilitating the accurate identification of the fire line data by the subsequent image recognition module, and reducing the influence of interference factors on the fire recognition.
[0023] Preferably: The fire line recognition module also judges the fault condition of the image sensor module according to the geographical coordinate information corresponding to the surface image data a2. If a fire is recognized in the surface image data a2 captured by at least two image sensor modules around the faulty image sensor module, the fault condition of the faulty image sensor module is damaged due to the fire, and the geographical location information of the image sensor module damaged due to the fire is supplemented into the fire line data; otherwise, it is damaged due to other reasons and waits for subsequent maintenance management.
[0024] Through the above design, further judgment can be made on the fire situation and the fire line path according to the fault condition of the image sensor module, which can make up for the problem of missing response data caused by the damage of some equipment and improve the integrity of the fire line data. At the same time, it speeds up the process of subsequent management and maintenance of the corresponding equipment.
[0025] Preferably: The fire line recognition module also comprehensively judges the environmental wind direction information according to the flame tilt direction or the smoke floating trajectory and its horizontal displacement in the surface image data a2, combined with the position information of the corresponding image sensor module; the environmental wind direction information includes the environmental wind direction and the wind direction deviation angle, and the judgment process is as follows:
[0026] The live wire recognition module selects one image sensor module from the image sensor modules corresponding to the surface image data a2 containing live wire data as the forward image sensor module, obtains the orientation information of the forward image sensor module, and then, according to the orientation information, determines the flame tilt direction, flame height or smoke floating direction, smoke height in the surface image data a2 captured by the forward image sensor module, and determines the forward horizontal displacement of the flame or smoke according to the flame tilt direction, flame height or smoke floating direction, smoke height;
[0027] Taking the flame or smoke height in the surface image data a2 captured by the forward image sensor module as the standard height, selecting the image sensor module with the orientation information close to the flame tilt direction or smoke floating direction and perpendicular to the orientation information of the current image sensor module as the lateral image sensor module, and determining the lateral horizontal displacement of the flame or smoke in the surface image data a2 captured by the lateral image sensor module at the standard height;
[0028] According to the forward horizontal displacement and the lateral horizontal displacement, determine the environmental wind direction and the wind direction deviation angle. The calculation formula of the wind direction deviation angle is as follows:
[0029]
[0030] Among them, α represents the wind direction deviation angle, H1 represents the forward horizontal displacement of the flame or smoke, and H2 represents the lateral horizontal displacement of the flame or smoke.
[0031] Through the above design, while completing the fire situation recognition, the comprehensive judgment of the environmental wind direction information can be carried out, and the change of the environmental wind direction can be understood in real time. The change of the wind direction directly affects the spread direction of the fire. Therefore, understanding and adapting to the change of the wind direction is the key to improving the fire fighting and rescue efficiency and ensuring the safety of personnel.
[0032] Preferably: The live wire recognition module also judges the future development trend of the fire situation according to the surface environment parameter information and the environmental wind direction information, determines the current flame position information according to the surface temperature information, and calculates the flame ignition point and trend according to the surface humidity information.
[0033] Through the above design, combined with the surface temperature and humidity data and the wind direction information of the forest and grassland sites, the current situation and future development trend of the fire situation can be accurately predicted, providing data support for subsequent fire fighting and disaster relief.
[0034] Preferably: In the step 3, the preprocessing operation includes preprocessing operations such as geometric correction, atmospheric correction, image mosaicking, image fusion, image cropping and shadow removal; among them, the geometric correction includes rough correction, geometric fine correction and ortho-rectification.
[0035] Among them, geometric correction is to adjust the image to correct geometric distortion, which involves correcting transformations such as rotation, scaling, and translation of the image, so that the objects in the image are correctly displayed in proportion and position.
[0036] Atmospheric correction is to improve the clarity and contrast of the image by removing the effects in the atmosphere.
[0037] Image mosaicking is to splice multiple overlapping or adjacent images together to form a seamless whole to obtain a wider coverage area.
[0038] Image fusion is to fuse image information from different sensors or sources to obtain richer information.
[0039] Image cropping is to extract the region of interest from the original image according to needs and remove the irrelevant parts.
[0040] Shadow removal is to remove the shadows in the image to improve the usability of the image.
[0041] In the process of remote sensing imaging, factors such as the instability of the sensor and atmospheric radiation may cause problems such as radiation distortion and geometric distortion of the image, which have a serious impact on the quality of remote sensing images. Pretreatment operations can effectively eliminate or reduce the noise, distortion, and distortion in the image, thereby improving the overall quality of remote sensing images.
[0042] Preferably: in the step 4, the feature extraction module adopts a multi-scale fusion algorithm to fuse the remote sensing image data a1 and the surface image data a2 corresponding to the geographical coordinates, and extract multi-level features in the fused image.
[0043] This process performs multi-level feature extraction on the image through a convolutional neural network CNN to enhance the visual clarity of the fire line.
[0044] Preferably: in the step 6, the sensor data in the unmanned aerial vehicle log includes the heading angle, the downward shooting angle, the flight altitude, and the GPS coordinates.
[0045] Centered on the shooting point, the value range of the heading angle is 0° to 360°. If the fire line is long, multi-view coverage is performed by adjusting the heading angle multiple times, with each change of 30°. The value range of the downward shooting angle is -30° to -90°, which can capture the environmental details in front of and around the fire line. The flight altitude range is 50 to 150 meters, which can capture clearer image details.
[0046] Preferably, in step 6, the live wire data includes live wire images from at least two shooting perspectives. The live wire length calculation module uses a neural network to perform depth estimation on the live wire images from each shooting perspective respectively, calculates the three-dimensional coordinates of each live wire pixel point from each shooting perspective, and then fuses and optimizes the three-dimensional coordinates of the same live wire point from all shooting perspectives to obtain the three-dimensional coordinates of the live wire data in the global coordinate system;
[0047] The shooting perspective is determined by the heading angle, the downward shooting angle, the flight altitude and the GPS coordinates when the unmanned aerial vehicle takes pictures.
[0048] The initial shooting calculation is centered on the shooting point. With the fusion of three-dimensional data and the optimization of the live wire positions, the system will convert these coordinates into a global coordinate system to ensure the consistency and accuracy of multi-perspective data.
[0049] Through the above design, first, independent three-dimensional depth estimation is performed on the data captured from multiple perspectives, and the three-dimensional coordinates of each live wire pixel point are calculated. Then, the three-dimensional coordinates from multiple perspectives are fused, and multi-perspective optimization is performed on the same live wire points. By fusing the live wire data from multiple perspectives, the single-perspective error is reduced, and a more accurate three-dimensional live wire position is achieved.
[0050] The two-dimensional coordinates of each pixel point in the live wire data identified by the live wire recognition module are (u, v). Combining the depth information Z, the two-dimensional coordinates (u, v) of each pixel point are converted into three-dimensional coordinates (X w , Y w , Z w ) and the expression is as follows:
[0051]
[0052] Z w =Z
[0053] where u and v are the image coordinates of the pixel point; Z is the depth in the camera coordinate system, representing the distance between the point and the camera; c x , c y are the coordinates of the principal point of the camera, that is, the optical center; f x , f y is the pixel unit length of the camera focal length, which is determined by the camera internal parameters;
[0054] The depth information Z is estimated through the binocular camera disparity formula:
[0055]
[0056] where B is the baseline length between the binocular cameras, and d is the pixel disparity between the left and right cameras.
[0057] The live wire recognition module is a YoLoV4 model trained with multi-light fusion image data.
[0058] Combined with depth estimation and sensor data in the unmanned aerial vehicle log, the three-dimensional coordinates of the live wire points are further adjusted to ensure that all points are accurately positioned in the global coordinate system. The accuracy of the live wire point positions is optimized through images from different perspectives to reduce the error caused by single-perspective estimation.
[0059] Preferably: In step 6, a B-spline curve is used to fit the live wire data curve, and the three-dimensional coordinates of the live wire data in the global coordinate system are (P0, P1, …, P n ), and the expression of the cubic B-spline curve is as follows:
[0060] B(t) = (1 - t) 3 P0 + 3(1 - t) 2 tP1 + 3(1 - t)t 2 P2 + t 3 P3, t ∈ [0, 1]
[0061] Among them, P0, P1, P2, P3 are the control points of the B-spline curve, used to fit the curve shape of the live wire; t is a parameter used to generate the coordinates of any point on the curve; by selecting appropriate control points P0, P1, P2, P3, a smooth live wire curve representation can be obtained.
[0062] According to the result of the B-spline curve fitting, the length of the live wire is calculated by piecewise calculation; among them, the length of each small segment of the curve is calculated using the Euclidean distance formula. For two consecutive points on the B-spline curve (x i , y i , z i ) and (x i+1 , y i+1 , z i+1 ), then the length L i of each small segment is calculated as follows:
[0063]
[0064] The total length L of the live wire is the sum of the lengths of each segment, and the expression is as follows:
[0065]
[0066] The beneficial effects of the present invention: It realizes the dynamic monitoring of the fire situation and the live wire, calculates the specific length of the live wire, and can also predict the subsequent fire development trend and spread speed according to the current wind direction and surface temperature and humidity data, not only significantly improving the accuracy and response speed of forest and grassland fire monitoring, but also improving the efficiency of emergency response and providing relevant basis for the formulation of fire extinguishing plans. Description of the Drawings
[0067] Figure 1 is a schematic flowchart of the present invention;
[0068] Figure 2 is a structural block diagram of the forest and grassland fire front monitoring system in the present invention;
[0069] Figure 3 is a schematic diagram of the coordinate for calculating the environmental wind direction angle in the embodiment. Detailed Embodiment
[0070] The present invention will be further described in detail below with reference to the drawings and specific examples. The following examples or drawings are used to illustrate the present invention, but not to limit the scope of the present invention.
[0071] As Figure 1 shown: A method for dynamically monitoring the fire front of forest and grassland fires by an unmanned aerial vehicle includes the following steps:
[0072] Step 1: Construct a forest and grassland fire front monitoring system. The forest and grassland fire front monitoring system is provided with a collection module, which is sequentially connected to a preprocessing module, a feature extraction module, a fire front identification and calculation module, and a fire front length calculation module; the collection module includes an unmanned aerial vehicle in the air and a flame detector fixed on the ground surface, as Figure 2 shown;
[0073] Step 2: The unmanned aerial vehicle collects real-time remote sensing image data a1 of the forest and grassland site and transmits it to the preprocessing module, and the flame detector collects real-time ground surface image data a2 of the forest and grassland site and ground surface environmental parameter information and transmits it to the preprocessing module;
[0074] Step 3: After the preprocessing module preprocesses the remote sensing image data a1, the ground surface image data a2, and the ground surface environmental parameter information, it transmits them to the feature extraction module;
[0075] Step 4: The feature extraction module performs feature fusion and extraction on the remote sensing image data a1, the ground surface image data a2, and the ground surface environmental parameter information to obtain multi-light fusion image data and corresponding environmental parameter features, and transmits them to the fire front identification and calculation module;
[0076] Step 5: The fire front identification module detects and identifies the fire front boundary according to the multi-light fusion image data and the corresponding environmental parameter features to obtain fire front data, and transmits it to the fire front length calculation module;
[0077] Step 6: The fire front length calculation module uses a neural network to perform depth estimation on the fire front data to obtain the depth information Z of each fire front pixel point, and combines the sensor data in the unmanned aerial vehicle log to calculate the fire front length data.
[0078] In step 2, the unmanned aircraft carries a dual - light imaging device, which includes an ultraviolet light imaging device and a visible light imaging device;
[0079] The ultraviolet light imaging device is used to capture ultraviolet spectrum pictures in the forest and grassland site, and the visible light imaging device is used to capture remote sensing visible light pictures in the forest and grassland site;
[0080] The ultraviolet spectrum pictures and the remote sensing visible light pictures constitute the remote sensing image data a1;
[0081] The flame detector is provided with m image sensor modules, and each image sensor module includes a visible light image sensor, an infrared image sensor, and a temperature and humidity sensor; the m image sensor modules are distributed in an array in the forest and grassland site;
[0082] The visible light image sensor is used to capture surface visible light pictures in the forest and grassland site, and the infrared image sensor is used to capture infrared thermal imaging pictures in the forest and grassland site and the temperature values of each pixel in the pictures;
[0083] The surface visible light pictures and the infrared thermal imaging pictures constitute the surface image data a2;
[0084] The temperature and humidity sensor is used to collect the surface temperature and humidity data of the forest and grassland site, that is, the surface environmental parameter information.
[0085] The fire line recognition module also judges the fault condition of the image sensor module according to the geographic coordinate information corresponding to the surface image data a2. If the surface image data a2 captured by at least two image sensor modules around the faulty image sensor module is recognized as a fire, the fault condition of the faulty image sensor module is damaged due to the fire, and the geographic location information of the image sensor module damaged due to the fire is supplemented into the fire line data; otherwise, it is damaged due to other reasons, and waiting for subsequent maintenance management.
[0086] The fire line recognition module also comprehensively judges the environmental wind direction information according to the flame tilt direction or the smoke floating trajectory and its horizontal displacement in the surface image data a2, combined with the position information of the corresponding image sensor module; the environmental wind direction information includes the environmental wind direction and the wind direction deviation angle, and the judgment process is as follows:
[0087] The live wire recognition module selects one image sensor module from the image sensor modules corresponding to the surface image data a2 containing live wire data as the forward image sensor module, obtains the orientation information of the forward image sensor module, and then, according to the orientation information, determines the flame tilt direction, flame height or smoke floating direction, smoke height in the surface image data a2 captured by the forward image sensor module, and determines the forward horizontal displacement of the flame or smoke according to the flame tilt direction, flame height or smoke floating direction, smoke height;
[0088] Taking the flame or smoke height in the surface image data a2 captured by the forward image sensor module as the standard height, selecting an image sensor module with orientation information close to the flame tilt direction or smoke floating direction and perpendicular to the orientation information of the current image sensor module as the lateral image sensor module, and determining the lateral horizontal displacement of the flame or smoke in the surface image data a2 captured by the lateral image sensor module at the standard height;
[0089] According to the forward horizontal displacement and the lateral horizontal displacement, determine the environmental wind direction and the wind direction deviation angle. The calculation formula of the wind direction deviation angle is as follows:
[0090]
[0091] Wherein, α represents the wind direction deviation angle, H1 represents the forward horizontal displacement of the flame or smoke, and H2 represents the lateral horizontal displacement of the flame or smoke.
[0092] As Figure 3 shown: In this embodiment, the forward image sensor module faces east and the lateral image sensor module faces south.
[0093] The live wire recognition module also judges the future development trend of the fire situation according to the surface environment parameter information and the environmental wind direction information, determines the current flame position information according to the surface temperature information, and calculates the flame ignition point and trend according to the surface humidity information.
[0094] In the step 3, the preprocessing operation includes preprocessing operations such as geometric correction, atmospheric correction, image mosaicking, image fusion, image cropping and shadow removal; among them, the geometric correction includes rough correction, geometric fine correction and orthorectification.
[0095] In the step 4, the feature extraction module adopts a multi-scale fusion algorithm to fuse the remote sensing image data a1 and the surface image data a2 corresponding to the geographic coordinates, and extracts multi-level features in the fused image.
[0096] In the step 6, the sensor data in the unmanned aerial vehicle log includes heading angle, downward shooting angle, flight altitude and GPS coordinates.
[0097] Centered at the shooting point, the range of the heading angle is 0° to 360°. If the fire line is long, multi-view coverage can be achieved by adjusting the heading angle multiple times, with each change being 30°. The range of the downward shooting angle is -30° to -90°, which can capture the environmental details in front of and around the fire line. The flight altitude range is 50 to 150 meters, which can capture clearer image details.
[0098] In step 6, the fire line data includes fire line images at at least two shooting perspectives. The fire line length calculation module uses a neural network to perform depth estimation on the fire line images at each shooting perspective respectively, calculates the three-dimensional coordinates of each fire line pixel point at each shooting perspective, and then fuses and optimizes the three-dimensional coordinates of the same fire line points at all shooting perspectives to obtain the three-dimensional coordinates of the fire line data in the global coordinate system.
[0099] The two-dimensional coordinates of each pixel point in the fire line data identified by the fire line recognition module are (u, v). Combining the depth information Z, the two-dimensional coordinates (u, v) of each pixel point are converted into three-dimensional coordinates (X w , Y w , Z w ) and the expression is as follows:
[0100]
[0101] Z w = Z
[0102] where u, v are the image coordinates of the pixel point; Z is the depth in the camera coordinate system, representing the distance between the point and the camera; c x , c y is the principal point coordinate of the camera, i.e., the optical center; f x , f y is the pixel unit length of the camera focal length, which is determined by the camera internal parameters.
[0103] The depth information Z is estimated through the binocular camera disparity formula:
[0104]
[0105] where B is the baseline length between the binocular cameras, and d is the pixel disparity between the left and right cameras.
[0106] In step 6, a B-spline curve is used to fit the fire line data curve. The three-dimensional coordinates of the fire line data in the global coordinate system are (P0, P1,..., P n ), and the expression of the cubic B-spline curve is as follows:
[0107] B(t) = (1 - t) 3 P0 + 3(1 - t)2 tP1 + 3(1 - t)t 2 P2 + t 3 P3, t ∈ [0, 1]
[0108] Where P0, P1, P2, P3 are the control points of the Bezier curve, used to fit the curve shape of the fire line; t is a parameter used to generate the coordinates of any point on the curve;
[0109] According to the fitting result of the Bezier curve, the length of the fire line is calculated by piecewise calculation; among them, the length of each small segment of the curve is calculated using the Euclidean distance formula. For two consecutive points on the Bezier curve (x i , y i , z i ) and (x i+1 , y i+1 , z i+1 ), then the length L of each small segment i The calculation expression is as follows:
[0110]
[0111] The total length L of the fire line is the sum of the lengths of each segment, and the expression is as follows:
[0112]
[0113] The above is only the preferred embodiment of the present invention and is not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles, characterized in that: The following steps are involved: Step 1: construct a forest and grass fire line monitoring system, wherein the forest and grass fire line monitoring system is provided with a collection module, which is sequentially connected to a preprocessing module, a feature extraction module, a fire line identification calculation module and a fire line length calculation module; the collection module includes an unmanned aerial vehicle in the air and a flame detector fixed on the surface; Step 2: The unmanned aerial vehicle collects remote sensing image data a1 of the forest and grassland site in real time and sends it to the preprocessing module, and the flame detector collects surface image data a2 and surface environmental parameter information of the forest and grassland site in real time and sends them to the preprocessing module; Step 3: The preprocessing module performs preprocessing operations on the remote sensing image data a1, the surface image data a2 and the surface environmental parameter information, and then transmits them to the feature extraction module; Step 4: The feature extraction module performs feature fusion and extraction on the remote sensing image data a1, the surface image data a2 and the surface environmental parameter information to obtain multi-light fusion image data and corresponding environmental parameter features, and transmits them to the fire line recognition calculation module; Step 5: The fire line recognition module detects and recognizes the fire line boundary according to the multi-light fusion image data and the corresponding environmental parameter characteristics, obtains the fire line data, and transmits it to the fire line length calculation module; Step 6: The fire line length calculation module uses a neural network to perform depth estimation on the fire line data to obtain the depth information Z of each fire line pixel, and combines the sensor data in the unmanned aerial vehicle log to calculate the fire line length data.
2. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 1 is characterized by: In step 2, the unmanned aerial vehicle is equipped with a dual-light camera device, which includes an ultraviolet light camera device and a visible light camera device; The ultraviolet light camera device is used to capture ultraviolet spectrum images in the forest and grassland site, and the visible light camera device is used to capture remote sensing visible light images in the forest and grassland site; The ultraviolet spectrum picture and the remote sensing visible light picture constitute the remote sensing image data a1; The flame detector is provided with m image sensor modules, each of which includes a visible light image sensor, an infrared image sensor and a temperature and humidity sensor; the m image sensor modules are arrayed and distributed in the forest and grassland field; The visible light image sensor is used to capture visible light images of the ground surface in the forest and grassland site, and the infrared image sensor is used to capture infrared thermal imaging images of the forest and grassland site and the temperature value of each pixel in the image; The surface visible light picture and infrared thermal imaging picture constitute the surface image data a2; The temperature and humidity sensor is used to collect surface temperature and humidity data of forest and grassland sites, that is, surface environmental parameter information.
3. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 2 is characterized by: The fire line identification module also determines the fault condition of the image sensor module based on the geographic coordinate information corresponding to the surface image data a2. If the surface image data a2 taken by at least two image sensor modules around the faulty image sensor module are identified as a fire, the fault condition of the faulty image sensor module is damage due to the fire, and the geographic location information of the image sensor module damaged by the fire is added to the fire line data; otherwise, it is damaged due to other reasons and awaits subsequent maintenance management.
4. The method for dynamic monitoring of forest and grass fire lines based on a data twin system according to claim 2 is characterized in that: The fire line recognition module also comprehensively determines the environmental wind direction information according to the flame tilt direction or smoke drift trajectory and horizontal displacement in the surface image data a2, combined with the position information of the corresponding image sensor module; the environmental wind direction information includes the environmental wind direction and the wind direction deviation angle, and the judgment process is as follows: The fire line recognition module selects an image sensor module as a forward image sensor module from the image sensor modules corresponding to the surface image data a2 containing the fire line data, and obtains the orientation information of the forward image sensor module, and then determines the flame inclination direction, flame height or smoke drifting direction, and smoke height in the surface image data a2 captured by the forward image sensor module according to the orientation information, and determines the forward horizontal displacement of the flame or smoke according to the flame inclination direction, flame height or smoke drifting direction, and smoke height; The flame or smoke height in the ground surface image data a2 captured by the forward image sensor module is used as the standard height, and an image sensor module whose orientation information is close to the flame inclination direction or smoke drifting direction and is perpendicular to the orientation information of the current image sensor module is selected as the lateral image sensor module, and the lateral horizontal displacement of the flame or smoke in the ground surface image data a2 captured by the lateral image sensor module at the standard height is determined; According to the forward horizontal displacement and the lateral horizontal displacement, the ambient wind direction and the wind direction deviation angle are determined. The calculation formula of the wind direction deviation angle is as follows: Among them, α represents the wind direction deviation angle, H1 represents the positive horizontal displacement of the flame or smoke, and H2 represents the lateral horizontal displacement of the flame or smoke.
5. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 4 is characterized by: The fire line identification module also determines the future development trend of the fire according to the surface environmental parameter information and the environmental wind direction information, determines the current flame position information according to the surface temperature information, and calculates the flame ignition point and trend according to the surface humidity information.
6. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 1, characterized in that: In step 3, the preprocessing operations include image geometric correction, atmospheric correction, image mosaicking, image fusion, image cropping and shadow removal preprocessing operations; wherein geometric correction includes coarse correction, fine geometric correction and orthorectification.
7. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 1, characterized in that: In step 4, the feature extraction module uses a multi-scale fusion algorithm to fuse the remote sensing image data a1 and the surface image data a2 corresponding to the geographic coordinates, and extracts multi-level features in the fused image.
8. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 1, characterized in that: In step 6, the sensor data in the UAV log includes heading angle, pitch angle, flight altitude and GPS coordinates.
9. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 8, characterized in that: In step 6, the fire line data includes fire line images under at least two shooting perspectives, and the fire line length calculation module uses a neural network to perform depth estimation on the fire line images under each shooting perspective, and calculates the three-dimensional coordinates of each fire line pixel point under each shooting perspective, and then fuses and optimizes the three-dimensional coordinates of the same fire line point under all shooting perspectives to obtain the three-dimensional coordinates of the fire line data in the global coordinate system; The two-dimensional coordinates of each pixel point in the fire line data identified by the fire line identification module are (u, v), and the two-dimensional coordinates of each pixel point (u, v) are converted into three-dimensional coordinates (X w ,Y w ,Z w ) is as follows: WITH w =Z Among them, u, v are the image coordinates of the pixel point; Z is the depth in the camera coordinate system, indicating the distance between the point and the camera; c x ,c y is the principal point coordinate of the camera, i.e. the optical center; f x ,f y is the pixel unit length of the camera focal length, which is determined by the camera intrinsic parameters; The depth information Z is estimated by the binocular camera parallax formula: Where B is the baseline length between the binocular cameras, and d is the pixel disparity between the left and right cameras.
10. The method for dynamic monitoring of forest and grass fire lines by unmanned aerial vehicles according to claim 9, characterized in that: In step 6, a Bezier curve is used to fit the fire line data curve, and the three-dimensional coordinates of the fire line data in the global coordinate system are (P0, P1, ..., P n ), the cubic Bezier curve expression is as follows: B(t)=(1-t) 3 P0+3(1-t) 2 tP1+3(1-t)t 2 P2+t 3 P3,t∈[0,1] Among them, P0, P1, P2, P3 are the control points of the Bezier curve, which are used to fit the curve shape of the fire line; t is a parameter, which is used to generate the coordinates of any point on the curve; According to the result of Bezier curve fitting, the length of the fire line is obtained by segment calculation; the length of each small segment of the curve is calculated using the Euclidean distance formula, and the length of two consecutive points on the Bezier curve is (x i ,y i ,z i ) and (x i+1 ,y i+1 ,z i+1 ), then the length of each segment is L i The calculation expression is as follows: The total length L of the live wire is the cumulative sum of the lengths of each segment, expressed as follows: