Forest and grassland fire early identification and positioning method based on infrared thermal imaging and visible light fusion

Abstract

The application provides a forest and grassland fire early identification and positioning method based on infrared thermal imaging and visible light fusion, and belongs to the technical field of forest and grassland resource protection and ecological safety monitoring. The method comprises the following steps: synchronously collecting infrared and visible light images and recording the space-time information of a monitoring platform; performing pixel-level registration on the two images; extracting a thermal anomaly suspected area from the infrared image based on a dynamic background temperature field, and extracting a smoke suspected area from the visible light image based on color, texture and motion features; performing spatial correlation analysis on the suspected areas extracted from the two channels, and combining the area growth rate and temperature rise rate of continuous multiple frames to construct a fire evolution model to confirm a fire point; calculating the three-dimensional geographic coordinates of the fire point based on the photogrammetry collinear equation and a digital elevation model; and generating and outputting alarm information containing the three-dimensional coordinates and fire intensity. The application realizes early and accurate identification and positioning of forest and grassland fires, and significantly reduces the false alarm rate.

Description

Technical Field

[0001] This invention relates to the field of forest and grassland resource protection and ecological security monitoring technology, and in particular to a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light. Background Technology

[0002] Forest and grassland fires are characterized by their suddenness, rapid spread, and high destructiveness. Early and accurate identification and location are crucial for effective firefighting and minimizing losses. Traditional fire monitoring relies mainly on manual lookout and visible light video surveillance. The former is inefficient and prone to missed reports, while the latter is greatly affected by lighting and weather conditions, making it difficult to detect early fires or fires at night.

[0003] In existing technologies, infrared thermal imaging technology is used for forest fire monitoring due to its independence from light and its ability to detect thermal radiation. However, relying solely on infrared technology is susceptible to interference from non-fire-related high-temperature objects such as rocks heated by sunlight and heat source equipment, leading to a high false alarm rate. Meanwhile, visible light analysis alone is ineffective for smoke identification in complex weather conditions and against similar terrain. To improve reliability, methods combining infrared and visible light information have emerged, such as first detecting suspected fire points using infrared imaging and then verifying them using visible light images for color analysis. These methods reduce false alarms to some extent, but still have significant limitations: First, their essence is a serial verification logic, which is insufficiently sensitive to very early stages of a fire (such as when only weak thermal anomalies or a small amount of smoke are produced); second, most are based on static feature analysis of single frames or short time series, lacking modeling of the dynamic development patterns of fires, making it difficult to effectively distinguish between continuously developing real fires and transient interference; third, the positioning accuracy is mostly limited to a two-dimensional plane, failing to fully consider the impact of mountainous terrain undulations on positioning, making it difficult to meet the needs of precise firefighting dispatch.

[0004] Therefore, there is an urgent need in this field to develop an early fire identification method that can deeply integrate infrared and visible light information, effectively model the dynamic evolution characteristics of fires, and achieve accurate three-dimensional geolocation, in order to solve the problems of high false alarm rate, weak early detection capability, and insufficient positioning accuracy of existing technologies. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light, in order to solve or improve the technical problems existing in the prior art.

[0006] The technical solution of this invention is implemented as follows: a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light, comprising the following steps: S1. Synchronous Image Acquisition and Information Recording: Using an infrared thermal imager and a visible light camera mounted on the monitoring platform, the original infrared image and the original visible light image of the area to be monitored are acquired simultaneously, and the geographical location, attitude angle and imaging timestamp of the monitoring platform at the time of acquisition are recorded. S2. Image preprocessing and pixel-level registration: The original infrared image and the original visible light image are preprocessed respectively; then, based on the preprocessed images, the infrared image and the visible light image are pixel-level registered to obtain a registered fused image pair; S3. Dual-path parallel feature extraction: In the registered fused image pair, the following extraction is performed in parallel: From the infrared image, a continuous region exceeding a first temperature threshold is extracted based on the dynamic background temperature field as a suspected thermal anomaly region; From the visible light image, a region that conforms to the preset smoke color and texture features is extracted as a suspected smoke region. S4. Spatiotemporal correlation analysis and fire confirmation: Perform spatial correlation analysis on the suspected thermal anomaly area and the suspected smoke area extracted in step S3; if there are spatially overlapping or adjacent correlated areas, and the correlated area satisfies the preset fire evolution model in a series of consecutive image frames, then the correlated area is determined to be a confirmed fire point. S5. Three-dimensional geolocation: Based on the pixel coordinates of the confirmed fire point in the image, the spatiotemporal information of the monitoring platform, and the preset geographic information model, calculate and output the three-dimensional geographic coordinates of the confirmed fire point; S6. Intelligent Alarm and Information Output: Generate and output fire alarm information containing the three-dimensional geographic coordinates of the confirmed fire point, the fire intensity, and the imaging timestamp.

[0007] Compared with existing technologies, the beneficial effects of this invention are as follows: by pixel-level registration of infrared thermal imaging and visible light images and dual-path parallel feature extraction, deep spatial alignment and complementary analysis of multi-source heterogeneous data are achieved, significantly improving the joint detection sensitivity of very early weak fire sources and smoke; by adopting dynamic background temperature field modeling and a multi-feature spatiotemporal fusion strategy for smoke, false alarm interference from environmental heat sources and visual false targets is effectively suppressed; by introducing a fire evolution model to perform dynamic trend discrimination of continuous frame sequences, the limitations of single-frame or short-term static features in identifying persistent fires and instantaneous interference are overcome; by combining photogrammetric collinearity equations and digital elevation models, high-precision three-dimensional geographic coordinates of fire points are calculated, making up for the spatial uncertainty of two-dimensional planar positioning in complex terrain.

[0008] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is the core flowchart for fire confirmation in this invention. Detailed Implementation

[0011] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0012] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0013] This invention proposes a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light. This method achieves early and accurate identification and location of forest and grassland fires by simultaneously acquiring data from multiple sensors, image registration, dual-path parallel feature extraction, spatiotemporal correlation analysis and fire confirmation, and combining three-dimensional geolocation and intelligent alarm mechanism.

[0014] It should be noted that a monitoring platform refers to a carrier used to mount image acquisition equipment for mobile or fixed observation, such as a fixed observation tower, a tethered drone, or a drone with automatic cruise capabilities. This platform provides a stable observation view and the necessary sensor interfaces. An infrared thermal imager is a device that detects infrared radiation emitted by objects and converts it into a visible image. Its working principle does not rely on visible light, making it suitable for heat source detection in low-visibility environments such as nighttime or smoky conditions. A visible light camera is a device used to acquire images within the visible spectrum. Its imaging principle is similar to that of the human eye, capturing visual features such as color and texture of objects, and is used to identify visible phenomena such as smoke. Pixel-level registration refers to aligning images acquired by different sensors (such as infrared thermal imagers and visible light cameras) at the pixel level, so that objects at the same geographical location correspond to the same pixel coordinates in different images, thereby achieving information fusion. A dynamic background temperature field refers to a background temperature reference that varies with time and space by performing real-time or near-real-time statistical analysis of the temperature of non-target areas in infrared images, used to distinguish between real thermal anomalies and ambient temperature fluctuations. A suspected thermal anomaly area refers to a continuous pixel region in an infrared image whose temperature is significantly higher than the surrounding dynamic background temperature, and is initially judged to be an area where a heat source may exist. A suspected smoke area refers to a continuous pixel region in a visible light image whose color, texture, and / or motion characteristics meet preset smoke discrimination criteria, and is initially judged to be an area where smoke may exist. A fire evolution model is a model that describes the regular changes in the area, temperature, and other characteristics of a fire over time after it occurs, used to determine whether a suspected fire has a trend of continuous development and spread. A confirmed fire point refers to the area that is finally determined to be the actual location of a fire after multi-source information fusion, spatiotemporal correlation analysis, and verification by a fire evolution model. A geographic information model is a system for organizing, managing, analyzing, and representing geospatial data of the Earth's surface and its surface, such as a digital elevation model (DEM), which can provide geographic environmental information such as surface elevation, slope, and aspect.

[0015] This embodiment provides a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light. The specific implementation process is as follows: In step S1, synchronous image acquisition and information recording are performed. Using an infrared thermal imager and a visible light camera mounted on the monitoring platform, synchronous image acquisition is performed on the predetermined monitoring area to obtain raw infrared and raw visible light images. Simultaneously, the geographical location, attitude angle, and imaging timestamp of the monitoring platform at the time of acquisition are recorded. For example, two independent cameras are used to acquire images separately, and a time synchronization module ensures that the timestamps of the two images are consistent. The geographical location and attitude angle information of the monitoring platform can be obtained through the built-in Global Positioning System (GPS) and Inertial Measurement Unit (IMU) and stored along with the image data. As an alternative, image acquisition can be performed by asynchronous devices, with subsequent rough alignment done manually or based on timestamp matching of image features.

[0016] In step S2, image preprocessing and pixel-level registration are performed. First, the original infrared and visible light images acquired in step S1 are preprocessed, for example, by noise filtering, contrast enhancement, or distortion correction. Then, based on the preprocessed images, pixel-level registration is performed between the infrared and visible light images to eliminate spatial discrepancies between the different sensors, resulting in a registered fused image pair. For example, several prominent landmarks in the two images can be manually matched, and then the infrared image can be mapped to the coordinate system of the visible light image through affine or polynomial transformation. This method can provide preliminary registration results when the position and attitude of the monitoring platform change relatively little.

[0017] In step S3, dual-path parallel feature extraction is performed. In the registered fused image pair, suspected fire features are extracted in parallel from both the infrared and visible light images. Specifically, from the infrared image, based on a preset global background temperature value, all continuous regions whose temperature exceeds the sum of the global background temperature and a first temperature threshold are extracted as suspected thermal anomaly regions. For example, a fixed temperature value (e.g., 35℃) can be set as the background temperature, and a threshold (e.g., 5℃) can be added to mark all regions with temperatures above 40℃ as thermal anomalies. Simultaneously, from the visible light image, regions that conform to preset smoke color and texture characteristics can be extracted as suspected smoke regions. For example, a fixed grayscale range can be set to identify smoke, and a simple edge detection algorithm can be combined to determine texture features.

[0018] In step S4, spatiotemporal correlation analysis and fire confirmation are performed. Spatial correlation analysis is conducted on the suspected thermal anomaly area and the suspected smoke area extracted in step S3. If spatial overlap is found, a fire is preliminarily considered to be possible. Further, to confirm the fire, it can be checked whether the correlated area exists in two consecutive image frames and whether its temperature or area has increased. For example, if an area is identified as a suspected area in both the current and previous frames, and its average temperature has increased, it can be determined as a confirmed fire point. This simple judgment based on short time series helps to eliminate transient interference.

[0019] In step S5, three-dimensional geolocation is performed. Based on the pixel coordinates of the confirmed fire point in the image, the spatiotemporal information of the monitoring platform, and a preset geographic information model, the three-dimensional geographic coordinates of the confirmed fire point are calculated and output. For example, a simple triangulation principle can be used, combined with the height of the monitoring platform and the camera's viewing angle, to convert the pixel coordinates of the fire point in the image into two-dimensional planar coordinates on the ground. This method can provide approximate location information in flat terrain areas.

[0020] In step S6, intelligent alarm and information output are performed. Fire alarm information is generated, including the three-dimensional geographic coordinates of the confirmed fire point, the fire intensity, and the imaging timestamp, and then output. For example, this information can be displayed on the local control screen of the monitoring platform or transmitted to a nearby receiving device via a wired connection. The fire intensity can be simply estimated based on the highest temperature or area of ​​the fire point.

[0021] The method proposed in this embodiment achieves deep fusion of multi-source information by simultaneously acquiring infrared and visible light images and performing pixel-level registration, effectively overcoming the limitations of a single sensor in complex environments. Dual-path parallel feature extraction combined with spatiotemporal correlation analysis and a fire evolution model significantly improves the sensitivity and accuracy of early-stage forest and grassland fire identification, reducing false alarm rates. Furthermore, the use of 3D geolocation technology provides precise 3D coordinates of fire points, compensating for the shortcomings of traditional 2D positioning and providing reliable technical support for rapid fire response and precise fire suppression.

[0022] In the above embodiments of the present invention, after preprocessing the original infrared image and the original visible light image, it is necessary to perform pixel-level registration between the infrared image and the visible light image to obtain the registered fused image pair. However, due to the different imaging principles of infrared images and visible light images, their image features differ greatly. In addition, the monitoring platform may experience posture changes during the acquisition process, resulting in complex geometric distortions between the images. If registration is performed directly, it is often difficult to achieve high precision, thereby affecting the accuracy of subsequent fire identification and positioning.

[0023] To address this, the present invention further proposes a specific method for pixel-level registration of infrared and visible light images, comprising: extracting scale-invariant feature transformation feature points of the visible light image and finding corresponding temperature gradient feature points in the infrared image; calculating a homography transformation matrix based on successfully matched feature point pairs, and using the homography transformation matrix to map the infrared image to the coordinate system of the visible light image to complete the preliminary registration; and using the attitude angle parameters of the monitoring platform to perform geometric correction on the preliminary registration result to obtain the final registration result.

[0024] Specifically, during pixel-level registration, scale-invariant feature transform (SIFT) feature points are first extracted from the visible light image. SIFT is a method for finding local image features in different scale spaces. The extracted feature points possess advantages such as scale invariance and rotation invariance, effectively handling changes in the image at different viewpoints and scales. These feature points are typically pixels with significant local features, such as corner points and spots. Simultaneously, corresponding temperature gradient feature points are searched in the infrared image. Since infrared images reflect the thermal radiation information of objects, their features are often manifested in areas of rapid temperature change, i.e., areas with large temperature gradients. Therefore, by calculating the temperature gradient of the infrared image and selecting gradient peaks or areas with significant gradient changes as feature points, a physical correspondence between the infrared image features and the visible light image features can be ensured, facilitating subsequent matching.

[0025] After acquiring feature points from both types of images, the homography transformation matrix is ​​calculated based on the successfully matched feature point pairs. Homography is a mathematical model describing the perspective projection relationship between two planes, mapping points on one plane to another. Using the matched feature point pairs, algorithms such as least squares or RANSAC (Random Sample Consensus) can be used to calculate the homography transformation matrix describing the geometric transformation relationship between the infrared image and the visible light image. Subsequently, this homography transformation matrix is ​​used to map the infrared image to the coordinate system of the visible light image, thus completing the initial image registration. This step adjusts the pixel positions of the infrared image according to the calculated geometric relationship, making it approximately spatially aligned with the visible light image.

[0026] To further improve registration accuracy, the initial registration results are geometrically corrected using the attitude angle parameters of the monitoring platform to obtain the final registration result. When the monitoring platform acquires images, its attitude angles (such as pitch, roll, and yaw angles) affect the projection relationship and distortion of the images. Although homography transformation can handle most in-plane perspective distortions, for complex three-dimensional spatial projection errors caused by platform attitude changes, correction using attitude angle parameters provides more accurate geometric constraints. For example, more accurate camera extrinsic parameters can be constructed based on the attitude angle parameters, or a secondary geometric transformation can be performed on the initially registered image to eliminate residual geometric errors, thereby obtaining higher-precision pixel-level registration results.

[0027] Through the above technical solution, this invention effectively solves the registration problem between infrared and visible light images caused by differences in imaging principles and platform attitude changes. By combining scale-invariant feature transformation with temperature gradient feature points for matching, reliable correspondences between images of different modalities can be found. Preliminary alignment is then performed through homography transformation, followed by fine geometric correction using the attitude angle parameters of the monitoring platform, ensuring high-precision alignment of infrared and visible light images at the pixel level. This enables more accurate determination of overlap or adjacency relationships between suspected thermal anomaly and smoke areas during subsequent spatial correlation analysis, significantly improving the accuracy and reliability of early fire identification and laying a solid foundation for subsequent fire confirmation and three-dimensional geolocation.

[0028] In the above embodiments of the present invention, a method is proposed to extract continuous regions exceeding a first temperature threshold from infrared images based on a dynamic background temperature field as suspected areas of thermal anomalies. However, in actual forest and grassland fire monitoring, the ambient background temperature is significantly non-uniform and dynamically changing due to the influence of various factors such as sunlight, topography, and vegetation type. If a fixed or global temperature threshold is simply used for judgment, it is difficult to accurately distinguish between real fire points and environmental thermal noise, which can easily lead to a high false alarm rate or missed alarm, thereby affecting the accuracy and reliability of early fire identification.

[0029] To address this, the present invention further proposes a specific method for extracting continuous regions exceeding a first temperature threshold based on a dynamic background temperature field, comprising: First, a sliding window statistical method is used to perform block-based statistical analysis on the current infrared image. Specifically, for each pixel in the image... Set a center around it A sliding window of varying size is used. This sliding window is used for localized analysis of infrared images to accommodate variations in background temperature across different geographical locations. To ensure that the statistically analyzed background temperature accurately reflects the local environment and to avoid interference from potential fire points or strong heat sources, this method excludes the highest-temperature areas within the window. The pixels. For example, if k is set to 5%, the top 5% of pixels with the highest temperature within the window will be removed. Then, the average temperature of the remaining pixels is calculated and used as the local background temperature for that location. In this way, a dynamic background temperature field can be constructed that precisely reflects the actual background heat distribution in different regions of an infrared image, rather than a single global value. This represents the width of the sliding window, and its size can be adjusted according to the size of the fire point and the gradient of background temperature changes in the actual monitoring scene. This is a preset exclusion ratio used to control the degree of rejection of abnormally high-temperature pixels within the window.

[0030] Secondly, the measured temperature values ​​of each pixel are... With the corresponding local background temperature Compare the measured temperature values ​​of the pixels. With local background temperature The difference is greater than the third preset threshold. That is, satisfying If the temperature difference is too high, the pixel is marked as a high-temperature pixel. This step introduces a relative temperature difference judgment mechanism, so that fire point identification is no longer limited to absolute temperature values, but focuses on the degree of temperature anomaly of the pixel relative to its local environment, thereby improving the detection capability of weak fire sources. Third preset threshold The setting determines the sensitivity to identify thermal anomalies, and its value can be adjusted according to the tolerance for false alarms and missed alarms based on the actual application scenario.

[0031] Finally, connected component analysis is performed on the high-temperature pixels. Connected component analysis aims to cluster spatially adjacent high-temperature pixels into one or more independent regions. After completing the connected component analysis, this method further filters pixels with areas greater than a minimum area threshold. The connected regions were identified as the suspected thermal anomaly regions. A minimum area threshold was set. It can effectively filter out isolated high-temperature pixels caused by sensor noise, instantaneous environmental interference, or tiny hot spots that are not fire sources, ensuring that the identified thermal anomaly areas have a certain spatial scale and persistence, thereby improving the reliability of the identification results.

[0032] Through the above technical solution, this invention no longer relies on a single global temperature threshold, but instead constructs a dynamic background temperature field using a sliding window statistical method, excluding pixels with the highest temperature within the window, effectively avoiding interference from the complex and variable environmental background temperature on fire point identification. The measured pixel temperature... With local background temperature By comparing these data, regions with significantly higher relative temperatures to the background can be identified more accurately, improving the sensitivity and accuracy of thermal anomaly detection. Furthermore, by performing connected component analysis on high-temperature pixels and combining this with a minimum area threshold... Screening can effectively filter out sporadic high-temperature points caused by noise or non-fire sources, thereby significantly reducing the false alarm rate and improving the robustness and reliability of early identification of forest and grassland fires.

[0033] In the above embodiments of the present invention, spatial correlation analysis of suspected thermal anomaly areas and suspected smoke areas can preliminarily determine potential fire situations. However, relying solely on spatial correlation at a single moment may not effectively distinguish between actual fires and incidental heat sources or smoke, easily leading to false alarms or missed alarms. Especially in complex and ever-changing environments, it is difficult to accurately capture the dynamic evolution characteristics of fire situations, thereby affecting the accuracy and reliability of early identification.

[0034] In response, the present invention further proposes that in step S4, the fire evolution model that satisfies the preset parameters specifically includes: assuming that the associated areas are in continuous... The pixel area sequence in the frame image sequence is Calculate its area growth rate It needs to exceed the first preset threshold. ,Right now: Meanwhile, assume that the region is continuous The average temperature sequence in the frame is Calculate its temperature rise rate It needs to exceed the second preset threshold. ,Right now: ;in, It is a positive integer greater than or equal to 3. This is the start time index of the frame sequence.

[0035] Specifically, the sequence of pixel areas of the associated region in a consecutive N-frame image sequence This refers to recording and tracking the number of pixels occupied by the same potential fire point region in consecutive image frames. This reflects the two-dimensional size change of the fire point in the image. By analyzing this sequence, the expansion or contraction trend of the fire point over time can be quantified. In terms of implementation, after identifying the associated region in each image frame, the total number of pixels it contains is calculated, and these area values ​​are stored in chronological order. Area growth rate It is an indicator that measures the rate of change of pixel area in a correlated region across a consecutive N-frame image sequence. Its calculation method is as follows: .in, It is the area of ​​the starting frame of the sequence. It is the area of ​​the end frame of the sequence. This refers to the number of consecutive frames. This metric directly reflects the spread trend of a fire point over a certain period of time. When it exceeds a preset first threshold... This indicates that the fire is rapidly expanding, a key characteristic of a fire. First preset threshold. This threshold is used to determine whether the increase in the fire area reaches the fire standard. This threshold needs to be set based on the actual application scenario, monitoring environment, and typical fire evolution patterns. For example, a reasonable threshold can be determined through historical fire data analysis, expert experience, or simulation experiments. The value is used to ensure that when the fire area grows to a certain extent, it can be accurately identified as a fire.

[0036] Meanwhile, the region is in continuous Average temperature sequence in frames This refers to averaging the temperature values ​​of all pixels within the same potential fire point region across consecutive image frames and recording these average temperature values ​​chronologically. This reflects the thermal intensity change of the fire point over time. By analyzing this sequence, the rising or falling trend of the fire point temperature can be quantified. In practice, after identifying the relevant region in each image frame, the temperature values ​​of all pixels within that region are extracted from the infrared image, their average is calculated, and then these average temperature values ​​are stored. (Temperature rise rate) It measures the correlation of regions in a continuous An index representing the rate of average temperature change in a sequence of frames. Its calculation method is as follows: .in, It is the average temperature of the first frame of the sequence. It is the average temperature of the last frame of the sequence. This refers to the number of consecutive frames. This metric directly reflects the heat accumulation and release trend of a fire point over a certain period of time. When it exceeds a preset second threshold... At this point, it indicates that the ignition temperature is continuously rising, which is an important thermodynamic characteristic of fire occurrence. Second preset threshold. It is used to determine whether the temperature rise at the ignition point reaches the critical value required for fire safety. Similar to the first preset threshold Pt, The settings also need to comprehensively consider the actual environment, the performance of the infrared thermal imager, and the temperature evolution pattern of the fire. A reasonable... The value can effectively distinguish between ambient temperature fluctuations and heat accumulation caused by actual fires, thus improving the accuracy of fire assessment. A positive integer greater than or equal to 3, representing the number of consecutive image frames used to analyze the fire evolution model. (Selection) Positive integers greater than or equal to 3 are used to ensure sufficient time-series data to capture the dynamic changes in the fire, avoiding misjudgments due to instantaneous fluctuations in a single or two frames. Longer... The value can provide a more stable trend judgment, but it will also increase the amount of calculation and response time, therefore The selection of [the appropriate method / mechanism] requires a trade-off between accuracy and real-time performance.

[0037] By employing the aforementioned technical solution, which introduces dynamic analysis of the pixel area sequence and average temperature sequence of the associated region in a series of consecutive image frames, this invention effectively overcomes the limitations of relying solely on spatial correlation at a single moment for fire situation determination. Specifically, this is achieved by calculating the area growth rate. and with the first preset threshold Compare and calculate the rate of temperature rise. and the second preset threshold In comparison, this invention can capture the dynamic evolution of fires over time. This dual discrimination mechanism based on temporal changes enables the system to more accurately distinguish between real fires and non-fire heat sources or smoke, significantly reducing the false alarm rate. For example, some transient hotspots or smoke may resemble fires spatially, but their area and temperature do not increase rapidly over time; this solution can effectively exclude them. Furthermore, for initial fires, even if their spatial characteristics are not obvious, as long as their area and temperature show a continuous upward trend, they can be identified in a timely manner, thereby improving the accuracy and reliability of early fire identification and providing a more solid foundation for subsequent fire location and alarm.

[0038] The above implementation proposes calculating and confirming the three-dimensional geographic coordinates of fire points using image pixel coordinates, spatiotemporal information of the monitoring platform, and a geographic information model. However, in practical applications, accurately and reliably mapping the fire point location in the image to the real-world three-dimensional geographic coordinates is crucial for achieving high-precision fire location. Traditional simple geometric projection methods may suffer from insufficient positioning accuracy due to factors such as camera intrinsic and extrinsic parameters, platform attitude, and terrain undulations, making it difficult to meet the needs of early identification and precise location of forest and grassland fires.

[0039] To address this, the present invention further proposes a specific method for calculating the three-dimensional geographic coordinates of the confirmed fire point. This method, based on the photogrammetric collinearity equation, calculates the image point coordinates of the confirmed fire point. Convert to ground coordinates The collinearity equation is expressed as: ; ; in, Let the principal point coordinates be... For camera focal length, To monitor the spatial location of the platform at the time of data collection, These are the rotation matrix elements calculated based on the attitude angle of the monitoring platform. Based on this, the surface elevation represented by the digital elevation model in the aforementioned geographic information model is incorporated. Solving the above equations simultaneously yields the geodetic latitude and longitude coordinates of the confirmed fire location. and elevation value .

[0040] Specifically, the photogrammetric collinearity equation is a mathematical model connecting image point coordinates, camera intrinsic and extrinsic parameters, ground point coordinates, and the camera's spatial position and orientation. Based on the principle of central projection, it describes the strict collinearity in space between ground points, the camera projection center, and the corresponding image points. This equation is fundamental to high-precision 3D positioning, enabling accurate correlation between 2D image information and 3D spatial information. (Image point coordinates) This refers to confirming the two-dimensional location of the fire point on the image sensor plane, typically in pixels or converted to millimeters. Ground coordinates. This refers to confirming the three-dimensional spatial location of a fire point in a specific geodetic coordinate system (such as the WGS84 coordinate system or a local independent coordinate system), where X and Y represent planar positions and Z represents elevation. Using collinearity equations, it is possible to perform inverse calculations from two-dimensional observations on an image to the real-world three-dimensional location. (Image principal point coordinates) It is the projection point of the camera's optical center onto the image plane, usually located near the image center, and is one of the camera's intrinsic parameters. Camera focal length. This is the distance from the camera's optical center to the image plane, and a key component of the camera's intrinsic parameters. These two parameters are obtained during camera calibration and are crucial for establishing accurate collinearity equations, determining the image's geometric distortion characteristics and projection scale. The spatial position of the monitoring platform at the time of acquisition... This refers to the precise three-dimensional geographic coordinates of an infrared thermal imager or visible light camera at the moment of image capture. This data is typically provided by a high-precision Global Navigation Satellite System (GNSS) receiver combined with an inertial measurement unit (IMU), ensuring the accuracy of the camera's external parameters and serving as essential input for three-dimensional positioning. Rotation matrix elements It is calculated based on the attitude angles of the monitoring platform (such as pitch, roll, and yaw angles) and is used to describe the rotation relationship between the camera coordinate system and the ground coordinate system. This rotation matrix transforms vectors in the camera coordinate system to the ground coordinate system and is a key parameter in the collinearity equation connecting the camera attitude and the position of ground points. Accurate attitude angle data is usually provided by the IMU and obtained through attitude calculation. A Digital Elevation Model (DEM) is a model that represents ground elevation information in discrete digital form, providing the functional relationship between the Z-coordinate (elevation) and X and Y coordinates (planar position) of any point on the Earth's surface. In the process of 3D positioning, the collinearity equation usually contains three unknowns. However, there are only two equations, so it is necessary to introduce... As a third constraint, it uniquely determines the three-dimensional coordinates of the ground point. The accuracy of the measurement directly affects the accuracy of the final positioning result. This is achieved by combining the photogrammetric collinearity equation with the digital elevation model. Solve the system of equations simultaneously to form a system containing three equations and three unknowns. The system of nonlinear equations. This system of equations can be solved using iterative methods (such as Newton's method or Gauss-Newton's method) to obtain the precise three-dimensional coordinates of the confirmed fire point in the ground coordinate system. Subsequently, these rectangular coordinates It can be further converted into more commonly used geodetic latitude and longitude coordinates. and elevation value This is to facilitate display and application in a Geographic Information System (GIS).

[0041] Through the above technical solution, this invention can achieve high-precision three-dimensional geolocation of confirmed fire points. Specifically, by employing the collinearity equation of photogrammetry, the pixel coordinates of the fire point in the image are closely combined with the precise spatiotemporal information of the monitoring platform (including spatial position and attitude angle) and the camera's intrinsic parameters (focal length, principal point coordinates), establishing a rigorous geometric projection relationship. Based on this, a digital elevation model (DEM) from the geographic information model is introduced as an additional geometric constraint, effectively solving the degree of freedom problem of the collinearity equation in three-dimensional positioning, enabling the three-dimensional coordinates of ground points to be uniquely and accurately calculated. This method fully utilizes the advantages of multi-source data (images, GNSS / IMU, DEM) and overcomes the limitations of traditional methods in terms of insufficient positioning accuracy under complex terrain and variable attitudes. The final output geodetic latitude and longitude coordinates and elevation values ​​are not only highly accurate but also compatible with geographic information systems, providing reliable and accurate spatial basis data for subsequent fire situation analysis, rescue resource allocation, and fire spread trend prediction, significantly improving the practicality and decision support capabilities of early identification and location of forest and grassland fires.

[0042] In the above embodiments of the present invention, a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light is proposed, which can confirm the fire point and output its three-dimensional geographic coordinates. However, in actual forest and grassland fire fighting, simply obtaining the current location information of the fire point may not be sufficient to support efficient decision-making. The rapid spread of fire makes the prediction of the future fire situation crucial.

[0043] To address this, the present invention further proposes a step S5a following step S5, which includes predicting the fire spread trend. This step S5a aims to obtain the current wind speed... and wind direction Meteorological data, combined with the confirmed location of the fire. The meteorological data and the slope α, aspect β, and vegetation type coefficients in the geographic information model. To predict The position of the front line in the fire after time Specifically, meteorological data can be acquired through meteorological stations deployed within the monitoring area, meteorological sensors mounted on drones, or satellite remote sensing data, ensuring the real-time nature and accuracy of the data. The location of the confirmed fire point... This involves projecting the three-dimensional geographic coordinates calculated in step S5 onto a two-dimensional plane. The geographic information model includes slope information. Slope aspect and vegetation type coefficient Environmental parameters, such as those obtained from a pre-established digital elevation model, can be derived from this model. Extracted from land cover classification maps. Among them, slope... and slope This characterizes the impact of topography on fire spread, while the vegetation type coefficient... This reflects the type, density, and moisture content of the combustible material. Based on these input parameters, the location of the fire front is predicted using the following formula: ; ; in, The spread rate coefficient is a comprehensive factor considering slope, aspect, and type of combustible material. Its value is calculated based on a specific fire spread model (e.g., an empirical formula or physical model based on the Rothermel model), reflecting the speed of fire spread under specific terrain and vegetation conditions. This is the correction angle for wind direction due to terrain, used to correct for the influence of complex terrain on local wind fields and ensure the accuracy of wind direction parameters in fire spread prediction. By integrating the above formula, we can obtain... Predicted position of the fire front after time Finally, this prediction result is appended to the fire alarm information for subsequent transmission and display.

[0044] Through the above technical solution, this invention can transform static fire location information into dynamic early warning of fire development trends. This allows fire alarm information to include not only the current location of the fire point but also the predicted location of the future fire front, thereby greatly improving the early warning capability and decision support level of the fire monitoring system. Fire commanders can plan firefighting strategies, deploy rescue forces, and evacuate threatened areas in advance based on the predicted spread trend, effectively preventing the fire from getting out of control and minimizing the losses caused by the fire. This forward-looking information output significantly enhances the efficiency and safety of early identification and response to forest and grassland fires, providing crucial support for scientific fire decision-making and precise firefighting.

[0045] In some of the above embodiments, although suspected smoke areas are identified by extracting regions that match preset smoke color and texture characteristics from visible light images, relying on only a single or limited feature may lead to false alarms or missed alarms. For example, some non-fire smoke (such as water vapor or dust) may be similar in color or texture to fire smoke, while true early fire smoke may be difficult to accurately identify due to low concentration, complex background, or other reasons, thus affecting the accuracy and reliability of early fire identification.

[0046] In response, this invention further proposes that in step S3, the extraction of regions conforming to preset smoke color and texture features specifically includes: converting the visible light image to the YCrCb color space, and defining the color range of suspected smoke that satisfies: and Within the stated chromaticity range, the texture contrast of the region is further calculated based on the gray-level co-occurrence matrix. With entropy , must meet and ,in and The texture threshold is used; for multiple consecutive frames of visible light images, the motion vector field of the region is calculated using the optical flow method, and the average motion direction consistency is statistically analyzed. It must be greater than the motion consistency threshold. Based on the combined conditions of chroma, texture, and motion consistency, the suspected smoke area is determined and extracted.

[0047] Specifically, this method first preliminarily filters out potential smoke areas using color information. Visible light images are usually represented in the RGB color space, but the RGB space has a high coupling between luminance and chrominance, making it difficult to stably extract the color features of smoke under different lighting conditions. The YCrCb color space combines luminance (Y) with two chrominance components (Y, Y, and C). , ) separation, in which This represents the difference between the red component and its brightness. This represents the difference between the blue component and brightness. Smoke typically appears grayish-white or pale blue in visible light images; in the YCrCb color space, these color features are... and The components exhibit a relatively concentrated distribution range. Through preset... , , , Thresholds can effectively exclude most non-smoke areas, reduce the computational load of subsequent processing, and improve the efficiency of smoke recognition. These thresholds can be obtained through statistical analysis and machine learning training on a large number of real smoke image samples.

[0048] Building upon this, this method introduces texture features to further distinguish smoke from non-smoke areas, building upon color filtering. Smoke, due to the diffusion and movement of its particles, visually exhibits a unique, blurry, irregular texture lacking sharp edges. Gray-level co-occurrence matrix (GLCM) is a commonly used texture analysis method that describes image texture by statistically analyzing the frequency of gray values ​​between two pixels with a certain spatial relationship (such as distance or orientation). Contrast ( ) reflects the sharpness and texture of an image; smoke typically has low contrast; entropy ( This measures the complexity and randomness of image texture; the diffusion characteristics of smoke give it a high entropy value. Contrast within a chromaticity range is calculated. Entropy and with the preset texture threshold and By comparing, non-smoke areas with clear structures or regular textures, such as clouds, fog, or buildings, can be effectively excluded, thereby improving the accuracy of smoke recognition.

[0049] Simultaneously, this method also utilizes the dynamic characteristics of smoke for identification. Smoke is a mixture of gas and particulate matter, exhibiting continuous and directional movement as it diffuses in the air. Optical flow is a method for calculating pixel motion vectors in an image sequence, capable of capturing the instantaneous velocity of objects moving in an image. By applying optical flow to suspected smoke regions in multiple consecutive frames of visible light images, the motion vectors of each pixel within that region can be calculated. In genuine smoke regions, the motion direction of pixels often exhibits high consistency, showing an overall trend of diffusion or drifting. The average consistency of these motion vectors is statistically analyzed. and consistent with the preset motion consistency threshold. By comparing the smoke with stationary or randomly moving objects in the background, such as swaying leaves or ripples on the water, the robustness of smoke recognition can be further enhanced.

[0050] Ultimately, this method fuses and determines the aforementioned multi-dimensional features. Single color, texture, or motion features can have limitations. For example, some clouds may resemble smoke in color and texture, but their motion patterns may differ; while some industrial smoke emissions may differ in color from natural smoke. By comprehensively judging these three independent features—chroma, texture, and motion consistency—a region is ultimately determined to be a suspected smoke region only when it simultaneously meets preset chroma ranges, texture thresholds, and motion consistency thresholds.

[0051] Through the above technical solution, this invention, when extracting suspected smoke regions from visible light images, no longer relies solely on single color or texture features, but introduces a multi-dimensional feature fusion strategy. First, by converting the visible light image to the YCrCb color space and defining the chromaticity range, most non-smok background can be effectively filtered out, initially focusing on potential smoke regions. Based on this, the texture contrast is further calculated using the gray-level co-occurrence matrix. Entropy And combined with a preset texture threshold and It can distinguish the unique blurry and irregular texture of smoke from other objects in the background with clear structures or regular textures. Furthermore, by performing optical flow analysis on multiple consecutive frames of images, the motion vector field of suspected regions is calculated, and the consistency of their average motion direction is statistically analyzed. This method effectively utilizes the dynamic characteristics of smoke diffusion and drift to distinguish it from stationary or randomly moving interference. Ultimately, by comprehensively considering three factors—color, texture, and motion consistency—it significantly improves the accuracy and robustness of smoke identification, effectively avoiding false alarms and missed alarms that might result from relying on a single feature. This provides more reliable information on suspected smoke areas for subsequent fire confirmation, and is of great significance for the early identification of forest and grassland fires.

[0052] In some of the above embodiments, a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light is proposed. This method synchronously acquires images and records information through a monitoring platform. However, the forest and grassland environment is complex and diverse, placing different demands on the deployment flexibility, coverage, and continuous operation capability of the monitoring platform. If the monitoring platform type is singular or inappropriately selected, it may lead to low monitoring efficiency and difficulty in achieving effective coverage of vast or remote areas, thereby affecting the timeliness and accuracy of early fire identification.

[0053] In this regard, the present invention further proposes that the monitoring platform can be one of a fixed observation tower, a tethered drone, or a drone with automatic cruise function.

[0054] Specifically, when the monitoring platform is a fixed observation tower, it is typically deployed at strategically important high points, such as mountaintops or high-rise buildings. Its main advantage lies in providing a stable and long-term observation base. By installing infrared thermal imagers and visible light cameras on the observation tower, continuous, fixed-point monitoring of specific areas can be achieved. Power supply and data transmission are usually wired, ensuring system reliability and data transmission stability. This deployment method is particularly suitable for routine monitoring of known high-risk areas or important protected targets.

[0055] When the monitoring platform is a tethered drone, it is connected to a ground base station via a physical cable. This cable not only provides the drone with a continuous power supply, greatly extending its loiter time, but also serves as a high-speed data transmission channel. This type of drone can hover or move within a certain altitude range for extended periods, providing a longer single-mission endurance than traditional drones. Its deployment is flexible; it can quickly ascend to a designated altitude and conduct long-term aerial monitoring of specific areas, making it suitable for scenarios requiring a continuous high-altitude perspective while also maintaining a certain degree of maneuverability.

[0056] When the monitoring platform is a drone with autonomous cruise capability, this drone integrates an advanced navigation and control system, enabling it to autonomously execute flight missions according to preset routes or task plans. Its characteristics include high maneuverability and wide coverage, allowing it to quickly reach remote and terrain-complex areas for patrol and monitoring. Equipped with infrared thermal imagers and visible light cameras, the drone can efficiently acquire images over a large area and adjust its cruise path based on real-time data or preset strategies, achieving dynamic monitoring of potential fire zones. This platform is particularly suitable for scenarios requiring rapid response and flexible coverage of large areas.

[0057] Through the above technical solutions, this invention can flexibly select the most suitable monitoring platform type according to the needs of different forest and grassland fire monitoring scenarios. Fixed lookout towers are suitable for long-term, stable, fixed-point monitoring of fixed areas, ensuring basic coverage and continuity; tethered drones provide longer endurance and more flexible perspectives in scenarios requiring long-term loitering and a certain degree of maneuverability; while drones with automatic cruise capabilities can achieve large-scale, rapid-response patrol monitoring, especially suitable for remote and terrain-complex areas. This diversified platform selection allows the early fire identification and location method to better adapt to various complex geographical environments and monitoring tasks, significantly improving the flexibility, coverage efficiency, and response speed of fire monitoring, thereby effectively solving the problem of insufficient adaptability of a single platform in different scenarios and improving the accuracy and timeliness of early fire identification.

[0058] In the above embodiments of the present invention, a method for early identification and location of forest and grassland fires is proposed. However, in its implementation, simply generating fire alarm information may not be sufficient to support an efficient emergency response. How to transmit this crucial fire information to decision-makers and frontline personnel in a timely, accurate, and intuitive manner is key to ensuring that fires can be dealt with rapidly.

[0059] To address this, the present invention further proposes a specific method for outputting fire alarm information in step S6. This method includes uploading the fire alarm information to the monitoring center server via a wireless communication network. This uploading process aims to ensure that the fire alarm information can be transmitted from the monitoring platform to the core system for centralized processing and management in a timely and reliable manner. The wireless communication network can employ various technologies, such as cellular mobile communication networks (e.g., 4G / 5G), satellite communication networks, wireless local area networks (Wi-Fi), or dedicated wireless data transmission links, depending on the deployment environment, coverage area, and data transmission requirements of the monitoring platform. The monitoring center server is the central node for receiving, storing, processing, and distributing fire alarm information, possessing powerful data processing capabilities and storage capacity, and capable of further analyzing, archiving, and managing the received fire information. The uploading process typically involves data encapsulation, encryption, and the application of transmission protocols (e.g., MQTT, HTTP / HTTPS) to ensure data integrity, security, and real-time performance.

[0060] Based on this, the confirmed fire points are plotted in real-time on the electronic map of the monitoring center, aiming to provide intuitive and visual fire situation awareness. The electronic map of the monitoring center can be a professional map platform based on Geographic Information System (GIS), or a customized application integrating map services (such as Gaode Map, Baidu Map, Google Maps, etc.). Real-time plotting means that once a fire alarm is received, the system can immediately mark the precise three-dimensional geographic coordinates (longitude, latitude, and elevation) of the confirmed fire point on the map using specific layers, icons, or colors. The plotting of the three-dimensional location can more accurately reflect the actual location of the fire point in complex terrain, helping commanders to quickly determine the fire distribution, plan rescue routes, and allocate resources. The plotting information can also include additional attributes such as fire intensity and timestamps, which are displayed through different legends or information boxes.

[0061] Simultaneously, alarm notifications containing geolocation links are sent to pre-defined responsible personnel terminals, enabling precise delivery of fire information and rapid response. These terminals can be mobile smartphones, tablets, walkie-talkies, or dedicated alarm devices, held by forest fire command personnel, rescue team members, and patrol personnel. Alarm notifications can be sent via SMS, instant messaging applications (such as WeChat and DingTalk), email, or dedicated app push notifications. The geolocation link included in the notification allows recipients to directly view the fire's location on a map application, quickly obtaining navigation information or understanding the surrounding environment, significantly reducing information transmission and decision-making response time. The pre-defined mechanism ensures that alarm information is accurately sent to personnel with the appropriate permissions and responsibilities, avoiding information overload or omissions.

[0062] Through the aforementioned technical solution, fire alarm information is uploaded to the monitoring center server, achieving centralized management and efficient processing of fire data, ensuring the integrity and traceability of information. Simultaneously, the three-dimensional location of the fire point is plotted in real-time on the electronic map of the monitoring center, providing commanders with intuitive and accurate fire situation awareness. This helps in quickly assessing the fire's scale, spread trend, and surrounding environment, enabling more scientific firefighting and rescue plans. Furthermore, sending alarm notifications containing geographic location links to pre-defined responsible personnel terminals significantly shortens the information transmission chain, allowing frontline rescue personnel to obtain the precise location of the fire point immediately and quickly navigate to the scene using the geographic location link, significantly improving fire response speed and rescue efficiency. This multi-channel, visualized, and precisely pushed alarm mechanism effectively solves the problems of delayed, unintuitive, and slow response in traditional fire information transmission, comprehensively improving the practicality and emergency response capabilities of early identification and location methods for forest and grassland fires.

[0063] In the above embodiments of the present invention, although a method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light is proposed, which can effectively identify and locate fire points and output alarm information, in practical applications, once a fire is confirmed, especially when the fire intensity is high, simply outputting alarm information may not be sufficient to cope with the rapid development and changes of the fire. For UAV monitoring platforms with automatic cruise capabilities, their dynamism and flexibility are not fully utilized, and they cannot perform more refined and real-time tracking and monitoring of confirmed fire points, which may delay the judgment of the fire spread trend and emergency response.

[0064] In response, this invention further proposes that when the monitoring platform is a drone with automatic cruise capability, the method further includes: after determining the existence of a confirmed fire point, if the intensity of the fire at the confirmed fire point is... Exceeding the fourth preset threshold The system then controls the drone to adjust its flight path and hovering position, and the fire intensity... The calculation formula is: ;in, To confirm the highest temperature of the ignition point, For global background temperature, The fire point area, For the area growth rate, The weighting coefficients are used; the UAV performs close-range tracking and monitoring based on the updated fire point location information, and supplements the analysis process of steps S3 and S4 in real time with the returned image data.

[0065] Specifically, drones, serving as monitoring platforms, possess autonomous navigation capabilities, meaning they can fly along preset routes and adjust their flight paths and hovering positions as needed based on commands. This flexibility gives them a unique advantage in forest and grassland fire monitoring, enabling rapid response and close approach to the fire site for detailed observation, rather than being limited to passive monitoring from fixed locations or preset paths. After the fire identification and location methods confirm the fire site, the system further assesses the fire intensity. Fire intensity It is a comprehensive indicator used to quantify the severity and development potential of a fire. Its calculation formula is as follows: ,in, This represents the highest temperature at which the fire point is confirmed, reflecting the degree of heat of the fire source; This serves as the global background temperature, providing a reference point. The fire point area represents the scale of the fire. The area growth rate represents the speed at which the fire spreads. These are weighting coefficients used to balance the contributions of different factors in fire intensity assessment. When the calculated fire intensity... Exceeding the preset fourth threshold When the fire intensity reaches a preset threshold, the system sends a command to the drone to detach it from its original patrol mission. Based on the latest confirmed fire location information, the drone dynamically plans a new flight path and adjusts to the optimal hovering position. This adjustment allows the drone to observe the fire more closely and stably, acquiring clearer and more detailed image data for subsequent analysis and decision-making. After adjusting its flight path and hovering position, the drone will continue to closely track and monitor the confirmed fire. The drone will update its position in real time based on the fire's movement or spread, always remaining within effective observation range. This proactive tracking and monitoring ensures that the monitoring data remains highly timely and accurate even as the fire situation changes dynamically. Infrared and visible light image data collected by the drone during close-range tracking and monitoring are transmitted back to the ground control system in real time via a wireless communication network. This real-time data is then incorporated into steps S3 (dual-path parallel feature extraction) and S4 (spatiotemporal correlation analysis and fire confirmation) of the early fire identification and location method. In this way, the system can continuously and dynamically assess and confirm the fire situation based on the latest and more refined data, forming a closed-loop monitoring and feedback mechanism, thereby achieving real-time control over the evolution of the fire.

[0066] Through the above technical solution, this invention effectively solves the problem that existing methods, after fire confirmation, especially when the fire intensity is high, are limited to alarm output and cannot perform dynamic, refined tracking and monitoring. When the monitoring platform is a drone with automatic cruise capability, this solution can adjust the monitoring based on the fire intensity. Based on the assessment results, the system intelligently controls the drone to adjust its flight path and hovering position, enabling close-range tracking and monitoring of confirmed fire points. This allows the system to acquire closer, higher-resolution fire point image data and supplement it in real time to subsequent feature extraction and fire confirmation analysis processes, thus forming a dynamic and continuously optimized fire monitoring closed loop. This significantly improves the real-time perception capability and early warning accuracy of fire development, providing more timely and accurate information support for emergency response. It effectively avoids the risk of fire getting out of control due to information lag, and greatly enhances the practicality and effectiveness of early identification and location methods for forest and grassland fires.

[0067] The present invention also provides an embodiment of the practical application of the method of the present invention. I. Application Scenarios A mixed larch and birch forest belt in a northern forest area was selected (the terrain is low hills, with an average altitude of 420m, facing southeast, and an average slope of 12°). The total forest area is 32,000 mu (approximately 2,800 hectares), and the fire prevention level is Class I. The monitoring platform uses a DJI M300RTK drone equipped with an H20T thermal imaging dual-light gimbal camera (infrared resolution 640×512, visible light 4K, focal length 13.5mm, supporting RTK positioning + IMU attitude output). Routine automatic patrol missions are performed every 2 hours to record the entire process of early identification and location of a fire caused by "tourist's illegal use of fire leading to smoldering of dry branches and fallen leaves on the ground." Monitoring period: March 15, 2026, 14:20-14:45, weather: sunny turning cloudy, temperature 11℃~14℃, northwest wind level 3.

[0068] II. Specific Implementation and Numerical Calculations at Each Stage: (a) S1: Synchronous image acquisition and information recording 1. Monitoring platform parameters and data collection time records (Table 1) 2. Original image quality record: Infrared image: Temperature drift value calibrated, NETD (noise equivalent temperature difference) <50mK Visible light image: Illumination intensity approximately 4200 lux, no cloud or fog obstruction. (ii) S2: Image preprocessing and pixel-level registration 1. SIFT Feature Point Extraction and Matching: Number of scale-invariant feature transform (SIFT) feature points extracted from visible light images: 247; Number of temperature gradient feature points extracted from infrared images: 189.

[0069] The nearest neighbor distance ratio (NNDR) strategy was used for matching, with a threshold of 0.75. 83 feature point pairs were successfully matched. The homography matrix was calculated from 20 evenly distributed pairs.

[0070] 2. Calculation of homography transformation matrix The homography matrix H is calculated using the RANSAC algorithm (maximum 1000 iterations, interior point threshold 1.5 pixels): use The infrared image is mapped to the visible light coordinate system to complete the initial registration.

[0071] 3. Attitude angle geometric correction Construct a rotation matrix based on the attitude angles recorded by the IMU. (Pitch angle) Roll angle Yaw angle ): By using a rotation matrix to perform secondary projection correction on the initial registration results, the registration accuracy was improved from 3.2 pixels to 0.7 pixels, meeting the pixel-level fusion requirements.

[0072] (III) S3: Dual-path parallel feature extraction 1. Infrared Image - Dynamic Background Temperature Field Construction Parameter settings: Sliding window width (pixels); Exclusion ratio (Remove the hottest 5% of pixels within the window); Third preset threshold K; minimum area threshold Pixel.

[0073] Example of local background temperature calculation (taking the center pixel (312, 244) of the suspected fire point as an example): Take this pixel as the center Window (225 pixels), remove the one with the highest temperature. After the first 214 pixels, the average temperature is calculated using the remaining 214 pixels: Pixel measured temperature Temperature difference determination: Marked as high-temperature pixel.

[0074] Connected component analysis: Connected component labeling was performed on the entire infrared image frame, and three candidate connected regions were extracted. One of these regions has a pixel area... pixels, more than This area has been identified as a suspected area of ​​thermal anomaly. The average temperature in this area is... highest temperature .

[0075] 2. Visible light image - extraction of suspected smoke regions Color space conversion: Convert the visible light image from RGB to YCrCb color space. The chromaticity range of the smoke image was set based on sample statistics as follows: Extract pixels that meet the chromaticity range to obtain the initial smoke mask.

[0076] Texture feature calculation: The gray-level co-occurrence matrix (GLCM, distance d=1, orientation 0°) is calculated for the mask region, yielding: Contrast (threshold) That is, satisfying entropy (threshold) That is, satisfying Motion consistency analysis: The motion vector field was calculated using the Farneback optical flow method for three consecutive frames (14:23:17, 14:23:27, 14:23:37). The regional average motion direction consistency index is as follows: Motion consistency threshold → Satisfy Based on comprehensive assessment, this region simultaneously meets the criteria for color, texture, and motion consistency, and is therefore identified as a suspected smoke area. This region has a pixel area of ​​58 pixels and a spatial overlap rate of 82% with the suspected thermal anomaly area.

[0077] (iv) S4: Spatiotemporal correlation analysis and fire confirmation 1. Continuous multi-frame tracking Select continuous The frame image sequence (time span: 14:23:17-14:24:27, sampling interval 10 seconds) is used to track the associated region extracted by S3.

[0078] Pixel area sequence (Unit: pixels): Area growth rate calculate: First preset threshold → This meets the conditions for area growth.

[0079] Average temperature series (Unit: K): Rate of temperature rise Calculation: Frame Second preset threshold frame → This satisfies the condition for a temperature rise.

[0080] 2. Fire situation assessment: The associated regions simultaneously meet the following criteria: spatial overlap / adjacency (overlap rate > 80%); and area growth rate. ; Rate of temperature rise ; The associated area was determined to be a confirmed fire point.

[0081] (v) S5: Three-dimensional geolocation 1. Establishment of collinearity equations: Confirm the image coordinates of the fire point. Pixels, converted to millimeters (pixel size 5.5μm): Principal coordinates Pixels (1.760, 1.408) mm Camera focal length mm Monitoring platform spatial location (obtained by RTK): Coordinate system: Rotation matrix elements Taken from matrix S2 .

[0082] Substituting into the photogrammetric collinearity equation: 2. DEM Constraints and Joint Solution: Simultaneous Digital Elevation Model (DEM) Constraints: ; The DEM grid resolution for this area is 5m, and the elevation function is obtained through cubic spline interpolation. Newton's iterative method is used to solve the problem (initial values ​​are taken from the ground point directly below the platform, 6 iterations, tolerance 0.01m).

[0083] The ground coordinates are obtained as follows: 3. Latitude and longitude conversion UTM coordinates back-projected to the WGS84 geographic coordinate system: Positioning error assessment: Compared with the actual fire point coordinates measured by RTK after the event, the horizontal error is 2.3m and the vertical error is 1.8m.

[0084] (vi) S5a: Fire spread trend prediction 1. Meteorological data acquisition: Current wind speed ,wind direction (Northwest wind, 0° north is the reference point, clockwise) 2. Extraction of geographical environmental parameters: Extract fire location parameters from DEM and vegetation distribution map: slope ; slope direction (Southeast slope); Vegetation type coefficient (Larch-birch mixed forest, ground litter load 2.8 kg / m²) 2 ).

[0085] 3. Spread rate and wind direction correction: The spread rate coefficient considering slope, aspect, and type of combustible material The empirical formula is simplified using the Rothermel model: Substitute the values: Terrain-induced wind deflection Empirical formula (valley topography effect): 4. Predicted positions of the fire front predict Forward position in the firefighting zone: Prediction: The fire front will spread approximately 2.82 km southeast in 10 minutes. 2 Range, forward arrival position (UTM:415711.5,4503971.2).

[0086] (vii) S6: Intelligent alarm and information output 1. Fire intensity calculation: Global background temperature The mode of temperature for all pixels in the infrared image is 285.2K (12.1℃). Fire intensity Calculation formula (weighting coefficients) ): Fourth preset threshold → This triggers the drone's close-range tracking mode.

[0087] 2. Alarm Information Generation and Output The system generates structured alarm information: Output method: The data is uploaded to the forest farm monitoring center server via 4G / 5G network (latency 0.8s). The monitoring center's electronic map plots the three-dimensional location of the fire point in real time, overlays the spread prediction sector, and pushes alarm notifications containing geographical links (Gaode Map URI) to the terminals of 5 pre-set forest rangers and 3 forest farm commanders. (viii) S10: Adaptive cruise and close-range tracking of unmanned aerial vehicles 1. Threshold determination and mode switching: Fire intensity The drone automatically switches routes: Original cruising route: 3km radius loop, speed 8m / s, altitude 180m Adjusted flight path: Target fire point (40.6887°N, 118.3764°E), hovering altitude 120m, approach distance 150m 2. Dynamic tracking and data feedback: The drone arrived over the fire point at 14:25:10, adjusted the gimbal pitch angle to -45°, and activated zoom tracking mode. Image data was transmitted back in real-time to supplement the S3 and S4 analysis processes.

[0088] Enhanced data performance: After close approach, the ground resolution of the infrared image is improved from... Increased to 0.18m / pixel, fire point pixel area The resolution was increased from 47 pixels to 213 pixels. Three independent heat cores were identified in the high-temperature zone (>350K) inside the fire point. The deviation between the smoke plume diffusion direction and the wind speed prediction was <8°.

[0089] III. Verification and Comparison of Early Warning Effects: 1. Results of the fire response: At 14:27:30 (3 minutes after the fire was confirmed), an 8-person semi-professional firefighting team from the forest farm arrived at the scene with backpack wind-powered fire extinguishers and high-pressure water mist spray. An investigation revealed that the fire originated from embers of a campfire left unextinguished by tourists, which ignited a layer of dry branches and fallen leaves; the burned area was approximately 11.6 square meters. 2 The fire line was approximately 4.5 meters long. The fire was completely extinguished at 14:35:00, with no reignition.

[0090] 2. Comparison of effects with traditional methods (Table 2) 3. Standardized Case Records (Table 3) IV. Conclusion: This application fully validated the technical effectiveness of the proposed method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light. Through the full-process implementation of a real fire scenario using a drone patrol in a forest farm, the following conclusions were drawn: Very early detection capability: When the fire source is only in the smoldering stage on the surface (burned area <0.5m²) 2 Automatic identification is achieved when the temperature reaches a maximum of 51.6℃, which is 8-12 minutes earlier than traditional manual observation or single-spectrum methods, thus gaining a critical window of opportunity for "early detection and early intervention". High-precision 3D positioning: Through the joint solution of collinear equations and DEM, the planar positioning error is 2.3 meters and the elevation error is 1.8 meters, which meets the precise navigation needs of fire fighting teams; Extremely low false alarm rate: The dynamic background temperature field effectively eliminates interference from environmental heat sources, and the dual-spectrum + smoke motion consistency verification eliminates false targets such as clouds, fog, and dust, with the average monthly false alarm rate controlled at 4.2%; Intelligent dynamic response: After the fire intensity exceeds the threshold, the drone autonomously switches to close-range tracking mode to acquire high-resolution fire scene images, forming a closed loop of "discovery-confirmation-tracking-feedback"; Forward-looking decision support: A fire spread prediction model that couples meteorological and geographic information has a 10-minute prediction error of less than 15%, providing a scientific basis for command and dispatch.

[0091] This method significantly outperforms existing technologies in four dimensions: sensitivity of early identification of forest and grassland fires, positioning accuracy, false alarm suppression, and dynamic response capability. It has engineering feasibility and practical value for large-scale deployment in forest areas.

[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light, characterized in that, Includes the following steps: S1. Synchronous Image Acquisition and Information Recording: Using an infrared thermal imager and a visible light camera mounted on the monitoring platform, the original infrared image and the original visible light image of the area to be monitored are acquired simultaneously, and the geographical location, attitude angle and imaging timestamp of the monitoring platform at the time of acquisition are recorded. S2. Image preprocessing and pixel-level registration: The original infrared image and the original visible light image are preprocessed respectively; then, based on the preprocessed images, the infrared image and the visible light image are pixel-level registered to obtain a registered fused image pair; S3. Dual-path parallel feature extraction: In the registered fused image pair, the following extraction is performed in parallel: From the infrared image, a continuous region exceeding a first temperature threshold is extracted based on the dynamic background temperature field as a suspected thermal anomaly region; From the visible light image, a region that conforms to the preset smoke color and texture features is extracted as a suspected smoke region. S4. Spatiotemporal correlation analysis and fire confirmation: Perform spatial correlation analysis on the suspected thermal anomaly area and the suspected smoke area extracted in step S3; if there are spatially overlapping or adjacent correlated areas, and the correlated area satisfies the preset fire evolution model in a series of consecutive image frames, then the correlated area is determined to be a confirmed fire point. S5. Three-dimensional geolocation: Based on the pixel coordinates of the confirmed fire point in the image, the spatiotemporal information of the monitoring platform, and the preset geographic information model, calculate and output the three-dimensional geographic coordinates of the confirmed fire point; S6. Intelligent Alarm and Information Output: Generate and output fire alarm information containing the three-dimensional geographic coordinates of the confirmed fire point, the fire intensity, and the imaging timestamp.

2. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light as described in claim 1, characterized in that, In step S2, the pixel-level registration of the infrared image and the visible light image specifically includes: Extract scale-invariant feature transformation feature points from the visible light image and find corresponding temperature gradient feature points in the infrared image; Based on the successfully matched feature point pairs, the homography transformation matrix is ​​calculated, and the infrared image is mapped to the coordinate system of the visible light image using the homography transformation matrix to complete the initial registration. The attitude angle parameters of the monitoring platform are used to perform geometric correction on the preliminary registration results to obtain the final registration results.

3. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, In step S3, the extraction of continuous regions exceeding the first temperature threshold based on the dynamic background temperature field specifically involves: The sliding window statistical method is used to divide the current infrared image into blocks for statistical analysis. For each block, the statistical analysis is performed on pixels. Centered Window, excluding the one with the highest temperature inside the window. After determining the pixel, the average temperature of the remaining pixels is calculated as the local background temperature at that location. This forms a dynamic background temperature field; among which, For window width, This is the preset exclusion ratio; The measured temperature values ​​of each pixel With the corresponding local background temperature A comparison will satisfy The pixels marked as high-temperature pixels; among them, The third preset threshold; Perform connected component analysis on the high-temperature pixels, and select pixels with areas greater than the minimum area threshold. The connected regions were identified as the suspected thermal anomaly regions.

4. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light as described in claim 1, characterized in that, In step S4, the fire evolution model that satisfies the preset parameters specifically includes: Suppose the associated region is continuous The pixel area sequence in the frame image sequence is Calculate its area growth rate It needs to exceed the first preset threshold. ,Right now: ； At the same time, assume that the region is continuous The average temperature sequence in the frame is Calculate its temperature rise rate It needs to exceed the second preset threshold. ,Right now: ； in, It is a positive integer greater than or equal to 3. This is the start time index of the frame sequence.

5. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, In step S5, calculating the three-dimensional geographic coordinates of the confirmed fire point specifically includes: Based on the collinearity equation in photogrammetry, the image point coordinates of the confirmed fire point are... Convert to ground coordinates The collinear equation is expressed as: ； ； in, Let the principal point coordinates be... For camera focal length, To monitor the spatial location of the platform at the time of data collection, These are the elements of the rotation matrix calculated based on the attitude angles of the monitoring platform; The surface elevation represented by the digital elevation model in the aforementioned geographic information model Solving the above equations simultaneously yields the geodetic latitude and longitude coordinates of the confirmed fire point. and elevation value .

6. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, The process after step S5 also includes: S5a. Fire Spread Trend Prediction: Obtain Current Wind Speed and wind direction Meteorological data; based on the confirmed location of the fire point The meteorological data and the slope in the geographic information model. Slope aspect and vegetation type coefficient ,predict The position of the front line in the fire after time : ； ； in, This is a propagation rate coefficient that takes into account slope, aspect, and type of combustible material. The terrain corrects for the wind direction at a certain angle; the prediction results are then appended to the fire alarm information.

7. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, In step S3, extracting the region that conforms to the preset smoke color and texture features specifically includes: The visible light image is converted to the YCrCb color space, and the chromaticity range of the suspected smoke is defined to satisfy the following: and ; Within the stated chroma range, the texture contrast of the region is further calculated based on the gray-level co-occurrence matrix. With entropy , must meet and ,in and Texture threshold; For multiple consecutive frames of visible light images, the motion vector field of the region is calculated using the optical flow method, and the average consistency of its motion direction is statistically analyzed. It must be greater than the motion consistency threshold. ; Based on the combined conditions of chroma, texture, and motion consistency, the suspected smoke area is determined and extracted.

8. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light as described in claim 1, characterized in that... The monitoring platform is one of the following: a fixed observation tower, a tethered drone, or a drone with automatic cruise capability.

9. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, In step S6, the output of fire alarm information specifically includes: The fire alarm information is uploaded to the monitoring center server via a wireless communication network; The three-dimensional location of the confirmed fire point is plotted in real time on the electronic map of the monitoring center; It also sends an alarm notification containing a geolocation link to the terminal of the designated responsible personnel.

10. The method for early identification and location of forest and grassland fires based on the fusion of infrared thermal imaging and visible light according to claim 1, characterized in that, When the monitoring platform is a drone with automatic cruise capability, the method further includes: After determining the existence of a confirmed fire point, if the intensity of the fire at the confirmed fire point is... Exceeding the fourth preset threshold The system then controls the drone to adjust its flight path and hovering position, and the fire intensity... The calculation formula is: ； in, To confirm the highest temperature of the ignition point, For global background temperature, The fire point area, The area growth rate as defined in claim 4, These are weighting coefficients; The drone performs close-range tracking and monitoring based on the updated fire location information, and supplements the analysis process in steps S3 and S4 in real time with the transmitted image data.

Images