Forest fire prevention monitoring system and method based on image data

Abstract

The invention provides a forest fire prevention monitoring system and method based on image data, and relates to the technical field of forest fire prevention monitoring and early warning, and the system comprises a multi-view-angle access module which is used for accessing image data streams of at least two forest region monitoring imaging units which are arranged in a spatial separation manner and have different view angles, according to the forest fire prevention monitoring system and method based on the image data, through integration of a multi-view imaging system and a visibility evaluation method, accurate forest fire monitoring is achieved, and the forest fire prevention monitoring system and method based on the image data have the advantages of being high in practicability and high in practicability. The image data from the monitoring imaging units at different positions are accessed, visibility calculation is carried out based on terrain and vegetation data, the system accurately evaluates the effective view field of each imaging unit, it is ensured that the system dynamically selects the optimal view angle to carry out fire monitoring, and the coverage range and accuracy of fire detection are improved.

Description

Technical Field

[0001] This invention relates to the field of forest fire monitoring and early warning technology, specifically to a forest fire monitoring system and method based on image data. Background Technology

[0002] In existing forest fire monitoring technologies, forest areas typically employ methods such as lookout tower monitoring, manual patrols, video surveillance using fixed-point cameras or PTZ cameras, drone inspections, satellite remote sensing, and monitoring of environmental parameters such as temperature, humidity, and wind speed to detect and issue alarms for fires. Among these, image-based monitoring solutions continuously acquire visible light images or video streams through front-end cameras. These are combined with image enhancement, background modeling, motion detection, color and texture analysis, and temporal change feature analysis, or deep learning target detection and classification models, to automatically identify fire targets such as smoke and flames. Alarm information is then uploaded to a command platform for manual review, tiered handling, and necessary coordinated responses. Some solutions also overlay geographic information displays and multi-point collaborative observation to assist in fire location and trend analysis.

[0003] However, in complex forest environments, the above technologies still have significant shortcomings: the single perspective is affected by the obstruction of mountains and vegetation, a large amount of manual confirmation is still required after the alarm is issued, and the spatial positioning reliability of the fire point is insufficient, making it difficult to reliably meet the needs of early and reliable forest fire monitoring. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a forest fire monitoring system and method based on image data. The technical problem this invention aims to solve is: how to address the issues of single-view occlusion and unreliable fire location in forest fire monitoring through multi-view collaborative imaging and visibility assessment methods.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a forest fire monitoring system based on image data, comprising: Multi-view access module: used to access image data streams from at least two spatially separated forest area monitoring imaging units with different viewing angles, and to assign imaging unit identifiers and timestamp information to the image data streams. The multi-view access module obtains the calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation position coordinates. Visibility assessment module: used to acquire forest spatial structure data covering a preset monitoring area. The forest spatial structure data includes three-dimensional terrain elevation data and vegetation height data. Based on the camera pose parameters, the installation location coordinates and the forest spatial structure data, the module performs line-of-sight visibility calculation on the preset monitoring area to obtain the set of visible areas and visibility score corresponding to each imaging unit. Based on the set of visible areas and visibility score, the module determines the set of effective viewing angles for fire confirmation and location. Cross-view collaborative imaging module: When any imaging unit detects a suspected target of smoke or flame in its image data stream, it generates a trigger event. The trigger event includes the trigger imaging unit identifier, timestamp, and pixel position of the suspected target in the image data stream. It acquires the trigger image frame corresponding to the timestamp, determines the target line of sight based on the pixel position and the calibration parameters of the corresponding imaging unit, and selects at least one non-triggered imaging unit from the effective view set to issue a linkage imaging command. The linkage imaging command includes a synchronous imaging time window and the target line of sight. The non-triggered imaging unit converts the target line of sight into the gimbal azimuth and pitch angles according to its own calibration parameters and completes the pointing. It acquires a verification image sequence, performs time alignment between the trigger image frame and the verification image sequence, and generates a collaborative imaging record. The collaborative imaging record includes the participating imaging unit identifier, timestamp, gimbal azimuth, pitch angle, and corresponding image frame. Spatial positioning and reliability calculation module: used to determine the line of sight of each imaging unit based on the installation position coordinates of at least two imaging units in the collaborative imaging record, the azimuth angle and the pitch angle of the gimbal, and to calculate the intersection point of the line of sight to obtain the spatial position coordinates of the fire. The module also calculates the positioning reliability based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units. Alarm output module: When the positioning confidence reaches a preset confidence threshold, it outputs alarm information, which includes the fire spatial location coordinates, positioning confidence, and evidence frames recorded by the collaborative imaging. When the positioning confidence does not reach the preset confidence threshold, it outputs verification information, which includes the trigger imaging unit identifier, the imaging unit identifier participating in the linkage, and the reason code for not reaching the threshold.

[0006] Preferably, the forest area monitoring imaging unit includes a fixed wide-angle imaging unit and a rotatable pan-tilt imaging unit. The triggering event is generated by the fixed wide-angle imaging unit, and the linkage imaging command is sent to the rotatable pan-tilt imaging unit to obtain the verification image sequence.

[0007] Preferably, the spatial structure data of the forest area is expressed in raster form and registered with a geographic coordinate system. The line-of-sight visibility calculation includes performing segment-by-segment occlusion determination on the three-dimensional terrain elevation data and vegetation height data along the line-of-sight direction of the imaging unit, and outputting the set of visible areas.

[0008] Preferably, the segment-by-segment occlusion determination includes: discretizing the line of sight from the imaging unit to the preset monitoring area into multiple sampling segments according to a preset sampling step size; acquiring corresponding three-dimensional terrain elevation data and vegetation height data for each sampling segment; comparing the three-dimensional terrain elevation data and vegetation height data with the height of the line of sight at the corresponding position; determining that the line of sight is occluded when any sampling segment meets the requirements of terrain elevation or vegetation height being higher than the height of the line of sight; and generating the set of visible areas based on the set of unoccluded lines of sight.

[0009] Preferably, the visibility score is determined by the coverage ratio of the visible area set within the preset monitoring area, where the coverage ratio is the ratio of the area covered by the visible area set to the area of ​​the preset monitoring area, and the effective viewing angle set consists of imaging units with a visibility score not lower than a preset visibility threshold.

[0010] Preferably, the intersection point calculation adopts a multi-line-of-sight minimum deviation intersection method. The deviation is a statistical measure of the minimum distance from the fire spatial location coordinates to each line of sight. The positioning reliability is calculated by the number of imaging units participating in the intersection point calculation, the line-of-sight intersection deviation, and the visibility score of the participating imaging units according to a preset weight. When the line-of-sight intersection deviation is greater than a preset deviation threshold or the number of imaging units participating in the intersection point calculation is less than a preset number threshold, the positioning reliability is determined to be below the preset reliability threshold.

[0011] Preferably, the reason code for not reaching the threshold is triggered and output by the following conditions: the visibility score is lower than the preset visibility threshold, corresponding to the first reason code; the number of imaging units participating in the intersection point calculation is less than the preset number threshold, corresponding to the second reason code; the deviation of the intersection point calculation is greater than the preset deviation threshold, corresponding to the third reason code. The evidence frame includes the trigger image frame and the image frame in the verification image sequence that is time-aligned with the trigger image frame, and is output together with the corresponding imaging unit identifier, timestamp, gimbal azimuth angle and pitch angle.

[0012] A forest fire monitoring method based on image data includes: S1. Access image data streams from at least two spatially separated forest area monitoring imaging units with different viewing angles, and assign imaging unit identifiers and timestamp information to the image data streams, and obtain calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation position coordinates; S2. Obtain forest spatial structure data covering a preset monitoring area, perform line-of-sight visibility calculation based on the camera pose parameters, the installation location coordinates and the forest spatial structure data, obtain the set of visible areas and visibility score corresponding to each imaging unit, and determine the set of effective viewing angles based on the set of visible areas and visibility score; S3. When any imaging unit detects a suspected target of smoke or flame in its image data stream, a trigger event is generated. The trigger event includes the trigger imaging unit identifier, timestamp, and pixel position of the suspected target. A trigger image frame corresponding to the timestamp is acquired. The target line of sight is determined based on the pixel position and the calibration parameters of the trigger imaging unit. At least one non-triggered imaging unit is selected from the effective viewpoint set to issue a linkage imaging command. The linkage imaging command includes a synchronous imaging time window and the target line of sight. The non-triggered imaging unit converts the target line of sight into the gimbal azimuth and pitch angles and completes the pointing. A verification image sequence is acquired. The trigger image frame and the verification image sequence are time-aligned to generate a collaborative imaging record. The collaborative imaging record includes the participating imaging unit identifier, timestamp, gimbal azimuth, pitch angle, and corresponding image frame. S4. Based on the installation position coordinates of at least two imaging units in the collaborative imaging record, the azimuth and pitch angles of the pan-tilt unit are used to determine the line of sight of each imaging unit, and the intersection point of the line of sight is calculated to obtain the spatial position coordinates of the fire; the positioning reliability is calculated based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units. S5. When the positioning confidence reaches a preset confidence threshold, an alarm message is output. The alarm message includes the spatial coordinates of the fire, the positioning confidence, and the evidence frame recorded by the collaborative imaging. When the positioning confidence does not reach the preset confidence threshold, a verification message is output. The verification message includes the trigger imaging unit identifier, the imaging unit identifier participating in the linkage, and the reason code for not reaching the threshold.

[0013] This invention provides a forest fire monitoring system and method based on image data. It has the following beneficial effects: This image-based forest fire monitoring system and method achieves precise forest fire monitoring by integrating a multi-view imaging system and a visibility assessment method. It accesses image data from monitoring imaging units at different locations and performs visibility calculations based on terrain and vegetation data. The system accurately assesses the effective field of view of each imaging unit, ensuring that the system dynamically selects the best viewpoint for fire monitoring and improving the coverage and accuracy of fire detection.

[0014] Employing cross-view collaborative imaging technology ensures that when the monitoring system detects suspected targets such as smoke or flames, it quickly initiates a coordinated imaging command. By selecting effective imaging units based on the target's line-of-sight direction, the system simultaneously acquires and verifies images, performs time alignment, and generates collaborative imaging records. This allows multiple imaging units to jointly confirm the target from different angles, improving the accuracy of fire location. Combining intersection point calculation and location reliability assessment, the system reliably determines the fire location under different conditions and promptly outputs alarm information when the reliability reaches a preset threshold, supporting rapid fire response. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a flowchart of the cross-view collaborative imaging process of the present invention; Figure 3 This is a flowchart of the overall monitoring method of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1 like Figure 1-3 As shown, this embodiment of the invention provides a forest fire monitoring system based on image data, comprising: Multi-view access module: Used to access image data streams from at least two spatially separated forest area monitoring imaging units with different viewing angles, and to assign imaging unit identifiers and timestamp information to the image data streams. The multi-view access module obtains the calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation position coordinates. The forest area monitoring imaging unit includes a fixed wide-angle imaging unit and a rotatable pan-tilt imaging unit. Triggering events are generated by the fixed wide-angle imaging unit, and linkage imaging commands are sent to the rotatable pan-tilt imaging unit to obtain verification image sequences.

[0018] The visibility assessment module acquires forest spatial structure data covering a pre-defined monitoring area. This data includes 3D terrain elevation data and vegetation height data. Based on camera pose parameters, installation location coordinates, and the forest spatial structure data, it performs line-of-sight visibility calculations on the pre-defined monitoring area, obtaining the visible area set and visibility score for each imaging unit. Based on the visible area set and visibility score, it determines the effective set of viewing angles for fire confirmation and location. The forest spatial structure data is rasterized and registered with a geographic coordinate system. The line-of-sight visibility calculation includes segment-by-segment occlusion determination of the 3D terrain elevation data and vegetation height data along the line-of-sight direction of the imaging unit, outputting the visible area set. The segment-by-segment occlusion determination includes: discretizing the line of sight from the imaging unit to the preset monitoring area into multiple sampling segments according to a preset sampling step size; acquiring corresponding 3D terrain elevation data and vegetation height data for each sampling segment; comparing the 3D terrain elevation data and vegetation height data with the height of the line of sight at the corresponding position; determining that the line of sight is occluded when any sampling segment meets the requirement that the terrain elevation or vegetation height is higher than the height of the line of sight; and generating a set of visible areas based on the set of unoccluded lines of sight. The visibility score is determined by the coverage ratio of the set of visible areas within the preset monitoring area, which is the ratio of the area covered by the set of visible areas to the area of ​​the preset monitoring area. The effective viewing angle set consists of imaging units with a visibility score not lower than a preset visibility threshold.

[0019] To improve the accuracy of fire detection, the system can incorporate deep learning algorithms to extract and analyze the features of smoke or flames, thereby optimizing the detection results.

[0020] The system can combine data from infrared sensors and temperature and humidity sensors and perform fusion processing during visibility assessment to improve the system's fire monitoring capabilities in complex environments.

[0021] Cross-view collaborative imaging module: When any imaging unit detects a suspected target of smoke or flame in its image data stream, it generates a trigger event. The trigger event includes the triggering imaging unit identifier, timestamp, and the pixel position of the suspected target in the image data stream. It acquires the trigger image frame corresponding to the timestamp, determines the target's line of sight direction based on the pixel position and the corresponding imaging unit's calibration parameters, and selects at least one non-triggered imaging unit from the effective view set to issue a linkage imaging command. The linkage imaging command includes a synchronized imaging time window and the target's line of sight direction. The non-triggered imaging unit converts the target's line of sight direction into the gimbal azimuth and pitch angles according to its own calibration parameters and completes the pointing. It acquires a verification image sequence, performs time alignment between the trigger image frame and the verification image sequence, and generates a collaborative imaging record. The collaborative imaging record includes the participating imaging unit identifier, timestamp, gimbal azimuth and pitch angles, and the corresponding image frame.

[0022] If an imaging unit fails to provide data on time, the system will automatically switch to a backup device for monitoring to ensure uninterrupted monitoring.

[0023] Image data streams are uploaded to the central server in real time, and are compressed, noise-reduced, and encrypted on local devices. A distributed storage solution is used to ensure data security and efficient transmission.

[0024] The spatial positioning and reliability calculation module is used to determine the lines of sight of each imaging unit based on the installation position coordinates of at least two imaging units in the collaborative imaging record, as well as the azimuth and elevation angles of the pan-tilt unit. It then calculates the intersection points of these lines of sight to obtain the spatial location coordinates of the fire. The module calculates the positioning reliability based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units. The intersection point calculation uses a multi-line-of-sight minimum deviation intersection method. The deviation is a statistical measure of the minimum distance from the fire's spatial location coordinates to each line of sight. The positioning reliability is calculated by weighting the number of imaging units participating in the intersection point calculation, the line-of-sight intersection deviation, and the visibility score of the participating imaging units according to preset weights. If the line-of-sight intersection deviation exceeds a preset deviation threshold or the number of imaging units participating in the intersection point calculation is less than a preset number threshold, the positioning reliability is determined to have failed to reach the preset reliability threshold.

[0025] The system incorporates fire intensity assessment, combining data such as temperature and smoke concentration to evaluate the intensity and spread trend of the fire.

[0026] To ensure data security, the system uses the AES encryption algorithm to encrypt image and communication data, and employs an authentication mechanism to ensure that only authorized personnel can access the monitoring data.

[0027] Alarm Output Module: This module outputs alarm information when the positioning reliability reaches a preset reliability threshold. The alarm information includes the fire's spatial location coordinates, positioning reliability, and evidence frames recorded by collaborative imaging. When the positioning reliability does not reach the preset reliability threshold, it outputs verification information, including the triggering imaging unit identifier, the identifiers of the participating imaging units, and a reason code for not reaching the threshold. The reason code for not reaching the threshold is triggered and output under the following conditions: a visibility score lower than a preset visibility threshold (corresponding to the first reason code); the number of imaging units participating in the intersection point calculation less than a preset number threshold (corresponding to the second reason code); and the deviation in the intersection point calculation greater than a preset deviation threshold (corresponding to the third reason code). The evidence frames include the trigger image frame and image frames in the verification image sequence that are time-aligned with the trigger image frame, and are output along with the corresponding imaging unit identifier, timestamp, PTZ azimuth angle, and elevation angle.

[0028] The system includes a user feedback mechanism, automatically optimizes the recognition model after a fire is confirmed, and improves recognition accuracy by continuously updating the training set.

[0029] A forest fire monitoring method based on image data includes: S1. Access the image data streams of at least two spatially separated forest area monitoring imaging units with different viewing angles, and assign imaging unit identifiers and timestamp information to the image data streams. Obtain the calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation location coordinates.

[0030] S2. Acquire forest spatial structure data covering the preset monitoring area, perform line-of-sight visibility calculation based on camera pose parameters, installation location coordinates and forest spatial structure data, obtain the set of visible areas and visibility score corresponding to each imaging unit, and determine the set of effective viewing angles based on the set of visible areas and visibility score.

[0031] S3. When any imaging unit detects a suspected target of smoke or flame in its image data stream, a trigger event is generated. The trigger event includes the trigger imaging unit identifier, timestamp, and pixel position of the suspected target. The trigger image frame corresponding to the timestamp is acquired. The target line of sight is determined based on the pixel position and the calibration parameters of the trigger imaging unit. At least one non-triggered imaging unit is selected from the effective viewpoint set to issue a linkage imaging command. The linkage imaging command includes a synchronous imaging time window and the target line of sight. The non-triggered imaging unit converts the target line of sight into the gimbal azimuth and pitch angles and completes the pointing. A verification image sequence is acquired. The trigger image frame and the verification image sequence are time-aligned to generate a collaborative imaging record. The collaborative imaging record includes the identifier of the participating imaging unit, timestamp, gimbal azimuth, pitch angle, and corresponding image frame.

[0032] S4. Based on the installation position coordinates of at least two imaging units in the collaborative imaging record, the azimuth and elevation angles of the pan-tilt unit are used to determine the line of sight of each imaging unit, and the intersection point of the lines of sight is calculated to obtain the spatial position coordinates of the fire. The positioning reliability is calculated based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units.

[0033] S5. When the positioning confidence level reaches the preset confidence threshold, an alarm message is output. The alarm message includes the spatial coordinates of the fire, the positioning confidence level, and evidence frames recorded by the collaborative imaging. When the positioning confidence level does not reach the preset confidence threshold, a verification message is output. The verification message includes the triggering imaging unit identifier, the identifier of the imaging unit involved in the linkage, and the reason code for not reaching the threshold.

[0034] Example 2 This embodiment describes a forest fire monitoring system and method based on image data. The system assesses the visibility of forest fire monitoring to ensure effective coverage of the monitoring area and improve the accuracy of fire detection and location. Specific implementation details are as follows: 1. Data Acquisition and Processing Acquisition of spatial structure data for forest areas: The forest area monitored in this embodiment is located in a provincial forest reserve, with a total area of ​​100 square kilometers. The terrain and vegetation data within the area were rasterized using a geographic information system (GIS) to obtain 1-meter × 1-meter raster resolution data. Specific data are as follows: Topographic elevation data: The terrain data was obtained through lidar scanning, and the terrain elevation data covers an area ranging from 300 meters to 1200 meters, with multiple mountains and valleys in the overall area. Specific terrain data was provided by the National Remote Sensing Center.

[0035] Vegetation height data: Vegetation height data was acquired through high-resolution satellite imagery, with the highest point of vegetation measuring 30 meters and the lowest point at 0 meters. The data was generated using remote sensing image processing technology and covers the entire forest reserve.

[0036] Remote sensing data is updated quarterly. If fires occur frequently or the monitored area experiences significant seasonal changes, the data update frequency needs to be increased to ensure more timely monitoring.

[0037] Camera calibration parameters were obtained: Fixed wide-angle cameras and rotatable gimbal cameras were deployed within the forest reserve.

[0038] The calibration of the two cameras was conducted by Company A's technical team, using a self-calibration method. The calibration board used during the calibration process was 30cm x 30cm, and it was ensured that the calibration board was photographed at different azimuth and pitch angles, with a calibration error tolerance range of ±0.5mm. Specific calibration parameters and installation locations are as follows: Imaging Unit 1: A fixed wide-angle camera, installed in the northwest corner of the forest reserve at a height of 50 meters.

[0039] The specific installation location is: latitude 30.6545°N, longitude 118.2456°E, altitude 50 meters.

[0040] Camera parameters: focal length 35mm, pixel size 0.005mm, field of view 90°, and after calibration, the camera pose is 30° pitch and 90° azimuth.

[0041] Imaging Unit 2: A rotatable gimbal camera, installed in the southeast corner of the forest reserve at a height of 60 meters.

[0042] The specific installation location is: latitude 30.6670°N, longitude 118.2250°E, altitude 60 meters.

[0043] Camera parameters: focal length 50mm, pixel size 0.004mm, angle of view 75°, and after calibration, the camera pose is 45° pitch and 135° azimuth.

[0044] The ambient temperature during camera calibration was controlled between 20℃ and 2℃ to eliminate interference from environmental factors. The tolerance for calibration error was set at ±0.5mm, and experiments verified that the impact of the error range on line-of-sight calculation was within acceptable limits.

[0045] 2. Line-of-sight visibility calculation Line-of-sight model construction: Based on the installation position and pose parameters of the imaging units, the line-of-sight direction of each imaging unit is calculated using a line-of-sight model. The line-of-sight direction is determined by the camera's azimuth and pitch angles. A spherical coordinate transformation method is used during calculation to ensure the accuracy of the line-of-sight calculation. The camera's azimuth and pitch angles are converted into line-of-sight vectors in three-dimensional space to avoid errors in the line-of-sight direction calculation.

[0046] Imaging unit 1 directs its line of sight to a specific area in the western part of the forest, with an elevation angle of 30° and an azimuth angle of 90°. Imaging unit 2 directs its line of sight to the eastern area, with an elevation angle of 45° and an azimuth angle of 135°.

[0047] Occlusion determination: Occlusion determination is performed on each line of sight using a discretization method.

[0048] Each sampling segment is 50 meters long. Using rasterized terrain and vegetation data, the line of sight in each sampling segment is determined to be obstructed. The following are the obstruction results calculated based on actual terrain data: Occlusion determination of imaging unit 1: At the first sampling point, 50 meters away, the terrain elevation was 700 meters, the vegetation height was 15 meters, the line of sight was 60 meters, and the line of sight was unobstructed.

[0049] At the second sampling point, 100 meters away, the terrain elevation was 800 meters, the vegetation height was 18 meters, the line of sight was 90 meters, and the line of sight was unobstructed.

[0050] At the third sampling point, 150 meters away, the terrain elevation is 1100 meters, the vegetation height is 20 meters, the line of sight is 120 meters, and the line of sight is obstructed.

[0051] Occlusion determination of imaging unit 2: Using the same 50-meter step size, similar segment-by-segment calculations were performed based on the camera's installation position and line-of-sight direction. The final results show that the line of sight of imaging unit 2 is obstructed by mountains in some areas.

[0052] The occlusion determination method employs linear interpolation to handle topographic variations in each sampling segment. Normalized Difference Vegetation Index (NDVI) data is used to dynamically estimate vegetation height. For complex terrain areas, a weighted occlusion determination method is employed, which weights the occlusion in each sampling segment based on slope and terrain features, thereby improving the accuracy of occlusion determination.

[0053] Viewport generation: Imaging unit 1's visible area covers 50 square kilometers of forest, while imaging unit 2's visible area covers 60 square kilometers. There is a 40 square kilometer overlap between their visible areas, resulting in a combined visible area of ​​80 square kilometers for both cameras.

[0054] The overlapping area of ​​the visible regions of the two imaging units is calculated using a geometric intersection algorithm. A polygon intersection algorithm is used for precise calculation, which ensures the accuracy of the overlapping area calculation and avoids errors in complex terrain.

[0055] 3. Visibility score calculation Calculate the coverage ratio: Visibility score of imaging unit 1: By calculating the coverage ratio of the visible area to the total monitoring area, the visibility score of imaging unit 1 is obtained as follows: Visibility score of imaging unit 2: By calculating the ratio of the coverage area of ​​imaging unit 2 to the total monitoring area, the visibility score of imaging unit 2 is obtained as follows: Through multiple experiments and verification using historical data, the threshold for visibility scoring was determined by setting it to 50% based on historical data. The verification results indicate that viewing angles with visibility scores below imaging unit 2 contribute less to the accuracy of fire detection and location.

[0056] Effective perspective determined: According to the preset visibility scoring threshold, the viewing angles of imaging unit 1 and imaging unit 2 both meet the requirements of effective viewing angles and are included in the effective viewing angle set.

[0057] 4. Results Output Imaging unit 1 has a visible area of ​​50 square kilometers and a viewing angle score of 50%, while imaging unit 2 has a visible area of ​​60 square kilometers and a viewing angle score of 60%.

[0058] Effective viewing angle set: The viewing angles of imaging unit 1 and imaging unit 2 are both effective viewing angles, covering 80 square kilometers of the monitoring area.

[0059] The system supports dynamic adjustment. When the monitoring imaging unit malfunctions or cannot cover certain areas, the system will automatically select the nearest effective viewing angle imaging unit to fill the gap, ensuring the continuity of the monitored area.

[0060] Through the above steps, this embodiment ultimately determines the effective set of viewing angles, providing support for fire confirmation and location. The system also features dynamic adjustment capabilities, responding in real-time to monitoring imaging unit malfunctions or changes in viewing angle, ensuring continuous coverage of the monitoring area, providing reliable technical support for forest fire prevention monitoring, and improving the accuracy and real-time performance of fire monitoring.

[0061] Example 3 This embodiment describes a forest fire monitoring system and method based on image data. Through multi-view imaging and real-time data processing technology, it improves the accuracy and efficiency of fire monitoring and location. The specific implementation method is as follows: 1. System Configuration and Installation In a forest reserve in Fujian Province, covering an area of ​​approximately 10 square kilometers, with a monitoring area 1000 meters long and 1000 meters wide, three imaging units were installed. The specific configuration is as follows: Imaging Unit A: Fixed wide-angle imaging unit, installed at (0, 0, 30), that is, installed in the southwest corner of the monitored area at a height of 30 meters.

[0062] The imaging unit uses a commercial Canon camera. The camera's intrinsic parameters have been calibrated in practice, including focal length, sensor size, distortion coefficient, etc. The transformation matrix and camera pose parameters have been obtained through calibration. The focal length is 24mm, the horizontal field of view is 84°, and the vertical field of view is 54°.

[0063] Imaging Unit B: Fixed wide-angle imaging unit, installed at (1000, 0, 30), that is, installed in the northeast corner of the monitored area at a height of 30 meters.

[0064] Imaging unit B uses the same Canon camera as A. The camera calibration parameters have been obtained through the calibration system and are consistent with the coordinate system. The focal length is 24mm, the horizontal field of view is 84°, and the vertical field of view is 54°.

[0065] Imaging Unit C: A rotatable pan-tilt imaging unit, installed at (500, 500, 30), that is, installed in the center of the monitoring area slightly southwest, at a height of 30 meters.

[0066] Imaging unit C uses a Sony camera, and the camera's intrinsic parameters are obtained through a calibration program. The focal length is 35mm, the horizontal field of view is 72°, and the vertical field of view is 48°. It has high autofocus accuracy, and the calibration results have been incorporated into the system configuration.

[0067] All imaging units are calibrated through the multi-view access module to obtain camera intrinsic parameters, camera pose parameters, and installation position coordinates.

[0068] 2. Spatial structure data of forest areas Spatial structure data of the monitored area is obtained by remote sensing satellites and lidar scanning.

[0069] Topographic elevation data: Elevation data obtained from LiDAR scanning, in 10m x 10m grid format, covering the monitored area. The terrain elevation ranges from 10 meters to 50 meters; specific data is as follows: In the southwest of the monitoring area, near imaging unit A, the terrain elevation ranges from 12 meters to 30 meters.

[0070] In the northeastern part of the monitoring area, near imaging unit B, the terrain elevation ranges from 18 meters to 45 meters.

[0071] The central part of the monitoring area, near imaging unit C, has a terrain elevation ranging from 15 meters to 40 meters.

[0072] Vegetation height data: acquired from multispectral imagery of a remote sensing satellite, with a resolution of 10 meters × 10 meters. The vegetation height data range for the monitored area is as follows: The vegetation height in the southwest ranges from 2 to 25 meters, in the northeast from 5 to 30 meters, and in the central region from 3 to 20 meters.

[0073] Based on the above spatial data, the system performs line-of-sight visibility calculations and obtains the visible area coverage of each imaging unit: The visible area coverage of imaging unit A is 82%, that of imaging unit B is 87%, and that of imaging unit C is 78%.

[0074] 3. Triggering the event At 12:05:30 on January 15, 2026, imaging unit A detected a suspected fire target in the image data stream using image analysis technology. The target is located at (400, 200) within the monitored area, specifically in the northwest of the monitored area. The triggering event includes the following data: The trigger imaging unit is A, the timestamp is January 15, 2026, 12:05:30, and the suspected target location is within the monitoring area at coordinates (400, 200) pixels.

[0075] 4. Calculation of target line of sight direction Imaging unit A calculates the target's line-of-sight direction using a geometric model and image processing algorithms, based on calibration parameters such as focal length and sensor size, as well as the target's pixel position. The specific calculation process is as follows: The azimuth is 45°, meaning the target is located in the northwest direction, and the elevation angle is 10°, meaning the target is relatively close to the ground.

[0076] The calculations are based on camera intrinsic parameters, target pixel coordinates, and known camera pose parameters.

[0077] 5. Gimbal linkage commands Based on the line-of-sight direction of imaging unit A, the system selected imaging units B and C for coordinated imaging. The coordinated imaging commands include the following: Synchronous imaging time window: January 15, 2026, 12:05:30 to 12:05:45.

[0078] The target's line-of-sight direction is 45° azimuth and 10° elevation. Imaging units B and C adjust the gimbal's azimuth and elevation angles based on the calculated line-of-sight direction. The adjustment results are as follows: The gimbal for imaging unit B has an azimuth angle of 50° and a pitch angle of 8°, while the gimbal for imaging unit C has an azimuth angle of 45° and a pitch angle of 10°.

[0079] Imaging units B and C adjust the gimbal and point it at the target based on the calculation results, in preparation for acquiring and verifying the image.

[0080] 6. Cooperative imaging record generation During the linked imaging process, the trigger image and the verification image sequences were time-aligned. The specific process is as follows: The trigger image timestamp for imaging unit A is 12:05:30, the verification image timestamp for imaging unit B is 12:05:35, and the verification image timestamp for imaging unit C is 12:05:40.

[0081] The collaborative imaging record contains the following data: the participating imaging units are identified as A, B, and C.

[0082] Timestamps: 12:05:30 is the trigger image, 12:05:35 is the B verification image, and 12:05:40 is the C verification image.

[0083] The gimbal azimuth angles are 50° at B and 45° at C, and the gimbal elevation angles are 8° at B and 10° at C. Image frame information: Path information between the trigger image and the verification image frame.

[0084] 7. Fire location and credibility calculation Through intersection calculations, the system calculates the spatial coordinates of the fire location based on the installation position of the participating imaging units and the azimuth and elevation angles of the pan-tilt unit: Fire location coordinates: 420 meters, 210 meters, height 32 meters.

[0085] The intersection calculation uses the minimum deviation intersection method, which calculates the intersection point of the fire sources through multiple lines of sight and uses the minimum distance from each line of sight to the intersection point as the deviation. The formula for calculating the location reliability is as follows: The purpose of intersection calculation is to determine the spatial coordinates of the fire by intersecting the lines of sight from different imaging units. Each imaging unit calculates the line of sight pointing to the fire source based on its installation position, pan-tilt azimuth angle, and elevation angle.

[0086] The deviation in intersection calculation refers to the minimum distance between the intersection point and each line of sight. It is typically optimized using the least squares method to obtain the optimal intersection point. The formula is: The visibility score reflects whether the line of sight of each imaging unit can effectively cover the fire area. When calculating the visibility score, terrain elevation, vegetation height, and the field of view of the imaging unit are taken into account.

[0087] If there are three imaging units involved, namely imaging units A, B, and C, with visibility scores of 82%, 87%, and 78% respectively, and a crossover deviation of 0.5 meters, the confidence calculation process is as follows: In forest fire monitoring, a high degree of system positioning accuracy is usually required. Based on previous test data, engineering experience, and industry standards, a preset positioning reliability threshold of 90% is set to ensure the system's reliability and timeliness.

[0088] Because the location reliability exceeded the preset threshold of 90%, the system output an alarm message: Fire location coordinates: 420 meters, 210 meters, 32 meters, with a positioning reliability of 95%.

[0089] The evidence frames recorded by the co-imaging system include path information for the trigger image and the verification image frames.

[0090] If the positioning reliability does not reach the preset threshold, the system will output verification information, including the triggering imaging unit identifier, the imaging unit identifier involved in the linkage, and the reason code for not reaching the threshold.

[0091] In summary, this embodiment achieves collaborative operation of multiple imaging units. Through precise target detection, line-of-sight calculation, and positioning reliability assessment, the system improves the accuracy and responsiveness of fire monitoring, demonstrating its efficiency and innovation in practical applications.

[0092] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A forest fire monitoring system based on image data, characterized in that, include: Multi-view access module: used to access image data streams from at least two spatially separated forest area monitoring imaging units with different viewing angles, and to assign imaging unit identifiers and timestamp information to the image data streams. The multi-view access module obtains the calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation position coordinates. Visibility assessment module: used to acquire forest spatial structure data covering a preset monitoring area. The forest spatial structure data includes three-dimensional terrain elevation data and vegetation height data. Based on the camera pose parameters, the installation location coordinates and the forest spatial structure data, the module performs line-of-sight visibility calculation on the preset monitoring area to obtain the set of visible areas and visibility score corresponding to each imaging unit. Based on the set of visible areas and visibility score, the module determines the set of effective viewing angles for fire confirmation and location. Cross-view collaborative imaging module: When any imaging unit detects a suspected target of smoke or flame in its image data stream, it generates a trigger event. The trigger event includes the trigger imaging unit identifier, timestamp, and pixel position of the suspected target in the image data stream. It acquires the trigger image frame corresponding to the timestamp, determines the target line of sight based on the pixel position and the calibration parameters of the corresponding imaging unit, and selects at least one non-triggered imaging unit from the effective view set to issue a linkage imaging command. The linkage imaging command includes a synchronous imaging time window and the target line of sight. The non-triggered imaging unit converts the target line of sight into the gimbal azimuth and pitch angles according to its own calibration parameters and completes the pointing. It acquires a verification image sequence, performs time alignment between the trigger image frame and the verification image sequence, and generates a collaborative imaging record. The collaborative imaging record includes the participating imaging unit identifier, timestamp, gimbal azimuth, pitch angle, and corresponding image frame. Spatial positioning and reliability calculation module: used to determine the line of sight of each imaging unit based on the installation position coordinates of at least two imaging units in the collaborative imaging record, the azimuth angle and the pitch angle of the gimbal, and to calculate the intersection point of the line of sight to obtain the spatial position coordinates of the fire. The module also calculates the positioning reliability based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units. Alarm output module: When the positioning confidence reaches a preset confidence threshold, it outputs alarm information, which includes the fire spatial location coordinates, positioning confidence, and evidence frames recorded by the collaborative imaging. When the positioning confidence does not reach the preset confidence threshold, it outputs verification information, which includes the trigger imaging unit identifier, the imaging unit identifier participating in the linkage, and the reason code for not reaching the threshold.

2. The forest fire monitoring system based on image data according to claim 1, characterized in that: The forest area monitoring imaging unit includes a fixed wide-angle imaging unit and a rotatable pan-tilt imaging unit. The triggering event is generated by the fixed wide-angle imaging unit, and the linkage imaging command is sent to the rotatable pan-tilt imaging unit to obtain the verification image sequence.

3. A forest fire monitoring system based on image data according to claim 1, characterized in that: The spatial structure data of the forest area is expressed in raster form and registered with the geographic coordinate system. The line-of-sight visibility calculation includes performing segment-by-segment occlusion determination on the three-dimensional terrain elevation data and vegetation height data along the line-of-sight direction of the imaging unit, and outputting the set of visible areas.

4. A forest fire monitoring system based on image data according to claim 3, characterized in that: The segmented occlusion determination includes: discretizing the line of sight from the imaging unit to the preset monitoring area into multiple sampling segments according to a preset sampling step size; acquiring corresponding three-dimensional terrain elevation data and vegetation height data for each sampling segment; comparing the three-dimensional terrain elevation data and vegetation height data with the height of the line of sight at the corresponding position; determining that the line of sight is occluded when any sampling segment meets the requirements of terrain elevation or vegetation height being higher than the height of the line of sight; and generating the set of visible areas based on the set of unoccluded lines of sight.

5. A forest fire monitoring system based on image data according to claim 1, characterized in that: The visibility score is determined by the coverage ratio of the visible area set within the preset monitoring area. The coverage ratio is the ratio of the area covered by the visible area set to the area of ​​the preset monitoring area. The effective viewing angle set consists of imaging units with a visibility score not lower than a preset visibility threshold.

6. A forest fire monitoring system based on image data according to claim 1, characterized in that: The intersection point calculation adopts a multi-line-of-sight minimum deviation intersection method. The deviation is a statistical measure of the minimum distance from the fire spatial location coordinates to each line of sight. The positioning reliability is calculated by the number of imaging units participating in the intersection point calculation, the line-of-sight intersection deviation, and the visibility score of the participating imaging units according to a preset weight. When the line-of-sight intersection deviation is greater than a preset deviation threshold or the number of imaging units participating in the intersection point calculation is less than a preset number threshold, the positioning reliability is determined to be below the preset reliability threshold.

7. A forest fire monitoring system based on image data according to claim 1, characterized in that: The reason code for not reaching the threshold is triggered and output under the following conditions: the visibility score is lower than the preset visibility threshold, corresponding to the first reason code; the number of imaging units participating in the intersection calculation is less than the preset number threshold, corresponding to the second reason code; the deviation of the intersection calculation is greater than the preset deviation threshold, corresponding to the third reason code. The evidence frame includes the trigger image frame and the image frame in the verification image sequence that is time-aligned with the trigger image frame, and is output together with the corresponding imaging unit identifier, timestamp, gimbal azimuth angle and pitch angle.

8. A forest fire monitoring method based on image data, implemented according to any one of claims 1-7, characterized in that, include: S1. Access image data streams from at least two spatially separated forest area monitoring imaging units with different viewing angles, and assign imaging unit identifiers and timestamp information to the image data streams, and obtain calibration parameters of the imaging units, including camera intrinsic parameters, camera pose parameters, and installation position coordinates; S2. Obtain forest spatial structure data covering a preset monitoring area, perform line-of-sight visibility calculation based on the camera pose parameters, the installation location coordinates and the forest spatial structure data, obtain the set of visible areas and visibility score corresponding to each imaging unit, and determine the set of effective viewing angles based on the set of visible areas and visibility score; S3. When any imaging unit detects a suspected target of smoke or flame in its image data stream, a trigger event is generated. The trigger event includes the trigger imaging unit identifier, timestamp, and pixel position of the suspected target. A trigger image frame corresponding to the timestamp is acquired. The target line of sight is determined based on the pixel position and the calibration parameters of the trigger imaging unit. At least one non-triggered imaging unit is selected from the effective viewpoint set to issue a linkage imaging command. The linkage imaging command includes a synchronous imaging time window and the target line of sight. The non-triggered imaging unit converts the target line of sight into the gimbal azimuth and pitch angles and completes the pointing. A verification image sequence is acquired. The trigger image frame and the verification image sequence are time-aligned to generate a collaborative imaging record. The collaborative imaging record includes the participating imaging unit identifier, timestamp, gimbal azimuth, pitch angle, and corresponding image frame. S4. Based on the installation position coordinates of at least two imaging units in the collaborative imaging record, the azimuth and pitch angles of the pan-tilt unit are used to determine the line of sight of each imaging unit, and the intersection point of the line of sight is calculated to obtain the spatial position coordinates of the fire; the positioning reliability is calculated based on the number of imaging units participating in the intersection point calculation, the deviation of the intersection point calculation, and the visibility score of the participating imaging units. S5. When the location confidence level reaches a preset confidence threshold, an alarm message is output. The alarm message includes the fire spatial location coordinates, the location confidence level, and the evidence frames recorded by the collaborative imaging. When the positioning reliability does not reach the preset reliability threshold, verification information is output. The verification information includes the trigger imaging unit identifier, the imaging unit identifier involved in the linkage, and the reason code for not reaching the threshold.

Images