A forest fire warning method and system fusing panoramic video and PTZ video

Abstract

The application provides a forest fire warning method and system fusing panoramic video and PTZ video, relates to the technical field of forest fire monitoring and warning, and solves the problems that the forest fire monitoring in the prior art cannot consider both wide coverage and detail capture, and the intelligent level of fire identification and warning is low. The method comprises the following steps: acquiring a panoramic video stream and performing fire target detection to obtain a suspected fire target and a pixel coordinate; performing line-of-sight tracking and terrain intersection calculation based on the pixel coordinate and prior three-dimensional terrain data to obtain a geographic spatial coordinate and generate linkage control parameters; controlling a target PTZ to collect a local detail video stream based on the linkage control parameters and extracting micro detail features; extracting macro trend features based on the panoramic video stream, constructing the macro trend features and the micro detail features into independent evidence bodies respectively; and performing uncertainty fusion decision on the independent evidence bodies to generate a fire warning signal. The application is used in the forest fire warning process.

Description

Technical Field

[0001] This application relates to the field of forest fire monitoring and early warning technology, and in particular to a forest fire early warning method and system that integrates panoramic video and PTZ video. Background Technology

[0002] Forest fires are characterized by their suddenness, destructiveness, and difficulty in response and rescue. Early and accurate detection and rapid location of fires are crucial for effectively controlling the fire and minimizing losses. Currently, forest fire prevention video monitoring systems mainly suffer from the following problems: Existing systems typically deploy panoramic cameras or pan-tilt-zoom (PTZ) cameras separately: panoramic cameras have a wide field of view but limited resolution, and are insufficient in detecting small target flames and initial smoke; while PTZ cameras can acquire high-definition details, a single device cannot cover a wide area, resulting in blind spots and making it easy to miss the initial stage of a fire.

[0003] Existing methods for linking panoramic cameras with PTZ cameras are mainly designed for indoor or flat urban terrain. In complex terrains such as forests and mountains, the simple linear mapping from pixel coordinates to PTZ angles results in large positioning errors, and the target is easily obscured by mountains and trees, leading to low success rate and poor practicality of the linkage.

[0004] In terms of fire detection, traditional algorithms rely on a single video source. In panoramic images, the characteristics of small, distant smoke or flames are weak, making them prone to missed detections or false alarms. PTZ (Potentially Transmitted Light) images lack global context, making it difficult to determine the overall scale and spread of the fire. Even when two types of cameras are deployed simultaneously, their information is often processed independently, lacking deep fusion and failing to comprehensively utilize both macroscopic trends and microscopic details for integrated analysis. Furthermore, existing systems typically only issue a uniform alarm signal, failing to provide tiered warnings based on the fire's development stage, resulting in a lack of targeted emergency response. Summary of the Invention

[0005] This application provides a forest fire early warning method and system that integrates panoramic video and PTZ video, which solves the problems in the existing technology that forest fire monitoring cannot achieve both large-scale coverage and detailed capture, and that the level of intelligence in fire identification and early warning is low.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, a forest fire early warning method integrating panoramic video and PTZ video is provided, including: Acquire panoramic video streams captured by panoramic cameras, perform fire target detection on the panoramic video streams, and obtain suspected fire targets and their corresponding pixel coordinates; Based on the pixel coordinates and prior 3D terrain data, line-of-sight tracking and terrain intersection calculation are performed to obtain the geospatial coordinates of the suspected fire target, and the linkage control parameters of the target PTZ are generated based on the geospatial coordinates. Based on the aforementioned linkage control parameters, the target PTZ is controlled to acquire local detail video streams, and microscopic detail features are extracted based on the local detail video streams; Based on the panoramic video stream, macroscopic situation features are extracted, and the macroscopic situation features and the microscopic detail features are respectively constructed into macroscopic situation evidence body and microscopic detail evidence body; The macroscopic situation evidence and the microscopic detail evidence are fused together, and a fire warning signal is generated based on the fusion result.

[0007] Based on the above technical solution, in the forest fire early warning method integrating panoramic video and PTZ video provided in this application, the panoramic video stream and the PTZ local detail video stream work together. The panoramic end achieves large-scale uninterrupted monitoring and initial screening of suspected targets, while the PTZ end achieves high-definition detail capture and precise confirmation of the target area. Furthermore, prior 3D terrain data is used for line-of-sight tracking and terrain intersection calculation, converting panoramic pixel coordinates into geospatial coordinates, thereby achieving high-precision spatial positioning and coordinated control under complex terrain conditions. On this basis, macroscopic situational features and microscopic detailal features are respectively constructed as macroscopic situational evidence and microscopic detailal evidence, and uncertainty fusion decision-making is performed. This allows for deep fusion of the global contextual information from the panoramic end and the local fine information from the PTZ end at the decision-making level, effectively improving the accuracy and robustness of fire early warning.

[0008] In conjunction with the first aspect above, in one possible implementation, the step of performing line-of-sight tracking and terrain intersection calculation based on the pixel coordinates and prior 3D terrain data includes: Based on the pose parameters of the panoramic camera and the pixel coordinates, the line of sight corresponding to the suspected fire target is determined; Based on the camera installation position in the pose parameters, extension points are generated by gradually extending along the line of sight at a preset step size. The elevation of the extension points is then compared with the terrain elevation of the corresponding position in the prior three-dimensional terrain data. When the elevation of the extension point is lower than the terrain elevation, the position of the extension point is determined as a terrain intersection point; Spatial interpolation is performed on the terrain intersections to obtain the geographic spatial coordinates.

[0009] The aforementioned scheme simulates the propagation path of light in complex terrain by gradually extending along the line of sight and comparing its elevation with prior 3D terrain data. This accurately determines the first intersection point between the line of sight and the terrain surface, thus converting pixel coordinates in the panoramic image into geospatial coordinates with actual geographical significance. This method fully considers the impact of terrain undulations such as forest ridges and valleys on the line of sight, avoiding the significant positioning errors that traditional linear mapping methods produce under non-flat terrain conditions.

[0010] In conjunction with the first aspect above, in one possible implementation, the step of generating the linkage control parameters for the target PTZ based on the geospatial coordinates includes: Based on the geospatial coordinates and the installation location of the target PTZ, calculate the initial control parameters; Based on the prior 3D terrain data, the terrain slope and terrain roughness index of the region where the geospatial coordinates are located are extracted; The pixel coordinates, terrain slope, and terrain roughness index are input into a pre-trained residual compensation network to output the pose residual. The initial control parameters are corrected based on the attitude residual to obtain the linkage control parameters.

[0011] The above scheme corrects the initial control parameters through a residual compensation network, and uses the terrain slope and terrain roughness index as network inputs, enabling the network to perceive the terrain features of the target area. This compensates for attitude deviations caused by nonlinear factors such as atmospheric refraction and optical distortion, further improving the accuracy of the linkage control parameters and ensuring that the PTZ camera can accurately point to the target area.

[0012] In conjunction with the first aspect above, in one possible implementation, before generating the linkage control parameters for the target PTZ based on the geospatial coordinates, the method further includes: Based on the prior 3D terrain data, a score is calculated to indicate the degree of line-of-sight occlusion of multiple candidate PTZs pointing to the geospatial coordinates; The observation geometry score and response time score are calculated based on the installation locations of multiple candidate PTZs and the geospatial coordinates. Based on the line-of-sight occlusion score, the observation geometry score, and the response time score, calculate the comprehensive score for each candidate PTZ; The candidate PTZ with the highest comprehensive score is determined as the target PTZ.

[0013] The above scheme selects the optimal target PTZ from multiple candidate PTZs through a multi-factor comprehensive scoring method. The line-of-sight occlusion score uses prior 3D terrain data to predict whether there are mountains or trees obstructing the view when each candidate PTZ points to the target location. The observation geometry score assesses the quality of the observation angle, and the response time score evaluates the linkage efficiency. This method effectively avoids linkage failures caused by occlusion and improves the success rate of PTZ linkage under complex terrain conditions in forest areas.

[0014] In conjunction with the first aspect above, in one possible implementation, the extraction of micro-detail features based on the local detail video stream includes: Fire features are extracted from the local detail video stream using a multi-task analysis network; the fire features include flame morphology features, flame dynamic features, smoke translucency features, and smoke diffusion features. Based on the flame morphology characteristics and the smoke translucency characteristics, the ratio of total combustion area to flame-smoke area is calculated; The overall area change rate is calculated based on the total burning area in the local detail video stream across multiple consecutive frames. The fire characteristics, the total burning area, the flame-smoke area ratio, and the overall area change rate are combined to form the microscopic detail features.

[0015] The aforementioned scheme extracts multi-dimensional fire characteristics from PTZ high-definition video streams, including flame morphology, flame dynamics, smoke translucency, and smoke diffusion. Based on these characteristics, it calculates comprehensive indicators such as total burning area, flame-to-smoke area ratio, and overall area change rate. These features and indicators characterize the type, scale, and dynamic evolution trend of the fire from different perspectives, providing ample micro-level information to support subsequent fusion-based decision-making.

[0016] In conjunction with the first aspect above, in one possible implementation, the step of constructing the macro-level situational characteristics and the micro-level detailal characteristics into a macro-level situational evidence body and a micro-level detailal evidence body, respectively, includes: Based on the fire location, impact range, and environmental context information in the macro-situation characteristics, the first basic probability allocation of each preset fire level is calculated to construct the macro-situation evidence body. Based on the fire type, fire intensity, and hazard source information in the micro-detail features, the second basic probability allocation for each preset fire level is calculated, and the micro-detail evidence body is constructed.

[0017] The aforementioned scheme constructs a macro-level situational evidence body from the fire location, impact range, and environmental context information provided by the panoramic terminal, and a micro-level detailed evidence body from the fire type, fire intensity, and hazard source information provided by the PTZ terminal, enabling each information source to form an evidence body with independent probability allocation. This construction method preserves the uncertainties of each information source and provides a standardized input format for subsequent fusion decision-making.

[0018] In conjunction with the first aspect above, in one possible implementation, generating a fire early warning signal based on the fusion result includes: The first basic probability allocation and the second basic probability allocation are synthesized using basic probability synthesis rules to obtain a synthesized basic probability allocation as the fusion result, and the conflict coefficient is calculated. When the conflict coefficient is greater than the preset conflict threshold, an evidence conflict alarm message is generated, and a manual review instruction is awaited. When the conflict coefficient is less than or equal to the preset conflict threshold, the preset fire level with the highest probability value in the fusion result is determined as the fire warning signal.

[0019] The above scheme fuses two independent pieces of evidence using basic probabilistic synthesis rules and calculates a conflict coefficient to quantify the degree of contradiction between the evidence. When the conflict coefficient exceeds a preset threshold, the system does not directly output a decision result, but instead generates an evidence conflict alarm and waits for manual review, thus avoiding the risk of making incorrect decisions in cases of severe evidence conflict. When the conflict coefficient is within an acceptable range, the fire level with the highest probability value after synthesis is used as the warning signal, achieving a reasonable fusion of uncertain information.

[0020] In conjunction with the first aspect above, in one possible implementation, after generating the fire early warning signal based on the fusion result, the method further includes: The macroscopic situation characteristics and microscopic details are reacquired at a preset cycle, and the fire intensity index is calculated to obtain the fire intensity index. When the fire intensity index exceeds the preset upper limit threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal will be upgraded. When the fire intensity index falls below the preset lower threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal will be downgraded. Set a minimum duration for each preset fire level, and restrict changes to the fire warning signal within the minimum duration.

[0021] The aforementioned scheme achieves dynamic updates to the warning level through a mechanism of periodic reassessment and continuous multiple sampling. The design of triggering an upgrade only when the number of consecutive preset counts exceeds the upper threshold, and triggering a downgrade only when the number of consecutive preset counts falls below the lower threshold, effectively avoids frequent level jumps caused by sensor noise or short-term fluctuations. Simultaneously, setting a minimum duration for each level further ensures the stability of the warning level output, guaranteeing that emergency response decisions are based on reliable warning information.

[0022] In conjunction with the first aspect above, in one possible implementation, the fire target detection of the panoramic video stream includes: The panoramic video stream is divided into multiple region sub-maps with overlapping areas; Feature extraction and target detection are performed on each of the aforementioned sub-regions using a target detection network to obtain sub-region detection results; wherein, the target detection network includes an asymmetric multi-scale receptive field enhancement module and an ultra-high resolution detection head; Non-maximum suppression fusion is performed on the detection results of each sub-image to obtain the suspected fire target and the corresponding pixel coordinates.

[0023] The aforementioned scheme avoids the problem of missing small targets caused by directly downsampling a large image by dividing the high-resolution panoramic image into multiple sub-images with overlapping regions and detecting them separately. The asymmetric multi-scale receptive field enhancement module in the target detection network can adapt to the large scale differences between smoke and flame targets in forest scenes, while the ultra-high resolution detection head improves the sensitivity to small smoke points and initial flames in the distance. The detection results of each sub-image are fused through non-maximum suppression, achieving accurate detection and localization of suspected fire targets while maintaining the original image resolution.

[0024] Secondly, a forest fire early warning system integrating panoramic video and PTZ video is provided, including: The panoramic video acquisition and detection module is used to acquire the panoramic video stream captured by the panoramic camera, perform fire target detection on the panoramic video stream, and obtain the suspected fire target and the corresponding pixel coordinates. The spatial positioning and linkage control module is used to perform line-of-sight tracking and terrain intersection calculation based on the pixel coordinates and prior three-dimensional terrain data to obtain the geospatial coordinates of the suspected fire target, and generate linkage control parameters for the target PTZ based on the geospatial coordinates. The detail acquisition and feature extraction module is used to control the target PTZ to acquire local detail video streams based on the linkage control parameters, and to extract micro-detail features based on the local detail video streams; An independent evidence body construction module is used to extract macro-situational features based on the panoramic video stream, and to construct the macro-situational features and the micro-detail features into macro-situational evidence bodies and micro-detail evidence bodies, respectively. The fusion decision and early warning output module is used to fuse the macroscopic situation evidence and the microscopic detail evidence, and generate a fire early warning signal based on the fusion result.

[0025] The aforementioned system, through the collaborative work of its various functional modules, organically integrates panoramic video acquisition and detection, spatial positioning and linkage control, detail collection and feature extraction, independent evidence construction, and fusion decision-making and early warning output. It achieves a complete closed-loop process from panoramic monitoring, target detection, spatial positioning, PTZ linkage, detail capture to fusion decision-making, providing systematic technical support for graded early warning of forest fires.

[0026] Beneficial effects: 1. Expanding the monitoring range and improving response speed. This application utilizes the coordinated deployment of panoramic cameras and PTZ cameras. The panoramic cameras continuously monitor vast forest areas. Upon detecting a suspected fire target, line-of-sight tracking and terrain intersection calculations are performed based on prior 3D terrain data. The panoramic pixel coordinates are converted into geospatial coordinates, and PTZ linkage control parameters are generated accordingly. This enables the PTZ cameras to quickly and accurately point to the target area for detailed capture. This method reduces the fire confirmation time from the traditional minutes to the seconds, gaining valuable time for early response.

[0027] 2. Significantly improves the accuracy of fire detection and greatly reduces the false alarm rate. This application adopts a two-level collaborative detection mechanism of initial screening at the panoramic end and fine-tuning at the PTZ end. The panoramic end aims for high recall to ensure that no suspected targets are missed, while the PTZ end aims for high accuracy for secondary confirmation. On this basis, by constructing macroscopic situation features and microscopic detail features into macroscopic situation evidence bodies and microscopic detail evidence bodies respectively, and performing uncertainty fusion decision-making, false alarms caused by changes in light, animal activity, cloud drift, etc. are effectively filtered out, making the overall accuracy of the system significantly better than a single panoramic detection scheme.

[0028] 3. Achieve intelligent fire situation classification and optimize emergency resource allocation. This application generates fire early warning signals through uncertainty fusion decision-making and sets up a dynamic update mechanism for warning levels, matching differentiated response strategies to different fire levels. Managers can quickly assess the urgency of the fire based on the warning level and accurately allocate emergency resources, avoiding resource waste caused by the uniform alarm method in traditional systems, and improving the scientific nature and efficiency of forest fire prevention.

[0029] 4. Improved positioning accuracy in complex terrain conditions. This application introduces prior 3D terrain data for line-of-sight tracking and terrain intersection calculation, and uses a residual compensation network to correct initial control parameters, effectively solving the interference of mountainous terrain undulations on positioning accuracy. Under complex terrain conditions, the positioning accuracy of this application is significantly better than traditional pure calibration methods, ensuring that the PTZ camera can place the target in the center of the frame on the first linkage without additional search scanning, greatly improving the linkage success rate.

[0030] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0031] Figure 1 A flowchart illustrating a forest fire early warning method integrating panoramic video and PTZ video provided in this application embodiment; Figure 2 A decision-making flowchart for spatial positioning and PTZ linkage control provided in the embodiments of this application; Figure 3 This is a schematic diagram of the data flow for microscopic detail feature extraction provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the process of evidence construction, fusion decision-making, and dynamic update of early warning provided in the embodiments of this application; Figure 5 This is a system architecture diagram of a forest fire early warning system that integrates panoramic video and PTZ video, provided as an embodiment of this application. Detailed Implementation

[0032] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0033] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0034] Example 1 like Figure 1 As shown, this embodiment provides a forest fire early warning method that integrates panoramic video and PTZ video. This method is executed collaboratively by a panoramic camera and one or more PTZ cameras. The panoramic camera is responsible for continuous, large-scale monitoring of the vast forest area, while the PTZ camera is responsible for capturing high-definition details of suspected target areas. Through the connection of spatial positioning and fusion decision-making processes, a complete fire early warning closed loop is formed.

[0035] Step S100: Obtain the panoramic video stream captured by the panoramic camera, perform fire target detection on the panoramic video stream, and obtain the suspected fire targets and their corresponding pixel coordinates.

[0036] The suspected fire targets output by the detection model can be suspected smoke points or suspected fire points. The corresponding pixel coordinates are usually the position of the center point of the target bounding box or the centroid of the smoke area in the panoramic image.

[0037] In some implementations, panoramic cameras are deployed at high points or observation posts in forest areas to continuously acquire panoramic video streams covering key fire prevention areas using a fixed viewing angle or a perimeter scan method. The panoramic video streams are then fed into a computing device connected to the near end of the panoramic camera, where a fire target detection model deployed on the device analyzes each frame in real time to identify potential smoke or flame areas.

[0038] The core objective of this step is to complete the initial screening with a high recall rate across a wide area, that is, to avoid missing any suspicious targets as much as possible. Whether the target is a real fire is then confirmed by the subsequent PTZ fine inspection stage. In this embodiment, the panoramic end and the PTZ end have a functional division of labor: the panoramic end solves the problem of seeing a wide area, while the PTZ end solves the problem of seeing clearly.

[0039] Step S200: Based on pixel coordinates and prior 3D terrain data, line-of-sight tracking and terrain intersection calculation are performed to obtain the geospatial coordinates of the suspected fire target, and the linkage control parameters of the target PTZ are generated based on the geospatial coordinates.

[0040] Among them, the prior 3D terrain data refers to the pre-acquired and stored digital elevation model of the forest area or other 3D data that can describe the undulation of the terrain surface; the geospatial coordinates can be in the form of latitude and longitude plus altitude, or they can be 3D coordinates in a local coordinate system; the linkage control parameters are used to instruct how the PTZ camera rotates and zooms so that the target enters the central area of ​​the PTZ image.

[0041] In some implementations, due to the significant topographical undulations in forest areas, pixel coordinates in the panoramic image cannot be simply converted into PTZ attitude parameters through linear mapping; otherwise, significant positioning errors would occur in areas such as ridges and valleys. A line of sight is determined from the panoramic camera's pose parameters and pixel coordinates, passing through the target pixel and originating from the camera's optical center. This line of sight is then intersected with prior 3D terrain data to obtain the first intersection point between the line of sight and the terrain surface. This intersection point represents the location of the suspected fire target in geographic space. After obtaining the geographic coordinates, the required linkage control parameters for the target PTZ to point to these geographic coordinates are calculated, based on the target PTZ's installation location and attitude reference.

[0042] In this embodiment of the application, by introducing prior three-dimensional terrain data, a reliable correspondence is established between the pixel space of the panoramic end and the real geographic space, providing a spatial reference for subsequent PTZ linkage.

[0043] Step S300: Based on the linkage control parameters, control the target PTZ to acquire local detail video streams, and extract micro-detail features based on the local detail video streams.

[0044] Among them, the resolution of local detail video streams is usually higher than that of the corresponding area in panoramic video streams, and can present detailed information such as flame shape and smoke texture; micro-detail features are used to characterize the type, intensity and dynamic changes of fire at the local scale, such as flame area, smoke diffusion rate, and flame flicker characteristics.

[0045] In some implementations, the linkage control parameters are sent to the selected target PTZ camera via the network. The target PTZ camera then performs horizontal rotation, vertical tilt, and optical zoom actions according to the parameters, aiming its lens at the area suspected of being a fire and continuously acquiring a high-definition video stream of that area—a local detail video stream. Subsequently, feature extraction is performed on the local detail video stream to obtain microscopic detail features. Through this transformation from broad initial screening to localized fine-tuning, the high-definition information from the PTZ provides detailed evidence for subsequent fusion decisions.

[0046] Step S400: Extract macro-situational features based on the panoramic video stream, and construct macro-situational features and micro-detail features into macro-situational evidence body and micro-detail evidence body, respectively.

[0047] Among them, macro-situation characteristics are global information extracted from the panoramic video stream, used to describe the location, impact range, and surrounding environmental context of the fire in the overall monitoring scene.

[0048] In some implementations, macro-level situational features can include the geographical location of the fire target, the actual coverage area corresponding to the pixel size of the target in the panoramic image, and whether there are residential areas or important facilities around the target. Micro-level detail features, derived from step S300, describe the specific manifestation of the fire at a local scale. Constructing macro-level and micro-level detail features as independent evidence means that each information source forms an evidence structure with independent probability allocation or confidence expression, rather than simply splicing them together at the feature level. This construction method preserves the inherent uncertainty of each information source, enabling subsequent fusion decisions to be made at the evidence level, rather than at the raw data level.

[0049] Step S500: The macro-level situational evidence and the micro-level detailed evidence are integrated, and a fire warning signal is generated based on the integration result.

[0050] Among them, fusion refers to making uncertain fusion decisions on macro-level situational evidence and micro-level detailed evidence. Uncertainty fusion decision-making adopts fusion methods that can handle information uncertainty and evidence conflict, and comprehensively evaluates macro-level situational evidence and micro-level detailed evidence.

[0051] In some implementations, fire warning signals can include fire severity level information, such as multiple preset levels, with different levels corresponding to different response strategies and notification ranges. Because the confidence level of a panoramic camera detecting small targets at long distances may be low, while the confidence level of a PTZ camera in high-definition images is usually high, there may be differences or even conflicts in the confidence levels of these two pieces of evidence. Uncertainty fusion decision-making can reasonably allocate decision weights at the evidence level, avoiding unreasonable results when evidence conflicts arise, as can be achieved using methods such as simple weighted averaging.

[0052] Through steps S100 to S500, this embodiment establishes a complete workflow for collaborative work between large-scale initial screening at the panoramic end and detailed inspection at the PTZ end. The panoramic camera continuously monitors a vast forest area. Upon detecting a suspected target, it uses prior 3D terrain data to convert pixel coordinates to geographic spatial coordinates, and accordingly controls the PTZ camera to quickly point at the target area for high-definition confirmation. The macroscopic situational information from the panoramic end and the microscopic detail information from the PTZ end undergo uncertainty fusion decision-making at the evidence level, ultimately outputting a tiered fire warning signal. This workflow reduces fire confirmation time from minutes in traditional manual inspections or single-camera solutions to seconds. Simultaneously, through two-level collaborative detection and evidence fusion, it effectively reduces the false alarm rate, providing systematic support for early detection and tiered response to forest fires.

[0053] Example 2 like Figure 2 As shown, this embodiment, based on step S200 of embodiment 1, further elaborates on the complete implementation process of spatial positioning and PTZ linkage control parameter generation. As explained in embodiment 1, line-of-sight tracking and terrain intersection calculation are performed based on pixel coordinates and prior 3D terrain data to obtain the geospatial coordinates of the suspected fire target, and linkage control parameters for the target PTZ are generated based on these geospatial coordinates. This embodiment subdivides this process into three sequentially connected steps: first, determining the geospatial coordinates through line-of-sight tracking and terrain intersection calculation; second, selecting the target PTZ from multiple candidate PTZs; and finally, generating linkage control parameters after residual compensation.

[0054] Step S201: Determine the line of sight corresponding to the suspected fire target based on the pose parameters and pixel coordinates of the panoramic camera.

[0055] The pose parameters of the panoramic camera refer to a set of parameters describing the camera's position and orientation in three-dimensional space, including at least the camera mounting position, orientation angle, and field of view. The camera mounting position can be the latitude, longitude, and altitude of the camera's optical center in a global geographic coordinate system, or it can be the three-dimensional coordinates in a local engineering coordinate system. The orientation angle includes azimuth and pitch angles, used to determine the direction of the camera's optical axis in space. The field of view determines the correspondence between each pixel in the panoramic image and the spatial direction. Pixel coordinates are the positions of the suspected fire targets detected in step S100 within the panoramic image, typically taken as the center point of the target's bounding box or the centroid of the smoke area.

[0056] In some implementations, based on the camera projection model, the angular offset of the incident ray relative to the camera's optical axis can be determined from the pixel coordinates and the field of view. Combined with the orientation angle, the direction vector of the incident ray in the global coordinate system, i.e., the viewing direction, can then be obtained. It should be noted that the specific form of the camera projection model depends on the type of lens used in the panoramic camera, such as equidistant projection, equal solid angle projection, or cylindrical projection.

[0057] Step S202: Based on the camera installation position in the pose parameters, extend the line of sight step by step to generate extension points, and compare the elevation of the extension points with the elevation of the corresponding position in the prior 3D terrain data.

[0058] The prior 3D terrain data can be a pre-acquired digital elevation model of the forest area, or other 3D data that can describe the undulation of the terrain surface, such as point cloud data generated by LiDAR scanning or 3D terrain mesh generated by photogrammetry. The selection of the preset step size needs to strike a balance between computational accuracy and computational efficiency: too small a step size will result in too many extension points and increased computational load, while too large a step size may cause terrain intersections to be skipped and the positioning accuracy to decrease.

[0059] In some implementations, taking a digital elevation model (DEM) as an example, the topographic elevation values ​​of various locations in the forest area are stored in the form of a regular grid or triangular mesh. Starting from the camera installation position of the panoramic camera, along the line of sight determined in step S201, the camera extends forward step by step according to a preset step size. Each extension step generates an extension point, and the topographic elevation of the corresponding position of the extension point in the prior 3D topographic data is calculated. In this embodiment, the preset step size can be taken as a ground distance of 5 meters, that is, the horizontal distance between two adjacent extension points is approximately 5 meters. It should be noted that the preset step size can also be adjusted according to the resolution of the prior 3D topographic data and the actual degree of topographic undulation. For example, the step size can be appropriately reduced when the topographic undulation is severe, and appropriately increased when the topographic undulation is relatively gentle.

[0060] Step S203: When the elevation of the extension point is lower than the terrain elevation, the location of the extension point is determined as the terrain intersection point.

[0061] Specifically, as the line of sight extends gradually, the elevation of each extension point is compared with the corresponding elevation in the prior 3D terrain data. When the extension point is above the terrain surface, its elevation is higher than the terrain elevation, indicating that the line of sight has not yet reached the ground; when the extension point's elevation is lower than the terrain elevation, it indicates that the line of sight has penetrated the terrain surface, and this extension point is the intersection of the line of sight and the terrain surface. Since the extension points are generated discretely according to a preset step size, the actual terrain intersection is usually located between the last extension point above the terrain surface and the first extension point below the terrain surface.

[0062] Step S204: Perform spatial interpolation on the terrain intersections to obtain geospatial coordinates.

[0063] In some implementations, spatial interpolation involves linearly interpolating between the last extension point above the terrain surface and the first extension point below the terrain surface, based on the ratio of the difference between their elevations and the corresponding terrain elevations. This more accurately determines the intersection of the line of sight and the terrain surface, and the interpolated intersection point becomes the geospatial coordinates of the suspected fire target. These geospatial coordinates can be in the form of latitude, longitude, and altitude, or they can be three-dimensional coordinates within a local engineering coordinate system. Through steps S201 to S204, the pixel coordinates in the panoramic image are converted into geospatial coordinates with actual geographical significance, providing a spatial reference for the subsequent generation of PTZ linkage control parameters.

[0064] It should be noted that the above-described method for calculating the intersection of line of sight and terrain fully considers the impact of terrain undulations such as forest ridges and valleys on line of sight propagation. Compared with the traditional linear mapping method, this embodiment no longer assumes that the target is located on a certain ideal plane, but simulates the actual propagation path of light in complex terrain by comparing elevations point by point, thereby obtaining more accurate positioning results under non-flat terrain conditions.

[0065] Before generating the linkage control parameters, the target PTZ needs to be determined from multiple candidate PTZs.

[0066] Step S205: Calculate the line-of-sight occlusion score of multiple candidate PTZ pointing to geospatial coordinates based on prior 3D terrain data.

[0067] Candidate PTZs refer to PTZ cameras deployed in the system that are capable of pointing to the geospatial coordinates. The line-of-sight occlusion score is used to evaluate whether the line of sight of each candidate PTZ is obstructed by features such as mountains and trees when pointing to the geospatial coordinates.

[0068] In some implementations, starting from the installation location of the candidate PTZ, line-of-sight tracking is performed along the direction pointing to the geospatial coordinates, similar to step S202. The intersection between the line of sight and the terrain surface is detected in the prior 3D terrain data. If the line of sight intersects the terrain surface before reaching the target location, it indicates occlusion, and the line-of-sight occlusion score is low; if the line of sight remains above the terrain surface throughout its path, it indicates no occlusion, and the line-of-sight occlusion score is high. The line-of-sight occlusion score can be a continuous score or a discrete level.

[0069] Step S206: Calculate the observation geometry score and response time score based on the installation location and geospatial coordinates of multiple candidate PTZs.

[0070] The observation geometry score is used to evaluate the quality of the observation angle when a candidate PTZ observes the geospatial coordinates. Specifically, it calculates the pitch and azimuth angles when the candidate PTZ points towards the geospatial coordinates. A pitch angle closer to the horizontal generally results in better observation, as an excessively large pitch angle causes perspective compression of the target in the image, hindering detail recognition. The azimuth angle is used to determine the presence of unfavorable lighting conditions such as backlighting. The observation geometry score is calculated by combining the pitch and azimuth angles. The response time score evaluates the time required for the candidate PTZ to rotate from its current position to the target position. It is estimated based on the horizontal and vertical angular displacements between the candidate PTZ's current attitude and the target's attitude, combined with the PTZ's rotational angular velocity, and also considers the time required for zooming. Smaller angular displacements, faster rotational angular velocities, and shorter response times result in a higher response time score.

[0071] Step S207: Calculate the comprehensive score of each candidate PTZ based on the line-of-sight occlusion score, observation geometry score, and response time score.

[0072] In some implementations, the overall score can be calculated using a weighted summation method. For example, the overall score S can be expressed as S = λ1·Q1 + λ2·Q2 + λ3·Q3, where Q1 is the line-of-sight obstruction score, Q2 is the observation geometry score, Q3 is the response time score, and λ1, λ2, and λ3 are the corresponding weight coefficients. The specific values ​​of the weight coefficients can be adjusted according to the actual deployment scenario. For example, in forest areas with severe obstruction, the value of λ1 can be appropriately increased, and in scenarios with high requirements for response speed, the value of λ3 can be appropriately increased. In this embodiment, λ1 is 0.4, λ2 is 0.3, and λ3 is 0.3.

[0073] Step S208: The candidate PTZ with the highest comprehensive score is determined as the target PTZ.

[0074] In some implementations, after calculating the comprehensive score of each candidate PTZ, they are sorted from highest to lowest comprehensive score, and the candidate PTZ with the highest comprehensive score is determined as the target PTZ. The target PTZ is the PTZ camera that subsequently performs linkage control and acquires local detail video streams. Through the above steps S205 to S208, the system automatically selects the PTZ with the best observation conditions and the highest response efficiency from multiple candidate PTZs, avoiding linkage failures caused by occlusion or poor observation angles.

[0075] In other implementations, if the initial linkage fails due to occlusion or other reasons, the system can automatically downgrade and select the candidate PTZ with the second-highest overall score to re-execute the linkage. For example, if the target PTZ rotates to its designated position according to the linkage control parameters, but no valid fire target is detected in the captured image, or the image quality is insufficient for subsequent feature extraction due to occlusion, the system can determine that the initial linkage has failed and select the PTZ with the second-highest overall score from the remaining candidate PTZs to re-execute the linkage. This downgraded retry mechanism further improves the success rate of PTZ linkage under complex terrain conditions in forest areas.

[0076] Step S209: Calculate the initial control parameters based on the geospatial coordinates and the installation location of the target PTZ.

[0077] The initial control parameters refer to the attitude parameters required for the target PTZ to point towards the geographic coordinates, without considering nonlinear factors such as atmospheric refraction and optical distortion. The initial control parameters include at least the horizontal rotation angle, vertical pitch angle, and zoom ratio. The horizontal rotation angle and vertical pitch angle can be calculated based on the geometric relationship between the installation position of the target PTZ and the geographic coordinates. Specifically, based on the horizontal distance and height difference between the two, the required azimuth and pitch angles for the target PTZ to point towards the geographic coordinates can be determined, and then converted into the horizontal rotation angle and vertical pitch angle by combining the installation orientation reference of the target PTZ. The zoom ratio is automatically calculated based on the target distance and the desired pixel size of the target in the image. For example, if the desired target occupies one-tenth to one-fifth of the image height, the appropriate zoom ratio is determined based on the target distance and the focal length range of the PTZ lens.

[0078] Step S210: Extract the terrain slope and terrain roughness index of the region where the geospatial coordinates are located based on prior three-dimensional terrain data.

[0079] The terrain slope refers to the degree of inclination of the terrain surface at a given geospatial coordinate location relative to the horizontal plane, which can be calculated from the elevation gradient of that location in prior 3D terrain data. The terrain roughness index reflects the severity of terrain undulation and can be obtained by calculating the standard deviation or mean absolute difference of terrain elevation within a certain window of the geospatial coordinate area. Terrain slope and terrain roughness index characterize the terrain features of the target area from different perspectives: terrain slope reflects the overall tilt trend of the terrain, while terrain roughness index reflects the severity of local terrain undulation. Both serve as inputs to the residual compensation network, enabling the network to perceive the terrain conditions of the target area.

[0080] Step S211: Input the pixel coordinates, terrain slope and terrain roughness index into the pre-trained residual compensation network and output the pose residual.

[0081] The residual compensation network is a pre-trained neural network model used to compensate for attitude deviations caused by nonlinear factors not considered in the initial control parameters. These nonlinear factors include, but are not limited to, atmospheric refraction and optical distortion. Atmospheric refraction causes light to bend as it passes through the atmosphere, resulting in a deviation between the apparent position and the actual position of distant targets. Optical distortion causes geometric deformation in lens imaging, affecting the precise correspondence between pixel coordinates and spatial orientation. The inputs to the residual compensation network are pixel coordinates, terrain slope, and terrain roughness index, and the output is the attitude residual, which refers to the difference between the initial control parameters and the actual required control parameters, including horizontal rotation angle residual, vertical pitch angle residual, and zoom magnification residual.

[0082] In some implementations, the residual compensation network adopts a three-layer fully connected network structure: the input layer contains four neurons, corresponding to the two components of the pixel coordinates, the terrain slope, and the terrain roughness index; the hidden layer contains 32 neurons, using the ReLU activation function; and the output layer contains three neurons, corresponding to the horizontal rotation residual, the vertical pitch residual, and the zoom magnification residual. The residual compensation network is trained as follows: using landmark calibration data as the training set, the pixel coordinates, terrain slope, and terrain roughness index of the landmarks are used as inputs, and the difference between the true PTZ parameters of the landmarks and the initial control parameters is used as the training objective. The network parameters are optimized by minimizing the mean square error between the network output and the training objective. Training can use the Adam optimizer, with a learning rate of 0.001, and the number of training epochs can be determined based on the convergence of the validation set loss.

[0083] Step S212: Correct the initial control parameters based on the attitude residual to obtain the linkage control parameters.

[0084] Specifically, the corrected linkage control parameters are obtained by superimposing the initial control parameters calculated in step S209 with the attitude residuals output in step S211. These linkage control parameters are the parameters included in the final control commands sent to the target PTZ, used to control the target PTZ to perform horizontal rotation, vertical pitch, and optical zoom. Through residual compensation, the linkage control parameters can more accurately reflect the true attitude required for the target PTZ to point to geospatial coordinates, thus ensuring that the target PTZ can place the suspected fire target in the center of the image on its first linkage.

[0085] Through steps S201 to S212, this embodiment completes a full link from pixel coordinates to geospatial coordinates, from geospatial coordinates to target PTZ optimization, and from target PTZ optimization to the generation of linkage control parameters after residual compensation. These three links are sequentially connected, jointly ensuring the accuracy and success rate of spatial linkage between the panoramic end and the PTZ end under complex terrain conditions in forest areas.

[0086] Example 3 like Figure 3 As shown, this embodiment, based on step S300 of embodiment 1, further elaborates on the specific implementation process of extracting micro-detail features based on local detail video streams. As explained in embodiment 1, the target PTZ is controlled to acquire local detail video streams based on linkage control parameters, and micro-detail features are extracted based on these local detail video streams. This embodiment subdivides this extraction process into three sequentially connected steps: first, fire features are extracted through a multi-task analysis network; then, the ratio of total burning area to flame and smoke area is calculated based on the fire features; finally, the comprehensive area change rate is calculated based on the total burning area of ​​multiple consecutive frames, and the above items are combined into micro-detail features.

[0087] Step S301: Fire features are extracted from the local detail video stream using a multi-task analysis network to obtain fire features.

[0088] Among them, fire characteristics include flame morphology characteristics, flame dynamic characteristics, smoke translucency characteristics, and smoke diffusion characteristics; multi-task analysis network refers to a neural network model that can perform both flame analysis and smoke analysis tasks simultaneously. Compared with training two independent models separately, multi-task analysis network reduces overall computational overhead while maintaining high accuracy by sharing the underlying feature extraction structure, which is beneficial for real-time processing of PTZ high-definition video streams on central servers or high-performance edge computing nodes; local detail video stream refers to the high-definition video stream acquired by the target PTZ camera in step S300. Its resolution is usually not less than 1920×1080 and the frame rate is not less than 30fps, which can present details such as flame morphology and smoke texture that are difficult to distinguish in panoramic images.

[0089] In some implementations, multi-task analysis networks employ a dual-attention architecture, comprising a flame confirmation network and a smoke confirmation network. These two networks share some low-level convolutional features but have independent output branches. The flame confirmation network performs pixel-level segmentation of flame regions in the local detail video stream, outputting a segmentation mask for the flame region and a global flame confidence score. The smoke confirmation network performs pixel-level segmentation of smoke regions in the local detail video stream, outputting a segmentation mask for the smoke region and a global smoke confidence score. The segmentation mask is a probability map of the same size as the input image, where the value of each pixel represents the probability that the pixel belongs to flame or smoke. The confidence score is a scalar representing the overall confidence level of the presence of flame or smoke in the current frame. The attention mechanism in the dual-attention architecture allows the network to automatically focus on the most discriminative feature locations in the flame or smoke region, suppressing interference from complex forest backgrounds.

[0090] Flame morphology features are used to describe the geometric shape and spatial distribution characteristics of a flame region. Specifically, flame morphology features can include the area, perimeter, circularity, and number of peaks of the flame region. Circularity can be calculated by the ratio of the flame region's area to the square of its perimeter, and is used to measure the degree to which the flame outline deviates from a circular shape. The number of peaks can be obtained by performing convex hull analysis on the flame region's outline, and is used to reflect the irregularity of the flame edge. Real flames typically have irregular peak shapes, while fire-like disturbances such as red light and sunsets often have more regular outlines. Therefore, flame morphology features play an important role in distinguishing real flames from fire-like disturbances.

[0091] Flame dynamics characteristics are used to describe the dynamic behavior of a flame over time. Specifically, flame dynamics characteristics can include the temporal rate of change of flame area and the dominant flicker frequency. The temporal rate of change of flame area can be calculated by comparing the amplitude of flame area changes over multiple consecutive frames; the dominant flicker frequency can be obtained by performing a frequency domain transformation on the time series of flame area and taking the frequency component with the largest amplitude. Real flames have natural flicker frequencies, typically in the range of 0.5 to 3 Hz, while static fire-like interference objects do not possess this dynamic characteristic.

[0092] Smoke translucency is used to describe the degree to which smoke obscures the background and its light transmission characteristics. Specifically, smoke translucency can be estimated by the visibility of background texture within the smoke area, for example, by calculating the gradient attenuation rate within the smoke area. Real smoke is translucent, and background texture is still partially visible under smoke coverage, while opaque or near-opaque meteorological phenomena such as clouds and haze can cause background texture to almost completely disappear. Therefore, smoke translucency helps distinguish real smoke from interference such as clouds and haze.

[0093] Smoke diffusion features are used to describe the way and speed at which smoke diffuses in space. Specifically, smoke diffusion features can include the outward expansion speed of the smoke region contour in different directions and the blurriness of the smoke contour. The outward expansion speed of the smoke contour can be calculated by matching and tracking the smoke region contour in multiple consecutive frames; the blurriness of the smoke contour can be measured by the gradient entropy of the smoke region edge. Real smoke typically has relatively blurry edges, low gradient entropy, and exhibits anisotropic diffusion characteristics, meaning that the diffusion speed varies in different directions.

[0094] Step S302: Calculate the total combustion area and the ratio of flame to smoke area based on flame morphology and smoke translucency.

[0095] The total combustion area refers to the sum of the flame area and the smoke area. The flame area can be obtained by counting the number of pixels with a probability greater than a preset threshold in the flame segmentation mask output by the flame confirmation network, and the smoke area can be obtained by counting the number of pixels with a probability greater than a preset threshold in the smoke segmentation mask output by the smoke confirmation network. The preset threshold can be adjusted according to the actual deployment scenario, for example, set to 0.5. The total combustion area reflects the overall scale of the fire at a local scale and is one of the important reference indicators for subsequent graded early warning.

[0096] The flame-to-smoke area ratio (FAR) is the ratio between the flame area and the smoke area. When calculating the FAR, a very small positive number can be added to the smoke area to avoid division by zero. For example, when the smoke area is zero or close to zero, a larger preset value is used for the FAR. The FAR reflects the stage of combustion: in the early stages of a fire, combustion is mainly smoldering and smoke release, with a smaller flame area and a larger smoke area, resulting in a lower FAR; as the fire develops, open flames gradually become dominant, the flame area increases while the smoke area relatively decreases, and the FAR increases accordingly. Therefore, the FAR provides a valuable reference for judging the development stage of a fire.

[0097] Step S303: Calculate the overall area change rate based on the total burning area in the local detail video stream of consecutive multiple frames.

[0098] The overall area change rate describes how quickly the total burning area changes over time. It can be calculated by taking the difference between the total burning area of ​​the current frame and the total burning area of ​​the previous frame or several previous frames, dividing this difference by the time interval between the two frames, and obtaining the change in total burning area per unit time. For example, if the total burning area of ​​the current frame is Area(t), the total burning area of ​​the previous frame is Area(t-Δt), and the time interval between the two frames is Δt, then the overall area change rate can be expressed as (Area(t)-Area(t-Δt)) / Δt. The overall area change rate reflects the speed of fire spread: the larger the rate of change, the faster the fire spreads and the higher the urgency of the fire.

[0099] Step S304: Combine fire characteristics, total burning area, flame-smoke area ratio, and area change rate into microscopic detail features.

[0100] Among them, microscopic detail features are a multidimensional feature set. These features and indicators characterize the type, scale, and dynamic evolution trend of the fire from different perspectives: flame morphology features and smoke translucency features focus on describing the static appearance attributes of the fire; flame dynamic features and smoke diffusion features focus on describing the dynamic behavior attributes of the fire; the total burning area and the ratio of flame to smoke area focus on describing the scale and burning stage of the fire; and the comprehensive area change rate focuses on describing the spread speed of the fire. After combining the above items into microscopic detail features, they can be used as input information for constructing the microscopic detail evidence body in the subsequent step S400.

[0101] Through steps S301 to S304 described above, this embodiment completes the entire process of extracting microscopic detail features from the PTZ local detail video stream. Unlike Embodiment 1, which only explained the principles at the higher level, this embodiment specifically illustrates the structure of the multi-task analysis network and the types of fire features, as well as the calculation logic for the total burning area, the flame-to-smoke area ratio, and the comprehensive area change rate. These microscopic detail features provide refined evidence that the panoramic end cannot provide for subsequent fusion decision-making, enabling the system to comprehensively utilize macroscopic situational information and microscopic detail information at the evidence level for fire situation assessment.

[0102] Example 4 like Figure 4 As shown, this embodiment, based on steps S400 and S500 of Embodiment 1, further elaborates on the complete implementation process of independent evidence body construction, uncertainty fusion decision-making, and dynamic updating of the warning level. Embodiment 1 has already explained that macroscopic situation features are extracted based on the panoramic video stream, and these macroscopic situation features and microscopic detail features are respectively constructed as macroscopic situation evidence bodies and microscopic detail evidence bodies. The macroscopic situation evidence bodies and the microscopic detail evidence bodies are then fused, and a fire warning signal is generated based on the fusion result. This embodiment subdivides this process into three sequentially connected steps: first, macroscopic situation evidence bodies and microscopic detail evidence bodies are formed according to the evidence body construction rules; then, the two evidence bodies are fused and decided using basic probability synthesis rules; finally, the warning level is dynamically updated after the fire warning signal is generated.

[0103] Step S401: Based on the fire location, impact range and environmental context information in the macro situation characteristics, calculate the first basic probability allocation for each preset fire level and construct the macro situation evidence body.

[0104] Among them, the macro-situation characteristics refer to the global information extracted from the panoramic video stream in step S400 of Example 1. The fire location refers to the location of the suspected fire target in geographic space. It can be represented by the geographic spatial coordinates calculated by the intersection of line-of-sight tracking and terrain in step S200, or by further estimating the actual fire coverage area based on the geographic spatial coordinates combined with the pixel size of the target's bounding box in the panoramic image and the installation parameters of the panoramic camera. The impact range refers to the spatial range that the fire target may affect at the current moment. For example, it can be estimated based on the pixel size of the target in the panoramic image, the distance between the panoramic camera and the target, and the field of view parameters of the lens, to determine the actual coverage radius or area of ​​the target on the ground. Environmental context information refers to the surrounding environmental data related to fire assessment automatically retrieved from the geographic information database, such as vegetation type, terrain slope, distance to the nearest water source, distance to the nearest residential area or important facility, and location of firebreaks. Vegetation type affects the combustion characteristics of combustibles, terrain slope affects the speed of fire spread, the distance to the nearest water source affects the convenience of water access for firefighting, the distance to the nearest residential area or important facilities affects the urgency of personnel safety assessment, and the location of firebreaks affects the possibility of fire being contained.

[0105] The preset fire severity levels refer to a set of fire severity levels predefined by the system. In this embodiment, the preset fire severity levels can be divided into four levels: Level I, Level II, Level III, and Level IV, where Level I corresponds to the lowest risk and Level IV corresponds to the highest risk. It should be noted that the specific number of preset fire severity levels and the definition of each level can be adjusted according to the actual deployment scenario. For example, more levels can be set in forest areas with high fire risk to refine the response strategy, while fewer levels can be set in areas with low fire risk to simplify the operation process.

[0106] The first basic probability allocation refers to a basic probability value assigned to each preset fire level by the macro-situational evidence body. It represents the credibility of determining whether a fire belongs to each level based solely on macro-situational information. Basic probability allocation is a fundamental concept in evidence theory, with values ​​ranging from 0 to 1, and the sum of the basic probability allocations for each preset fire level does not exceed 1. The first basic probability allocation can be calculated by matching various indicators from the fire location, impact range, and environmental context information with the threshold intervals corresponding to each preset fire level, and then combining the matching results to obtain a probability distribution for each preset fire level. For example, when the impact range is small, there are no important facilities nearby, and the vegetation type is non-flammable broadleaf forest, the probability value for level I in the first basic probability allocation is higher; when the impact range is large, there are residential areas nearby, and the vegetation type is flammable pine forest, the probability values ​​for level III or IV in the first basic probability allocation increase accordingly. The macro-situational evidence body is the evidence structure constituted by the first basic probability allocation, which completely preserves the credibility distribution of each preset fire level in the panoramic information.

[0107] Step S402: Based on the fire type, fire intensity and hazard source information in the micro-detail features, calculate the second basic probability allocation for each preset fire level and construct the micro-detail evidence body.

[0108] Among them, the micro-detail features refer to the multi-dimensional feature set extracted from the local detail video stream in Example 3, including fire characteristics, total burning area, flame-to-smoke area ratio, and comprehensive area change rate. Fire type refers to the judgment result of the nature of the fire, such as open flame, smoldering, or mixed type. Open flame refers to a burning state with visible flames, smoldering refers to a slow burning state with no visible flames but continuous smoke release, and mixed type refers to a state with both open flame and smoldering. The fire type can be determined by the confidence scores output by the flame confirmation network and smoke confirmation network, as well as the flame-to-smoke area ratio. Fire intensity refers to the severity of the fire at a local scale, which can be characterized by indicators such as total burning area, flame area, and smoke area. Hazard source information refers to the information of sensitive targets that may be threatened by fire detected in the PTZ high-definition image, such as personnel, vehicles, buildings, substations, gas stations, etc., and the distance between these sensitive targets and the fire point.

[0109] The second basic probability allocation refers to the basic probability values ​​assigned to each preset fire level by the micro-detail evidence body. It represents the credibility of determining whether a fire belongs to each level based solely on micro-detail information. The calculation method for the second basic probability allocation is similar to that of the first basic probability allocation, the difference being the source of the information. For example, when the fire type is smoldering, the total burning area is small, the overall area change rate is low, and no hazard source is detected, the probability value for level I or II in the second basic probability allocation is higher; when the fire type is dominated by open flame, the total burning area is large, the overall area change rate is high, and the detected hazard source is close to the fire point, the probability value for level III or IV in the second basic probability allocation increases accordingly. The micro-detail evidence body is the evidence structure constituted by the second basic probability allocation.

[0110] Step S403: Treat the macro-level situational evidence and the micro-level detailed evidence as independent evidence.

[0111] Independent evidence refers to two evidence bodies that are formed independently during the construction process, without depending on each other's output. The macro-level situational evidence body is constructed solely based on macro-level situational features extracted from the panoramic video stream, while the micro-level detail evidence body is constructed solely based on micro-level detail features extracted from the local detail video stream. The two remain independent at the evidentiary level. The significance of this independence lies in the fact that when one source of information has a lower confidence level due to environmental limitations, the higher confidence level information from the other source will not be diluted or interfered with during the evidence construction stage. Instead, it will be fully preserved until the fusion decision stage, where the fusion algorithm will process it appropriately according to the rules of evidence theory.

[0112] Step S501: The first basic probability allocation and the second basic probability allocation are synthesized by the basic probability synthesis rule to obtain the synthesized basic probability allocation as the fusion result, and the conflict coefficient is calculated.

[0113] The basic probability synthesis rule refers to the mathematical rule in evidence theory used to combine the basic probability allocations of multiple independent pieces of evidence into a comprehensive basic probability allocation. In this embodiment, the basic probability synthesis rule can adopt the Dempster synthesis rule. The calculation method of the Dempster synthesis rule is as follows: for any preset fire level C, its synthesized basic probability allocation m(C) is equal to the sum of the products of the basic probability allocations of all evidence combinations that satisfy the intersection of C, divided by a normalization factor. This can be expressed as: ; in, This represents the first basic probability allocation of macro-level situation evidence to the preset fire level A. Let A represent the second basic probability assignment of the micro-level detailed evidence to the presupposed fire level B. The summation in the numerator iterates through all evidence combinations that satisfy the intersection of A and B, which is C. K is the conflict coefficient, calculated as follows: ; The physical meaning of the conflict coefficient K is the degree of contradiction between two pieces of evidence. When the two pieces of evidence have a high degree of consistency in their basic probability allocations for each preset fire level, the conflict coefficient K is close to 0; when the probability allocations given by the two pieces of evidence are almost completely contradictory, the conflict coefficient K is close to 1. The larger the conflict coefficient, the greater the discrepancy between the panoramic end and the PTZ end in their judgment of the fire level, and the lower the reliability of the fusion result.

[0114] It should be noted that, to prevent the fusion result from failing due to severe conflict between two sources of evidence (such as K→1), the system introduces a conflict suppression mechanism: when the K value is greater than a preset conflict threshold (such as 0.85), the current fusion is deemed invalid, the system abandons the output of a specific grade, and instead issues a "sensor logic self-check alarm," forcibly requesting manual review of the current image source. This mechanism can effectively prevent contradictory evidence from being fabricated into a false grade due to normalization processing.

[0115] Step S502: When the conflict coefficient is greater than the preset conflict threshold, generate evidence conflict alarm information and wait for manual review instructions.

[0116] The preset conflict threshold is a critical value pre-set by the system to determine whether evidence conflicts are acceptable. Its specific value can be adjusted according to the actual deployment scenario; for example, it can be 0.85. When the conflict coefficient exceeds the preset conflict threshold, it indicates that the discrepancy between the macro-level situational evidence and the micro-level detailed evidence has exceeded the scope that automatic fusion decision-making can reliably handle. In this case, if the system still automatically outputs a warning signal based on the synthesized result, there is a risk of making an incorrect decision. Therefore, in this situation, the system does not directly output a fire warning signal but generates an evidence conflict alarm message, indicating that there is a significant discrepancy between the information from the panoramic end and the PTZ end, requiring manual intervention for review. The manual review instruction refers to the confirmation or correction instruction issued by the monitoring personnel to the system after reviewing and comparing the original information of the two evidences. Before receiving the manual review instruction, the system remains in a waiting state and does not execute subsequent warning outputs or linkage operations.

[0117] Step S503: When the conflict coefficient is less than or equal to the preset conflict threshold, the preset fire level with the highest probability value in the fusion result is determined as the fire warning signal.

[0118] Specifically, when the conflict coefficient is within an acceptable range, the fusion result integrates information from both evidence sources, and the preset fire level with the highest probability value is the output of the fusion decision. The fire warning signal may include identification information of the preset fire level, such as Level I, Level II, Level III, or Level IV, and may further include auxiliary information such as the probability values ​​and conflict coefficients of each level in the fusion result, so that monitoring personnel can understand the confidence level of the fusion decision.

[0119] Step S504: Reacquire macroscopic situation characteristics and microscopic detail characteristics according to a preset cycle, and calculate the fire index to obtain the fire index.

[0120] The preset period refers to the time interval at which the system periodically re-executes the fire assessment process. In this embodiment, the preset period can be set to 30 seconds. Re-acquiring macroscopic situation features and microscopic detail features means re-executing steps S100 to S400 in Embodiment 1 to acquire the latest panoramic video stream and local detail video stream, and extracting the updated macroscopic situation features and microscopic detail features from them. Fire intensity indicators refer to a set of quantitative indicators used to measure the severity of a fire, which may include total burning area, flame-to-smoke area ratio, and comprehensive area change rate, etc., and their calculation method is consistent with the calculation method of microscopic detail features in Embodiment 3.

[0121] Step S505: When the fire intensity index exceeds the preset upper limit threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal is upgraded.

[0122] The preset upper limit threshold refers to the upper limit value of the fire intensity index allowed by the preset fire level corresponding to the current fire warning signal. When the fire intensity index exceeds this upper limit value, it indicates that the fire has developed beyond the range described by the current level, and it is necessary to consider raising the warning level. The consecutive preset number of times refers to the number of times the fire intensity index exceeds the upper limit threshold consecutively. For example, the consecutive preset number of times can be 2. The purpose of the design of consecutive multiple judgments is to avoid triggering false escalation due to temporary index anomalies caused by sensor noise, instantaneous fluctuations in image processing, or environmental interference. Only when the fire intensity index exceeds the upper limit threshold in multiple consecutive samplings is it considered that the fire has indeed substantially deteriorated, and at this time, the fire warning signal level is raised by one level.

[0123] Step S506: When the fire intensity index falls below the preset lower limit threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal is downgraded.

[0124] The preset lower threshold refers to the lower limit of the fire intensity index allowed by the preset fire level corresponding to the current fire warning signal. When the fire intensity index is lower than this lower limit, it indicates that the fire has weakened to a level below the range described by the current level, and the warning level can be considered for downgrading. For example, the preset number of consecutive downgrading judgments can be 3, meaning that the downgrading operation is only performed when the fire intensity index is below the lower threshold for 3 consecutive samples. The number of consecutive downgrading judgments can be greater than the number of consecutive escalation judgments, because erroneous downgrading may lead to complacency regarding a still developing fire, and the risk of this is usually higher than the waste of resources caused by erroneous escalation. Therefore, a more cautious strategy is adopted in downgrading judgments.

[0125] Step S507: Set the minimum duration for each preset fire level, and restrict changes to the fire warning signal within the minimum duration.

[0126] The minimum duration refers to the shortest time that each preset fire level must remain after being output. Within the minimum duration, even if the upgrade or downgrade conditions in step S505 or S506 are met, the system does not actually change the fire warning signal level, but only records the changes in relevant indicators in the system background. The specific value of the minimum duration can be set according to the importance of each preset fire level. For example, the minimum duration for levels I and II can be set to 2 minutes, and the minimum duration for levels III and IV can be set to 5 minutes. The purpose of setting the minimum duration is to prevent the fire warning signal from frequently changing in a short period of time. For example, under a preset period of 30 seconds, if the fire index fluctuates around the threshold, it may cause the warning level to switch repeatedly between adjacent levels. Such frequent changes will make it difficult for monitoring personnel to judge the true trend of the fire and will also lead to confusion in notification and linkage operations. By setting the minimum duration, the system has better stability when outputting changes in the warning level, ensuring that emergency response decisions are based on reliable warning information.

[0127] Through steps S401 to S507, this embodiment completes the entire decision-making chain from constructing independent evidence bodies to uncertainty fusion decision-making and dynamic updating of warning levels. The macro-level situational evidence body and the micro-level detail evidence body respectively retain the uncertainties of information from the panoramic end and the PTZ end. The Dempster synthesis rule achieves reasonable fusion of the two information sources at the evidence level. The conflict coefficient mechanism avoids making erroneous decisions when there are serious conflicts in the evidence. The dynamic updating mechanism of the warning level ensures that the fire warning signal can be adjusted in a timely manner according to the actual changes in the fire situation. At the same time, the design of multiple consecutive judgments and a minimum duration ensures the stability of the warning output.

[0128] Example 5 This embodiment, based on step S100 of Embodiment 1, further elaborates on the specific implementation process of fire target detection in a panoramic video stream. As described in Embodiment 1, a panoramic video stream is acquired from a panoramic camera, and fire target detection is performed on the panoramic video stream to obtain suspected fire targets and their corresponding pixel coordinates. This embodiment subdivides this detection process into three sequentially connected steps: first, the panoramic video stream is divided into regional sub-images; then, feature extraction and target detection are performed on each regional sub-image using a target detection network; finally, the detection results of each sub-image are fused to obtain suspected fire targets and their corresponding pixel coordinates.

[0129] Step S101: Divide the panoramic video stream into multiple sub-maps with overlapping regions.

[0130] In a panoramic video stream, each frame typically has a high resolution, such as at least 4K. If the entire panoramic image is directly fed into the object detection network for inference, the limited memory and computing power of edge computing devices often necessitates downsampling the image to a lower resolution. This downsampling process causes small smoke points or flames in the distance to shrink further or even disappear from the image, thus reducing the detection recall rate. To address this issue, this embodiment employs a slice-assisted inference method, dividing each frame of the panoramic image into multiple region sub-images. Each region sub-image is then fed into the object detection network for inference. The region sub-images overlap to prevent targets from being truncated at sub-image boundaries.

[0131] Specifically, if there is no overlap between the sub-images, a suspected fire target that crosses the sub-image boundary will be divided into two parts, appearing at the edges of two adjacent sub-images. Each part may be too small or have incomplete features to be recognized by the target detection network, resulting in missed detection. By setting overlapping regions, the same target crossing the boundary appears in a complete or nearly complete form in both adjacent sub-images, thus ensuring the continuity of detection. The width of the overlapping region can be set according to the input size of the target detection network and the resolution of the panoramic image. For example, when dividing a 3840×1920 panoramic image into four 1920×960 sub-images, the width of the overlapping region between adjacent sub-images can be 100 pixels.

[0132] Step S102: Feature extraction and target detection are performed on each region sub-image using an object detection network to obtain the sub-image detection results.

[0133] The target detection network refers to a neural network model deployed on the near-end computing device of the panoramic camera. It is used to extract features and detect targets in the input regional sub-map, and output the location and category information of suspected fire targets in the sub-map. The sub-map detection results can include the coordinates of the target bounding box in the sub-map, the target category, and the corresponding confidence score. The target detection network includes an asymmetric multi-scale receptive field enhancement module and an ultra-high resolution detection head in terms of structure. The two improve the network's ability to detect forest fire targets from different perspectives.

[0134] The asymmetric multi-scale receptive field enhancement module is used to enhance the network's adaptability to fire targets with significant scale differences in forest scenes. In forest fire scenarios, initial smoke points may only occupy 0.01% to 0.5% of the image pixel area, while flames in the development stage may occupy more than 5%, resulting in a huge range of target scales. The standard convolutional receptive field in traditional target detection networks is usually of a fixed size, making it difficult to effectively respond to both small smoke points and large flame areas simultaneously. The asymmetric multi-scale receptive field enhancement module extracts multi-scale features by setting multiple dilated convolutions with different dilation rates in parallel, such as 1, 3, 5, and 7. Then, it adaptively fuses the feature responses of each scale through a channel attention mechanism, enabling the network to be sensitive to both small smoke points and large flame areas simultaneously.

[0135] It should be noted that the asymmetric design is adopted because the texture feature scales in the vertical direction (from tree canopy to ground) and horizontal direction (from ridge to valley) of forest areas differ significantly. Asymmetric sampling can better adapt to the spatial anisotropy of forest scenes. In other words, the asymmetric multi-scale receptive field enhancement module does not simply use the same receptive field expansion method in all directions, but rather differentiates the texture scale differences in different spatial directions within the forest scene, thereby more effectively extracting the spatial features of fire targets.

[0136] An ultra-high resolution detection head is used to improve the network's sensitivity to small, distant targets. In target detection networks, feature maps at different levels have different resolutions: shallow feature maps have high resolution and weak semantic information, suitable for detecting small targets; deep feature maps have low resolution and strong semantic information, suitable for detecting large targets. Traditional target detection networks typically start setting up the detection head from deeper feature maps, underutilizing shallow high-resolution feature maps, thus limiting the ability to detect small targets. The ultra-high resolution detection head in this embodiment utilizes shallow high-resolution feature maps for detection, for example, using a feature map downsampled by 4 times from the original image, specifically for detecting small, distant targets. Because this detection head uses higher resolution feature maps, small smoke points retain more spatial detail information on the feature map, thereby improving the network's sensitivity to initial flames and tiny smoke points. Simultaneously, to balance computational load, the ultra-high resolution detection head can employ a lightweight convolutional structure, activating only when detecting small targets, avoiding excessive impact on overall inference speed.

[0137] It's important to note that the target detection network prioritizes high recall, allowing for a certain sacrifice in precision. This is because the target detection network in this embodiment performs the initial screening task at the panoramic level. Its responsibility is to ensure that no suspicious targets are missed within a broad range. Whether a target represents a real fire is then determined by the subsequent PTZ (Potentially Persistent Threatened Detection) refinement stage. If the target detection network filters out low-confidence suspected targets in pursuit of high precision, it may cause initial fires to be missed due to weak features, resulting in missed opportunities for optimal response. Therefore, when training the target detection network, recall can be prioritized, for example, setting a recall target of at least 95%, while allowing for a slightly lower precision target, such as at least 75%. In this way, the panoramic level can complete the initial screening with high recall, providing sufficient candidate targets for subsequent PTZ refinement.

[0138] Step S103: Non-maximum suppression fusion is performed on the detection results of each sub-image to obtain the suspected fire target and its corresponding pixel coordinates.

[0139] In some implementations, due to overlapping areas between sub-images, the same suspected fire target may be detected by multiple sub-images, resulting in multiple overlapping detection boxes. Non-maximum suppression fusion (NMS) aims to deduplicate and merge these redundant detection boxes, retaining the detection box with the highest confidence and suppressing other detection boxes whose overlap exceeds a preset threshold. After NMS fusion, each suspected fire target corresponds to a unique detection box. The position of this detection box in the panoramic image is the pixel coordinate of the suspected fire target, typically taken as the center point of the detection box or the centroid of the smoke area within the panoramic image.

[0140] Through steps S101 to S103, this embodiment completes a full detection chain from panoramic video stream to suspected fire targets and their corresponding pixel coordinates. Region sub-image partitioning avoids the problem of small target loss caused by direct downsampling of high-resolution panoramic images. The asymmetric multi-scale receptive field enhancement module and the ultra-high-resolution detection head improve detection capabilities from two dimensions: multi-scale feature extraction and small target sensitivity, respectively. Non-maximum suppression fusion eliminates redundant detection results caused by overlapping regions. This detection chain prioritizes high recall as its core optimization objective, providing a reliable source of candidate targets for subsequent spatial localization, PTZ linkage, and fusion decision-making.

[0141] Example 6 like Figure 5 As shown, this embodiment provides a forest fire early warning system 600 that integrates panoramic video and PTZ video, used to implement the forest fire early warning method integrating panoramic video and PTZ video described in any of the preceding embodiments. Unlike Embodiment 1, which focuses on the sequential flow of method steps, this embodiment focuses on the system's modular architecture, the data flow relationship between modules, and the hardware carrier of each module. The system includes a panoramic video acquisition and detection module 601, a spatial positioning and linkage control module 602, a detail acquisition and feature extraction module 603, an independent evidence construction module 604, and a fusion decision and early warning output module 605.

[0142] The panoramic video acquisition and detection module 601 is used to acquire the panoramic video stream captured by the panoramic camera, perform fire target detection on the panoramic video stream, and obtain the suspected fire targets and their corresponding pixel coordinates.

[0143] Specifically, the panoramic video acquisition and detection module 601 serves as the data entry point between the system and the panoramic camera. This module can be deployed on an edge computing device, which refers to a computing device with a certain computing power located near the panoramic camera, such as an embedded computing node deployed on a forest lookout tower or monitoring pole. The significance of deploying the panoramic video acquisition and detection module 601 on an edge computing device rather than a central server is that the panoramic video stream has a large data volume. If all of it were transmitted back to the central server for processing, it would not only consume a large amount of network bandwidth but also introduce additional transmission latency. Performing fire target detection directly at the edge allows for the transmission of only lightweight results such as detected suspected fire targets and their corresponding pixel coordinates, thereby reducing dependence on network transmission conditions and improving the system's real-time response capability. The fire target detection performed by this module can be implemented using methods such as region subgraph partitioning, target detection network inference, and non-maximum suppression fusion as described in Example 5. The output suspected fire targets and their corresponding pixel coordinates serve as inputs to the subsequent spatial positioning and linkage control module 602.

[0144] The spatial positioning and linkage control module 602 is used to perform line-of-sight tracking and terrain intersection calculation based on pixel coordinates and prior 3D terrain data to obtain the geospatial coordinates of the suspected fire target, and generate linkage control parameters for the target PTZ based on the geospatial coordinates.

[0145] Specifically, the spatial positioning and linkage control module 602 receives the pixel coordinates output by the panoramic video acquisition and detection module 601, and combines them with pre-acquired and stored prior 3D terrain data to perform a conversion from pixel space to geographic space. The specific implementation of line-of-sight tracking and terrain intersection calculation can employ methods such as step-by-step extension along the line-of-sight direction, elevation comparison, and spatial interpolation as described in Example 2. After obtaining the geographic spatial coordinates, this module further generates linkage control parameters for the target PTZ. The generation process of linkage control parameters can include two stages: target PTZ optimization and residual compensation. Target PTZ optimization can employ a comprehensive scoring method based on line-of-sight occlusion scoring, observation geometry scoring, and response time scoring as described in Example 2. Residual compensation can employ a posture residual correction method based on a residual compensation network as described in Example 2. The hardware carrier of this module can be an edge computing device or a central server. When the amount of prior 3D terrain data is large and line-of-sight tracking calculations are frequent, this module can be deployed on a central server with strong computing power. When the edge computing device has sufficient storage and computing power, this module and the panoramic video acquisition and detection module 601 can also be deployed on the same edge computing device to reduce the latency of data transmission between modules.

[0146] The detail acquisition and feature extraction module 603 is used to control the target PTZ to acquire local detail video streams based on linkage control parameters, and to extract micro-detail features based on the local detail video streams.

[0147] Specifically, the detail acquisition and feature extraction module 603 receives the linkage control parameters output by the spatial positioning and linkage control module 602, and sends the linkage control parameters to the target PTZ camera via the network. This controls the target PTZ camera to perform horizontal rotation, vertical pitch, and optical zoom, aligning the camera's lens with the suspected fire target area and continuously acquiring a high-definition video stream of that area, i.e., a local detail video stream. After acquiring the local detail video stream, this module performs feature extraction to obtain microscopic detail features. The extraction of microscopic detail features can be achieved using methods described in Example 3, such as multi-task analysis network, total combustion area calculation, flame-smoke area ratio calculation, and area comprehensive change rate calculation. The hardware support for this module is typically a central server or a high-performance edge computing node, because the computing power required for multi-task analysis of the PTZ high-definition video stream is usually higher than that required for panoramic target detection.

[0148] The independent evidence body construction module 604 is used to extract macro-situation features based on panoramic video streams, and to construct macro-situation features and micro-detail features into macro-situation evidence bodies and micro-detail evidence bodies, respectively.

[0149] Specifically, the independent evidence body construction module 604 receives two inputs: one is the panoramic video stream acquired by the panoramic video acquisition and detection module 601, and the other is the microscopic detail features output by the detail acquisition and feature extraction module 603. For the panoramic video stream, this module extracts macroscopic situation features, which may include global information such as fire location, impact range, and environmental context. For the microscopic detail features, this module constructs them and the macroscopic situation features into a macroscopic situation evidence body and a microscopic detail evidence body, respectively. The construction of the two independent evidence bodies can adopt the calculation methods of the first and second basic probability allocations described in Example 4. The hardware host of this module can be a central server, because constructing the evidence body requires simultaneous access to the panoramic video stream data and the microscopic detail feature data, and these two types of data are usually generated at the edge and center sides, respectively. It is more convenient for the central server to collect and process them uniformly.

[0150] The fusion decision and early warning output module 605 is used to fuse macro-level situational evidence and micro-level detailed evidence, and generate fire early warning signals based on the fusion results.

[0151] Specifically, the fusion decision and early warning output module 605 receives two independent evidence bodies output by the independent evidence body construction module 604, and performs an uncertainty fusion decision on the two independent evidence bodies. The uncertainty fusion decision adopts the basic probability synthesis rule described in Example 4. The fire early warning signal can be further output to the display device, notification push system, or emergency response platform of the monitoring center. The hardware carrier of this module is usually a central server, because the fusion decision needs to aggregate evidence bodies from different information sources, and the output of the early warning signal needs to interface with various peripheral systems of the monitoring center.

[0152] Through the collaborative work of the five functional modules described above, the system provided in this embodiment organically integrates panoramic video acquisition and detection, spatial positioning and linkage control, detail acquisition and feature extraction, independent evidence construction, and fusion decision-making and early warning output into a complete closed-loop process. This modular architecture clearly defines the responsibilities of each component of the system, facilitating flexible deployment and expansion in actual forest scenarios based on equipment and network conditions.

[0153] The above description is merely a specific implementation of this application, but the scope of protection of this application is not limited thereto. It should be understood that the specific parameter values, network structures, module division methods, preset thresholds, and preset periods given in the above embodiments are all exemplary descriptions used to help those skilled in the art understand the technical solutions of this application, and are not intended to limit the scope of protection of this application. Any adjustments, changes, or substitutions made by those skilled in the art to the above parameters, structures, or division methods within the scope of the technology disclosed in this application, based on actual deployment scenarios, as long as they do not depart from the core concept of the technical solutions of this application, should be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A forest fire early warning method integrating panoramic video and PTZ video, characterized in that, include: Acquire panoramic video streams captured by panoramic cameras, perform fire target detection on the panoramic video streams, and obtain suspected fire targets and their corresponding pixel coordinates; Based on the pixel coordinates and prior 3D terrain data, line-of-sight tracking and terrain intersection calculation are performed to obtain the geospatial coordinates of the suspected fire target, and the linkage control parameters of the target PTZ are generated based on the geospatial coordinates. Based on the aforementioned linkage control parameters, the target PTZ is controlled to acquire local detail video streams, and microscopic detail features are extracted based on the local detail video streams; Based on the panoramic video stream, macroscopic situation features are extracted, and the macroscopic situation features and the microscopic detail features are respectively constructed into macroscopic situation evidence body and microscopic detail evidence body; The macroscopic situation evidence and the microscopic detail evidence are fused together, and a fire warning signal is generated based on the fusion result.

2. The forest fire early warning method integrating panoramic video and PTZ video according to claim 1, characterized in that, The process of performing line-of-sight tracking and terrain intersection calculation based on the pixel coordinates and prior 3D terrain data includes: Based on the pose parameters of the panoramic camera and the pixel coordinates, the line of sight corresponding to the suspected fire target is determined; Based on the camera installation position in the pose parameters, extension points are generated by gradually extending along the line of sight at a preset step size. The elevation of the extension points is then compared with the terrain elevation of the corresponding position in the prior three-dimensional terrain data. When the elevation of the extension point is lower than the terrain elevation, the position of the extension point is determined as a terrain intersection point; Spatial interpolation is performed on the terrain intersections to obtain the geographic spatial coordinates.

3. The forest fire early warning method integrating panoramic video and PTZ video according to claim 1, characterized in that, The linkage control parameters for generating the target PTZ based on the geospatial coordinates include: Based on the geospatial coordinates and the installation location of the target PTZ, calculate the initial control parameters; Based on the prior 3D terrain data, the terrain slope and terrain roughness index of the region where the geospatial coordinates are located are extracted; The pixel coordinates, terrain slope, and terrain roughness index are input into a pre-trained residual compensation network to output the pose residual. The initial control parameters are corrected based on the attitude residual to obtain the linkage control parameters.

4. A forest fire early warning method integrating panoramic video and PTZ video according to claim 3, characterized in that, Before generating the linkage control parameters for the target PTZ based on the geospatial coordinates, the method further includes: Based on the prior 3D terrain data, a score is calculated to indicate the degree of line-of-sight occlusion of multiple candidate PTZs pointing to the geospatial coordinates; The observation geometry score and response time score are calculated based on the installation locations of multiple candidate PTZs and the geospatial coordinates. Based on the line-of-sight occlusion score, the observation geometry score, and the response time score, calculate the comprehensive score for each candidate PTZ; The candidate PTZ with the highest comprehensive score is determined as the target PTZ.

5. A forest fire early warning method integrating panoramic video and PTZ video according to claim 1, characterized in that, The extraction of micro-detail features based on the local detail video stream includes: Fire features are extracted from the local detail video stream using a multi-task analysis network; the fire features include flame morphology features, flame dynamic features, smoke translucency features, and smoke diffusion features. Based on the flame morphology characteristics and the smoke translucency characteristics, the ratio of total combustion area to flame-smoke area is calculated; The overall area change rate is calculated based on the total burning area in the local detail video stream across multiple consecutive frames. The fire characteristics, the total burning area, the flame-smoke area ratio, and the overall area change rate are combined to form the microscopic detail features.

6. A forest fire early warning method integrating panoramic video and PTZ video according to claim 1, characterized in that, The step of constructing the macro-level situational features and the micro-level detail features into a macro-level situational evidence body and a micro-level detail evidence body, respectively, includes: Based on the fire location, impact range, and environmental context information in the macro-situation characteristics, the first basic probability allocation of each preset fire level is calculated to construct the macro-situation evidence body. Based on the fire type, fire intensity, and hazard source information in the micro-detail features, the second basic probability allocation for each preset fire level is calculated, and the micro-detail evidence body is constructed.

7. A forest fire early warning method integrating panoramic video and PTZ video according to claim 6, characterized in that, The generation of fire early warning signals based on the fusion results includes: The first basic probability allocation and the second basic probability allocation are synthesized using basic probability synthesis rules to obtain a synthesized basic probability allocation as the fusion result, and the conflict coefficient is calculated. When the conflict coefficient is greater than the preset conflict threshold, an evidence conflict alarm message is generated, and a manual review instruction is awaited. When the conflict coefficient is less than or equal to the preset conflict threshold, the preset fire level with the highest probability value in the fusion result is determined as the fire warning signal.

8. A forest fire early warning method integrating panoramic video and PTZ video according to claim 7, characterized in that, After generating the fire early warning signal based on the fusion result, the following is also included: The macroscopic situation characteristics and microscopic details are reacquired at a preset cycle, and the fire intensity index is calculated to obtain the fire intensity index. When the fire intensity index exceeds the preset upper limit threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal will be upgraded. When the fire intensity index falls below the preset lower threshold corresponding to the current fire warning signal for a preset number of consecutive preset times, the level of the fire warning signal will be downgraded. Set a minimum duration for each preset fire level, and restrict changes to the fire warning signal within the minimum duration.

9. A forest fire early warning method integrating panoramic video and PTZ video according to claim 1, characterized in that, The fire target detection of the panoramic video stream includes: The panoramic video stream is divided into multiple region sub-maps with overlapping areas; Feature extraction and target detection are performed on each of the aforementioned sub-regions using a target detection network to obtain sub-region detection results; wherein, the target detection network includes an asymmetric multi-scale receptive field enhancement module and an ultra-high resolution detection head; Non-maximum suppression fusion is performed on the detection results of each sub-image to obtain the suspected fire target and the corresponding pixel coordinates.

10. A forest fire early warning system integrating panoramic video and PTZ video, characterized in that, include: The panoramic video acquisition and detection module is used to acquire the panoramic video stream captured by the panoramic camera, perform fire target detection on the panoramic video stream, and obtain the suspected fire target and the corresponding pixel coordinates. The spatial positioning and linkage control module is used to perform line-of-sight tracking and terrain intersection calculation based on the pixel coordinates and prior three-dimensional terrain data to obtain the geospatial coordinates of the suspected fire target, and generate linkage control parameters for the target PTZ based on the geospatial coordinates. The detail acquisition and feature extraction module is used to control the target PTZ to acquire local detail video streams based on the linkage control parameters, and to extract micro-detail features based on the local detail video streams; An independent evidence body construction module is used to extract macro-situational features based on the panoramic video stream, and to construct the macro-situational features and the micro-detail features into macro-situational evidence bodies and micro-detail evidence bodies, respectively. The fusion decision and early warning output module is used to fuse the macroscopic situation evidence and the microscopic detail evidence, and generate a fire early warning signal based on the fusion result.

Images