Forest fire detection method and device based on laser radar and camera fusion

Abstract

The invention relates to the field of forest fire detection, in particular to a forest fire detection method and device based on laser radar and camera fusion. The detection method comprises the steps of taking an image of a target area, and identifying a fire point image position in the image; converting the fire point image position into a fire point geographic position in a geographic space; an aerosol laser radar scans the geographic position of a fire point in a geographic space; after a laser radar transmitting signal passes through the atmosphere, an extinction coefficient and a depolarization ratio are returned; judging whether the extinction coefficient exceeds the calibration coefficient or not, if so, then performing the next step, otherwise, judging that no fire occurs at the fire point geographic position; and judging whether the depolarization ratio is within a calibration range, if so, determining the geographic position of the fire alarm fire point, otherwise, introducing an artificial rechecking mechanism. According to the invention, the detection data of the laser radar is utilized to enhance the accuracy of image recognition, and the panoramic information of the image is utilized to make up for the insufficiency of the coverage range of the laser radar, thereby remarkably improving the accuracy, real-time performance and stability of fire smoke detection.

Description

Technical Field

[0001] The present invention relates to a forest fire detection method and a forest fire detection device using the method in the field of forest fire detection, and in particular to a forest fire detection method based on the fusion of a laser radar and a camera and a device using the method. Background Art

[0002] Most of the existing fire video detection technologies are AI smoke target detection technologies based on deep learning, which analyze the characteristics of fire points in videos or images to achieve fire early warning. Such methods and systems usually use convolutional neural networks (CNN) or target detection algorithms (such as YOLO, Faster R-CNN) to identify the visual patterns of fire points (fire targets), including color, texture, diffusion dynamics and other features. Due to the translucent and variable shape of smoke, advanced detection methods will also combine time series analysis and use optical flow or 3D convolution to capture the motion characteristics of fire points to improve detection accuracy. However, although deep learning models perform well in laboratory environments, due to the complexity of actual scenes, image fire detection methods still face a high false alarm rate in reality, as follows Figure 1 As shown. The false alarm problem mainly stems from the following three aspects: First, there are many interferences in nature that are similar to the visual characteristics of smoke, such as steam, haze, dust and even fast-moving clouds, which may be mistakenly identified as smoke by the system. Secondly, environmental factors such as lighting changes, camera shake, reflections or lens stains can also affect image-based detection results. In addition, the limitation of training data is also a key factor - most fire point detection models rely on limited data sets for training, which makes it difficult to cover all possible real-world scenarios, resulting in insufficient generalization capabilities when facing unknown environments. The smoke in the early stages of a fire is often thin and the features are not obvious, and increasing the sensitivity of the model to reduce missed reports will bring more false alarms, which means that the system still needs to rely on manual review to ensure reliability in actual applications.

[0003] Aerosol lidar (LIDAR) offers significant penetration capabilities and the ability to quantitatively analyze aerosol particles for fire smoke detection. It can effectively identify the concentration and spatial distribution of particles in smoke, providing a scientific basis for early fire warning. However, in practical applications, LIDAR still has certain limitations. Due to its point-by-point or line-by-line scanning mechanism, its spatial coverage per unit time is low, making it difficult to quickly and comprehensively scan and monitor large areas in real time. Especially in large or open environments such as forests, industrial parks, and tunnels, LIDAR's effective detection area per unit time is significantly smaller than that of video surveillance equipment. This limits its efficiency and response speed for time-sensitive monitoring tasks. In contrast, image detection technology offers the advantages of a wide field of view and fast acquisition speed, enabling it to cover larger areas per unit time and achieve rapid, global monitoring of fire smoke.

[0004] Therefore, fire image detection technology may have certain errors when identifying real smoke and interference objects (fire point detection) under complex meteorological conditions; although lidar has high accuracy in detecting smoke, its spatial coverage rate within a certain period of time is low. Compared with the vast forest area, it is difficult to achieve comprehensive spatial coverage in a short period of time even with high-power lidar. Moreover, the higher the power of the lidar, the more likely it is to have problems (for example, it may itself be a fire hazard), making it difficult to quickly achieve comprehensive scanning and real-time monitoring of large areas. Summary of the Invention

[0005] In order to solve the technical problem that the existing forest fire detection technology has unsatisfactory real-time performance and accuracy of fire detection results, the present invention provides a forest fire detection method based on the fusion of laser radar and camera and a device using the method.

[0006] The present invention utilizes the following technical solution: a forest fire detection method based on the fusion of laser radar and camera, comprising the following steps: capturing an image of a target area and identifying the location of a fire point in the image; converting the image location of the fire point into the geographic location of the fire point in geographic space; and issuing a fire alarm at the geographic location of the fire point. Prior to the fire alarm, the method also includes: scanning the geographic location of the fire point in geographic space using an aerosol laser radar; transmitting a signal through the atmosphere, which returns an extinction coefficient and depolarization ratio; determining whether the extinction coefficient exceeds a calibration coefficient; if so, proceeding to the next step; otherwise, determining that no fire has occurred at the geographic location of the fire point; and determining whether the depolarization ratio is within a calibration range; if so, issuing a fire alarm; otherwise, initiating a manual review mechanism.

[0007] As a further improvement of the above solution, the forest fire detection method further includes: correcting the position of the fire point image; wherein the method for correcting the position of the fire point image includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's pitch angle T and horizontal angle P, as well as ΔV and ΔH, the fire point image position relative to the camera's optimized pitch angle θ and horizontal angle β is calculated: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

[0008] As a further improvement of the above scheme, a method for converting the position of a fire point image into the geographical location of the fire point includes the following steps: constructing a spatial observation ray of the fire point image position in the geographic space according to the pitch angle T and the horizontal angle P of the camera; obtaining the profile line of P through the DEM elevation data; and taking the first intersection of the spatial observation ray and the profile line as the geographical location of the fire point.

[0009] As a further improvement of the above scheme, the forest fire detection method also includes: confirming whether there is any obstruction in the laser transmission path of the aerosol lidar, and if so, determining that the data is invalid and abandoning subsequent analysis; otherwise, determining whether the extinction coefficient exceeds the calibration coefficient.

[0010] As a further improvement of the above solution, a dual-channel camera is used to capture images of the target area.

[0011] The present invention also provides a forest fire detection device based on the fusion of laser radar and camera, which includes a dual-channel camera, an aerosol laser radar, an electric pan-tilt head, and an edge box. The edge box is used to: control the electric pan-tilt head to drive the dual-channel camera to rotate periodically along a set patrol path to capture images of the target area; identify the image location of the fire point in the image; convert the image location of the fire point into the geographical location of the fire point in geographic space; control the electric pan-tilt head to drive the aerosol laser radar to scan the geographical location of the fire point in geographic space, and the laser radar transmits a signal that returns an extinction coefficient and depolarization ratio after passing through the atmosphere; determine whether the extinction coefficient exceeds the calibration coefficient, and if so, continue to determine whether the depolarization ratio is within the calibration range; otherwise, determine that there is no fire at the geographical location of the fire point; if the depolarization ratio is determined to be within the calibration range, a fire alarm is issued to the geographical location of the fire point; otherwise, a manual review mechanism is introduced.

[0012] As a further improvement of the above solution, the electric gimbal is also used to adjust the pitch angle of the aerosol lidar until the lidar transmission signal emitted by the lidar is incident on the smoke at the geographical location of the fire.

[0013] As a further improvement of the above solution, when the edge box detects the position of the fire point image, it drives the electric pan-tilt head to dynamically adjust the direction and elevation angle to achieve multi-angle re-shooting and tracking of the fire point image position.

[0014] As a further improvement of the above solution, the forest fire detection device also includes a bracket for carrying a dual-channel camera, an aerosol lidar, an electric pan-tilt head, and an edge box.

[0015] As a further improvement of the above solution, the edge box is further used to correct the position of the fire point image; wherein the method for correcting the position of the fire point image includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's horizontal angle T and pitch angle P, as well as ΔV and ΔH, calculate the fire point image position relative to the camera's optimized horizontal angle θ and pitch angle β: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

[0016] As a further improvement of the above scheme, the edge box is also used to: construct the spatial observation ray of the fire point image position in the geographic space according to the horizontal angle T and pitch angle P of the camera; obtain the profile line of P through the DEM elevation data; and take the first intersection of the spatial observation ray and the profile line as the geographic location of the fire point.

[0017] The present invention also provides a forest fire detection method based on the fusion of laser radar and camera, which includes the following steps: capturing an image of the target area and identifying the image position of the fire point in the image; correcting the image position of the fire point; converting the corrected image position of the fire point into the geographical location of the fire point in geographic space; and generating a fire alarm at the geographical location of the fire point. The method for correcting the image position of the fire point includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's horizontal angle T and pitch angle P, as well as ΔV and ΔH, calculate the fire point image position relative to the camera's optimized horizontal angle θ and pitch angle β: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

[0018] Compared to existing technologies, this invention combines the high-sensitivity detection capabilities of aerosol lidar with the wide-angle, rapid capture advantages of image detection. This approach complements the shortcomings of each technology, leveraging lidar detection data to enhance image recognition accuracy and leveraging panoramic image information to compensate for the lidar's limited coverage. This significantly improves the accuracy, real-time nature, and stability of fire detection. This invention is primarily characterized by the following two key aspects.

[0019] (1) Integrate aerosol lidar to identify fire smoke.

[0020] Considering the long cycle time associated with all-around laser radar scanning, this invention leverages smoke information captured by a front-mounted dual-channel camera to achieve rapid laser radar response and key area monitoring. This fusion recognition method significantly improves the timeliness of aerosol laser radar fire smoke detection and forms the key technical foundation and core preventive protection component of this invention.

[0021] (2) Fusion alarm method based on video images and aerosol lidar.

[0022] This method fully integrates the spatial scope and real-time advantages of image recognition with the quantitative perception of aerosol concentration changes by lidar. By extracting fire point features from visible or infrared images and combining them with aerosol parameters returned by lidar in real time, this method enables cross-verification and joint judgment of suspected fires, significantly improving the accuracy and reliability of fire alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 Schematic diagrams showing examples of various scenarios in which false alarms may occur in fire image detection using existing technologies.

[0024] Figure 2 A schematic flow chart of a forest fire detection method based on the fusion of lidar and camera provided in Example 1 of the present invention.

[0025] Figure 3 For application Figure 2 Flowchart of the method for correcting the fire point image position in the forest fire detection method.

[0026] Figure 4 For application Figure 3 Schematic diagram of fire sight angle correction after the correction method shown.

[0027] Figure 5 For application Figure 2 Schematic diagram of DEM fire point positioning line of sight for forest fire detection method.

[0028] Figure 6 For Figure 4 Corresponding fire point, section line and sight line relationship diagram.

[0029] Figure 7 For application Figure 2 Flowchart of the forest fire detection method using lidar to detect fire smoke.

[0030] Figure 8 For application Figure 2 Flowchart of the forest fire detection method using dual-channel camera and aerosol lidar fusion for fire detection.

[0031] Figure 9 A schematic structural diagram of a forest fire detection device based on the fusion of lidar and camera provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] It should be noted that when a component is referred to as being "mounted on" another component, it may be directly on the other component or there may be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a central component. When a component is considered to be "fixed to" another component, it may be directly fixed to the other component or there may be a central component.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0035] Example 1 See also Figure 2 , which is a flow chart of a forest fire detection method based on laser radar and camera fusion provided by Example 1 of the present invention. The forest fire detection method comprises the following steps S11 to S18.

[0036] S11, capturing an image of the target area and identifying the location of the fire point in the image.

[0037] A dual-channel camera can be used to capture images of the target area, which can be the forest area to be monitored or a specific monitoring area within the forest. The dual-channel camera uses both thermal and visible light channels to capture image data of the monitored area. Deep learning-based image processing algorithms can then be used to locate the fire point and determine whether a fire has occurred. Therefore, the dual-channel image data can be used to determine whether a fire has occurred using an image detection algorithm, while the positioning algorithm can also determine the location of the area to be detected.

[0038] For the identification and location of fire spots, the image detection principle can be applied using existing technologies. Image object detection is a key technology in the field of computer vision. Its core task is to automatically identify and locate objects of a specific category from the input image, and output the category label and precise bounding box coordinates of each detected object. The implementation of this technology relies on a deep understanding of the image content and involves multiple key links such as feature extraction, target location, classification, and bounding box regression. Early target detection methods were mainly based on manually designed features such as HOG (Histogram of Oriented Gradients) and SIFT (Scale-Invariant Feature Transform), combined with a sliding window strategy to search for targets at different locations and scales in the image. However, due to low computational efficiency and limited generalization ability, such methods have gradually been replaced by deep learning-based methods.

[0039] With the rise of convolutional neural networks (CNNs), object detection technology has undergone a revolutionary breakthrough. Modern object detection algorithms can be broadly categorized into two main groups: two-stage detectors and one-stage detectors. Faster R-CNN, a representative example of a two-stage detector, employs a two-step workflow: first, generating a series of candidate regions (region proposals) that may contain an object, typically performed by a region proposal network (RPN); then, performing precise classification and bounding box regression on these candidate regions. This approach, due to the refined processing performed during the candidate region screening stage, generally achieves high detection accuracy, but suffers from high computational overhead, making it difficult to meet real-time requirements. In contrast, one-stage detectors such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) directly uniformly sample prediction points across the image and perform classification and regression simultaneously, omitting the candidate region generation step. This results in faster speed, but often offers slightly lower accuracy than two-stage methods.

[0040] The core idea behind both two-stage and one-stage detectors is to use deep neural networks to extract multi-level feature representations from images and use these features to predict the category and location of objects. During the feature extraction stage, a pre-trained CNN (such as ResNet, VGG, or EfficientNet) is typically used as the backbone network to extract feature maps at different levels. Shallow feature maps contain rich detail information and are suitable for detecting small objects, while deep feature maps have stronger semantic information and are suitable for detecting large objects. To fully utilize multi-scale information, many modern detectors have introduced feature pyramid networks (FPNs), which fuse features from different levels through top-down paths and lateral connections to improve detection performance.

[0041] For object localization, most detectors use an anchor box mechanism. This involves presetting a set of reference boxes of varying scales and aspect ratios at each spatial location in the image. These boxes are then regressed to predict their offsets, resulting in the final detection box. This approach effectively reduces the search space and improves detection efficiency. Furthermore, non-maximum suppression (NMS) techniques are widely used in post-processing to eliminate redundant detection boxes and retain only the most confident predictions. In recent years, some research has attempted to improve NMS algorithms, such as Soft-NMS and Adaptive NMS, to alleviate the problem of missed detections in dense object detection.

[0042] In addition to traditional CNN architectures, Transformer structures have also been introduced into the field of object detection. For example, the DETR (Detection Transformer) model dispenses with anchor boxes and NMS, directly utilizing a global attention mechanism to model the relationships among all objects in an image and outputting detection results through collective prediction. This approach simplifies the detection process but typically requires longer training times and greater computational resources. Furthermore, some research has explored anchor-free detection methods, such as CenterNet and FCOS, which achieve detection by predicting the center point or key points of an object, further simplifying model design.

[0043] The object detection training process typically employs a multi-task loss function, optimizing both classification and regression losses. Classification losses (such as cross-entropy loss or focal loss) ensure the model correctly identifies the object category, while regression losses (such as smooth-L1 loss or intersection over union loss) optimize the bounding box localization accuracy. Furthermore, data augmentation techniques (such as random cropping, rotation, and color transformation) are widely used to improve model generalization. Transfer learning can leverage large-scale pre-trained models (such as classification models on ImageNet) to accelerate convergence and improve detection performance.

[0044] With the continuous advancement of technology, object detection technology has demonstrated unique application value in the field of fireworks detection. Given the dynamic characteristics of flames and smoke, modern fireworks detection systems typically employ spatiotemporal fusion detection strategies, analyzing motion features between consecutive frames through 3D convolution or temporal attention mechanisms. Regarding feature extraction, an improved feature pyramid network combined with a channel-wise attention mechanism effectively captures the color characteristics of flames and the diffusion pattern of smoke. Given the large scale variation of fireworks targets, multi-scale feature fusion and adaptive anchor box design are particularly important. The lightweight YOLO series of models, due to their excellent speed-accuracy balance, has become the preferred solution for real-time fireworks detection. Furthermore, Transformer-based detection models, through a global attention mechanism, can better model the diffusion trajectory of smoke. In practical deployments, the combination of multimodal data fusion from infrared thermal imaging and NMS post-processing optimized for fireworks characteristics further improves the system's detection accuracy. These technological advances have made fireworks detection systems increasingly important in scenarios such as forest fire prevention and urban security.

[0045] In fire monitoring systems, fire point detection based on video images is a key technology for ensuring accurate fire detection. Cameras (such as dual-channel cameras) are mounted on fixed platforms such as observation towers, high poles, or communication towers to continuously capture video of distant forests. Because cameras capture two-dimensional images, while the actual location of fire points resides in three-dimensional geographic space, the fire point coordinates obtained through image recognition cannot be directly mapped to their actual geographic location. To accurately convert the two-dimensional image to three-dimensional space, the actual spatial orientation of the fire point must be calculated based on its position in the image, the camera's field of view, the camera's mounting posture, and other parameters. This calculation, known as angle correction, provides the critical observation direction parameter for spatially locating the fire point. While the image captured by the camera is a two-dimensional plane (pixel coordinate system), the fire point actually resides in three-dimensional space. Therefore, the geographical location of the fire point obtained by traditional fire monitoring often has large errors, such as the research on forest fire positioning algorithm based on video monitoring system; Zhang Jian, Wang Yuanyuan, Han Ning, Liang Yi; School of Engineering, Beijing Forestry University; Journal of Safety and Environment, Vol. 9, No. 1, February 2009; Article Number: 1009-6094(2009)01-0127-04; P127~130, which constructs the spatial observation ray of the fire point image position in geographic space based on the camera's pitch angle T and horizontal angle P; obtains the profile line of P through DEM elevation data; and takes the first intersection of the spatial observation ray and the profile line as the geographical location of the fire point.

[0046] Through research, the present invention has found that the fire points in the image often deviate from the center point, and its offset on the image must be converted into the actual spatial direction angle (horizontal angle β, pitch angle θ) before subsequent spatial inversion can be performed. Angle correction establishes a connection between the image detection results and the real space geometric relationship. Without angle correction, image recognition can only remain at the pixel level detection. Only with angle correction can the ability of spatial positioning and accurate response to fire be achieved, and it is the basic supporting technology for realizing intelligent and spatial fire monitoring systems. Inferring the spatial angle from the image pixel offset is the key factor for the present invention to achieve accurate fire point geographic location reporting. The forest fire detection method of the present invention corrects the camera's pitch angle T and horizontal angle P: corrects the fire point image position.

[0047] Please combine Figure 3 As shown, the method for correcting the fire point image position includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's pitch angle T and horizontal angle P, as well as ΔV and ΔH, the fire point image position relative to the camera's optimized pitch angle θ and horizontal angle β is calculated: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

[0048] In this embodiment, the image positioning principle adopted by the fire point image position correction function is as follows.

[0049] (1) Calculation of fire image position and acquisition of camera parameters.

[0050] Use an image recognition algorithm to identify the fireworks area and obtain the horizontal pixel offset Δx and vertical pixel offset Δy from the center of the image. Also obtain camera distance L (unit: m), horizontal angle P (unit: °), pitch angle T (unit: °), and the mapping relationship between focal length and field of view angle.

[0051] (2) Correction of the fire point sight angle.

[0052] like Figure 4 As shown, assuming that the camera's pitch angle T, horizontal angle P, horizontal field of view HFOV, and vertical field of view VFOV, the image resolution is x×y (unit: pixel), the horizontal pixel offset is Δx, and the vertical pixel offset is Δy, the horizontal angle β and pitch angle θ of the fire point relative to the camera can be calculated.

[0053] (I) Fire point image pitch angle and horizontal angle correction values ​​ΔV and ΔH: (II) The pitch angle θ and horizontal angle β of the fire point relative to the camera: θ = T + ΔV, β = P + ΔH (3) Calculate the fire location using DEM.

[0054] like Figure 5 As shown, the pitch angle θ and horizontal angle β are obtained through part (2). Figure 6 As shown, the profile line of the horizontal angle β is obtained in combination with the DEM elevation data (5KM / 10KM [according to the monitoring distance supported by the camera] profile line with the camera coordinate as the starting point), and the first intersection of the fire point line of sight and the profile line is calculated. This intersection is the smoke position. Figure 6 where a is the placement of the device, and θ is the pitch angle obtained in part 2.

[0055] The significance of the method for correcting the position of the fire point image is: 1. Improve the spatial positioning accuracy of fire points. Through angle correction, the fire point locations identified in the image can be accurately mapped to the direction lines in real space. Combined with terrain data such as DEM, high-precision geographic location inversion can be further achieved, avoiding positioning deviations caused by angle errors.

[0056] 2. Supports cross-platform, universal deployment. The angle correction logic is decoupled from the image recognition model, making it adaptable to cameras of different resolutions, fields of view, and postures, facilitating widespread deployment and promotion of the algorithm in practical systems.

[0057] 3. Build the foundation for a true 3D observation model. The angle-corrected (β, θ) provides the basis for constructing spatial observation rays for the fire point. Combined with the camera installation point coordinates, this can be used in expansion modules such as 3D inversion algorithms, spatial intersection, and binocular ranging.

[0058] 4. High engineering feasibility. This method does not rely on external positioning devices or complex models, has low computational complexity, and is easy to deploy in embedded environments such as edge computing boxes, intelligent front-ends, and drones.

[0059] S12, converting the fire point image position into the fire point geographic location in geographic space.

[0060] As mentioned above, the implementation of this step will not be described in detail.

[0061] S13, aerosol lidar 2 scans the geographical location of the fire in geographic space: the lidar transmits the signal and returns the extinction coefficient and depolarization ratio after passing through the atmosphere.

[0062] LiDAR is an active remote sensing technology that relies on the scattering effect between a laser beam and atmospheric particles. This technology emits short laser pulses into the atmosphere and receives the backscattered signal returned from the target area, thereby obtaining information on atmospheric composition, particle concentration, and distribution. In fire detection applications, LiDAR can identify smoke particles released in the early stages of a fire and rapidly respond to abnormal aerosol changes. As laser light travels through the atmosphere, it scatters from aerosol particles, primarily through Rayleigh scattering and Mie scattering. Particles in fire smoke are mostly large, primarily causing Mie scattering. This scattering has a certain intensity in the backward direction. The LiDAR receiving system captures these backscattered signals using an optical telescope and then extracts the spatial distribution of the particles through photoelectric conversion and signal processing. Based on the echo signal intensity and propagation time, the atmospheric aerosol concentration distribution at different distances can be inferred. To improve its ability to detect fire smoke, LiDAR systems use a laser source with polarization properties. The emitted linearly polarized laser light undergoes a change in polarization when it encounters non-spherical particles (such as charcoal particles and soot produced by combustion). The system incorporates a polarization separation device in the receiving path, dividing the echo light into two channels: one channel receives scattered light parallel to the original polarization direction of the laser, and the other receives the component perpendicular to it. By comparing the echo signals from these two channels, the system can calculate the depolarization ratio, a parameter that is closely related to the particle shape and composition. When signs of fire appear in the monitored area, the depolarization ratio is typically significantly higher than the background value because most particles in the smoke are non-spherical. The LiDAR system utilizes this feature, combined with spatial distribution and concentration trends, to rapidly identify and locate fire smoke. This detection method offers the advantages of non-contact, long-range, high spatial resolution, and rapid response, making it particularly suitable for scenarios such as forest fire early warning, tunnel fire monitoring, and smoke detection in industrial facilities.

[0063] like Figure 7The forest fire detection device of this embodiment first collects the extinction coefficient and depolarization ratio data of the laser radar signal after it passes through the atmosphere. The extinction coefficient reflects the energy attenuation caused by particle absorption and scattering during the propagation of laser light, and can reflect the concentration of particulate matter in the atmosphere. The depolarization ratio is used to identify the shape characteristics of particulate matter and can distinguish between natural aerosols (spherical particles such as haze and dust) and non-spherical smoke produced by combustion. The system's identification process first performs an obstruction check to determine if there are any obstructions in the laser transmission path. If obstruction is detected, the data is deemed invalid and further analysis is abandoned. If the path is unobstructed, the system then determines whether the extinction coefficient exceeds the normal atmospheric background range. If the extinction coefficient is within the normal range, it indicates normal weather conditions and no fire is occurring. If the extinction coefficient is significantly elevated, it indicates a high concentration of particulate matter in the atmosphere, and the next step is to determine the depolarization ratio. If the depolarization ratio remains outside the specified range, it may be caused by natural fog or dust, which the system excludes as an interfering factor and no fire is occurring. If the depolarization ratio is within the set range, it indicates the presence of a large number of non-spherical particles, consistent with the characteristics of fire smoke, and the system determines that fire smoke is present. The system then combines the extinction coefficient and depolarization ratio data from multiple monitoring points, using spatial distribution characteristics to infer the specific location and impact range of the fire, and triggers appropriate alarms or coordinated responses. Through this multi-parameter, multi-stage analysis process, the system not only accurately identifies fire smoke, avoiding false alarms caused by interfering factors such as natural aerosols, but also provides high-precision and high-stability fire detection capabilities.

[0064] S14, judging whether the extinction coefficient exceeds the calibration coefficient, if yes, proceed to S15, otherwise proceed to S16.

[0065] S15, judging whether the depolarization ratio is within the calibration range, if yes, proceeding to S17, otherwise proceeding to S18.

[0066] S16, determining that there is no fire at the geographical location of the fire point.

[0067] S17, geographical location of the fire alarm point.

[0068] S18, introduce manual review mechanism.

[0069] Steps S14 to S18 use a fusion detection method based on the fusion of laser radar and camera. Figure 8As shown, in the initial phase, a dual-channel camera performs a wide-area detection of the target area. A deep learning model analyzes the image for abnormal areas (such as smoke patterns and particle diffusion profiles). The algorithm uses dynamic enhancement technology to suppress haze or lighting interference, extracts texture and edge features of the target area, and maps them to a three-dimensional spatial coordinate system using camera calibration parameters to generate a probability distribution map of the target area. This phase provides spatial information for subsequent detection, narrowing the lidar's scanning range from a full-area search to high-probability areas, significantly reducing ineffective scans.

[0070] Based on the visual positioning results, the aerosol lidar initiates adaptive scanning. The system dynamically plans a scanning path based on the target area: dense scanning with small-angle steps is used within the camera's positioning area, while sparse sampling is used in peripheral areas. The lidar simultaneously collects multi-dimensional data, including extinction coefficient and depolarization ratio, to infer aerosol physical properties such as particle size distribution and mass concentration.

[0071] If smoke or flames are detected in the image and the LiDAR simultaneously measures extinction-depolarization parameters consistent with smoke characteristics, the system immediately triggers a high-confidence alarm. Using coordinate registration technology, the LiDAR polar coordinate data and image pixel coordinates are aligned to the geographic coordinate system. If only a single sensor triggers an alarm (e.g., the radar detects smoke but the image is normal), a low-confidence warning is initiated and a manual review mechanism is introduced to prevent false alarms caused by sunlight reflection or dust interference.

[0072] Example 2 See also Figure 9 , which is a schematic diagram of the structure of a forest fire detection device based on the fusion of laser radar and camera provided by the present invention. This embodiment is one implementation of the forest fire detection method in Example 1. The forest fire detection device in Example 2 demonstrates the excellent application of the forest fire detection method in Example 1. The forest fire detection device includes a dual-channel camera 1, an aerosol laser radar 2, a motorized pan / tilt head 3, an edge box 4, a bracket 5, and a plurality of rollers 6.

[0073] The electric pan-tilt platform 3 and edge box 4 are fixed to the bracket 5. The dual-channel camera 1 and aerosol lidar 2 are both fixed to the rotating axis of the electric pan-tilt platform 3, with the dual-channel camera 1 positioned above the aerosol lidar 2. Since forest fire detection devices are typically installed at a high location, such as on a fixed platform such as a watchtower, high pole, or communication tower, the pitch angles of the dual-channel camera 1 and aerosol lidar 2 are generally fixed during installation. The electric pan-tilt platform 3 can drive the dual-channel camera 1 and aerosol lidar 2 to rotate and adjust their horizontal angles. In other embodiments, the dual-channel camera and aerosol lidar can be designed to adjust their pitch angles, or the electric pan-tilt platform can be adjusted to achieve pitch adjustment. The roller 6 is mounted at the bottom of the bracket 5 and can be a self-locking roller. When mapping, the roller 6 must be adapted to the mountainous forest terrain to facilitate movement and positioning of the equipment. Of course, in other embodiments, the bracket 5 and the plurality of rollers 6 may not be provided, and the dual-channel camera 1, the aerosol lidar 2, the electric pan-tilt head 3, and the edge box 4 may all be installed on a fixed platform such as a watchtower, a high pole or a communication tower.

[0074] The dual-channel camera 1 uses thermal imaging and visible light channels to capture image data of the monitored area. Using a deep learning-based image processing algorithm, it locates the fire point and determines whether a fire has occurred. Using the dual-channel image data, the image detection algorithm determines whether a fire has occurred, while the positioning algorithm determines the location of the area to be inspected.

[0075] Aerosol LiDAR 2 uses a 532 nm or 1064 nm laser to detect aerosol particles (smoke). Key components of Aerosol LiDAR 2 include a laser transmitter and a laser receiver. The laser transmitter emits laser pulses into the monitored area. Its power and frequency can be adjusted according to monitoring requirements. By setting different emission frequencies, smoke detection can be achieved at different monitoring distances. When the laser is emitted into a smoke-prone area, the smoke interacts with the laser, generating a reflected echo that is completely opposite to the emitted laser. The laser receiver receives these reflected echoes and analyzes the echo intensity to determine the smoke concentration. The time from laser emission to receiver reception is used to calculate the distance of the smoke from the LiDAR (the time difference multiplied by the speed of light is the laser distance, and half of this time difference is the distance between the LiDAR and the smoke). Because different aerosol particles produce different LiDAR echo signals, and LiDAR has high smoke detection accuracy, it can detect smoke from fires.

[0076] The electric gimbal 3 has two functions: First, during routine patrols, it rotates the dual-channel camera and LiDAR it carries, enabling omnidirectional scanning. Second, when the dual-channel camera captures and identifies suspected fire smoke, it uses the specific smoke location determined by positioning to rotate the camera and LiDAR, adjusting the pitch angle until the laser emitted by the LiDAR can penetrate the smoke.

[0077] Edge box 4 integrates a pan / tilt rotation control system, a laser control switch, an image recognition deep learning algorithm, and a fire point identification algorithm. It controls the rotation of the motorized pan / tilt 3; processes image data captured by the dual-channel camera 1 and outputs the fire point location and fire detection results; after the image determines a fire has occurred and controls the laser radar 2 to focus on the smoke, it turns on the laser. After determining whether the smoke is fire smoke, it turns off the laser (normally, the laser is not on). Furthermore, the fire smoke identification algorithm combines the fire point detection results with the laser radar smoke data to accurately determine whether a fire has occurred.

[0078] Therefore, the forest fire detection device of the present invention is essentially a fire monitoring system integrating a dual-channel camera and an aerosol lidar. By integrating a dual-channel camera 1 (visible light and thermal infrared), the system enables intelligent patrol of the monitored area and target tracking. The motorized pan / tilt head 3 rotates periodically along a predefined patrol path, covering a wide viewing angle. When a suspected target is detected, the camera's direction and elevation are dynamically adjusted based on the recognition results, enabling multi-angle capture and precise tracking of key areas.

[0079] The Edge Box 4 captures real-time video streams from its dual-channel cameras, runs intelligent image recognition algorithms, and continuously analyzes fire points within the images. Once a suspected fire target is identified, the system immediately extracts the corresponding image coordinates and camera pose parameters, calculates the smoke's line of sight, and uses this information to rapidly rotate the gimbal to align with the smoke's location, further enhancing the accuracy of image acquisition and confirmation of the target area.

[0080] Based on this, the system then integrates an aerosol lidar. Based on the fire direction provided by image recognition, it initiates a directional scanning process to conduct high-resolution aerosol detection in the target area. The lidar transmits pulses and receives echoes, measuring the concentration, characteristics, and location of smoke within the area, assisting in determining whether it is fire smoke.

[0081] After image and LiDAR detection is complete, the system enters the multi-source information fusion phase. The fusion module combines the flame / smoke characteristics identified by the image with the aerosol parameters acquired by the radar to comprehensively determine the presence and spatial location of the fire source. If the recognition results of the two sensors are highly consistent, it is determined to be a high-confidence fire event and an alarm is automatically triggered. If the recognition results are inconsistent, a low-confidence warning state is entered. The system can then consider subsequent image changes or introduce a manual review process.

[0082] Through image-driven radar precise scanning, gimbal adaptive control and multi-source data fusion mechanism, this system significantly improves the real-time performance, spatial accuracy and environmental robustness of forest fire detection, effectively reduces the false alarm rate, and enhances the fire identification capability in all-weather and multi-terrain conditions.

[0083] The entire device can be installed on a stable bracket with self-locking rollers at the bottom. When mapping, the rollers need to adapt to the mountain and forest terrain to facilitate the movement and positioning of the equipment.

[0084] To sum up, the edge box 4 of this embodiment is used to: control the electric pan-tilt head 3 to drive the dual-channel camera 1 to rotate periodically according to the set patrol path to capture the target area; identify the image position of the fire point in the image; convert the image position of the fire point into the geographical location of the fire point in the geographic space; control the electric pan-tilt head 3 to drive the aerosol lidar 2 to scan the geographical location of the fire point in the geographic space, and the lidar transmits the signal and returns the extinction coefficient and depolarization ratio after passing through the atmosphere; judge whether the extinction coefficient exceeds the calibration coefficient, and if so, continue to judge whether the depolarization ratio is within the calibration range, otherwise it is judged that no fire has occurred at the geographical location of the fire point; when it is judged that the depolarization ratio is within the calibration range, a fire alarm is issued to the geographical location of the fire point, otherwise a manual review mechanism is introduced.

[0085] Of course, the functions of edge box 4 can also be loaded into dual-channel camera 1 via software, allowing dual-channel camera 1 to realize the functions of edge box 4 while capturing images. Similarly, the functions of edge box 4 can also be loaded into aerosol lidar 2 via software, allowing aerosol lidar 2 to realize the functions of edge box 4 while scanning and collecting data.

[0086] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0087] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A forest fire detection method based on laser radar and camera fusion, comprising the following steps: Capture the target area and identify the location of the fire point in the image; Convert the fire point image location into the fire point geographic location in geographic space; Geographical location of fire alarm point; The method is characterized in that, before the fire alarm is triggered, it also includes: Aerosol lidar scans the geographical location of fire points in geographic space: the laser radar transmits signals that pass through the atmosphere and return extinction coefficient and depolarization ratio; Determine whether the extinction coefficient exceeds the calibration coefficient, if yes, proceed to the next step, otherwise determine that there is no fire at the geographical location of the fire point; Determine whether the depolarization ratio is within the calibration range. If so, a fire alarm is triggered; otherwise, a manual review mechanism is introduced.

2. The forest fire detection method based on laser radar and camera fusion according to claim 1 is characterized in that: The forest fire detection method further comprises: Correct the fire point image position; The method for correcting the fire point image position includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's pitch angle T and horizontal angle P, as well as ΔV and ΔH, calculate the fire point image position relative to the optimized pitch angle θ and horizontal angle β of the camera: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

3. The forest fire detection method based on laser radar and camera fusion according to claim 1 is characterized in that: The method for converting the fire point image position to the fire point geographical location includes the following steps: According to the camera's pitch angle T and horizontal angle P, a spatial observation ray of the fire point image position in geographic space is constructed; Obtain the profile line of P through DEM elevation data; The first intersection of the space observation ray and the section line is taken as the geographical location of the fire point.

4. The forest fire detection method based on laser radar and camera fusion according to claim 1, characterized in that: The forest fire detection method further comprises: Confirm whether the laser transmission path of the aerosol lidar is blocked. If so, the data is deemed invalid and subsequent analysis is abandoned. Otherwise, determine whether the extinction coefficient exceeds the calibration coefficient. And / or, a dual-channel camera is used to capture images of the target area.

5. A forest fire detection device based on the fusion of laser radar and camera, characterized in that: It includes a dual-channel camera, aerosol lidar, an electric gimbal, and an edge box; Edge boxes are used for: Control the electric pan-tilt head to drive the dual-channel camera to rotate periodically along the set patrol path to capture the target area; Identify the location of the fire point in the image; Convert the fire point image location into the fire point geographic location in geographic space; Control the electric gimbal to drive the aerosol lidar to scan the geographical location of the fire in geographic space. The laser radar transmits signals that pass through the atmosphere and returns the extinction coefficient and depolarization ratio. Determine whether the extinction coefficient exceeds the calibration coefficient. If so, continue to determine whether the depolarization ratio is within the calibration range. Otherwise, determine that no fire has occurred at the geographical location of the fire point. When the depolarization ratio is judged to be within the calibration range, the geographical location of the fire point is determined by a fire alarm; otherwise, a manual review mechanism is introduced.

6. The forest fire detection device based on laser radar and camera fusion according to claim 5, characterized in that: The motorized gimbal is also used to adjust the pitch angle of the aerosol lidar until the lidar transmission signal emitted by the lidar is incident on the smoke at the geographical location of the fire; And / or, when the edge box detects the position of the fire point image, it drives the electric pan-tilt head to dynamically adjust the direction and elevation angle to achieve multi-angle re-shooting and tracking of the fire point image position.

7. The forest fire detection device based on laser radar and camera fusion according to claim 5 is characterized in that: The forest fire detection device also includes a bracket for carrying a dual-channel camera, an aerosol lidar, an electric pan-tilt head, and an edge box.

8. The forest fire detection device based on laser radar and camera fusion according to claim 5, characterized in that: Edge boxes are also used for: Correct the fire point image position; The method for correcting the fire point image position includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's horizontal angle T and pitch angle P, as well as ΔV and ΔH, calculate the fire point image position relative to the optimized horizontal angle θ and pitch angle β of the camera: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

9. The forest fire detection device based on laser radar and camera fusion according to claim 5, characterized in that: Edge boxes are also used for: According to the horizontal angle T and pitch angle P of the camera, the spatial observation ray of the fire point image position in the geographic space is constructed; Obtain the profile line of P through DEM elevation data; The first intersection of the space observation ray and the section line is taken as the geographical location of the fire point.

10. A forest fire detection method based on laser radar and camera fusion, comprising the following steps: Capture the target area and identify the location of the fire point in the image; Convert the fire point image location into the fire point geographic location in geographic space; Geographical location of fire alarm point; It is characterized by further comprising: Correct the fire point image position; The method for correcting the fire point image position includes the following steps: According to the camera's horizontal field of view HFOV, vertical field of view VFOV, image resolution x×y, horizontal pixel offset Δx, and vertical pixel offset Δy, calculate the image's pitch angle correction value ΔV and horizontal angle correction value ΔH: According to the camera's horizontal angle T and pitch angle P, as well as ΔV and ΔH, calculate the fire point image position relative to the optimized horizontal angle θ and pitch angle β of the camera: θ = T + ΔV, β = P + ΔH The corrected fire point image position is obtained according to θ and β.

Images