A method for identifying fire smoke by fusing temperature data and YOLO target detection

Abstract

This invention discloses a fire and smoke recognition method that integrates temperature data and YOLO target detection, belonging to the field of intelligent environmental monitoring technology. This invention creatively constructs a multi-channel YOLO input by accurately registering and fusing visible light and infrared temperature channels. This not only effectively eliminates the interference of photothermal artifacts in complex backgrounds but also significantly improves the robustness of fire and smoke boundary detection. Furthermore, by introducing a dark channel prior step in the detection box processing stage, it overcomes the limitation of traditional visual networks that can only perform qualitative box selection, achieving real-time calculation of dynamic physical concentration of smoke. This method deeply correlates the multi-dimensional fire attributes calculated at the front end with the back-end UAV, directly outputting 3D geographic positioning through coordinate transformation. It achieves full-chain automation from qualitative perception and quantitative measurement to spatial mapping, providing technical support with significant practical value for smart environmental protection.

Description

Technical Field

[0001] This invention belongs to the field of environmental intelligent monitoring technology, and specifically relates to a method for fire point and smoke recognition that integrates temperature data and YOLO target detection. Background Technology

[0002] With increasing global climate change and human activity, forest fires and air pollution pose serious threats to ecological security, making environmental monitoring and early warning crucial for disaster prevention. Traditional manual inspections or conventional visual detection technologies often fall short in complex field environments. Therefore, modern smart fire protection and environmental monitoring are developing towards multi-source data perception and multi-dimensional attribute analysis. However, existing intelligent monitoring methods typically rely solely on visible light or single infrared thermal imaging, making them highly susceptible to interference from complex lighting, water reflections, or false targets caused by high environmental temperatures. This not only makes it difficult to accurately distinguish between real fires and smoke but also often leads to extremely high false alarm and false alarm rates. Furthermore, existing target detection algorithms typically only provide qualitative two-dimensional bounding boxes at the image level. Moreover, traditional recognition systems are mostly isolated within the camera's pixel coordinate system, and after acquiring fire images, they cannot directly convert potential hazards into precise geographic latitude and longitude coordinates, limiting the actual effectiveness of environmental emergency response. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a method for fire point and smoke identification that integrates temperature data with YOLO target detection, thereby solving the aforementioned technical problems.

[0004] A method for fire point smoke recognition that integrates temperature data and YOLO target detection includes the following steps:

[0005] S1: Acquire the visible light image and the corresponding infrared temperature data synchronously collected by the UAV, and preprocess the visible light image and infrared temperature data.

[0006] S2: The preprocessed infrared temperature data is used as an independent temperature channel and fused with the R, G and B channels of the visible light image to construct multi-channel input data;

[0007] S3: Input the multi-channel input data into the pre-trained YOLO target detection model and output the detected fire candidate boxes and smoke candidate boxes;

[0008] S4: For the image region corresponding to the smoke candidate box, calculate the atmospheric transmittance of the image region, and deduce the smoke concentration in the smoke candidate box through the relationship between the atmospheric transmittance and absorbance;

[0009] S5: Convert the pixel coordinates of the fire candidate box and the smoke candidate box to the geographic coordinate system, and output the precise location of the fire target and the smoke area. The smoke area has a smoke concentration attribute.

[0010] Preferably, in step S1, the preprocessing specifically refers to: geometrically registering the visible light image with the infrared temperature data.

[0011] Preferably, step S2 includes the following steps:

[0012] The infrared temperature data after geometric registration are normalized.

[0013] The normalized infrared temperature data is used as the T channel and is then stitched together with the R, G, and B channels of the visible light image to form a multi-channel tensor format.

[0014] Preferably, step S3, when the YOLO object detection model is pre-trained, specifically includes the following steps:

[0015] Based on historical visible light images and corresponding historical infrared temperature data, perform the same operations as steps S1 and S2 to construct multi-channel sample data;

[0016] The multi-channel sample data is classified and labeled to construct a labeled dataset, and the labeled dataset is divided into a training set and a validation set.

[0017] The first convolutional layer of the initial YOLO object detection model was modified to adapt to the input structure of multi-channel data;

[0018] The modified initial YOLO object detection model is trained using the training set, and the model is evaluated based on the validation set to obtain a pre-trained YOLO object detection model.

[0019] Preferably, step S3, when outputting the detected fire point candidate box and smoke candidate box, specifically includes the following steps:

[0020] Obtain the accuracy and recall of the YOLO object detection model output at different confidence thresholds, and plot the PR curve;

[0021] The confidence level corresponding to the inflection point of the PR curve is extracted as the optimal confidence threshold;

[0022] When the YOLO object detection model outputs detection results, it filters the object detection boxes based on the optimal confidence threshold to complete the final selection of fire point candidate boxes and smoke candidate boxes.

[0023] Preferably, in step S4, the calculation of the atmospheric transmittance of the image region includes the following steps:

[0024] Extract the dark channel value of each pixel in the image region;

[0025] Based on the dark channel values, a dark channel feature map of the image region is obtained;

[0026] Extract the top N% of smoky pixels with the highest brightness values ​​in the dark channel feature map, where 0 < N ≤ 1;

[0027] The maximum pixel value corresponding to the smoke-containing pixel in the visible light image is used as the estimated atmospheric light value;

[0028] The atmospheric transmittance of the image region is calculated based on the atmospheric light value.

[0029] Preferably, the calculation formula for the dark channel is as follows:

[0030]

[0031] in, For pixels The dark channel value, In pixels The local window area centered on the center The local window region Traverse the pixels within, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

[0032] Preferably, the atmospheric transmittance The specific calculation formula is as follows:

[0033]

[0034] in, Atmospheric light value, The defogging retention coefficient, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

[0035] Preferably, in step S4, when deriving the smoke concentration within the smoke candidate frame based on the relationship between atmospheric transmittance and absorbance, the specific steps include:

[0036] Absorbance is calculated based on atmospheric transmittance. Calculated based on the following formula:

[0037]

[0038] The smoke concentration within the smoke candidate frame is estimated based on the obtained absorbance. Calculated based on the following formula:

[0039]

[0040] In the formula, The absorption coefficient of the smoke. The optical path length for light to penetrate smoke. Atmospheric transmittance.

[0041] Preferably, in step S5, when the pixel coordinates of the fire candidate box and the smoke candidate box are converted to the geographic coordinate system, the following steps are specifically included:

[0042] The position and attitude data of the drone during filming are acquired and used as filming parameters;

[0043] Retrieve pre-stored ground elevation data;

[0044] After integrating the shooting parameters with the ground elevation data, the pixel coordinates are converted into geographic latitude and longitude coordinates through a coordinate transformation algorithm.

[0045] The beneficial effects of this invention are as follows: This invention creatively constructs a multi-channel YOLO input by accurately registering and fusing visible light and infrared temperature channels. This not only effectively eliminates the interference of photothermal artifacts in complex backgrounds but also significantly improves the robustness of fire point and smoke boundary detection. Furthermore, by introducing a dark channel prior step in the detection box processing stage, it overcomes the limitation of traditional visual networks that can only perform qualitative box selection, achieving real-time calculation of dynamic physical concentration of smoke. This method deeply correlates the multi-dimensional fire attributes calculated at the front end with the back-end UAV, directly outputting geographic 3D positioning through coordinate transformation. This achieves full-chain automation from qualitative perception and quantitative measurement to spatial mapping, providing technical support with significant practical value for smart environmental protection. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of a method for fire point and smoke recognition that integrates temperature data and YOLO target detection provided by the present invention;

[0048] Figure 2 The original image is for the fire point and smoke recognition method that integrates temperature data and YOLO target detection provided by this invention.

[0049] Figure 3 The image shows the effect of the fire candidate box and smoke candidate box in the fire and smoke recognition method that integrates temperature data and YOLO target detection provided by the present invention.

[0050] Figure 4 Dark channel feature map of a fire point and smoke recognition method that integrates temperature data and YOLO target detection provided by the present invention;

[0051] Figure 5 A schematic diagram of atmospheric transmittance for a fire point smoke identification method that integrates temperature data and YOLO target detection, provided by the present invention.

[0052] Figure 6 A schematic diagram of smoke concentration for a fire point smoke identification method that integrates temperature data and YOLO target detection provided by the present invention;

[0053] Figure 7 This is a schematic diagram of the PR curve of a fire point and smoke recognition method that integrates temperature data and YOLO target detection provided by the present invention. Detailed Implementation

[0054] The following disclosure provides many different embodiments or examples for implementing various embodiments of the invention. To simplify the disclosure, specific embodiments are described below. Of course, these are merely examples and are not intended to limit the scope of the invention.

[0055] The embodiments of the invention will now be described in detail with reference to the accompanying drawings.

[0056] like Figure 1 As shown, a method for fire point smoke recognition that integrates temperature data and YOLO target detection includes the following steps:

[0057] S1: Acquire the visible light image and the corresponding infrared temperature data synchronously collected by the UAV, and preprocess the visible light image and infrared temperature data.

[0058] S2: The preprocessed infrared temperature data is used as an independent temperature channel and fused with the R, G and B channels of the visible light image to construct multi-channel input data;

[0059] S3: Input the multi-channel input data into the pre-trained YOLO target detection model and output the detected fire candidate boxes and smoke candidate boxes;

[0060] S4: For the image region corresponding to the smoke candidate box, calculate the atmospheric transmittance of the image region, and deduce the smoke concentration in the smoke candidate box through the relationship between the atmospheric transmittance and absorbance;

[0061] S5: Convert the pixel coordinates of the fire candidate box and the smoke candidate box to the geographic coordinate system, and output the precise location of the fire target and the smoke area. The smoke area has a smoke concentration attribute.

[0062] In the specific implementation process, pixel-level spatial geometric registration is first performed on the two sets of heterogeneous data to eliminate sensor parallax. The two sets of heterogeneous data are visible light image and infrared temperature data. The preprocessed infrared data is then converted into an independent temperature feature map, which is superimposed with the RGB channels to construct multi-channel input data, and input into a YOLO model adapted to this data dimension. The design principle of this data fusion is to utilize the physical complementarity between the optical morphology and thermal radiation properties of the target, and to strictly filter high-altitude clouds or water vapor that are visually highly similar to smoke but lack heat source support by using temperature gradient features, thereby outputting candidate boxes for fire points and smoke with high accuracy.

[0063] Subsequently, for the smoke candidate box image region identified by the YOLO object detection model, the dark channel minimum of local pixels in the region is extracted to estimate the atmospheric light value. Based on this, the atmospheric transmittance, which characterizes the ability of light to penetrate the medium, is calculated. Then, according to the attenuation law of light propagating in suspended particulate matter, the atmospheric transmittance is logarithmically converted into absorbance. Finally, the smoke concentration in the smoke candidate box is derived by combining the Beer-Lambert law.

[0064] Finally, the attitude and position data synchronously recorded by the UAV's onboard terminal are extracted. Based on the collinearity equation and digital elevation model, the pixel coordinates of the fire point candidate box and the smoke candidate box with smoke concentration attribute are reverse-calculated to the geographic coordinate system to generate accurate latitude and longitude positioning.

[0065] Compared to existing technologies that rely on single visible light recognition or manual visual inspection, this overall invention not only fundamentally avoids the high false alarm rate caused by complex environmental backgrounds, but also breaks through the technical limitations of traditional computer vision, which can only output two-dimensional qualitative bounding boxes. It realizes a closed loop from target capture, physical concentration inversion to real spatial geographic mapping with a complete automated process, providing indispensable multi-dimensional data support for quantitative assessment of pollution sources and precise fire fighting.

[0066] More specifically, in step S1, the preprocessing specifically refers to: geometrically registering the visible light image with the infrared temperature data.

[0067] In one specific implementation, an infrared camera and a visible light camera are fixedly mounted on the UAV flight platform, with their optical axes parallel and their relative positions fixed. A checkerboard calibration plate or a blackbody array calibration field is used to simultaneously acquire infrared and visible light images. The coordinates of corresponding corner points in the two images are extracted, and the homography transformation matrix H is calculated so that after applying transformation H to the infrared image, its pixel coordinates correspond one-to-one with the coordinates of the visible light image.

[0068] In another implementation, a scale-invariant feature transformation algorithm is used to detect key points and generate feature descriptors in both visible light images and infrared temperature data. Based on feature matching, pairs of corresponding feature points are extracted. The affine transformation matrix is ​​then solved using these matching point pairs. Finally, this affine transformation matrix is ​​applied to perform spatial resampling and interpolation operations on the infrared temperature data, achieving pixel-level spatial alignment between its coordinate system and the visible light image. The principle behind this design is that the imaging mechanisms and fields of view of visible light spectra and infrared thermal radiation have inherent physical differences. A unified two-dimensional mesh reference system must be constructed using a rigorous geometric mapping model to ensure that heterogeneous data can be seamlessly stitched together into a spatially semantically matched tensor in the channel dimension.

[0069] In specific field monitoring examples, when drones simultaneously collect images containing fire and smoke, feature point matching and affine resampling are performed by extracting the texture contours of ground features in the visible light image and the edges of thermal radiation distribution in the infrared image as registration benchmarks, so that the heat core array in the infrared data and the flame pixels in the visible light image completely overlap.

[0070] Compared to existing technologies that rely on joint judgment without strict alignment or depend on a single vision, this implementation completely eliminates feature offset errors caused by hardware binocular parallax, effectively eliminating interference from bright reflective artifacts in complex backgrounds and hot, fireless terrain. It provides the target detection network with an absolutely reliable multimodal underlying data source for positional relationships, thereby enabling the overall invention to not only significantly reduce the false alarm and false negative rates of fire monitoring through precisely fused four-channel data, but also ensuring the absolute accuracy of subsequent calculations of atmospheric transmittance and inversion of smoke concentration spatial physical boundaries based on dark channel extraction from the detection frame.

[0071] More specifically, in step S2, the fusion process specifically includes the following steps:

[0072] The infrared temperature data after geometric registration are normalized.

[0073] The normalized infrared temperature data is used as the T channel and is then stitched together with the R, G, and B channels of the visible light image to form a multi-channel tensor format.

[0074] In the specific implementation process, after obtaining the infrared temperature matrix with geometrically aligned coordinates, a mapping algorithm is first used to scale the extremely large absolute physical temperature values ​​so that they fall into the standard dimensionless range, thus completing the data normalization process. Subsequently, this normalized infrared temperature matrix is ​​directly defined as a newly added T channel, and in the channel dimension, it is concatenated with the R, G, and B channels of the visible light image to form a multi-channel tensor format, which serves as the basic input data for the YOLO object detection model. The principle of this data fusion design is that there is a significant difference in data scale between the actual temperature magnitude of the physical interface and the grayscale range of visible light pixels. If the input is directly mixed without processing, it is very easy for the neural network to produce numerical oscillations or non-convergence during the gradient descent process.

[0075] By unifying scale and cascading channel-level matrices, the convolutional kernels at the bottom layer of the YOLO target detection model can simultaneously extract the optical texture features and thermal radiation energy distribution of the target within the same spatial receptive field. Compared to the loose approach in existing technologies that input visible light and infrared images into two separate networks for later result comparison, this early-level data fusion strategy not only fundamentally ensures the computational stability of the cross-modal feature extraction process but also fosters a strong correlation between visual representation and physical thermal properties. The overall invention, through this physically meaningful and compact tensor construction method, effectively avoids the computational losses caused by complex computational branches and exhibits extremely high feature discrimination ability and robustness when dealing with cold source interference objects that are visually highly similar to smoke in the field.

[0076] More specifically, in step S3, the pre-training of the YOLO object detection model includes the following steps:

[0077] Based on historical visible light images and corresponding historical infrared temperature data, perform the same operations as steps S1 and S2 to construct multi-channel sample data;

[0078] The multi-channel sample data is classified and labeled to construct a labeled dataset, and the labeled dataset is divided into a training set and a validation set.

[0079] The first convolutional layer of the initial YOLO object detection model was modified to adapt to the input structure of multi-channel data;

[0080] The modified initial YOLO object detection model is trained using the training set, and the model is evaluated based on the validation set to obtain a pre-trained YOLO object detection model.

[0081] In practice, the first step is to extract visible light and infrared thermal imaging dual-modal images accumulated from historical UAV inspections. Spatial geometric registration and channel stitching fusion operations are then performed to generate a multi-channel sample tensor covering both visual and temperature attributes. Subsequently, for the actual flame core and smoke-filled areas within this sample set, bounding boxes and category attributes are classified and labeled to construct a standardized labeled dataset, which is then divided into training and validation sets. Next, the channel parameters of the first convolutional layer in the initial YOLO network structure are structurally reconstructed, enabling the network's receptive field to directly process the multi-channel fused tensor. Finally, the reconstructed model is driven to perform feature learning on the training set and generalization ability evaluation on the validation set, thus outputting a pre-trained model. The principle behind this design is that traditional standard deep learning architectures default to receiving a three-channel visible light visual matrix. By directly physically expanding the input dimension of the convolutional kernels at the network's bottom layer, the neural network establishes a non-linear coupling mapping between spatial texture and infrared thermal radiation during the initial feature mapping stage, thereby enabling the model to possess cross-modal joint feature extraction capabilities.

[0082] In specific forest fire identification cases, technicians accurately selected early surface fires and associated smoke plumes from historical multi-source images. After capturing the underlying features transformed into a multi-channel structure, the model autonomously learned the spatial composite features of high-temperature attributes and specific smoke textures using a backpropagation algorithm. Compared to existing single-vision detection techniques that rely solely on color and outlines, this implementation eliminates false alarms caused by low-altitude clouds or bright reflective objects that resemble smoke. Its effectiveness lies in giving the detection algorithm the physical discrimination capability against camouflaged targets in complex environments. This allows the overall invention to not only achieve deep semantic binding and highly robust detection of multi-source heterogeneous data in the front-end target acquisition stage, but also lay a highly reliable algorithmic model foundation for subsequent physical concentration inversion and spatial geographic mapping based on accurate detection boxes.

[0083] More specifically, in step S3, when outputting the detected fire point candidate box and smoke candidate box, the following steps are included:

[0084] Obtain the accuracy and recall of the YOLO object detection model output at different confidence thresholds, and plot the PR curve;

[0085] The confidence level corresponding to the inflection point of the PR curve is extracted as the optimal confidence threshold;

[0086] When the YOLO object detection model outputs detection results, it filters the object detection boxes based on the optimal confidence threshold to complete the final selection of fire point candidate boxes and smoke candidate boxes.

[0087] In the specific implementation process, during the model evaluation phase, the accuracy and recall of the YOLO object detection model output results under different confidence threshold settings are obtained, and plotted as follows through data fitting. Figure 7 The PR curve shown is used to extract the confidence level corresponding to the inflection point of the PR curve as the optimal confidence threshold. Then, in the actual inference stage, based on this optimal confidence threshold, low-scoring filtering is applied to the original object detection boxes generated by the network, and weighted non-maximum suppression is performed to remove redundant boxes, thereby completing the process as shown. Figure 3 The final selection of candidate fire and smoke boxes is shown. The principle behind this design is that the morphology and contrast of fire and smoke in the wild fluctuate significantly. Traditional fixed-confidence truncation methods struggle to balance false negatives and false positives. The inflection point of the PR curve accurately represents the limiting equilibrium state of the model's ability to distinguish target features in probabilistic statistics, dynamically locking the optimal decision boundary for classifying positive and negative samples. Simultaneously, by adaptively optimizing the feature dataset and applying this inflection point threshold, low-probability false bounding boxes generated by water reflections or thermal airflow disturbances can be directly eliminated. Furthermore, a spatial weighting algorithm merges multiple overlapping boxes with offset positions around the same real fire source into a single high-precision bounding box. Compared to existing technologies that rely on fixed parameters based on human experience, leading to a lack of generalization ability and candidate box overflow problems, this implementation completely eliminates the blindness of subjective parameter settings. Its implementation effect is that it provides an adaptive and rigorous boundary constraint mechanism for multi-scale disaster targets, thereby enabling the candidate region output by the front-end network of the overall invention scheme to have extremely strong spatial convergence. This fundamentally ensures the purity of pixel samples when the Beer-Lambert law is subsequently introduced to invert the atmospheric transmittance and physical concentration of smoke, and provides an absolutely reliable image reference positioning for the final multi-dimensional attribute geographic coordinate inversion.

[0088] More specifically, in step S4, the calculation of atmospheric transmittance of the image region includes the following steps:

[0089] Extract the dark channel value of each pixel in the image region;

[0090] Based on the dark channel values, a dark channel feature map of the image region is obtained;

[0091] Extract the top N% of smoky pixels with the highest brightness values ​​in the dark channel feature map, where 0 < N ≤ 1;

[0092] The maximum pixel value corresponding to the smoke-containing pixel in the visible light image is used as the estimated atmospheric light value;

[0093] The atmospheric transmittance of the image region is calculated based on the atmospheric light value.

[0094] In the specific implementation process, the first step is to address issues such as... Figure 2The smoke candidate region output by the target network is shown. The dark channel value of each pixel within it is extracted to ensure that the dark channel value of each pixel in the dark channel feature map is the minimum value of all pixels in the RGB three color channels within the local image patch where the corresponding pixel in the original RGB image is located. Subsequently, in the generated... Figure 4 In the dark channel feature map shown, the set of extremely bright smoky pixels with the top N% brightness distribution is selected and used as spatial coordinate index. The maximum pixel value corresponding to the smoky pixels in the visible light image is used as the estimated atmospheric light value. Finally, the atmospheric transmittance matrix, which characterizes the ability of light to penetrate the medium, is calculated based on the atmospheric attenuation model and the atmospheric light value.

[0095] This design is based on the physical dark channel prior theory. In smoke-free areas, pixels with extremely low grayscale response typically exist in their local spatial neighborhood. Atmospheric scattering caused by suspended smoke particles significantly increases the dark channel value of the corresponding area. Therefore, the brightest part of the dark channel image objectively reflects the core area with the densest smoke and strongest optical obstruction. Mapping its coordinates back to the original color space can restore the incident atmospheric light background of the real environment to the greatest extent. In an example of early plume monitoring in forest areas, after receiving the smoke area defined by the front-end network, a dark channel matrix is ​​constructed by performing local spatial and channel minimization filtering. This accurately identifies the few core pixels in the image that are most strongly scattered by particles. Then, the atmospheric light value is determined by looking up the visible light spectrum, and the atmospheric transmittance spectrum reflecting the degree of smoke obstruction is calculated pixel by pixel. Figure 5 As shown. Compared to existing technologies that rely on subjective visual judgment or pure image contrast enhancement and lack objective physical basis, this implementation transforms qualitative image color blocks into quantitative atmospheric transmittance parameters through physical optics derivation. Its implementation effect is that it completely eliminates the interference of complex environmental lighting fluctuations on the visual characteristics of smoke, thereby enabling the overall invention to use atmospheric transmittance as the core data bridge to seamlessly connect with the Beer-Lambert law to derive the absolute dynamic smoke concentration, achieving a technological leap from shallow image visual perception to deep physical property inversion.

[0096] More specifically, the atmospheric transmittance The specific calculation formula is as follows:

[0097]

[0098] in, Atmospheric light value, The defogging retention coefficient, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

[0099] In the specific implementation process, for the smoke candidate boxes locked by the target detection network, physical-level calculation of atmospheric transmittance is performed based on dark channel prior theory. Original image pixels within local image patches are extracted and divided by the atmospheric light value of the corresponding color channel to complete channel normalization. Subsequently, minimum value filtering is performed sequentially in the local spatial window dimension and the RGB channel dimension to generate a normalized dark channel feature map. This feature map is then multiplied by a fixed dehazing retention coefficient, and finally, the atmospheric transmittance of each pixel in the image is calculated by subtracting this product term from the ideal value of complete transmission.

[0100] The physical principle behind this computational model is that the light reflectance of non-sky surfaces in nature approaches zero in at least one color channel, while the scattering effect of smoke aerosol particles on ambient light significantly increases the brightness baseline of pixels. This makes the increase in the value of the dark channel map directly reflect the degree of light attenuation due to obstruction. Atmospheric transmittance is the inverse physical derivation of this spatial scattering effect.

[0101] In a specific case study of forest fire monitoring, when a drone captured images of a nascent forest fire plume above the tree canopy, an algorithm was used to traverse the two-dimensional pixel grid within the smoke boundary box. Minimum filtering was employed to extract the spectral attenuation base, and the atmospheric transmittance distribution matrix, characterizing the ability of light to penetrate the medium, was calculated. Simultaneously, a dehazing retention coefficient was used to preserve the physical depth of field of the real scene and the thin texture of the plume edge.

[0102] Compared to existing techniques that rely solely on image grayscale thresholding or color space conversion for qualitative smoke segmentation, this implementation method abandons empirical judgments based on apparent visual features and transforms them into a rigorous mathematical model of optical scattering. It eliminates interference from sudden changes in local ambient lighting and background terrain color, generating transmittance parameters with clear physical meaning.

[0103] More specifically, the calculation formula for the dark channel is as follows:

[0104]

[0105] in, For pixels The dark channel value, In pixels The local window area centered on the center The local window region Traverse the pixels within, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

[0106] To address the problem that existing technologies struggle to directly extract aerosol physical features from conventional color images in complex field scenarios, this invention further clarifies the specific calculation method for the dark channel based on a given formula. Specifically, when calculating the dark channel value of a target pixel in an image, a local window region is constructed centered on that pixel. The minimum pixel values ​​of all pixels within this local region in the red, green, and blue color channels are extracted. Subsequently, the global minimum value in spatial distribution is determined among these channel minima and used as the final dark channel value for the central pixel.

[0107] The physical principle behind this dual minimum filtering design is as follows: Based on the dark channel prior law in atmospheric optics, local image patches in smoke-free outdoor environments typically contain pixels with extremely low reflectivity in a certain color channel. The Mie scattering effect of smoke particles generates additional atmospheric light components, significantly increasing the local minimum value in this area. Therefore, by jointly minimizing the spatial and channel dimensions, the background albedo of background objects can be effectively separated, purely extracting the scattering increment determined by the concentration of suspended particles. In actual forest fire monitoring cases, when smoke plumes appear above the tree canopy in the drone image, a fixed rectangular neighborhood sliding window is used to scan the smoke candidate area pixel by pixel. The minimum value sequence of the three primary color channels within the sliding window's coverage area is extracted sequentially, and the infimum of this sequence is calculated to construct the dark channel feature map of the entire area. This process completely eliminates the inherent color interference from underlying green vegetation or dark rocks.

[0108] Compared to traditional empirical estimation techniques that rely solely on overall image grayscale or contrast thresholds, this implementation method precisely quantifies the optical occlusion effect of pollutant particles using a rigorous mathematical model. Its effectiveness lies in providing highly robust, low-level feature data for calculating atmospheric transmittance. Furthermore, this allows the overall invention to overcome the qualitative limitation of deep learning vision algorithms, which can only output bounding boxes, by combining this high-fidelity dark channel matrix with subsequent atmospheric light value derivation and physical calculations based on the Beer-Lambert law. This lays a solid mathematical foundation for the dynamic quantitative inversion of smoke concentration at disaster emergency sites.

[0109] More specifically, in step S4, when deriving the smoke concentration within the smoke candidate frame based on the relationship between atmospheric transmittance and absorbance, the following steps are included:

[0110] Absorbance is calculated based on atmospheric transmittance. Calculated based on the following formula:

[0111]

[0112] The smoke concentration within the smoke candidate frame is estimated based on the obtained absorbance. Calculated based on the following formula:

[0113]

[0114] In the formula, The absorption coefficient of the smoke. The optical path length for light to penetrate smoke. Atmospheric transmittance.

[0115] In the specific implementation process, for the atmospheric transmittance of the smoke image region calculated in the previous step, this invention uses the physical relationship between atmospheric transmittance and absorbance to invert the smoke concentration. First, the negative logarithm of the atmospheric transmittance parameter for each pixel is taken to calculate the absorbance matrix, which characterizes the ability of the medium to absorb and scatter light. Then, this absorbance is divided by the product of the smoke absorption coefficient of the specific medium and the optical path length of light penetrating the smoke, thereby rigorously deriving the absolute smoke concentration within the smoke candidate box. In this core calculation step, regarding the spatial acquisition of the optical path parameter, spatial geometric inverse calculation is performed by extracting the UAV flight altitude and gimbal pose data in real time, combined with the coverage area of ​​the smoke pixels in the field of view, to dynamically estimate the physical depth of the light beam penetrating the smoke plume. The principle of this inversion model design is based on the Beer-Lambert law, that is, the energy attenuation of light passing through a suspended aerosol medium is exponentially related to the concentration of medium particles and the length of the optical penetration path. By performing a logarithmic mathematical transformation on the nonlinear atmospheric transmittance to construct an absorbance parameter, the visual occlusion effect originally coupled to the two-dimensional image can be decoupled into the product of the material's inherent optical properties and the spatial propagation distance. This allows for the precise extraction of pure physical concentration indicators that characterize the degree of pollution after removing spatial scale variables.

[0116] In specific cases of aerial reconnaissance of forest fires, after the UAV locks onto the initial smoke plume in the valley area, it extracts the atmospheric transmittance grid, performs logarithmic calculations to generate pixel-level absorbance, and simultaneously analyzes the flight altitude and detector pitch angle in the telemetry downlink to calculate the spatial optical path. Substituting these into the empirical absorption coefficient of typical biomass combustion, a quantified spatial distribution map of smoke concentration is quickly output, such as... Figure 6 As shown.

[0117] Compared to existing technologies that heavily rely on fixed ground-based gas sensors or subjective assessments of fire intensity by monitoring personnel, this implementation method employs non-contact optical analysis, overcoming the physical limitations of complex and dangerous terrain. Its effectiveness lies in eliminating the misleading effects of changes in drone observation distance and perspective on target density, transforming the apparent two-dimensional image into objectively meaningful three-dimensional concentration data. This allows the overall invention to overcome the limitations of traditional computer vision, which can only output qualitative alarm signals, expanding from disaster location early warning to quantitative assessment of disaster intensity. It provides an irreplaceable and precise decision-making basis for judging fire spread trends and scientifically dispatching firefighting forces.

[0118] In another implementation, to address the issue of inaccurate spatial concentration calculations caused by the assumption of a constant smoke layer thickness when using the basic Beer-Lambert law to invert smoke concentration, a sequence of consecutive video frames with timestamps and synchronized UAV flight control data are extracted during continuous UAV inspections. The spatial baseline distance between two adjacent frames is calculated based on their spatial coordinates at the time of capture. Feature point extraction and matching are performed on the two adjacent image frames to calculate the number of pixels within the smoke candidate box region. Parallax between two frames Using the principle of triangulation, combined with the camera's focal length Calculate the depth value of the viewpoint at the corresponding pixel. .

[0119]

[0120] Subsequently, the depth of background features not obscured by smoke is extracted as the background depth in the boundary region of the smoke candidate box. The depth at the densest edge of the smoke is extracted as the depth of the smoke front surface. By calculating the depth difference along the line of sight, each pixel in the image is obtained. Corresponding actual smoke penetration thickness The calculation formula is as follows:

[0121]

[0122] Finally, the atmospheric transmittance of each pixel calculated in step S4 is... The actual smoke penetration thickness calculated dynamically Substituting the modified Beer-Lambert law formula, the smoke mass concentration is calculated. The calculation formula is as follows:

[0123]

[0124] in, The mass extinction coefficient of the smoke aerosol.

[0125] During flight, the drone continuously captures images of the same area, naturally constructing a moving binocular stereoscopic vision scene. Using spatial baselines and pixel parallax, a precise 3D depth map within the field of view can be reconstructed. Given the semi-transparent and dynamically diffusing physical properties of smoke, the actual physical distance at which light penetrates smoke plumes located at different pixels varies significantly. By separating the 3D coordinates of the smoke's front surface from the background features in 3D space, the actual optical path length of light scattering and absorption in aerosols can be accurately quantified. Compared to the concentration inversion distortion caused by the coarsely preset smoke thickness to a fixed value in existing technologies, this implementation deeply fuses the drone's spatiotemporal trajectory with the image's underlying features, completely eliminating the geometric projection error caused by estimating 3D volume from a 2D plane. It can realistically present the concentration gradient distribution of smoke in the spatial depth direction, significantly improving the quantitative accuracy of irregularly diffusing smoke plumes generated by early-stage small fires. This provides absolutely reliable data support for subsequent 3D modeling of fire spread trends and accurate disaster classification.

[0126] More specifically, in step S5, when the pixel coordinates of the fire candidate box and the smoke candidate box are converted to the geographic coordinate system, the following steps are included:

[0127] The position and attitude data of the drone during filming are acquired and used as filming parameters;

[0128] The system retrieves pre-stored ground elevation data; after integrating the shooting parameters with the ground elevation data, it uses a coordinate transformation algorithm to convert the pixel coordinates into geographic latitude and longitude coordinates.

[0129] To address the limitation of conventional target detection algorithms in existing technologies, which can only output two-dimensional image pixel coordinates and cannot provide the true physical location of a disaster, the present invention implements the following specific method in step S5 for inverting the fire point and smoke candidate boxes to a geographic coordinate system: Real-time acquisition of spatial positioning data from the UAV gimbal at the moment of shooting and attitude data recorded by the inertial navigation unit are used as shooting parameters; simultaneously, the onboard DEM database is called to obtain the true terrain elevation surface of the monitoring area. Subsequently, a spatial ray is constructed based on the collinearity equation of central projection, and the shooting parameters and digital elevation data are jointly analyzed. The intersection point of the spatial ray and the terrain surface is calculated through a coordinate transformation algorithm, thereby inverting the two-dimensional pixel coordinates of the target to absolute geographic latitude and longitude coordinates. Since single-eye optical imaging is essentially a dimensionality reduction mapping from the three-dimensional real world to a two-dimensional plane, it directly leads to the loss of depth and distance information. By introducing the exterior orientation element that records the absolute spatial pose of the camera, the true physical ray penetrating the target pixel can be reconstructed in reverse; further supplemented by a digital elevation model reflecting the surface undulations as the ray truncation boundary, the true three-dimensional position of the target can be restored through rigorous geometric intersection operations.

[0130] In a specific implementation case of patrolling rugged forest areas, after the multimodal network front end accurately outputs the fire point bounding box, the spatial pose of the UAV body is extracted simultaneously, and the DEM elevation benchmark of the corresponding forest area is retrieved. The actual intersection point of the line-of-sight ray and the mountain slope is calculated through the geographic transformation model, and the accurate latitude and longitude of the fire source is directly output.

[0131] Compared to existing technologies that rely on the assumption of flat terrain for coarse mapping, resulting in severe positioning errors in mountainous areas, this implementation method completely eliminates geometric projection distortion errors caused by terrain undulations. It provides high-precision target disaster coordinate guidance without secondary calculations, thereby enabling the overall invention to construct a complete closed loop from multimodal image perception and physical attribute inversion to three-dimensional spatial coordinate positioning, significantly improving the scientific nature of field fire fighting command and the accuracy of resource allocation.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for fire point and smoke recognition that integrates temperature data with YOLO target detection, characterized in that, Includes the following steps: S1: Acquire the visible light image and the corresponding infrared temperature data synchronously collected by the UAV, and preprocess the visible light image and infrared temperature data. S2: The preprocessed infrared temperature data is used as an independent temperature channel and fused with the R, G, and B channels of the visible light image to construct multi-channel input data; S3: The multi-channel input data is input into the pre-trained YOLO target detection model to output the detected fire candidate boxes and smoke candidate boxes; S4: For the image region corresponding to the smoke candidate box, calculate the atmospheric transmittance of the image region, and deduce the smoke concentration in the smoke candidate box through the relationship between the atmospheric transmittance and absorbance; S5: Convert the pixel coordinates of the fire candidate box and the smoke candidate box to the geographic coordinate system, and output the precise location of the fire target and the smoke region, wherein the smoke region has a smoke concentration attribute.

2. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 1, characterized in that, In step S1, the preprocessing specifically refers to geometrically registering the visible light image with the infrared temperature data.

3. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 2, characterized in that, In step S2, the fusion process specifically includes the following steps: normalizing the geometrically registered infrared temperature data; using the normalized infrared temperature data as the T channel, and stitching it with the R, G, and B channels of the visible light image into a multi-channel tensor format.

4. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 1, characterized in that, In step S3, the pre-training of the YOLO object detection model specifically includes the following steps: based on historical visible light images and corresponding historical infrared temperature data, perform the same operations as in steps S1 and S2 to construct multi-channel sample data; classify and label the multi-channel sample data to construct a labeled dataset, and divide the labeled dataset into a training set and a validation set; modify the first convolutional layer of the initial YOLO object detection model to adapt to the input structure of the multi-channel data; train the modified initial YOLO object detection model using the training set, and evaluate the model based on the validation set to obtain the pre-trained YOLO object detection model.

5. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 4, characterized in that, In step S3, when outputting the detected fire candidate boxes and smoke candidate boxes, the specific steps include: obtaining the accuracy and recall of the YOLO object detection model output results under different confidence thresholds, and plotting the PR curve; extracting the confidence corresponding to the inflection point of the PR curve as the optimal confidence threshold; when the YOLO object detection model outputs the detection results, filtering the target detection boxes based on the optimal confidence threshold to complete the final screening of fire candidate boxes and smoke candidate boxes.

6. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 1, characterized in that, In step S4, the calculation of atmospheric transmittance of the image region includes the following steps: extracting the dark channel value of each pixel in the image region; obtaining a dark channel feature map of the image region based on the dark channel value; extracting the top N% of smoky pixels in the dark channel feature map, where 0 < N ≤ 1; using the maximum pixel value corresponding to the smoky pixel in the visible light image as the estimated atmospheric light value; and calculating the atmospheric transmittance of the image region based on the atmospheric light value.

7. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 6, characterized in that, The specific formula for calculating the dark channel is as follows: in, For pixels The dark channel value, In pixels The local window area centered on the center The local window region Traverse the pixels within, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

8. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 7, characterized in that, Atmospheric transmittance The specific calculation formula is as follows: in, Atmospheric light value, The defogging retention coefficient, For the traversed pixels Color channels in visible light images The pixel values ​​in the image.

9. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 8, characterized in that, In step S4, when deriving the smoke concentration within the smoke candidate frame based on the relationship between atmospheric transmittance and absorbance, the specific steps include: calculating absorbance based on atmospheric transmittance, wherein the absorbance... Calculated based on the following formula: The smoke concentration within the smoke candidate frame is estimated based on the obtained absorbance. Calculated based on the following formula: In the formula, The absorption coefficient of the smoke. The optical path length for light to penetrate smoke. Atmospheric transmittance.

10. The method for fire point and smoke recognition by fusing temperature data and YOLO target detection according to claim 1, characterized in that, In step S5, when the pixel coordinates of the fire candidate box and the smoke candidate box are converted to the geographic coordinate system, the following steps are specifically included: obtaining the position and attitude data of the UAV during shooting as shooting parameters; calling the pre-stored ground elevation data; integrating the shooting parameters and the ground elevation data, and then converting the pixel coordinates to geographic latitude and longitude coordinates through a coordinate transformation algorithm.

Images