A fire source positioning method for fire reconnaissance unmanned aerial vehicle based on roof temperature field inversion

Abstract

This invention provides a method for fire source location by inverting the roof temperature field for fire reconnaissance drones, belonging to the field of fire source location technology. This invention uses a fire reconnaissance drone to collect visible light images and infrared temperature distribution maps of building roofs, performs radiometric correction on the infrared temperature distribution map, and compensates for roof surface emissivity and atmospheric transmission attenuation to generate a corrected, accurate roof temperature distribution map. This overcomes the limitations of smoke obstructing imaging, using the roof's heated temperature field to invert the fire source instead of relying on direct imaging through smoke, thus achieving effective location even in low smoke transmittance scenarios. Radiometric correction reduces emissivity interference, and perspective transformation generates a vertical top-down projection image and establishes a two-dimensional physical coordinate system. The spatial gradient vector of the temperature field is obtained using the central difference method, and a linear equation for the gradient direction is constructed. Gradient inversion replaces subjective visual estimation, improving coordinate accuracy.

Description

Technical Field

[0001] This invention relates to the field of building fire source location technology, specifically a method for fire source location based on roof temperature field inversion for fire reconnaissance drones. Background Technology

[0002] Large-span buildings, such as steel-structured factories and logistics warehouses, have spacious interiors, high combustible loads, and complex layouts. During a fire, they are accompanied by high temperatures, dense smoke, and the rapid spread of toxic and harmful substances, creating a harsh fire environment and making fire source location difficult. Due to their maneuverability and ability to carry various sensors, drones are increasingly being used in fire location.

[0003] However, existing drone-based fire source location methods still have the following shortcomings:

[0004] (1) In fires in large-span buildings, the high concentration of smoke generated by combustion accumulates rapidly under the ceiling, which has a strong absorption and scattering effect on visible light and infrared radiation. Existing vision-based fire source location methods are difficult to obtain effective image signals when the smoke transmittance is below the threshold, resulting in the inability to accurately identify the fire source location. (2) Although existing methods can determine high-temperature areas through infrared images, the coordinate accuracy of estimating the location of the fire source by visual observation through infrared images is limited when the building spans a large area. The measurement accuracy of infrared thermal imaging is also affected by multiple factors such as the emissivity and reflectivity of the surface of the object being measured and the ambient temperature, making it difficult to meet the requirements for accurate positioning. (3) Although the positioning method based on physical model inversion or long-term scanning modeling is relatively accurate, it is computationally complex and has a slow response, which cannot meet the needs of rapid reconnaissance in the early stage of a fire and real-time positioning of dynamic fire conditions, seriously affecting the timeliness of rescue decision-making.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method for fire source location by inverting the roof temperature field for fire reconnaissance drones, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for fire source location by inverting the roof temperature field for fire reconnaissance drones, the specific steps of which include:

[0009] S1. Collect visible light images and infrared temperature distribution maps of the building ceiling using a drone, and simultaneously collect gimbal attitude data and ceiling geometric parameters. The gimbal attitude data is used to reflect the direction and angle of the camera optical axis relative to the horizontal plane.

[0010] S2. Preprocess the visible light image to identify the ceiling boundary, and perform image registration and fusion of the visible light image and the infrared temperature distribution map to generate a fused image. The ceiling boundary is used to define the ceiling area and the non-ceiling area. The fused image is used to reflect the structural texture features of the building ceiling and the temperature distribution state of the corresponding position from the same viewpoint.

[0011] S3. Perform perspective transformation on the fused image, combine the ceiling geometric parameters to generate a vertical top-view projection image of the building ceiling, and establish a two-dimensional coordinate system on the vertical top-view projection image. The vertical top-view projection image is used to reflect the orthographic projection view of the building ceiling on the horizontal plane that integrates structural texture and temperature information.

[0012] S4. Based on the temperature gradient backtracking and thermal diffusion characteristics in the two-dimensional coordinate system, construct the fire source location inversion model and determine the coordinates of the fire source projection point.

[0013] Furthermore, the drone is equipped with an optical camera and an infrared thermal imaging camera, which are used to collect visible light images and infrared temperature distribution maps of the building ceiling, respectively. The drone's control console reads the gimbal's pitch angle, roll angle, and yaw angle, and the ceiling's geometric parameters are obtained through a laser rangefinder or photogrammetry.

[0014] Furthermore, the image registration and fusion process is as follows: the optical camera and the infrared thermal imaging camera are jointly calibrated to obtain the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix. Based on the distortion coefficient, the visible light image and the infrared temperature distribution map are distorted. Based on the extrinsic parameter matrix, the distorted infrared temperature distribution map is projected onto the visible light image imaging plane to achieve pixel spatial alignment. The infrared temperature distribution map is radiatively corrected. Fixed pattern noise is eliminated by the two-point correction method. After output, it is linearly converted to radiance. The surface radiance temperature is inverted using Planck's law. Compensation is performed by combining the emissivity of the ceiling surface and atmospheric transmission attenuation to generate the corrected ceiling true temperature distribution map. The visible light image and the ceiling true temperature distribution map are decomposed using Laplace pyramid. A weighted average fusion strategy is used for low-frequency components, and a strategy of taking the maximum absolute value is used for high-frequency components to reconstruct and generate the fused image.

[0015] Furthermore, the process of establishing a two-dimensional coordinate system is as follows: a perspective projection model is constructed based on gimbal attitude data and ceiling geometric parameters; control point pairs are selected on the fused image, the control point pairs including image pixel coordinates and corresponding ceiling physical coordinates; a perspective transformation homography matrix is ​​obtained based on the control point pairs; a perspective transformation is performed on the fused image based on the homography matrix to generate a vertical top-view projection image; and a two-dimensional physical coordinate system is established on the vertical top-view projection image based on the ceiling geometric parameters, the two-dimensional physical coordinate system having the ceiling corner point as the origin and the coordinate axes along the length and width directions, respectively.

[0016] Furthermore, a temperature threshold is set to divide the vertical top-view projection image into temperature regions, generating a binary mask image. Regions are marked based on the binary mask image, and high-temperature zones of candidate fire sources are retained. Based on the highest temperature point of the high-temperature zone of the fire source and the spatial gradient of the temperature field, the coordinates of the fire source projection point are constructed. The coordinates of the fire source projection point reflect the two-dimensional physical position of the fire source on the horizontal projection surface of the building ceiling.

[0017] Furthermore, the steps for constructing the fire source location inversion model are as follows: analyze the partial derivative of temperature with respect to the spatial step size in the physical coordinate system using the central difference method to obtain the spatial gradient vector of the temperature field, select a set of pixels with gradient magnitude greater than a preset threshold, construct the gradient direction line equation based on the gradient direction angle of the pixels, obtain the intersection points of several gradient direction lines using the least squares method, and use the coordinates of the intersection points as the coordinates of the fire source projection point.

[0018] Compared with the prior art, the beneficial effects of the present invention are:

[0019] 1. This invention uses a fire reconnaissance drone to collect visible light images and infrared temperature distribution maps of building roofs, performs radiometric correction on the infrared temperature distribution maps, and compensates for the emissivity of the roof surface and atmospheric transmission attenuation to generate a corrected true temperature distribution map of the roof. This overcomes the limitations of smoke obstructing the imaging and uses the heated temperature field of the roof to invert the fire source, rather than relying on direct imaging through the smoke. It can still obtain effective positioning in scenarios with low smoke transmittance.

[0020] 2. This invention reduces emissivity interference through radiometric correction, generates a vertical top-view projection image through perspective transformation and establishes a two-dimensional physical coordinate system, obtains the spatial gradient vector of the temperature field through the central difference method, constructs the linear equation of the gradient direction, and obtains the intersection point as the coordinates of the fire source projection point through the least squares method, thereby reducing interference from emissivity, reflectivity and other factors. The positioning is transformed from image pixels to actual physical size through the two-dimensional physical coordinate system, and gradient inversion is used to replace subjective visual estimation, thereby improving coordinate accuracy.

[0021] 3. A lightweight model based on the least squares method to obtain the intersection of gradient direction lines is used. Through perspective transformation and the establishment of a two-dimensional physical coordinate system, temperature thresholds are set to divide the region and mark the high-temperature zone of candidate fire sources. This avoids the computational burden of complex physical models and achieves a fast response from image acquisition to coordinate output, meeting the needs of real-time dynamic fire location in the early stage of a fire. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0024] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0025] Example:

[0026] Please see Figure 1 The present invention provides a technical solution:

[0027] A method for fire source location by inverting the roof temperature field for fire reconnaissance drones, the specific steps of which include:

[0028] Step 1: Collect visible light images and infrared temperature distribution maps of the building ceiling using a drone, and simultaneously collect gimbal attitude data and ceiling geometric parameters. The gimbal attitude data is used to reflect the direction and angle of the camera optical axis relative to the horizontal plane.

[0029] The drone hovers at a fixed point near the top of the building's roof, and the angle of the drone's gimbal is adjusted to select a shooting direction that avoids the side with dense smoke, ensuring that the building's roof is completely imaged in the picture and the image is clear. The drone is equipped with an optical camera and an infrared thermal imaging camera to collect visible light images and infrared temperature distribution maps of the building's roof, respectively. If it is nighttime or in a low-light environment, the drone's supplementary light source is turned on to provide auxiliary lighting to enhance the brightness and contrast of the visible light images. The drone's control console reads the gimbal's pitch angle, roll angle, and yaw angle, and the geometric parameters of the roof are obtained through a laser rangefinder or photogrammetry.

[0030] Step 2: Preprocess the visible light image to identify the ceiling boundary, and perform image registration and fusion of the visible light image and the infrared temperature distribution map to generate a fused image. The ceiling boundary is used to define the ceiling area and the non-ceiling area. The fused image is used to reflect the structural texture features of the building ceiling and the temperature distribution status of the corresponding position from the same viewpoint.

[0031] The preprocessing process includes: denoising the visible light image using Gaussian filtering or median filtering algorithms. Gaussian filtering achieves smoothing through a weighted average of pixel neighborhoods, with weights determined by a Gaussian function. Median filtering replaces the center pixel value with the median value of neighboring pixels, preserving edges while denoising. Histogram equalization or adaptive histogram equalization methods are used to stretch the image's grayscale distribution range. Histogram equalization maps pixel grayscale values ​​to cumulative distribution function values, resulting in a uniform grayscale distribution in the output image. Adaptive histogram equalization performs equalization in local image regions separately, avoiding over-enhancement caused by global processing. Gamma correction or the Retinex algorithm is used to eliminate the effects of non-uniform illumination. Gamma correction uses nonlinear transformation... Adjust the brightness. Improve details in dark areas, The input brightness value Perform exponentiation. The Retinex algorithm, based on color constancy theory, decomposes the image into illuminance and reflectance components, removing illuminance variations while preserving reflectance properties. It employs the Laplacian operator or unsharpened mask method to enhance high-frequency information at the ceiling edges. The Laplacian operator detects grayscale abrupt changes using the second derivative and superimposes the edge information back into the original image. The unsharpened mask first applies Gaussian blur to the original image to obtain a low-frequency image, then subtracts the low-frequency image from the original image to obtain high-frequency components, and finally weights and superimposes the high-frequency components onto the original image.

[0032] The image registration and fusion process is as follows: joint calibration of the optical camera and the infrared thermal imaging camera is performed to obtain the intrinsic parameter matrix, distortion coefficients and extrinsic parameter matrix. The intrinsic parameter matrix includes focal length, principal point coordinates and pixel size parameters, which are used to describe the imaging geometry of the camera itself. The distortion coefficients include radial distortion and tangential distortion parameters, which are used to correct the image distortion introduced by the lens. The extrinsic parameter matrix includes rotation matrix and translation vector, which describe the relative spatial position relationship between the optical camera and the infrared camera.

[0033] Distortion correction is performed on visible light images and infrared temperature distribution maps based on the distortion coefficients obtained from joint calibration. The distortion coefficients include radial distortion coefficients. , , and tangential distortion coefficient , Distortion correction is performed on the visible light image and the infrared temperature distribution map respectively. The correction formula is as follows: ; ;

[0034] Where (x, y) are the pixel coordinates in the distorted image. , The coordinates are corrected; based on the extrinsic parameter matrix, the distortion-corrected infrared temperature distribution map is projected onto the visible light image imaging plane to achieve pixel spatial alignment. For any pixel in the infrared temperature distribution map... Its projection onto the corresponding point in the visible light image satisfy: ,in, and These are the intrinsic parameter matrices for the optical camera and the infrared camera, respectively. As the extrinsic parameter matrix, after perspective transformation, the corresponding pixels in the visible light image and the infrared temperature distribution map have consistent spatial coordinates, achieving pixel alignment of the dual-modal images. Since the raw values ​​directly output by the infrared thermal imaging camera are digitally quantized values, and the measurement accuracy is affected by factors such as the surface emissivity of the measured object, ambient temperature, atmospheric transmission attenuation, and camera non-uniformity, radiometric correction of the infrared temperature distribution map is also necessary to reduce the degree of influence. Specifically, a two-point correction method is used to eliminate the fixed pattern noise caused by inconsistent responses of each detection unit in the infrared focal plane array, and the output is linearly converted to radiance. The mathematical model used is: ,in This represents the original output value of the detection unit in the i-th row and j-th column. The corrected output value. and These are the gain correction coefficient and the bias correction coefficient, respectively, which are calculated by acquiring image data from two reference temperature points: a high-temperature blackbody and a low-temperature blackbody. The quantized values ​​of the corrected infrared temperature distribution map are converted into true radiance values ​​using an established radiometric calibration model. The radiation temperature of the target surface is then obtained by inversion using Planck's law. The radiometric calibration model is a linear response model. In the formula, The digital quantization value of the infrared image. The corresponding radiance value, and The calibration coefficients are obtained through fitting experiments using blackbody radiation calibration at multiple temperature points, and the inversion temperature is... Given by Planck's law: In the formula, Let be Planck's constant. At the speed of light, Where λ is the Boltzmann constant and λ is the center wavelength of the infrared camera's operating band; the surface emissivity of the corrugated steel roof differs from that of an ideal blackbody, and the dense smoke from the fire causes absorption and scattering attenuation of infrared radiation transmission, requiring compensation and correction to determine the target's true radiance. With the radiance received by the camera The relationship is: ;

[0035] Atmospheric transmittance, The emissivity of the ceiling surface. For ambient radiant brightness, Atmospheric path radiance, and It can be estimated by combining atmospheric transport models with measured meteorological parameters. The corrected temperature distribution map of the ceiling can be obtained by referring to tables based on the material of the color steel plate or by on-site measurement. After compensation, the true temperature distribution map of the ceiling is obtained. The visible light image after geometric registration and the infrared temperature distribution map after radiation correction are fused pixel by pixel to generate a fused image that contains both visible light texture information and temperature value information. The visible light image and the infrared temperature distribution map are decomposed into Laplacian pyramids, decomposing each image into low-frequency components and multiple high-frequency components. First, the original image is decomposed into Gaussian pyramids. Each layer of the Gaussian pyramid is obtained by Gaussian filtering and downsampling the image of the previous layer. Then, each layer of the Gaussian pyramid is upsampled and subtracted from the image of the previous layer to obtain the Laplacian pyramid coefficients of the corresponding layer. After decomposition, the original image is represented as: , These are the coefficients at the top layer of the Gaussian pyramid (low-frequency components). Let l be the Laplace pyramid coefficient of the l-th level. The images are the original images to be fused. A weighted average fusion strategy is used for low-frequency component fusion. The low-frequency components of the visible light image retain the brightness and contrast distribution, while the low-frequency components of the infrared image reflect the heat distribution characteristics of the scene, exhibiting complementarity. The low-frequency fusion coefficient... for: ,in and These are the low-frequency coefficients of the visible light image and the infrared image, respectively. The weighting coefficients range from 0.5 to 0.7 and can be dynamically adjusted based on the degree of ceiling temperature distribution in a fire scene. The fusion of high-frequency components employs a strategy of maximizing the absolute value. High-frequency components correspond to details such as edges and textures in the images; maximizing the absolute value preserves the most significant edge features from both images. The high-frequency fusion coefficients... for: ;

[0036] in, and These are the high-frequency coefficients at pixel (x, y) in the visible light image and the infrared image, respectively. The fused low-frequency component is then superimposed on the high-frequency components of each layer using the Laplacian pyramid reconstruction process, i.e., upsampling layer by layer from the top layer and adding to the layers below to recover the final fused image. The reconstruction formula is: ;

[0037] in, For the first The result of layer reconstruction For the first The fusion coefficient of the layer, the final result This is a fused image. In the fused image, the visible light texture information clearly presents the structural features and boundary contours of the ceiling, while the infrared temperature information is presented in a color mapping overlay. Each pixel corresponds to a corrected temperature value, achieving a complete representation of the same building ceiling in both spatial location and temperature distribution dimensions.

[0038] Step 3: Perform perspective transformation on the fused image, combine the ceiling geometric parameters to generate a vertical top-view projection image of the building ceiling, and establish a two-dimensional coordinate system on the vertical top-view projection image. The vertical top-view projection image is used to reflect the orthographic projection view of the building ceiling on the horizontal plane that integrates structural texture and temperature information.

[0039] The process of establishing a two-dimensional coordinate system is as follows: A perspective projection model is constructed based on the gimbal attitude data and the roof geometry parameters. The gimbal attitude data includes the gimbal pitch angle, gimbal roll angle, and gimbal yaw angle. The pitch angle represents the angle between the camera's optical axis and the horizontal plane; the roll angle represents the camera's rotation angle around the optical axis; and the yaw angle represents the angle between the camera's orientation and true north. Simultaneously, the camera's intrinsic parameter matrix, including focal length and principal point coordinates, is read for subsequent coordinate transformations. The roof geometry parameters are obtained as follows: If the building structure design parameters are known, the roof type and its geometry parameters are directly input. For horizontal roofs, no tilt correction is required; for single-sided tilted roofs… Input the angle between the roof and the horizontal plane and the direction of tilt. For a gable roof, input the ridge line position, the tilt angles on both sides, and the ridge height. If the structural parameters are unknown, the roof tilt angle can be estimated using a laser rangefinder mounted on a UAV or a photogrammetric method based on multi-view images. The UAV collects roof images from multiple different locations, and the structure-reconstruction-motion algorithm is used to reconstruct the roof's 3D point cloud. The tilt parameters are obtained by fitting a plane. Establish the mapping relationship between the pixels of the original fused image and the 3D spatial points of the roof (roof 3D point cloud). Define three coordinate systems: an image coordinate system (u, v) with the upper left corner of the fused image as the origin and units in pixels; a camera coordinate system with the camera optical center as the origin, the Z-axis pointing along the optical axis towards the object, the X-axis pointing horizontally to the right, and the Y-axis pointing vertically downwards. A local coordinate system with the Z-axis perpendicular to the roof plane, using the plane containing the roof as a reference. For sloping roofs, the coordinate system plane is the roof itself; for horizontal projections, it is the horizontal plane, with the origin set at a corner of the roof. To unify the top-view projection and output standardized data, a separate global output reference, the world horizontal coordinate system, is added. Set as: based on the horizontal plane flat, With the axis pointing vertically upwards, the final vertical top-view projection image will be established in this horizontal coordinate system, considering the perspective projection model of the camera's pose: ;

[0040] Where K is the camera intrinsic parameter matrix and S is the scaling factor. and Let be the camera's rotation matrix and translation vector in the world coordinate system. Since the camera's attitude is determined by the gimbal angle when the UAV is hovering, ... The angles of the gimbal's pitch, roll, and yaw can be calculated using the Euler angle rotation formula. Let the height of a point on the roof be the height of the point on the roof. For a horizontal roof, For a sloping roof, the constant is... Follow Linear change.

[0041] Obtain a vertical top-view projection image of the roof on a horizontal plane, thus eliminating distortion caused by the roof's tilt, and establish a mapping from the roof's local coordinate system to the horizontal coordinate system. If the roof is horizontal, the roof plane is parallel to the horizontal plane, requiring no additional correction; the roof's local coordinate system is the same as the horizontal coordinate system, and the horizontal top view can be directly obtained from the fused image through perspective transformation. If the roof is a single-sided tilted roof, the roof is a plane with an angle of θ with the horizontal plane. The tilt direction is along one side of the roof, and a local coordinate system for the roof is established. Within the roof plane, Perpendicular to the roof plane, the coordinates of any point on the roof plane in the horizontal coordinate system are: ;

[0042] in, The rotation matrix from the roof plane to the horizontal plane is determined by the tilt angle. Determined by the tilt azimuth angle; Let be the coordinates of the origin of the local coordinate system of the roof in the world horizontal coordinate system. If the roof along If the axis is tilted, then ;

[0043] A little on the roof Corresponding horizontal coordinates This can be obtained by projecting it onto a horizontal plane, i.e., ignoring... Quantity, retain If the roof consists of two sloping surfaces, symmetrical or asymmetrical along the ridge line, for a gable roof, each sloping surface needs to be processed separately, projected onto a horizontal plane, and stitched together to form a complete horizontal top view. The position of the ridge line in the image is determined by the identified roof boundary. Combined with the input ridge line parameters, the fused image is segmented into two regions: a left slope and a right slope. A local roof coordinate system is established for each sloping surface. Let the inclination angle of the left slope be θ. The right slope inclination angle is The projection of the ridgeline onto the horizontal plane is a straight line. A point on the left slope... The coordinates projected onto the horizontal plane are: , The right slope is similar, but the direction of inclination is opposite: , ,in Measuring from the ridgeline to the other side, in practice, the two slopes can be affine transformed separately and then joined along the ridgeline. For the ridgeline itself at the intersection of the two slopes, the mapping result of either side or the average value can be taken.

[0044] Control point pairs are selected on the fused image. These control point pairs connect the image space and the physical space. Each control point pair includes the pixel coordinates of the original image and the corresponding physical coordinates of the roof. When selecting control point pairs on the original fused image, for a horizontal roof, four corner points or more feature points of the building's roof are selected; for a sloping roof, at least four points need to be selected in the roof plane. The projected coordinates of these points on the horizontal plane are known or can be calculated. The pixel coordinates of the control points... Obtain the corresponding horizontal projection physical coordinates from the fused image. The coordinates are determined as follows: For a horizontal roof, the actual dimensions (length L, width W) input directly provide the physical coordinates of the four corner points, for example, (0,0), (L,0), (0,W), (L,W); for a single-sided sloping roof, the coordinates are determined based on the actual roof dimensions and the sloping angle input. Calculate the horizontal projection dimensions. The projection length is the product of cosα and the slope length, while the projection width remains constant. This yields the physical coordinates of the corner points on the horizontal plane. For a gable roof, calculate the horizontal projection dimensions of the left and right slopes separately, and the projection widths on both sides of the ridge line. , They are respectively , The total projected width is , The sum of these values ​​determines the horizontal coordinates of each corner point. This represents the actual width of the slope on the left side of the gable roof. This represents the actual width of the right-side slope of a gable roof.

[0045] Based on at least 4 pairs of control points and To obtain the perspective transformation homography matrix, i is used to index the control point pairs, based on the following formula: ;

[0046] Where H is a 3×3 matrix with 8 degrees of freedom, used to map the homogeneous coordinates of pixels in the original fused image to projected coordinates on the horizontal plane or top-view grid coordinates, solved by the direct linear transformation method: ;

[0047] An overdetermined system of equations is constructed using many pairs of points, and the least squares method is used to solve matrix H, where, , , , , , , , These are the eight elements in matrix H.

[0048] Perspective transformation is performed on the fused image based on the homography matrix to generate a vertical top-down projection image. This process is applied to each pixel in the original fused image. Calculate its coordinates in the horizontal top view image based on matrix H. : ;

[0049] In perspective transformations in computer vision, to represent nonlinear mappings using matrix multiplication, the two-dimensional pixel coordinates need to be expanded to homogeneous coordinates. Multiplying matrix H by the homogeneous coordinates outputs a three-dimensional vector in homogeneous coordinates. , , All coordinates are homogeneous. When non-integer coordinates are obtained, bilinear interpolation or bicubic interpolation is used to determine the pixel values, and finally a new image is generated. The pixel coordinates directly correspond to the physical position of the roof on the horizontal plane (unit: meters). This image is the vertical top-view projection fusion image of the building roof.

[0050] A two-dimensional physical coordinate system is established on the vertical top-view projection image based on the ceiling's geometric parameters. The actual total length of the building ceiling on the horizontal plane is input through a human-computer interaction interface or pre-stored data. and total width For a gable roof, the total length and width after horizontal projection need to be input. A two-dimensional physical coordinate system is established with the roof corner as the origin, and the length and width directions as coordinate axes, respectively, to create an image pixel coordinate system. Simultaneously, a physical coordinate system is established: with a certain corner of the roof as the origin, and the axes along the roof length direction... The axis, along the width direction is Axis, unit is meters, physical coordinates pixel coordinates of top view image The conversion relationship between them is as follows: ;

[0051] in and These represent the width and height of the top-view projection image, respectively. The transformation relationship is used to establish a linear mapping from image space to real physical space, so that the pixels in the high-temperature area identified in subsequent steps can be directly converted into the real coordinates of the fire source on the horizontal projection surface of the roof. In some special cases, when the optical axis of the drone camera is not vertically downward, the perspective transformation naturally corrects the oblique distortion without additional processing. The gimbal attitude parameters are implicitly contained in the control point correspondence or can be directly used to construct the projection matrix. This invention does not require the drone to shoot vertically; any angle can be corrected using the above method. In cases where there is obstruction or the roof is partially invisible (e.g., completely covered by thick smoke), only the visible area can be processed during the perspective transformation. The invisible area is marked as missing in the top-view image and subsequently supplemented by temperature field interpolation or information from adjacent frames.

[0052] Step 4: Construct a fire source location inversion model based on temperature gradient backtracking and thermal diffusion characteristics in a two-dimensional coordinate system, and determine the coordinates of the fire source projection point.

[0053] The steps for constructing the fire source location inversion model are as follows: analyze the partial derivative of temperature with respect to the spatial step size in the physical coordinate system using the central difference method to obtain the spatial gradient vector of the temperature field, select a set of pixels with gradient magnitude greater than a preset threshold, construct the gradient direction line equation based on the gradient direction angle of the pixels, obtain the intersection points of several gradient direction lines using the least squares method, and use the coordinates of the intersection points as the coordinates of the fire source projection point.

[0054] A temperature threshold is set to divide the vertical top-view projection image into temperature regions, generating a binary mask image. Regions are marked based on the binary mask image, and high-temperature zones of candidate fire sources are retained. Based on the highest temperature point of the high-temperature zone of the fire source and the spatial gradient of the temperature field, the coordinates of the fire source projection point are generated. The coordinates of the fire source projection point reflect the two-dimensional physical position of the fire source on the horizontal projection surface of the building ceiling.

[0055] Based on the temperature value corresponding to each pixel in the vertical top-view projection image, a dynamic temperature threshold is set to segment high-temperature regions, generating a binary mask image; the temperature value corresponding to each pixel in the top-view projection fused image... ,in The coordinates are in the physical coordinate system, in meters, and a dynamic temperature threshold is set. : ;

[0056] in, The ambient temperature of the ceiling. The standard deviation of the temperature values ​​across the entire graph. This is an empirical coefficient, ranging from 2 to 5, applied to temperatures higher than... The pixels are marked as potential fire source areas, resulting in a binary mask image. : ;

[0057] The binary mask image is labeled with eight-neighbor connected components, and adjacent pixels labeled as 1 are grouped into the same connected component. Each connected component is calculated. area Maximum temperature and the coordinates of the set center 'i' is used to index connected regions, each corresponding to an independent high-temperature region. Based on prior physical knowledge of fires in large-span buildings, regions that are too small or have isolated abnormal temperatures are eliminated, as these regions may be caused by heat from electrical equipment, sunlight reflection, or sensor noise. Regions with reasonable area and temperature are retained as candidate high-temperature fire sources.

[0058] Calculate the spatial gradient of the temperature field for each pixel in the top-view projected fused image within the high-temperature zone of the candidate fire source, and the gradient vector. Defined as: ;

[0059] The partial derivatives are approximated using the central difference method: ;

[0060] , Let be the spatial step size in the physical coordinate system. According to heat conduction theory, under steady-state or quasi-steady-state conditions, the gradient vector of the ceiling temperature field is... The direction points to the direction of the fastest temperature increase, which is the direction of the fire source projection point. For a single fire source scenario, the isotherms of the temperature field are approximately concentric circles centered on the fire source projection point. The gradient direction lines all intersect at the fire source projection point, and the fire source projection position can be determined by the intersection of the gradient direction lines.

[0061] For each candidate ignition source high-temperature zone its highest temperature point As an initial estimate of the fire source projection point, that is: ;

[0062] , Let x and y be the initial estimated x and y coordinates of the projection point of the fire source corresponding to the high-temperature zone of the i-th candidate fire source, and let y be the independent variable. The value range is the i-th candidate high temperature region All pixels within, It is a pixel. The corresponding actual ceiling temperature value. This formula is used to determine the initial fire source projection point for each candidate fire source high-temperature zone: in the i-th region. Inside, traverse all pixels. Find the temperature value The coordinates of the pixel that reaches the maximum value are used as the initial estimate for the i-th fire source projection point. .

[0063] To improve positioning accuracy, the initial estimate is corrected using temperature field gradient information in the region. Select a set P of pixels with large gradient magnitudes within and around the area: ;

[0064] in The preset gradient magnitude threshold is used to filter points with significant temperature changes, and j is used to index the pixels in the set P. Let x and y be the x and y coordinates of the j-th pixel in the physical coordinate system. Let be the magnitude of the temperature field gradient at the j-th pixel, i.e., the intensity of the rate of temperature change. For each point... Its gradient direction angle is Draw a straight line through this point along the gradient direction. The equation of the line is: ;

[0065] If there are multiple high-temperature areas, the above positioning process is performed on each area to obtain the coordinates of multiple fire source projection points, which correspond to different fire source locations. Each fire source is calculated independently and does not interfere with each other.

[0066] Project the horizontal coordinates of the fire source Using the physical coordinate system established in step 3 as the reference, with the origin at a corner of the ceiling, the length direction as the X-axis, and the width direction as the Y-axis, in meters, the fire source coordinates are output in numerical form to the fire command terminal, and the fire source projection position is marked with a prominent symbol on the top-view projection fusion image. For multi-fire source scenarios, multiple coordinates are output separately and marked on the image respectively.

[0067] After determining the coordinates of the fire source projection point, the confidence score of the positioning result is calculated based on the area of ​​the high-temperature zone of the candidate fire source, temperature contrast, and gradient direction consistency index. When the confidence level falls below a threshold (e.g., 0.6), a warning is issued and it is recommended to recollect data or use other auxiliary positioning methods.

[0068] After determining the coordinates of the fire source projection point, the reliability of the positioning result needs to be quantitatively evaluated. Three core indicators for analyzing the high-temperature zone of the candidate fire source are extracted. First is the area score, which calculates the pixel area of ​​the high-temperature zone and maps it to physical dimensions. If the area is too small (e.g., less than 0.5 square meters, possibly an electrical hotspot) or too large (e.g., covering the entire ceiling, losing its positioning significance), the score for this item will be lowered; if it falls within the typical fire source size range, it receives full marks. Second is the temperature contrast score, which calculates the difference between the highest temperature in the high-temperature zone and the average temperature of the ceiling background. The larger the temperature difference, the stronger the heat source energy, and the higher the positioning reliability. If the temperature difference is weak, close to ambient temperature fluctuations, the score will be lower. Finally, there is the gradient direction consistency score, which analyzes the temperature gradient direction (i.e., heat flow direction) of all pixels within the high-temperature zone. Theoretically, all gradient lines should converge at the fire source projection point. If the gradient directions of each point are highly convergent (small standard deviation), it indicates a clear heat source point, and this item receives a high score; if the gradient directions are chaotic, it indicates possible random hot spots or noise, and this item receives a low score. The scores of the above three items are weighted and summed according to preset weights. The weights are set manually based on historical data patterns. The output confidence score is between 0 and 1. If the confidence score is lower than 0.6, the positioning is deemed unreliable. Visual and audible alarms are immediately sent to the fire command terminal, prompting the operator to control the drone to adjust its posture or switch to a backup observation point to re-collect data. If necessary, other reconnaissance methods are used for cross-verification.

[0069] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0070] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0072] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for fire source positioning by inversion of ceiling temperature field for fire reconnaissance unmanned aerial vehicle, characterized in that, The specific steps include: S1. Collect visible light images and infrared temperature distribution maps of the building ceiling using a drone, and simultaneously collect gimbal attitude data and ceiling geometric parameters. The gimbal attitude data is used to reflect the direction and angle of the camera optical axis relative to the horizontal plane. S2. Preprocess the visible light image to identify the ceiling boundary, and perform image registration and fusion of the visible light image and the infrared temperature distribution map to generate a fused image. The ceiling boundary is used to define the ceiling area and the non-ceiling area. The fused image is used to reflect the structural texture features of the building ceiling and the temperature distribution state of the corresponding position from the same viewpoint. S3. Perform perspective transformation on the fused image, combine the ceiling geometric parameters to generate a vertical top-view projection image of the building ceiling, and establish a two-dimensional coordinate system on the vertical top-view projection image. The vertical top-view projection image is used to reflect the orthographic projection view of the building ceiling on the horizontal plane that integrates structural texture and temperature information. S4. Based on the temperature gradient backtracking and thermal diffusion characteristics in the two-dimensional coordinate system, construct the fire source location inversion model and determine the coordinates of the fire source projection point.

2. The method for locating fire sources by inverting the roof temperature field for fire reconnaissance UAVs according to claim 1, characterized in that: The drone is equipped with an optical camera and an infrared thermal imaging camera, which are used to collect visible light images and infrared temperature distribution maps of the building ceiling, respectively. The drone's control console reads the gimbal's pitch angle, roll angle, and yaw angle, and the ceiling's geometric parameters are obtained through a laser rangefinder or photogrammetry.

3. The method for locating fire sources by inverting the roof temperature field for fire reconnaissance UAVs according to claim 2, characterized in that: The image registration and fusion process is as follows: joint calibration of the optical camera and the infrared thermal imaging camera to obtain the intrinsic parameter matrix, distortion coefficients, and extrinsic parameter matrix; distortion correction of the visible light image and the infrared temperature distribution map; projection of the distortion-corrected infrared temperature distribution map onto the imaging plane of the visible light image; radiometric correction of the infrared temperature distribution map; elimination of fixed pattern noise using the two-point correction method; linear conversion of the corrected digital quantization value into radiance; inversion of the surface radiance temperature using Planck's law; compensation based on the emissivity of the ceiling surface and atmospheric transmission attenuation; generation of the corrected true ceiling temperature distribution map; Laplace pyramid decomposition of the visible light image and the true ceiling temperature distribution map; weighted average fusion strategy for low-frequency components; and absolute value maximization strategy for high-frequency components; reconstructing and generating the fused image.

4. The method for locating fire sources by inverting the roof temperature field for fire reconnaissance UAVs according to claim 3, characterized in that: The process of establishing a two-dimensional coordinate system is as follows: a perspective projection model is constructed based on gimbal attitude data and ceiling geometric parameters. Control point pairs are selected on the fused image. The control point pairs include image pixel coordinates and corresponding ceiling physical coordinates. A perspective transformation homography matrix is ​​obtained based on the control point pairs. A perspective transformation is performed on the fused image based on the homography matrix to generate a vertical top-view projection image. A two-dimensional physical coordinate system is established on the vertical top-view projection image based on the ceiling geometric parameters. The two-dimensional physical coordinate system has the ceiling corner point as the origin and the length and width directions as coordinate axes, respectively.

5. The method for locating fire sources by inverting the roof temperature field for fire reconnaissance UAVs according to claim 4, characterized in that: A temperature threshold is set to divide the vertical top-view projection image into temperature regions, generating a binary mask image. Regions are marked based on the binary mask image, and high-temperature zones of candidate fire sources are retained. Based on the highest temperature point of the high-temperature zone of the fire source and the spatial gradient of the temperature field, the coordinates of the fire source projection point are generated. The coordinates of the fire source projection point reflect the two-dimensional physical position of the fire source on the horizontal projection surface of the building ceiling.

6. A method for locating fire sources by inverting the roof temperature field for fire reconnaissance UAVs according to claim 5, characterized in that: The steps for constructing the fire source location inversion model are as follows: analyze the partial derivative of temperature using the central difference method to obtain the spatial gradient vector of the temperature field, select a set of pixels with gradient magnitude greater than a preset threshold, construct the gradient direction line equation based on the gradient direction angle of the pixels, obtain the intersection points of several gradient direction lines using the least squares method, and use the coordinates of the intersection points as the coordinates of the fire source projection point.

Images