Method and system for wide monitoring of small-area forest fire of power transmission line based on multi-source sensing fusion

Abstract

The present application provides a power transmission line small-area mountain fire unmanned aerial vehicle wide-width monitoring method and system based on multi-source sensing fusion, relating to the technical field of data processing, the method comprising: dividing the smoke edge contour line into a diffuse smoke edge section and a sharp interference edge section according to the local bending curvature; integrating the visible light transmittance along the direction perpendicular to the ground surface for the pixel area surrounded by the diffuse smoke edge section to generate a smoke column optical thickness cumulative amount; registering the smoke column optical thickness cumulative amount and the temperature rise gradient field along the path in the geographical spatial coordinate system to generate a multi-source fusion fire point feature vector; and calculating the three-dimensional spatial geographical coordinates of the small-area mountain fire point according to the multi-source fusion fire point feature vector and the real-time flight attitude parameters of the unmanned aerial vehicle. The present application reduces the false alarm risk caused by the absolute value of temperature alone, and improves the detection sensitivity of the initial small-area fire point with an area of only a few square meters.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a wide-area drone monitoring method and system for small-area wildfires along power transmission lines based on multi-source sensor fusion. Background Technology

[0002] In recent years, drones equipped with dual-light pods have been gradually applied to power transmission line inspection. Dual-light pods can simultaneously acquire infrared thermal and visible light images, using the infrared channel to detect surface temperature anomalies and the visible light channel to identify smoke characteristics, providing a technological possibility for the rapid detection and precise location of small-area wildfires. However, existing dual-light drone inspection methods still have the following technical shortcomings in practical applications: First, existing methods typically only segment images from a single frame in an infrared thermal imaging sequence using temperature thresholds, directly identifying pixel areas with temperatures exceeding a preset threshold as fire points. This method, based on static temperature values ​​from a single frame, cannot distinguish between the sustained temperature rise caused by a real wildfire and the stable high-temperature areas formed by non-fire source objects such as exposed rocks and building roofs under sunlight, resulting in a persistently high false alarm rate. Especially when there are numerous heat-absorbing objects such as metal structures and concrete pavements along power transmission line corridors, relying solely on the absolute temperature value of a single frame is insufficient to accurately identify initial fire points of only a few square meters in area amidst numerous high-temperature interferences.

[0003] Second, existing methods for utilizing smoke regions in visible light images are limited to simply determining the presence of smoke. They rely solely on detecting the presence or absence of smoke regions to assist in confirming the fire situation, without quantitatively analyzing the optical characteristics of these smoke regions. For small-area wildfires, the smoke column produced by the combustion occupies only a few pixels in the visible light image. The thinness of the smoke varies drastically with the intensity of the fire, the type of combustible material, and meteorological conditions. Relying solely on the presence or absence of smoke regions cannot quantify the severity of the current fire, nor can it effectively distinguish thin smoke from non-fire-caused aerosols such as fog and dust commonly found in forests, resulting in both missed reports and false alarms. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a wide-range monitoring method and system for small-area wildfires on power transmission lines by drones based on multi-source sensor fusion, which reduces the risk of false alarms caused by judging solely by absolute temperature values ​​and improves the detection sensitivity of initial small-area fire points with an area of ​​only a few square meters.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a wide-area drone monitoring method for small-area wildfires along power transmission lines based on multi-source sensor fusion, the method comprising: Acquire the infrared thermal imaging sequence and the visible light image sequence from the corresponding viewpoints simultaneously collected by the UAV dual-light pod as it flies along the power transmission line corridor; Along the flight path of the UAV, the partial derivatives of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence are calculated to generate a temperature rise gradient field along the flight path. Extract connected regions with gradient values ​​greater than zero from the friction temperature rise gradient field and located within the protection zone below the transmission line tower to generate initial heat source candidate patches; Extract the smoke edge contour line of the corresponding region of the initial heat source candidate patch in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; Based on the local bending curvature, the smoke edge contour line is divided into diffuse smoke edge segments and sharp interference edge segments; For the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along the direction perpendicular to the ground surface to generate the cumulative optical thickness of the smoke column. The cumulative optical thickness of the smoke column is registered with the temperature rise gradient field along the path in the geospatial coordinate system to generate a multi-source fusion fire point feature vector. Based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of small-area wildfire fire points are calculated.

[0006] Secondly, a wide-area drone monitoring device for small-area wildfires along power transmission lines, based on multi-source sensor fusion, includes: The acquisition module is used to acquire the infrared thermal imaging sequence and the visible light image sequence of the corresponding viewpoint synchronously collected by the UAV dual-light pod as it flies along the power transmission line corridor; along the UAV flight path, the module calculates the partial derivative of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence to generate a temperature rise gradient field along the flight path. The extraction module is used to extract the connected regions in the friction temperature rise gradient field where the gradient value is greater than zero and located in the protection zone below the transmission line tower, and generate initial heat source candidate patches; extract the smoke edge contour line of the corresponding region of the initial heat source candidate patches in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; The calculation module is used to divide the smoke edge contour line into diffuse smoke edge segments and sharp interference edge segments according to the local bending curvature; and to integrate the visible light transmittance along the direction perpendicular to the ground surface for the pixel area surrounded by the diffuse smoke edge segments to generate the cumulative optical thickness of the smoke column. The registration module is used to register the cumulative optical thickness of the smoke column with the temperature rise gradient field along the path in the geographic coordinate system to generate a multi-source fused fire point feature vector; based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of the small-area wildfire fire point are calculated.

[0007] Thirdly, a computing device, comprising: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0008] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0009] The above-described solution of the present invention has at least the following beneficial effects: By generating a temperature gradient field along the flight path of a drone by taking the partial derivative of the pixel temperature values ​​of adjacent frames in an infrared thermal imaging sequence, the traditional discrimination method based on the static absolute temperature value of a single frame is transformed into a dynamic discrimination method based on the inter-frame temperature change rate. The temperature gradient field reflects the instantaneous rate of temperature change along the time axis at various locations on the Earth's surface. In a real wildfire burning area, the temperature rises continuously over time, with a positive inter-frame temperature rise rate. Conversely, the temperature of static high-temperature objects such as exposed rocks and metal structures remains stable over time or changes slowly only with sunlight, with an inter-frame temperature rise rate close to zero. Therefore, the temperature gradient field effectively highlights the heating signal of a real wildfire amidst numerous static high-temperature interferences, fundamentally reducing the risk of false alarms caused by discrimination based solely on absolute temperature values, and significantly improving the detection sensitivity for small initial fire spots with an area of ​​only a few square meters.

[0010] By integrating the visible light transmittance of the pixel region surrounded by the edge of diffuse smoke along a direction perpendicular to the ground surface, the cumulative optical thickness of the smoke column is generated, upgrading the traditional qualitative assessment of the existence of smoke regions to a quantitative measurement of optical thickness. The cumulative optical thickness of the smoke column reflects the cumulative attenuation of visible light by the smoke column within the entire vertical light column from the ground surface to the sensor receiving plane. Smoke particles generated by actual combustion have a significant scattering and absorption effect on visible light, and their optical thickness is positively correlated with combustion intensity and smoke column concentration. However, the optical thickness distribution characteristics of non-flammable aerosols such as fog and dust commonly found in forests are fundamentally different from those of combustion smoke columns. Therefore, the cumulative optical thickness of the smoke column can not only quantify the density of the smoke column at pixel-level resolution to assess the intensity of the fire, but also effectively eliminate the interference of fog and dust on smoke detection by measuring the range of optical thickness values, solving the problems of existing methods that easily miss thin smoke and easily misjudge non-flammable aerosols.

[0011] By extracting the local curvature of each node of the smoke edge contour line, the smoke edge contour line is divided into diffuse smoke edge segments and sharp interference edge segments. Utilizing the physical characteristics of the real smoke edge contour line, which naturally presents a meandering and diffused shape with a low curvature under the action of air turbulence, and the essential difference in the outline geometry of interference objects with regular and sharp edges such as artificial structures and mountain ridges, the accurate segmentation of the smoke area and the non-smoke interference area is achieved, avoiding the misinclusion of building outlines, road boundaries, etc. in the scope of smoke analysis.

[0012] By registering the accumulated optical thickness of the smoke column with the temperature rise gradient field along the path in a geospatial coordinate system to generate a multi-source fused fire point feature vector, pixel-by-pixel correspondence between the dynamic temperature change information of the infrared channel and the quantitative optical information of smoke from the visible light channel is achieved under the same geospatial reference. This fusion mechanism ensures that only spatial locations that simultaneously meet the dual physical conditions of continuous temperature rise and quantitative optical thickness of the smoke column will be identified as fire points, eliminating false alarms and missed alarms caused by incomplete information from single-source sensors. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion, provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of a wide-area drone monitoring device for small-area wildfires on power transmission lines based on multi-source sensor fusion, provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of the process for generating a temperature rise gradient field along the path, provided by an embodiment of the present invention. Detailed Implementation

[0014] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0015] Please see Figure 1 and Figure 3 The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion provided in this application includes: Acquire the infrared thermal imaging sequence and the visible light image sequence from the corresponding viewpoints simultaneously collected by the UAV dual-light pod as it flies along the power transmission line corridor; Along the flight path of the UAV, the partial derivatives of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence are calculated to generate a temperature rise gradient field along the flight path. Extract connected regions with gradient values ​​greater than zero from the friction temperature rise gradient field and located within the protection zone below the transmission line tower to generate initial heat source candidate patches; Extract the smoke edge contour line of the corresponding region of the initial heat source candidate patch in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; Based on the local bending curvature, the smoke edge contour line is divided into diffuse smoke edge segments and sharp interference edge segments; For the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along the direction perpendicular to the ground surface to generate the cumulative optical thickness of the smoke column. The cumulative optical thickness of the smoke column is registered with the temperature rise gradient field along the path in the geospatial coordinate system to generate a multi-source fusion fire point feature vector. Based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of small-area wildfire fire points are calculated.

[0016] The monitoring system in this embodiment consists of five parts: an unmanned aerial vehicle (UAV) flight platform, a dual-light pod, a navigation and positioning module, an airborne data processing unit, and a ground monitoring station. The following provides a detailed description of each component.

[0017] The drone flight platform uses a six-rotor industrial-grade drone with a maximum takeoff weight of 15kg, a payload capacity of 5kg, and a single flight time of 45 minutes (with a dual-light pod). The cruising speed can be adjusted from 8m / s to 15m / s according to the inspection task requirements. The drone is equipped with autonomous flight path capability, allowing it to perform reciprocating strip scanning along the power transmission line corridor according to a preset waypoint sequence. The flight bandwidth is 80m on each side of the power transmission line centerline. The flight altitude adaptively adjusts from 60m to 120m above the ground based on the terrain undulations of the power transmission line corridor. When the terrain undulations increase, the flight altitude increases accordingly to maintain a safe distance; when the terrain is flat, the flight altitude decreases to obtain higher ground sampling resolution, ensuring that the ground sampling distance of the infrared image is no worse than 15cm / pixel.

[0018] The dual-light pod integrates an infrared thermal imaging sensor and a visible light sensor. The optical axes of the two sensors are jointly calibrated before leaving the factory, and the spatial coordinate transformation parameters (including rotation matrix and translation vector) are stored in the non-volatile memory of the pod firmware for subsequent pixel coordinate transformation between the infrared image coordinate system and the visible light image coordinate system.

[0019] The infrared thermal imaging sensor operates in the long-wave infrared band from 8μm to 14μm, employing an uncooled vanadium oxide focal plane detector with a pixel pitch of 17μm. The image resolution is 640×512 pixels, and the thermal sensitivity (noise equivalent temperature difference, NETD) is better than 50mK (at f / 1.0 and a scene temperature of 30℃). The full frame rate is 30Hz, the temperature measurement range is -20℃ to +500℃, and the temperature measurement accuracy is ±2℃ or ±2% of the reading (whichever is greater). The infrared sensor lens has a focal length of 19mm and a field of view of 32°×26°.

[0020] The visible light sensor uses a 1 / 1.8-inch CMOS image sensor with an effective pixel count of 1920×1080. Its frame rate is synchronized and locked at 30Hz with the infrared sensor via a unified pulse trigger signal from the flight control system. The lens focal length is 35mm, and the field of view is approximately 28°×21°, which is basically matched with the infrared sensor's field of view. Both sensors operate in frame synchronization mode. At the beginning of each acquisition cycle, the flight control system simultaneously sends trigger pulses to both the infrared and visible light sensors. Both sensors complete the shutter action on the same trigger edge, ensuring that the acquisition time deviation between each infrared image frame and the corresponding visible light image frame does not exceed 1ms.

[0021] The navigation and positioning module includes an RTK differential GPS receiver and an inertial measurement unit (IMU). The RTK differential GPS module receives differential correction data broadcast from a base station, achieving real-time dynamic positioning with a horizontal positioning accuracy of ±2cm and a vertical positioning accuracy of ±5cm, with a position data update frequency of 20Hz. The IMU integrates a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. After fusion using a built-in extended Kalman filter algorithm, it outputs three attitude angle data: pitch, roll, and yaw, with an attitude angle measurement accuracy of ±0.1° and a data update frequency of 100Hz. The UAV flight control system uses GPS time as a unified time reference, aligning and binding the acquisition time of each frame of infrared and visible light images with the UAV's longitude, latitude, flight altitude (elevation), pitch, roll, and yaw angles at that time, forming a flight attitude parameter record corresponding one-to-one with each frame of the image sequence.

[0022] The airborne data processing unit is an embedded industrial control computer equipped with an Intel i7 processor, 16GB of RAM, and a 512GB solid-state drive, running a Linux operating system. The data processing unit receives real-time streams of infrared thermal imaging sequences and visible light image sequences from the data output of the dual-optical pod via a gigabit Ethernet interface, and simultaneously receives time series of flight attitude parameters from the flight control system via a serial communication interface. The data processing unit executes all the processing steps described in this embodiment, including generating a temperature gradient field along the path, extracting initial heat source candidate patches, extracting smoke edge contours and calculating curvature, classifying edge segments, generating cumulative optical thickness of the smoke column, generating multi-source fusion fire point feature vectors, and calculating the three-dimensional spatial geographic coordinates of the fire point. The processing results are transmitted back to the ground monitoring station in real time via a 4G / 5G wireless communication module.

[0023] Please see below. Figure 3 Along the flight path of the UAV, partial derivatives are calculated for the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence to generate a temperature gradient field along the path. This includes: extracting the preceding and following infrared images that are temporally adjacent from the infrared thermal imaging sequence; for each pixel located at the same pixel coordinate position in the two infrared images, reading the previous temperature value recorded at that position in the preceding infrared image and the following temperature value recorded at the same position in the following infrared image; calculating the difference between the following temperature value and the preceding temperature value to obtain the inter-frame temperature change at that pixel coordinate position. The method involves: measuring the ground projection distance of the UAV along its flight path during the time interval between acquiring the previous infrared image and acquiring the next infrared image; dividing the inter-frame temperature change by the ground projection distance to obtain the temperature rise rate per unit distance along the flight path for that pixel coordinate position; iterating through all valid pixel coordinate positions in the two infrared images and calculating the temperature rise rate per unit distance for each pixel coordinate position; arranging the temperature rise rates per unit distance for all pixel coordinate positions according to their row and column order in the infrared images to form the temperature rise gradient field along the flight path.

[0024] The process of extracting the smoke edge contour line of the corresponding region of the initial heat source candidate patch in the visible light image sequence and calculating the local bending curvature of each node of the smoke edge contour line includes: transforming the pixel coordinate boundary of the initial heat source candidate patch in the infrared image coordinate system to the visible light image coordinate system according to the pre-calibrated spatial coordinate transformation parameters between the infrared sensor and the visible light sensor of the UAV dual-light pod, to obtain the area to be detected in the visible light image; extracting pixel position points whose brightness amplitude changes more than a preset brightness jump threshold along the brightness change path between adjacent pixels in the area to be detected, forming an edge candidate point set; sequentially connecting the pixel position points of the eight neighboring regions in the edge candidate point set according to the connectivity relationship to form the smoke edge contour line; for each contour node on the smoke edge contour line, taking the Mth node forward and the Mth node backward along the extension direction of the contour line, taking the current node as the vertex, the Mth node before and the Mth node after as the endpoints, determining the planar arc passing through these three points, and taking the reciprocal of the radius of the planar arc as the local bending curvature of the current contour node.

[0025] The process of dividing the smoke edge contour line into diffuse smoke edge segments and sharp interference edge segments based on the local curvature includes: comparing the local curvature of each contour node on the smoke edge contour line with a preset curvature discrimination threshold; marking contour nodes with local curvature less than or equal to the preset curvature discrimination threshold as smooth nodes, and marking contour nodes with local curvature greater than the preset curvature discrimination threshold as bent nodes; traversing the node type markings of each contour node sequentially from the start end to the end end of the contour line along its extension direction; concatenating and merging contour line segments that are consecutively adjacent along the extension direction of the contour line and whose node type markings are all smooth nodes into diffuse smoke edge segments; and concatenating and merging contour line segments that are consecutively adjacent along the extension direction of the contour line and whose node type markings are all bent nodes into sharp interference edge segments.

[0026] Specifically, for the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along a direction perpendicular to the ground surface to generate a cumulative optical thickness of the smoke column. This includes: for the pixel region in the visible light image sequence corresponding to the edge segment of the diffuse smoke, determining the vertical light column range of the pixel region in geographic space, measured from the ground plane along a direction perpendicular to the ground surface, to the visible light sensor receiving plane of the UAV dual-light pod; within the vertical light column range, advancing layer by layer along a direction perpendicular to the ground surface from the ground plane to the sensor receiving plane, increasing the height of each layer... The product of the transmittance of the corresponding pixel at the specified position and the height step of the layer is used as the local optical thickness contribution value of the layer. The local optical thickness contribution values ​​of all height layers within the vertical light column range are accumulated one by one to obtain the smoke column optical thickness value at the pixel position corresponding to the vertical light column. All pixel positions in the pixel region surrounded by the edge segment of the diffuse smoke are traversed, and the smoke column optical thickness value corresponding to each pixel position is calculated. The smoke column optical thickness values ​​of all pixel positions are arranged according to the row and column positions of each pixel in the visible light image to generate the cumulative smoke column optical thickness.

[0027] The process involves registering the accumulated optical thickness of the smoke column with the temperature rise gradient field along the path in a geospatial coordinate system to generate a multi-source fusion fire point feature vector. This includes: transforming the optical thickness value of the smoke column at each pixel location in the accumulated optical thickness of the smoke column from the visible light image pixel coordinate system to the geodetic plane geographic coordinate system based on the visible light sensor imaging model of the UAV dual-light pod and the spatial position and attitude angle of the UAV at that moment, to obtain optical thickness geographic distribution data; transforming the unit distance temperature rise rate at each pixel location in the temperature rise gradient field along the path from the infrared image pixel coordinate system to the same geodetic plane geographic coordinate system based on the infrared sensor imaging model of the UAV dual-light pod and the spatial position and attitude angle of the UAV at that moment, to obtain temperature rise gradient geographic distribution data; merging the optical thickness geographic distribution data and the smoke column optical thickness value and unit distance temperature rise rate with the same spatial location in the temperature rise gradient geographic distribution data into a two-dimensional feature data pair according to their spatial location; and arranging all two-dimensional feature data pairs in the order of geographic grid rows and columns to generate a multi-source fusion fire point feature vector.

[0028] Specifically, the calculation of the three-dimensional spatial geographic coordinates of small-area wildfire points based on the multi-source fusion fire point feature vector and the real-time flight attitude parameters of the UAV includes: selecting spatial location points from the multi-source fusion fire point feature vector where the smoke column optical thickness value exceeds a preset fire point optical judgment threshold and the unit distance temperature rise rate exceeds a preset fire point temperature rise judgment threshold, as confirmed fire point locations; and extracting the longitude coordinates, latitude coordinates, flight altitude, pitch angle, roll angle, and heading angle of the UAV at the moment of collecting the image frame corresponding to the confirmed fire point location from the real-time flight attitude parameters of the UAV. The three-dimensional spatial center position of the UAV optical sensor is determined based on the longitude coordinates, latitude coordinates, and flight altitude. The spatial pointing direction of the sensor's line of sight is determined based on the pitch angle, roll angle, and heading angle. A spatial observation ray is determined by the center position and the spatial pointing direction of the line of sight. The intersection of this spatial observation ray and the surface surface of the pre-constructed terrain digital elevation model of the area where the power transmission line corridor is located is obtained. The longitude coordinates, latitude coordinates, and elevation value of this intersection point are taken as the three-dimensional spatial geographic coordinates of the small-area wildfire hotspot.

[0029] The topographic digital elevation model (DEM) of the area where the transmission line corridor is located was pre-built before the full system deployment. The DEM data comes from 3D point cloud data obtained by airborne LiDAR scanning. After filtering, classification and rasterization, a regular raster DEM with a spatial resolution of 5m / pixel is generated, with an elevation accuracy of no less than ±2m. The storage format is GeoTIFF, and the coverage area is a strip area of ​​500m on both sides of the centerline of the transmission line corridor.

[0030] During the monitoring mission, the UAV flew back and forth along the power transmission line corridor, scanning in a strip pattern 80 meters to either side of the line's centerline. Within each scanning segment, the UAV's dual-light pod simultaneously acquired infrared thermal imaging sequences and visible light image sequences at a frame rate of 30Hz. Taking a cruising speed of 10 m / s as an example, the interval between ground sampling positions along the flight path between two adjacent infrared images was approximately 0.33 meters. This spatial sampling density was sufficient to capture pixel-level temperature changes caused by small wildfires covering only a few square meters in the infrared images. The flight control system simultaneously recorded the longitude coordinates, latitude coordinates, flight altitude, pitch angle, roll angle, and heading angle at each acquisition moment, generating a flight attitude parameter sequence corresponding to each image frame. Each frame of the infrared thermal imaging sequence is a 640×512 two-dimensional temperature value matrix. Each element in the matrix is ​​the equivalent blackbody radiation temperature of the ground object at the corresponding pixel location in the 8μm to 14μm band, in degrees Celsius. Each frame of the visible light image sequence is a 1920×1080 three-channel RGB color image.

[0031] From the infrared thermal imaging sequence, in chronological order of frame acquisition timestamps, sequentially extract the preceding (i-th) and following infrared images (i+1) on the timeline to form an adjacent frame pair. For all N frames of infrared images in the infrared thermal imaging sequence, a total of N-1 adjacent frame pairs are formed, and the subsequent sub-steps are performed one by one.

[0032] For the i-th and (i+1)-th infrared images in the current adjacent frame pair, iterate through every pixel coordinate position (x, y) that has the same coordinates in both frames. For each pixel coordinate position (x, y), read the previous temperature value from that position in the i-th infrared image; and read the next temperature value from the same coordinate position (x, y) in the (i+1)-th infrared image. Since the two infrared images maintain the same spatial resolution and field of view during acquisition, the same pixel coordinate position (x, y) corresponds to the same spatial position on the ground in both frames.

[0033] For a pixel coordinate (x, y), the difference between the next temperature value and the previous temperature value is calculated to obtain the inter-frame temperature change at that pixel coordinate. A positive inter-frame temperature change indicates that the temperature at that ground location increased during the acquisition period from frame i to frame i+1; a zero or negative inter-frame temperature change indicates that the temperature remained stable or decreased.

[0034] From the flight attitude parameter sequence recorded by the flight control system, the three-dimensional spatial position (longitude coordinates, latitude coordinates, and flight altitude) of the UAV at the time of acquiring the i-th infrared image and the three-dimensional spatial position at the time of acquiring the (i+1)-th infrared image are extracted. These two three-dimensional spatial positions are projected onto the geoid to obtain two ground projection coordinate points. The distance between these two ground projection coordinate points along the geoid is calculated as the ground projection distance denoted as Δd, representing the distance the UAV moves along its flight path during the time interval between acquiring the previous and next infrared image frames, expressed in meters. For an infrared sensor frame rate of 30Hz, the time interval between adjacent frames is 1 / 30 second. At a cruising speed of 10 m / s, the ground projection distance is approximately 0.33 m.

[0035] For a pixel coordinate position (x, y), the inter-frame temperature change ΔT(x, y) is divided by the ground projection distance Δd to obtain the unit distance temperature rise rate G(x, y) = ΔT(x, y) / Δd along the flight path at that pixel coordinate position. The physical meaning of the unit distance temperature rise rate is: the change in surface temperature at that ground position when moving one unit ground distance (1m) along the UAV's flight path, expressed in ℃ / m. In a real wildfire burning area, the surface temperature rises continuously over time, so its unit distance temperature rise rate is positive and relatively large; while the temperature of static high-temperature objects such as exposed rocks and metal structures remains relatively stable for a short period of time, so their unit distance temperature rise rate is close to zero.

[0036] Traverse all valid pixel coordinates in the infrared images of frame i and frame i+1 (excluding invalid pixels due to sensor boundary effects, etc.) and calculate the unit distance temperature rise rate G(x,y) for each pixel coordinate position. Arrange the unit distance temperature rise rates of all pixel coordinate positions in order of their row (y-coordinate from smallest to largest) and column (x-coordinate from smallest to largest) in the infrared image to form a two-dimensional numerical matrix with the same resolution as the infrared image, which is the temperature rise gradient field along the path from frame i to frame i+1. The element value located in row y and column x of this temperature rise gradient field is the unit distance temperature rise rate at image coordinate (x,y).

[0037] In this embodiment, for special cases where the ground projection distance is zero (such as when the drone is hovering), the calculation of the temperature rise gradient field along the path of adjacent frames automatically skips the adjacent frame pair and processes the next set of valid adjacent frame pairs to ensure the effectiveness of the division operation and the physical rationality of the gradient field.

[0038] After generating the temperature rise gradient field along the transmission line, connected regions satisfying two conditions are extracted from this gradient field. The first condition is that the gradient value is greater than zero, that is, the temperature rise rate per unit distance G(x,y) at the pixel coordinate location is greater than 0℃, which indicates that the temperature at this ground location has actually risen between adjacent frames. The second condition is that the ground spatial location corresponding to the pixel coordinate location is located within the protection zone below the transmission line tower. The protection zone is defined as the projection of a circular area with a radius of 50m centered at the tower base onto the ground. Pixel coordinate locations that simultaneously satisfy the above two conditions are marked as candidate fire point pixels. Eight-neighborhood connected region analysis is performed on all marked candidate fire point pixels. If two candidate fire point pixels are adjacent in the image along the horizontal, vertical, or diagonal direction, they are considered to belong to the same connected region. Each connected region is treated as an independent initial heat source candidate patch, and the pixel coordinate range of the patch in the infrared image (the coordinates of the upper left and lower right corners of the smallest bounding rectangle) is recorded. Meanwhile, for each initial heat source candidate patch, the number of candidate fire point pixels it contains is calculated: if the number of pixels is less than the preset minimum fire point area threshold (set to 9 pixels in this embodiment, corresponding to a minimum fire point area of ​​about 2m² under the ground spatial resolution of infrared images), then the patch is discarded as noise.

[0039] The following details the complete process of extracting the smoke edge contour and calculating the local curvature.

[0040] For each initial candidate heat source patch, the range of infrared image pixel coordinates (the coordinates of the top-left and bottom-right corners of the smallest bounding rectangle) recorded in the previous step is taken. Using the pre-calibrated spatial coordinate transformation parameters (rotation matrix R and translation vector T) between the UAV dual-light pod's infrared sensor and visible light sensor, the pixel coordinate boundaries in the infrared image coordinate system are transformed to the visible light image coordinate system. The specific coordinate transformation process is as follows: The infrared image pixel coordinates (uir, vir) are transformed into normalized coordinates in the infrared sensor coordinate system using the intrinsic parameter matrix of the infrared sensor. Then, based on the rotation matrix R and translation vector T between the infrared sensor coordinate system and the visible light sensor coordinate system, the normalized coordinates are transformed into the visible light sensor coordinate system. Finally, the coordinates in the visible light sensor coordinate system are projected onto the visible light image pixel coordinate system using the intrinsic parameter matrix of the visible light sensor to obtain the corresponding visible light image pixel coordinates (uvis, vvis). The above coordinate transformation is performed sequentially on the coordinates of all four boundary corner points of the initial heat source candidate patch in the infrared image. The smallest bounding rectangle region of the four transformed corner point coordinates in the visible light image is taken as the detection area in the visible light image.

[0041] Within the detection area of ​​the visible light image, edge detection based on brightness gradient is performed. Specifically: The RGB channels of the visible light image are converted into a grayscale image. For each pixel location within the detection area of ​​the grayscale image, the change in brightness value between that pixel location and its horizontally adjacent pixels and its vertically adjacent pixels is calculated. If the change in brightness amplitude between that pixel location and its adjacent pixels in the horizontal or vertical direction exceeds a preset brightness jump threshold (set to a grayscale value difference of 30 in this embodiment, i.e., the absolute value of the difference between the grayscale values ​​of adjacent pixels is greater than 30), then that pixel location is marked as an edge candidate point. After traversing all pixel locations within the detection area, an edge candidate point set consisting of all edge candidate points is obtained.

[0042] For the set of edge candidate points, perform eight-neighbor connectivity analysis. If two edge candidate points are horizontally adjacent, vertically adjacent, or diagonally adjacent (i.e., the difference between their row and column coordinates is no more than 1), then these two edge candidate points are considered spatially connected. Starting from any unvisited edge candidate point in the set, visit connected edge candidate points sequentially along the eight-neighbor connectivity path until no further expansion is possible, forming a complete contour line. Repeat the above process until all edge candidate points in the set have been visited, resulting in one or more smoke edge contour lines. For each contour line, arrange the edge candidate points along the path in the order of visit, forming an ordered sequence of contour nodes for that contour line.

[0043] For each contour node on each smoke edge contour line, the local curvature is calculated using the three-point circular arc method. Specifically, the current contour node is set to the k-th node in the ordered node sequence of the contour line (denoted as...). ), taking the Mth node as the front reference point along the direction of the contour line extending forward of the node sequence (in the direction of increasing node number). Take the Mth node as the next reference point, moving forward in the direction of decreasing node numbers from the end of the node sequence. In this embodiment, M is set to 5, meaning that 5 contour nodes before and after are used as reference points, corresponding to the local morphology within an arc length of approximately 1.65m along the contour line under conditions of a 30Hz frame rate and a ground sampling distance of 0.33m. The current node... Previous reference point and subsequent reference points Three points uniquely define a planar arc passing through those three points in the visible light image plane. Calculate the radius of this planar arc. Take the radius reciprocal As the current contour node The local bending curvature.

[0044] The physical meaning of local bending curvature is: the degree to which the contour line deviates from the straight extension direction at the contour node. The larger the curvature, the more severe the bending, and the smaller the curvature, the straighter and smoother the contour line. For the first and last nodes of the contour line with less than M reference nodes at both ends, the contour line endpoints are used as reference points for truncation.

[0045] The distinction between diffuse smoke edge segments and sharp interference edge segments includes: The local curvature of each contour node on each smoke edge contour line is compared with a preset curvature distinction threshold. A comparison is made. In this embodiment, a preset curvature discrimination threshold is used. Set as This threshold was determined through statistical analysis of 50 sets of real wildfire smoke images and 50 sets of images of edges of non-fire-related interfering objects (including building corners, ridge outlines, road boundaries, high-voltage tower structures, etc.). The basis for selecting this threshold is that, under the influence of air turbulence, the edge outline of real combustion smoke exhibits a meandering and diffused natural shape, with the local curvature of the outline nodes concentrated in... to Within a lower range; while the sharp edges of man-made structures and natural terrain have regular geometric contours, and the local curvature of their edge nodes typically exceeds [a certain value]. .Will Setting it to the boundary value that distinguishes between diffuse natural contours and sharp regular contours can effectively differentiate between the two types of edges.

[0046] For each contour node, if its local curvature is less than or equal to a preset curvature discrimination threshold... (Right now If the contour node is flat, it is marked as a smooth node; if its local curvature is greater than the preset curvature discrimination threshold, it is marked as a smooth node. (Right now If the contour node is not marked as a zigzag node, then the contour node is marked as a zigzag node. After all contour nodes have been marked, each smoke edge contour line is converted into a sequence of node type marks consisting of smooth and zigzag node marks.

[0047] Along the extension direction of each smoke edge contour line (i.e., from the starting node to the ending node), the node type labels of each contour node are traversed sequentially. During the traversal, contour segments that are consecutively adjacent along the extension direction and have the same node type label are concatenated and merged. Specifically, for consecutively adjacent smooth node segments, that is, all nodes on the contour line starting from a certain smooth node and continuing for several subsequent contour nodes until the first zigzag node is encountered, these consecutive smooth nodes and their corresponding pixel coordinate positions in the visible light image are concatenated and merged into a diffuse smoke edge segment. Similarly, for consecutively adjacent zigzag node segments, that is, all nodes on the contour line starting from a certain zigzag node and continuing for several subsequent contour nodes until the first smooth node is encountered, these consecutive zigzag nodes and their corresponding pixel coordinate positions are concatenated and merged into a sharp interference edge segment.

[0048] In this embodiment, to further eliminate transient noise interference, a minimum segment length filtering condition is set for the merged edge segments: if a diffuse smoke edge segment or a sharp interference edge segment contains fewer than 3 contour nodes (i.e., the segment length is less than the coverage of the previous reference point M=5), the edge segment is discarded and does not proceed to subsequent steps. This filtering operation can effectively eliminate the interference of isolated edge pixels caused by image noise on the classification results.

[0049] For each pixel region corresponding to the edge segment of diffuse smoke in the visible light image sequence (i.e., the closed pixel region enclosed by the edge segment of diffuse smoke in the visible light image), determine the range of the vertical light column corresponding to that pixel region in geographic space. The vertical light column is defined as: a spatial column extending upwards from the ground plane (elevation zground) along a direction perpendicular to the ground surface (i.e., along the normal direction of the geoid) to the visible light sensor receiving plane of the UAV's dual-light pod (elevation zsensor = UAV flight altitude), with each geographic point (xg, yg) on ​​the ground plane as its base center. The horizontal cross-sectional area of ​​the vertical light column is equal to the ground projection area corresponding to one pixel in the visible light image at that pixel location (approximately 0.01m at flight altitudes of 60m to 120m and a focal length of 35mm). 2 up to 0.04m 2 ).

[0050] For each vertical light column, the light is advanced layer by layer from the ground plane to the sensor receiving plane along a direction perpendicular to the ground surface, with a layer height step size Δh set to 1m. For a height position h (h starts from the ground elevation zground and increases with a step size Δh until zsensor), the visible light image pixel transmittance T(h) corresponding to that layer is obtained as follows: Based on the imaging geometry of the visible light sensor, the ground space point at height h is projected backward onto the visible light image pixel coordinate system to obtain the corresponding pixel coordinates (uh, vh); the brightness value of this pixel coordinate position in the visible light image (the weighted gray value of the RGB three channels) is read, and the ratio of this brightness value to the reference brightness value of the same ground position under smoke-free conditions (pre-extracted from adjacent smoke-free areas or historical fire-free images) is used as the visible light transmittance T(h) of that height layer at pixel coordinates (uh, vh). The transmittance T(h) ranges from [0, 1]. The closer T(h) is to 1, the thinner the smoke in that layer; the closer it is to 0, the denser the smoke in that layer. The product of the transmittance T(h) at each height position h and the height step Δh of that layer, T(h)×Δh, is taken as the local optical thickness contribution value of that layer.

[0051] The local optical thickness contributions from the ground plane (h = zground) to the sensor receiving plane (h = zsensor) within the vertical light column are summed one by one to obtain the smoke column optical thickness value at the pixel location corresponding to the vertical light column. The physical meaning of the smoke column optical thickness value is: the total attenuation of visible light by the entire smoke column that runs through the vertical light column, and its value is equal to the sum of the negative natural logarithm of the transmittance along the path of the light column. The larger the value, the denser the smoke column, the stronger the light attenuation, and the more intense the fire at that location; the smaller the value, the thinner the smoke at that location.

[0052] Traverse all pixel positions within the pixel region surrounded by the diffuse smoke edge segment and calculate the optical thickness value of each smoke column. Arrange the optical thickness values ​​of the smoke column at all pixel positions according to the row order (from top to bottom) and column order (from left to right) of each pixel in the visible light image to generate the cumulative optical thickness of the smoke column corresponding to the diffuse smoke edge segment, resulting in a two-dimensional optical thickness matrix of smoke column values ​​corresponding to the pixel region size.

[0053] The optical thickness values ​​of the smoke column corresponding to each pixel position in the cumulative optical thickness of the smoke column are transformed from the visible light image pixel coordinate system (uvis, vvis) to the geodetic geographic coordinate system (Lon, Lat) based on the visible light sensor imaging model of the UAV dual-light pod and the spatial position (longitude coordinates, latitude coordinates, flight altitude) and attitude angles (pitch angle, roll angle, yaw angle) recorded by the UAV at the time of acquisition of the visible light image frame. The transformation process adopts the collinearity condition equation, which establishes a mapping function from the visible light image pixel coordinates to the geodetic geographic coordinates based on the collinear geometric relationship between the visible light sensor projection center, the image point position corresponding to the pixel coordinates, and the corresponding ground point. For each element in the cumulative optical thickness of the smoke column matrix, its visible light image pixel coordinates are substituted into the mapping function to obtain the geodetic geographic coordinates (Lon, Lat) of the ground spatial position corresponding to that element, while keeping the optical thickness value of the smoke column at that position unchanged. After all transformations are completed, the optical thickness geographic distribution data is obtained, a spatial distribution dataset indexed by the geodetic geographic coordinates (Lon, Lat) and with the optical thickness values ​​of the smoke column as the data values.

[0054] The temperature rise rate per unit distance corresponding to each pixel position in the temperature gradient field along the path is transformed from the infrared image pixel coordinate system (uir, vir) to the same geodetic geographic coordinate system (Lon, Lat) based on the infrared sensor imaging model of the UAV dual-light pod and the spatial position and attitude angle of the UAV at the time of acquisition of the infrared image frame. The imaging model of the infrared sensor is similar to that of the visible light sensor, both based on the collinearity condition equation, but the coordinate mapping uses the intrinsic parameter matrix (focal length, principal point coordinates, etc.) unique to the infrared sensor. After the transformation, the geographical distribution data of the temperature rise gradient is obtained, a spatially distributed dataset indexed by the geodetic geographic coordinates (Lon, Lat) and with the temperature rise rate per unit distance as the data value.

[0055] In the geographic coordinate system, the optical column optical thickness values ​​at each spatial location in the optical thickness geographic distribution data are paired with the unit distance temperature rise rate at the same spatial location in the temperature rise gradient geographic distribution data. Since the two sets of geographic distribution data use different sensor imaging models during generation, the geographic coordinates of the same ground spatial location in the two sets of data may have slight deviations (mainly due to residuals from the joint calibration of infrared and visible light sensors and instantaneous errors in GPS positioning). Therefore, in this embodiment, nearest neighbor spatial registration is implemented. For each spatial location point in the optical thickness geographic distribution data, the spatial location point closest to its geodetic line is searched in the temperature rise gradient geographic distribution data. If the nearest neighbor distance is less than a preset registration tolerance (set to 2m in this embodiment, approximately on the order of magnitude of GPS dynamic positioning accuracy and DEM spatial resolution), these two spatial location points are considered to correspond to the same ground location, and their optical column optical thickness values ​​and unit distance temperature rise rates are paired and combined into a two-dimensional feature data pair. If the nearest neighbor distance exceeds the registration tolerance, the optical thickness spatial location point has no corresponding point in the temperature rise gradient field and does not participate in subsequent fusion.

[0056] All registered two-dimensional feature data pairs are arranged sequentially according to their corresponding geodetic coordinates within a predefined geographic grid of the transmission line corridor. The geographic grid is based on the centerline of the transmission line corridor, with a grid spacing of 10m along the longitudinal direction and 5m along the transverse direction. The average smoke column optical thickness and the maximum temperature rise per unit distance for all two-dimensional feature data pairs within each grid cell are used as the fused feature value for that grid cell. The fused feature values ​​of all grid cells are then arranged in row (from left to right in the transverse direction of the corridor) and column (from the start to the end in the longitudinal direction of the corridor) to generate a multi-source fused fire point feature vector. ,in The total number of effective grid cells, Indicates the first The average optical thickness of the smoke column in each grid cell, Indicates the first The maximum temperature rise rate per unit distance of each grid cell.

[0057] From the multi-source fused fire point feature vector, spatial locations that simultaneously meet the following two criteria are selected as confirmed fire point locations. The first criterion is that the optical thickness of the smoke column exceeds a preset fire point optical determination threshold. In this embodiment The threshold was set to 0.3, determined through statistical analysis of the optical thickness of the smoke column region in 30 sets of real-world images of small-area wildfires. This corresponds to the minimum optical thickness level at which a stable, observable smoke column has formed. The second condition is that the temperature rise rate per unit distance exceeds the preset fire point temperature rise determination threshold. In this embodiment The threshold was set to 0.5℃ / m, determined through statistical analysis of the temperature rise rate along the path in the fire zone from 20 sets of real small-area wildfire infrared image sequences. A small fire point with a combustion area of ​​approximately 1 m² can produce a minimum temperature rise rate that is distinguishable from solar radiation temperature rise at a ground sampling distance of 0.33 m. Only spatial locations that simultaneously meet both of these conditions are confirmed as fire points. This dual determination mechanism cross-validates from two independent physical dimensions: dynamic changes in infrared temperature and optical quantitative analysis of visible light smoke, effectively eliminating false alarms and missed alarms caused by relying solely on information from a single sensor.

[0058] For each confirmed fire point location, the acquisition time of the corresponding infrared thermal imaging sequence frame and visible light image sequence frame is traced back. From the flight attitude parameter sequence recorded by the UAV flight control system, six flight attitude parameters corresponding to the acquisition time are extracted: longitude coordinates, latitude coordinates, flight altitude, pitch angle, roll angle, and heading angle. The flight altitude is the elevation value of the phase center of the RTK GPS antenna on the UAV, which needs to be subtracted from the vertical installation offset between the GPS antenna and the projection center of the optical sensor (in this embodiment, the offset is 0.15m) to obtain the elevation of the optical sensor projection center.

[0059] Based on the longitude coordinates, latitude coordinates, and elevation of the optical sensor projection center, the geographic coordinates of the UAV optical sensor projection center are converted into three-dimensional Cartesian coordinates in the geocentric-fixed (ECEF) coordinate system, serving as the center position of the optical sensor in three-dimensional space. Using the pitch angle (the tilt angle of the sensor's line of sight relative to the horizontal plane in the vertical plane, downward is positive), roll angle (the rotation angle of the sensor around the direction of the line of sight), and heading angle (the deflection angle of the sensor's line of sight projection in the horizontal plane relative to true north, clockwise is positive), three Euler angle rotation matrices are constructed, from the sensor's body coordinate system to the local northeast-northeast navigation coordinate system and then to the ECEF coordinate system. The unit direction vector of the line of sight in the sensor's body coordinate system is then rotated three times sequentially by roll, pitch, and heading to obtain the spatial pointing direction vector of the sensor's line of sight in the ECEF coordinate system.

[0060] A spatial observation ray is determined by the center position of the optical sensor and the spatial pointing direction vector of the line of sight, extending along the line of sight from the sensor center. This spatial observation ray is then intersected with the surface of a pre-constructed digital elevation model (DEM) of the power transmission corridor terrain. The intersection calculation uses a ray-stepping method, starting at t=0 (t being the distance parameter along the ray direction), and progressing step by step Δt=0.5m along the ray direction. At each step, the DEM surface elevation value corresponding to the current horizontal position (longitude, latitude) of the ray point is calculated, and the vertical relationship between the ray point's elevation value and the DEM surface elevation value is compared. When the ray point's elevation value changes from greater than the DEM surface elevation value to less than or equal to the DEM surface elevation value (i.e., the ray penetrates below the surface), the position between the two step points before and after this crossing point is recorded as the intersection point of the spatial observation ray and the surface. Linear interpolation is then performed on this intersection point to obtain more accurate intersection coordinates. If the ray does not intersect the DEM surface within a range of 2000m (corresponding to the maximum effective propagation distance of the ray under the condition of flight altitude of 60m to 120m), it indicates that there is no ground intersection point in that direction. The space observation ray corresponding to the confirmed fire point location is invalid because it points to the sky or for other reasons, and is therefore discarded.

[0061] The longitude, latitude, and elevation values ​​(i.e., the DEM surface elevation values ​​at the intersection of the spatial observation ray and the surface surface) are taken as the three-dimensional spatial geographic coordinates of the small-area wildfire hotspot. These three-dimensional coordinates are transmitted in real-time to the ground monitoring station via a 4G / 5G wireless communication module and displayed as a fire point icon overlaid on the transmission line corridor map in the geographic information system. The fire point number, detection time, and confidence level are also labeled for use by maintenance personnel in decision-making and firefighting resource allocation.

[0062] like Figure 2 As shown, this embodiment also provides a wide-area drone monitoring system for small-area wildfires along power transmission lines based on multi-source sensor fusion, including: The acquisition module is used to acquire the infrared thermal imaging sequence and the visible light image sequence of the corresponding viewpoint synchronously collected by the UAV dual-light pod as it flies along the power transmission line corridor; along the UAV flight path, the module calculates the partial derivative of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence to generate a temperature rise gradient field along the flight path. The extraction module is used to extract the connected regions in the friction temperature rise gradient field where the gradient value is greater than zero and located in the protection zone below the transmission line tower, and generate initial heat source candidate patches; extract the smoke edge contour line of the corresponding region of the initial heat source candidate patches in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; The calculation module is used to divide the smoke edge contour line into diffuse smoke edge segments and sharp interference edge segments according to the local bending curvature; and to integrate the visible light transmittance along the direction perpendicular to the ground surface for the pixel area surrounded by the diffuse smoke edge segments to generate the cumulative optical thickness of the smoke column. The registration module is used to register the cumulative optical thickness of the smoke column with the temperature rise gradient field along the path in the geographic coordinate system to generate a multi-source fused fire point feature vector; based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of the small-area wildfire fire point are calculated.

[0063] A computing device, comprising: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0064] A computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0065] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A wide-area UAV monitoring method for small-area wildfires along power transmission lines based on multi-source sensor fusion, characterized in that, The method includes: Acquire the infrared thermal imaging sequence and the visible light image sequence from the corresponding viewpoints simultaneously collected by the UAV dual-light pod as it flies along the power transmission line corridor; Along the flight path of the UAV, the partial derivatives of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence are calculated to generate a temperature rise gradient field along the flight path. Extract connected regions with gradient values ​​greater than zero from the friction temperature rise gradient field and located within the protection zone below the transmission line tower to generate initial heat source candidate patches; Extract the smoke edge contour line of the corresponding region of the initial heat source candidate patch in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; Based on the local bending curvature, the smoke edge contour line is divided into diffuse smoke edge segments and sharp interference edge segments; For the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along the direction perpendicular to the ground surface to generate the cumulative optical thickness of the smoke column. The cumulative optical thickness of the smoke column is registered with the temperature rise gradient field along the path in the geospatial coordinate system to generate a multi-source fusion fire point feature vector. Based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of small-area wildfire fire points are calculated.

2. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 1, characterized in that, Along the flight path of the UAV, partial derivatives are calculated for the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence to generate a temperature rise gradient field along the path, including: Extract the preceding and following infrared images that are temporally adjacent from the infrared thermal imaging sequence; For each pixel at the same pixel coordinate position in two infrared images, read the previous temperature value recorded at that position in the previous infrared image and the next temperature value recorded at the same position in the next infrared image. Calculate the difference between the next temperature value and the previous temperature value to obtain the inter-frame temperature change at the pixel coordinate position; The ground projection distance that the UAV moves along the flight path during the time interval between acquiring the previous frame of infrared image and acquiring the next frame of infrared image is obtained. Divide the inter-frame temperature change by the ground projection distance to obtain the unit distance temperature rise rate of the pixel coordinate position along the flight path direction; Traverse all valid pixel coordinates in two frames of infrared images, calculate the unit distance temperature rise rate for each pixel coordinate, and arrange the unit distance temperature rise rates of all pixel coordinates according to their row and column order in the infrared images to form the along-path temperature rise gradient field.

3. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 2, characterized in that, Extract the smoke edge contour line of the corresponding region of the initial heat source candidate patch in the visible light image sequence, and calculate the local curvature of each node of the smoke edge contour line, including: Based on the pre-calibrated spatial coordinate transformation parameters between the infrared sensor and the visible light sensor of the UAV dual-light pod, the pixel coordinate boundary of the initial heat source candidate patch in the infrared image coordinate system is transformed to the visible light image coordinate system to obtain the area to be detected in the visible light image. Within the detection area, along the brightness change path between adjacent pixels, pixel location points whose brightness amplitude changes exceed a preset brightness jump threshold are extracted to form an edge candidate point set. The pixel positions of the eight neighboring nodes in the edge candidate point set are connected in sequence according to the connectivity relationship to form the smoke edge contour line; For each contour node on the edge contour line of the smoke, take the Mth node forward and the Mth node backward along the extension direction of the contour line. With the current node as the vertex and the Mth node before and the Mth node after as the endpoints, determine the planar arc passing through these three points. Take the reciprocal of the radius of the planar arc as the local bending curvature of the current contour node.

4. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 3, characterized in that, Based on the local curvature, the smoke edge contour is divided into diffuse smoke edge segments and sharp interference edge segments, including: The local bending curvature of each contour node on the smoke edge contour line is compared with a preset curvature distinction threshold one by one; Contour nodes with local curvature less than or equal to the preset curvature discrimination threshold are marked as smooth nodes, and contour nodes with local curvature greater than the preset curvature discrimination threshold are marked as bent nodes. Along the extension direction of the smoke edge contour line, traverse the node type markers of each contour node sequentially from the start end to the end end of the contour line; The contour line segments that are consecutively adjacent along the extension direction of the contour line and whose node type is all flat nodes are connected in series and merged into the diffuse smoke edge segment. Concatenate and merge consecutive adjacent contour line segments along the contour line extension direction, where all node type labels are zigzag nodes, into sharp interference edge segments.

5. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 4, characterized in that, For the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along a direction perpendicular to the ground surface to generate the cumulative optical thickness of the smoke column, including: For the pixel region in the visible light image sequence that corresponds to the edge segment of the diffuse smoke, determine the vertical light column range of the pixel region in geographic space, measured from the ground plane along the direction perpendicular to the ground surface, to the visible light sensor receiving plane of the UAV dual-light pod. Within the vertical light column range, the light column advances layer by layer from the ground plane to the sensor receiving plane in a direction perpendicular to the ground surface. The product of the transmittance of the corresponding pixel at each layer height position and the layer height step size is used as the local optical thickness contribution value of the layer. The local optical thickness contribution values ​​of all height layers within the range of the vertical light column are accumulated one by one to obtain the smoke column optical thickness value at the pixel position corresponding to the vertical light column. Traverse all pixel positions in the pixel region surrounded by the edge segment of the diffuse smoke, calculate the optical thickness value of the smoke column corresponding to each pixel position, and arrange the optical thickness values ​​of the smoke column at all pixel positions according to the row and column positions of each pixel in the visible light image to generate the cumulative optical thickness of the smoke column.

6. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 5, characterized in that, The cumulative optical thickness of the smoke column is registered with the temperature rise gradient field along the path in a geospatial coordinate system to generate a multi-source fused fire point feature vector, including: The optical thickness values ​​of each pixel position in the cumulative optical thickness of the smoke column are transformed from the visible light image pixel coordinate system to the ground plane geographic coordinate system according to the visible light sensor imaging model of the UAV dual-light pod and the spatial position and attitude angle of the UAV at that moment, to obtain the geographic distribution data of optical thickness. The temperature rise rate per unit distance at each pixel position in the temperature rise gradient field along the path is transformed from the infrared image pixel coordinate system to the same geodetic plane geographic coordinate system based on the infrared sensor imaging model of the UAV dual-light pod and the spatial position and attitude angle of the UAV at that moment, to obtain the geographic distribution data of the temperature rise gradient. In the aforementioned geodetic coordinate system, the optical thickness geographic distribution data and the temperature rise rate per unit distance of smoke columns with the same spatial location in the temperature rise gradient geographic distribution data are merged into two-dimensional feature data pairs according to their spatial location. All two-dimensional feature data pairs are arranged in the order of rows and columns of the geographic grid to generate multi-source fusion fire point feature vectors.

7. The method for wide-area drone monitoring of small-area wildfires along power transmission lines based on multi-source sensor fusion according to claim 6, characterized in that, Based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of small-area wildfire points are calculated, including: From the multi-source fused fire point feature vector, select spatial locations where the smoke column optical thickness value exceeds the preset fire point optical determination threshold and the unit distance temperature rise rate exceeds the preset fire point temperature rise determination threshold, and use them as confirmed fire point locations. From the real-time flight attitude parameters of the UAV, extract the longitude coordinates, latitude coordinates, flight altitude, pitch angle, roll angle and heading angle of the UAV at the moment when the image frame corresponding to the location of the fire point is collected and confirmed; The three-dimensional spatial center position of the UAV optical sensor is determined based on the longitude coordinates, latitude coordinates, and flight altitude; the spatial pointing direction of the sensor's line of sight is determined based on the pitch angle, roll angle, and heading angle. A spatial observation ray is determined by the central position and the spatial pointing direction of the line of sight. The intersection of the spatial observation ray and the surface surface of the pre-constructed digital elevation model of the terrain in the area where the transmission line corridor is located is obtained. The longitude, latitude, and elevation of the intersection point are taken as the three-dimensional spatial geographic coordinates of the small-area wildfire hotspot.

8. A wide-area drone monitoring system for small-area wildfires along power transmission lines based on multi-source sensor fusion, wherein the system implements the method as described in any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire the infrared thermal imaging sequence and the visible light image sequence of the corresponding viewpoint synchronously collected by the UAV dual-light pod as it flies along the power transmission line corridor; along the UAV flight path, the module calculates the partial derivative of the pixel temperature values ​​of adjacent frames in the infrared thermal imaging sequence to generate a temperature rise gradient field along the flight path. The extraction module is used to extract the connected regions in the friction temperature rise gradient field where the gradient value is greater than zero and located in the protection zone below the transmission line tower, and generate initial heat source candidate patches; extract the smoke edge contour line of the corresponding region of the initial heat source candidate patches in the visible light image sequence, and calculate the local bending curvature of each node of the smoke edge contour line; The calculation module is used to divide the smoke edge contour line into diffuse smoke edge segments and sharp interference edge segments according to the local bending curvature; For the pixel region surrounded by the edge segment of the diffuse smoke, the visible light transmittance is integrated along the direction perpendicular to the ground surface to generate the cumulative optical thickness of the smoke column. The registration module is used to register the cumulative optical thickness of the smoke column with the temperature rise gradient field along the path in the geographic coordinate system to generate a multi-source fused fire point feature vector; based on the multi-source fused fire point feature vector and the real-time flight attitude parameters of the UAV, the three-dimensional spatial geographic coordinates of the small-area wildfire fire point are calculated.

9. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images