Forest fire unmanned aerial vehicle monitoring and early warning system and method

Abstract

The present application relates to the technical field of forest fire monitoring and early warning, and specifically discloses a kind of forest fire unmanned aerial vehicle monitoring and early warning system and method, by obtaining smoke plume image sequence, wind speed and direction and unmanned aerial vehicle attitude data, extract smoke plume profile fitting ellipse and solve equivalent barycenter coordinates, long axis deflection angle and eccentricity form tensor, according to attitude data stripping rotor interference restores background wind field vector, from background wind field vector and the previous frame form tensor recursion obtains the predicted tensor and dynamic envelope boundary under the pure wind drive, the deviation average amplitude of measured tensor and predicted tensor deviation fusion calculation is deviated, when it continues to exceed envelope boundary, activate azimuth solution, detect long axis deflection angle variance and according to the offset azimuth calibration spread main direction, according to the length of time and eccentricity decay rate classification warning;The present application solves the problem of low accuracy of fire line spread direction prediction in dense smoke environment, and realizes reliable classification warning.

Description

Technical Field

[0001] This invention relates to the field of forest fire monitoring and early warning technology, specifically to a forest fire monitoring and early warning system and method using unmanned aerial vehicles (UAVs). Background Technology

[0002] Forest fires are characterized by their sudden onset, rapid spread, and difficulty in extinguishing. High-oil-content forests, such as coniferous and eucalyptus forests, in particular, rapidly produce dense smoke columns during combustion, severely interfering with real-time monitoring of the fire situation. In recent years, with the rapid development of drone technology, using drones equipped with visible light cameras and infrared thermal imagers for aerial fire monitoring has become a crucial means of obtaining fire information in forest firefighting. Currently, drones typically determine the location and extent of fires by identifying flame pixels, smoke textures, or detecting high-temperature anomalies in infrared thermal images, and then transmit early warning information back to the command center.

[0003] Existing drone-based early warning methods for forest fires suffer from several drawbacks under dense smoke cover. They lack quantitative analysis of the macroscopic morphology of the smoke plume (spatial location, stretching direction, and dispersion), and fail to isolate the interference of drone motion and rotor downwash on wind field measurements. This results in large background wind field vector errors and inaccurate predictions of wind-driven plume motion. Furthermore, the single-frame threshold triggering method struggles to filter out noise generated by turbulence disturbances, leading to low accuracy in predicting the direction of fire spread and a high false alarm rate. Additionally, the warning signals lack clear directional direction and refined grading, failing to provide reliable and timely information on the fire's spread status for frontline command. Summary of the Invention

[0004] The purpose of this invention is to provide a forest fire monitoring and early warning system and method using unmanned aerial vehicles (UAVs) to solve the problems mentioned above.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for monitoring and early warning of forest fires using drones includes the following steps:

[0007] S1, acquire continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data;

[0008] S2, extract the plume contour from the continuous image sequence and fit it into an ellipse, solve the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a shape tensor, and perform rotor downwash stripping on the wind speed and wind direction values ​​based on the UAV attitude data to restore the background wind field vector.

[0009] S3, perform recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and its dynamic envelope boundary of the plume under pure wind drive in the next frame.

[0010] S4, perform deviation fusion between the actual measured morphological tensor and the predicted tensor in the current frame, calculate the average deviation magnitude, and activate the orientation solution when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window.

[0011] S5, in the orientation calculation, detect the recent variance of the major axis deviation angle. If the variance is lower than the threshold, the actual centroid offset orientation relative to the predicted centroid is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency warning signals are generated.

[0012] As a further aspect of the present invention: the constitutive morphological tensor specifically includes:

[0013] The dynamic plume region is obtained by extracting adjacent frames from the continuous image sequence and performing a difference operation.

[0014] Edge tracing is performed on the contour of the dynamic plume region to obtain the complete plume boundary;

[0015] Using the plume boundary as the fitting object, the least squares fitting algorithm is used to solve the equivalent centroid coordinates, major axis skew angle and eccentricity of the plume ellipse;

[0016] The equivalent centroid coordinates, major axis skew angle, and eccentricity are combined into a morphological tensor.

[0017] As a further aspect of the present invention: the restoration of the background wind field vector specifically includes:

[0018] Based on the pitch and roll angles in the UAV attitude data, the speed interference caused by the fuselage translational motion is separated from the airborne wind speed measurement.

[0019] Based on the yaw angle in the UAV attitude data, the vortex deflection component generated by the rotor rotation is extracted from the airborne wind direction measurement.

[0020] The wind speed and wind direction values ​​after separation and stripping are combined to form the background wind field vector.

[0021] As a further aspect of the present invention: S3 specifically includes:

[0022] Apply the background wind field vector of the current frame to the equivalent barycenter coordinates in the morphological tensor of the previous frame, and generate the predicted barycenter coordinates after vector translation mapping.

[0023] The direction of the background wind field vector in the current frame is conditionally balanced with the major axis offset angle in the morphological tensor of the previous frame to generate the predicted major axis offset angle.

[0024] The eccentricity in the previous frame's morphological tensor is combined with the magnitude of the current frame's background wind field vector to perform trend-preserving extension, generating the predicted eccentricity.

[0025] The predicted centroid coordinates, predicted major axis skew angle, and predicted eccentricity are combined into a prediction tensor. The horizontal and vertical envelope boundary values ​​are set according to the translation distance of the predicted centroid coordinates and the approximation range of the predicted major axis skew angle, respectively. The horizontal and vertical envelope boundary values ​​constitute the dynamic envelope boundary.

[0026] As a further aspect of the present invention: the generation of the predicted eccentricity specifically includes:

[0027] The eccentricity in the morphological tensor of the previous frame is compared with the eccentricity of the previous frame and the eccentricity of the consecutive frames before the previous frame to obtain the tilt angle of the eccentricity change.

[0028] The magnitude of the current frame background wind field vector is used as the perturbation intensity, and it is coupled with the eccentricity change angle to obtain the eccentricity drift under wind field perturbation.

[0029] The eccentricity drift is superimposed on the eccentricity in the previous frame's morphological tensor, and then the predicted eccentricity is generated after amplitude limiting.

[0030] As a further aspect of the present invention: the calculation of the average amplitude of the deviation specifically includes:

[0031] The centroid coordinates and major axis skew angles are solved from the predicted tensor and the measured tensor respectively. The absolute differences between the Euclidean spacing of the centroid coordinates and the major axis skew angle are normalized in the same direction and then weighted and synthesized to obtain the single-frame deviation amplitude.

[0032] The single-frame deviation amplitude is accumulated frame by frame according to the acquisition time sequence, and the total accumulated value is divided by the number of frames in the sliding window to obtain the average deviation amplitude.

[0033] Within the sliding window, a traversal scan is performed on the average amplitude of the deviation, and the out-of-bounds depth value that crosses the dynamic envelope boundary is captured. When the number of consecutive frames of the out-of-bounds depth value exceeds a preset frame threshold, the azimuth calculation is activated.

[0034] As a further aspect of the present invention: the boundary depth value specifically includes:

[0035] The average magnitude of the deviation in each frame within the sliding window is extracted one by one, and compared with the dynamic envelope boundary value of the same frame to obtain the difference sequence.

[0036] Extract positive differences from the difference sequence as positive differences, and record the frame position corresponding to each positive difference.

[0037] After sorting the positive differences according to their frame positions, the consecutive positive differences are accumulated and summed. The sum is then divided by the number of consecutive frames to obtain the out-of-bounds depth value.

[0038] As a further aspect of the present invention: S5 specifically includes:

[0039] The cumulative number of frames in which the average deviation amplitude exceeds the dynamic envelope boundary is calculated from the current frame and consecutive frames preceding the current frame. The cumulative number of frames is used as the duration of the exceedance. At the same time, the ratio of the decrease amplitude of the eccentricity from the current frame to the preset number of frames is calculated and used as the decay rate.

[0040] The duration of the overrun is compared in pairs with the decay rate:

[0041] If both the duration and decay rate exceed the low range, a watch signal is generated;

[0042] If both the duration and decay rate exceed the high value range, an alarm signal is generated;

[0043] An emergency warning signal is generated when the decay rate is in the high range and the duration exceeds the extremely high range.

[0044] As a further aspect of the present invention: the value ranges of the low value range, the high value range, and the extremely high value range are specifically as follows:

[0045] Using the duration of the overrun as the first dimension and the decay rate as the second dimension, the duration of the overrun in the first dimension is divided into short-term intervals and long-term intervals according to a preset frame number threshold, and the decay rate in the second dimension is divided into gradually changing intervals and steeply changing intervals according to a preset slope threshold.

[0046] The low-value range is the intersection of the short-term range and the gradually changing range; the high-value range is the intersection of the long-term range and the steeply changing range; and the extremely high-value range is the portion of the long-term range and the steeply changing range that exceeds the upper limit of the long-term range in terms of duration.

[0047] A forest fire monitoring and early warning system using unmanned aerial vehicles (UAVs) includes:

[0048] The data acquisition module acquires continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data.

[0049] The tensor construction and wind field restoration module extracts the plume contour from a continuous image sequence and fits it into an ellipse. It calculates the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a morphological tensor. Based on the UAV attitude data, it performs rotor downwash stripping on the wind speed and direction values ​​to restore the background wind field vector.

[0050] The recursive prediction module performs recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and its dynamic envelope boundary of the plume under pure wind drive in the next frame.

[0051] The deviation fusion and orientation calculation module performs deviation fusion between the actual measured morphological tensor and the predicted tensor of the current frame, calculates the average deviation magnitude, and activates orientation calculation when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window.

[0052] The graded early warning generation module detects the recent variance of the major axis deviation angle during the orientation calculation. If the variance is lower than the threshold, the actual center of gravity offset from the predicted center of gravity is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency early warning signals are generated.

[0053] The beneficial effects of this invention are:

[0054] (1) This invention fits the plume profile into an ellipse and calculates the equivalent centroid coordinates, major axis skew angle and eccentricity to form a morphological tensor, so that the overall spatial morphology of the plume is quantified into a unified data structure to participate in subsequent recursion and deviation calculation. At the same time, based on the pitch angle, roll angle and yaw angle of the UAV, the velocity interference generated by the fuselage translation motion and the wind direction deflection component generated by the rotor vortex are respectively removed to restore the background wind field vector. On this basis, the measured morphological tensor and the predicted tensor under pure wind drive are fused by deviation and the average deviation amplitude is calculated. The azimuth calculation is activated only when the average deviation amplitude continuously exceeds the dynamic envelope boundary within the sliding window, rather than triggering the warning based on the instantaneous deviation of a single frame. This filters out the high-frequency noise interference caused by gust disturbance, turbulence pulsation and the vibration of the UAV itself, reduces the false alarm rate and false alarm rate of the fire spread direction in the dense smoke environment, and enables the warning system to have a reliable ability to identify the breakthrough spread trend of the fire.

[0055] (2) This invention generates a prediction tensor and its dynamic envelope boundary under pure wind-driven conditions through recursive confidence prediction. The offset of the actual center of gravity relative to the predicted center of gravity is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, attention, alarm or emergency warning signals are generated in stages. The duration of the deviation reflects the degree of fire breaking through the wind-driven state, and the decay rate of the eccentricity reflects the trend of the smoke plume narrowing and elongating. The joint determination of the two can issue graded warning information with directional orientation before the fire spread direction changes substantially but before a large area of ​​fire is formed. This provides an operable lead time and clear directional basis for fire command, and enhances the practical guiding value of the warning results for fire fighting deployment. Attached Figure Description

[0056] The invention will now be further described with reference to the accompanying drawings.

[0057] Figure 1 This is a flowchart of the method of the present invention;

[0058] Figure 2This is a flowchart illustrating the process of constructing a morphological tensor in this invention.

[0059] Figure 3 This is a flowchart of the process for reconstructing the background wind field vector in this invention;

[0060] Figure 4 This is a system block diagram of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Please see Figure 1 As shown, this invention provides a method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs), comprising the following steps:

[0063] S1, acquire continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data;

[0064] S2, extract the plume contour from the continuous image sequence and fit it into an ellipse, solve the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a shape tensor, and perform rotor downwash stripping on the wind speed and wind direction values ​​based on the UAV attitude data to restore the background wind field vector.

[0065] S3, perform recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and dynamic envelope boundary of the plume under pure wind drive in the next frame.

[0066] S4, perform deviation fusion between the actual measured morphological tensor and the predicted tensor in the current frame, calculate the average deviation magnitude, and activate the orientation solution when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window.

[0067] S5, in the orientation calculation, detect the recent variance of the major axis deviation angle. If the variance is lower than the threshold, the actual centroid offset orientation relative to the predicted centroid is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency warning signals are generated.

[0068] In S1, continuous image sequences of the fire plume, environmental wind speed and direction values, and UAV attitude data are acquired, specifically including:

[0069] A dual-light pod is mounted on the underside of the drone via a gimbal. The dual-light pod integrates a visible light camera and an uncooled infrared thermal imager.

[0070] During the drone's patrol and monitoring of the fire area along a preset route, the visible light camera acquires a sequence of visible light images of the fire area and its surroundings at a frame rate of 30 frames per second, while the infrared thermal imager simultaneously acquires a sequence of infrared thermal images of the same field of view at a frame rate of 30 frames per second. Both image sequences are converted from analog to digital by the onboard image acquisition card and stored in the onboard data recording unit in digital image format. Under the control of the timestamp synchronization signal, the visible light image and the infrared thermal image acquired at the same time are grouped into an image pair, and multiple image pairs are arranged in chronological order to form a continuous image sequence.

[0071] When dense smoke in the fire area prevents visible light cameras from penetrating the smoke layer, the system automatically switches to infrared thermal image sequences as the primary data source for smoke plume extraction.

[0072] An onboard ultrasonic anemometer is installed on the top of the drone fuselage. This ultrasonic anemometer calculates the ambient wind speed and direction by measuring the time difference of ultrasonic waves propagating along the orthogonal axis, and continuously outputs digital wind speed and direction data to the onboard data recording unit at a refresh rate of 10 Hz.

[0073] The wind speed value is in meters per second, and the wind direction value is in degrees, with north as the reference and clockwise as the positive angle.

[0074] The UAV flight control computer integrates an inertial measurement unit, which includes a three-axis microelectromechanical gyroscope and a three-axis microelectromechanical accelerometer, and outputs the UAV's three-axis angular velocity data and three-axis acceleration data at a refresh rate of 100 Hz.

[0075] Based on the three-axis angular velocity data and three-axis acceleration data, the flight control computer generates the pitch angle, roll angle and yaw angle of the UAV in real time through attitude calculation algorithm, and outputs the pitch angle, roll angle and yaw angle as UAV attitude data to the onboard data recording unit.

[0076] The airborne data recording unit aligns the continuous image sequence, environmental wind speed and direction values, and UAV attitude data according to a unified timestamp benchmark to form a multi-source data packet containing image information, wind field information, and attitude information for subsequent steps to call.

[0077] Please see Figure 2 or Figure 3 As shown, in S2, the plume contour is extracted from the continuous image sequence and fitted into an ellipse. The equivalent centroid coordinates, major axis skew angle, and eccentricity of the ellipse are calculated to form a shape tensor. Based on the UAV attitude data, rotor downwash stripping is performed on the wind speed and direction values ​​to reconstruct the background wind field vector, specifically including:

[0078] The plume contour is extracted from a continuous image sequence and fitted as an ellipse. The equivalent centroid coordinates, major axis skew angle, and eccentricity of the ellipse are calculated to form a morphological tensor. Specifically, this is achieved by performing pixel-by-pixel grayscale difference operations on adjacent frames in the continuous image sequence. The set of pixels whose absolute difference value is greater than a preset grayscale threshold is taken as the dynamic plume region. The preset grayscale threshold is 20. This preset grayscale threshold of 20 is based on the statistical distribution difference between plume moving pixels and background noise pixels in the inter-frame difference grayscale amplitude of visible light / infrared image sequences under forest fire conditions. It is obtained by differential grayscale histogram analysis and calibration of multiple sets of sample images under classic fire conditions. Regions with inter-frame difference values ​​below 20 are mainly due to sensor noise and illumination fluctuations, while regions with values ​​above 20 correspond to actual movement changes at the plume edge. This value has been verified under multiple conditions to maintain a stable detection rate of moving regions under both low and high smoke density conditions.

[0079] Edge tracking is performed on the binarized image of the dynamic plume region. The specific process is as follows: scan line by line starting from the upper left corner of the image. When the first pixel belonging to the dynamic plume region is scanned, it is taken as the tracking starting point. The next boundary pixel is searched in a clockwise direction according to the eight neighbors. This search is repeated until the tracking starting point is returned. All boundary pixels recorded during the search process are arranged in order to form a complete plume boundary.

[0080] Using the position coordinates of all pixels on the plume boundary as the fitting object, the least squares fitting algorithm is used to solve the ellipse parameters that minimize the sum of squared distances to the plume boundary. That is, by searching for the x-coordinate of the ellipse center point, the y-coordinate of the ellipse center point, the angle of the major axis of the ellipse, and the ratio of the length of the minor axis to the major axis of the ellipse, the x-coordinate and y-coordinate of the ellipse center point are combined to form the equivalent centroid coordinates, the angle of the major axis of the ellipse is taken as the major axis skew angle, and the ratio of the length of the minor axis to the major axis of the ellipse is taken as the eccentricity.

[0081] The equivalent centroid coordinates, major axis skew angle, and eccentricity are combined in a fixed data arrangement order to form a morphological tensor. This morphological tensor represents the position, stretching direction, and dispersion degree of the plume in space in the current frame.

[0082] The process of stripping the wind speed and direction values ​​from the UAV attitude data to restore the background wind field vector is achieved in the following way: the pitch and roll angles at the current moment are read from the airborne data recording unit, and the wind speed and direction values ​​output by the airborne ultrasonic anemometer and the ground speed value output by the flight control computer are read simultaneously; based on the pitch and roll angles, the cosine of the pitch angle is multiplied by the ground speed value, and the cosine of the roll angle is multiplied by the ground speed value to obtain the speed interference caused by the UAV's pitch motion in the forward direction and the speed interference caused by the roll motion in the lateral direction. The wind speed value is then subtracted from the above two speed interference values ​​to obtain the wind speed value after stripping the interference from the fuselage translational motion.

[0083] The yaw angle at the current moment is read from the airborne data recording unit. The rotor rotation direction is determined based on the yaw angle. The absolute value of the sine of the yaw angle is taken as the eddy deflection intensity coefficient. The eddy deflection component is obtained by multiplying the eddy deflection intensity coefficient by a preset deflection constant. The wind direction value is obtained by subtracting the eddy deflection component from the wind direction value and removing the rotor eddy interference. The preset deflection constant is 15 degrees. This 15-degree value is the average deflection statistical value of the rotor eddy to the wind direction measurement obtained by comparing and calibrating the measured data of the airborne anemometer in the hovering state of the UAV with the ground reference wind field.

[0084] The wind speed value after stripping the fuselage translational motion interference and the wind direction value after stripping the rotor vortex interference are combined into a background wind field vector according to the vector synthesis rule. The background wind field vector is represented by the wind speed value as the magnitude and the wind direction value as the direction.

[0085] In S3, the background wind field vector of the current frame and the morphological tensor of the previous frame are recursively predicted with confidence to obtain the predicted tensor and dynamic envelope boundary of the plume under pure wind drive in the next frame, specifically including:

[0086] The recursive confidence prediction of the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and dynamic envelope boundary of the plume under pure wind drive in the next frame is specifically achieved in the following way.

[0087] The wind direction in the background wind field vector of the current frame is used as the translation direction, and the wind speed value in the background wind field vector of the current frame is used as the translation distance. This is applied to the equivalent centroid coordinates in the morphological tensor of the previous frame. That is, the product of the wind speed value and the cosine of the wind direction is superimposed on the horizontal coordinate value of the equivalent centroid coordinates, and the product of the wind speed value and the sine of the wind direction is superimposed on the vertical coordinate value of the equivalent centroid coordinates. After the above vector translation mapping, the predicted centroid coordinates are generated.

[0088] The confidence level in the recursive confidence prediction is determined by the temporal correlation covariance matrix between the current frame background wind field vector and the previous frame morphology tensor, and the confidence interval is formed by setting the horizontal and vertical bidirectional envelope boundary values ​​based on the translation distance of the predicted centroid coordinates and the approximation magnitude of the predicted major axis deviation angle.

[0089] The wind direction in the background wind field vector of the current frame is calculated by comparing the major axis deviation angle in the morphological tensor of the previous frame. If the absolute value of the difference is less than 90 degrees, the wind direction is taken as the predicted major axis deviation angle. If the absolute value of the difference is greater than or equal to 90 degrees, the major axis deviation angle in the morphological tensor of the previous frame plus half of the difference is taken as the predicted major axis deviation angle. After completing the conditional same-direction compromise, the predicted major axis deviation angle is generated.

[0090] The generation of the predicted eccentricity is specifically as follows: starting from the eccentricity in the morphological tensor of the previous frame, the eccentricity values ​​of 6 consecutive frames are retrieved backward. The eccentricity of the current frame is differed from the eccentricity of the previous frame to obtain the first difference value. The eccentricity of the previous frame is differed from the eccentricity of the previous two frames to obtain the second difference value. This process is repeated until the eccentricity of the previous five frames is differed from the eccentricity of the previous six frames to obtain the fifth difference value. The sum of these 5 difference values ​​is then divided by 5 to obtain the eccentricity change angle.

[0091] The wind speed value in the current frame background wind field vector is taken as the disturbance intensity. The product of the disturbance intensity and the eccentricity change angle is then multiplied by the disturbance scaling factor of 0.7 to obtain the eccentricity drift under the wind field disturbance.

[0092] The eccentricity drift caused by wind disturbance is superimposed on the eccentricity in the morphological tensor of the previous frame to obtain a preliminary predicted eccentricity value. Then, an amplitude limiting constraint is applied to the preliminary predicted eccentricity value to generate a predicted eccentricity. The amplitude limiting constraint is as follows: if the preliminary predicted eccentricity value is greater than 0.95, the value is 0.95; if the preliminary predicted eccentricity value is less than 0.05, the value is 0.05; otherwise, the preliminary predicted eccentricity value remains unchanged.

[0093] The mathematical formula used to calculate the predicted eccentricity after the amplitude limiting constraint is as follows: ;

[0094] In the formula, This represents the predicted eccentricity generated after amplitude limiting constraints. This represents the eccentricity value in the morphological tensor of the previous frame. This represents the eccentricity drift under wind field disturbance.

[0095] The predicted centroid coordinates, predicted major axis skew angle, and predicted eccentricity are combined into a prediction tensor, which characterizes the predicted spatial position, predicted stretching direction, and predicted dispersion degree of the plume in the next frame under pure wind drive.

[0096] Based on the translation distance of the predicted centroid coordinates and the approximation range of the predicted major axis skew angle, the horizontal and vertical envelope boundary values ​​are set respectively: 1.2 times the translation distance of the predicted centroid coordinates is taken as the horizontal envelope boundary value, and the absolute value of the approximation angle difference between the predicted major axis skew angle and the current frame background wind field vector direction is taken as the vertical envelope boundary value. The horizontal and vertical envelope boundary values ​​constitute the dynamic envelope boundary.

[0097] In S4, the actual measured morphological tensor and the predicted tensor of the current frame are fused to calculate the average deviation magnitude. When the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window, the orientation calculation is activated, specifically including:

[0098] The predicted centroid coordinates and predicted major axis offset angle are obtained from the predicted tensor. At the same time, the measured centroid coordinates and measured major axis offset angle are obtained from the measured tensor. The centroid offset distance is obtained by calculating the Euclidean distance between the predicted and measured centroid coordinates. The direction deviation angle is obtained by calculating the absolute difference between the predicted and measured major axis offset angles.

[0099] Take 0.5 times the length of the image diagonal as the centroid normalization reference value, divide the centroid offset distance by the centroid normalization reference value to obtain the normalized centroid offset metric, take 180 degrees as the angle normalization reference value, divide the direction deviation angle by the angle normalization reference value to obtain the normalized direction deviation metric.

[0100] The center of gravity offset weight is set to 0.6, and the direction deviation weight is set to 0.4. The normalized center of gravity offset metric is multiplied by the center of gravity offset weight to obtain the center of gravity weighted component. The normalized direction deviation metric is multiplied by the direction deviation weight to obtain the direction weighted component. The center of gravity weighted component and the direction weighted component are added together to obtain the single-frame deviation amplitude.

[0101] It should be noted that the weighting is based on the following: the Euclidean spacing of the barycentric coordinates directly represents the actual spatial displacement of the plume center in the fire scene, and has a more sensitive first-order response characteristic to the breakthrough spread of fire driven by wind, hence it is assigned a higher weight of 0.6; the major axis skew angle reflects the change in the plume stretching direction, which is a second-order geometric feature and is easily affected by local turbulence, causing small oscillations, hence it is assigned a lower weight of 0.4 to suppress the influence of directional noise on the bias fusion result. The above 6:4 weighting ratio was obtained by traversing and optimizing multiple sets of measured data under classic fire scene conditions.

[0102] A sliding window containing 60 consecutive frames of image data is set. The single-frame deviation amplitude of each frame in the sliding window is accumulated frame by frame according to the acquisition time sequence. The total accumulated value is divided by the number of frames in the sliding window, 60, to obtain the average deviation amplitude. This average deviation amplitude represents the degree to which the plume morphology deviates from the pure wind-driven prediction within the sliding window.

[0103] The horizontal envelope boundary value is set based on the translation distance of the predicted centroid coordinates in the current frame, and the vertical envelope boundary value is set based on the approximation magnitude of the predicted major axis offset angle in the current frame. The horizontal envelope boundary value is multiplied by the centroid offset weight value of 0.6 to obtain the horizontal boundary weighted component, and the vertical envelope boundary value is multiplied by the direction deviation weight value of 0.4 to obtain the vertical boundary weighted component. The horizontal boundary weighted component and the vertical boundary weighted component are added together to obtain the dynamic envelope boundary scalar value.

[0104] The average deviation magnitude of each frame from the start frame to the end frame within the sliding window is extracted one by one in frame order. The difference between the average deviation magnitude of each frame and the dynamic envelope boundary scalar value of the same frame is calculated to obtain a difference sequence composed of the differences of each frame arranged in frame order.

[0105] Extract the positive difference values ​​from the difference sequence that are greater than 0, and record the frame position number corresponding to each positive difference value in the difference sequence to obtain the positive difference set and its frame position record.

[0106] The positive difference values ​​in the positive difference set are reordered in ascending order of frame position number. The positive difference values ​​with consecutively adjacent frame positions are selected from the sorted positive difference values ​​as a continuous positive difference group. All positive difference values ​​in the continuous positive difference group are summed to obtain the positive difference sum. The positive difference sum is divided by the number of differences contained in the continuous positive difference group to obtain the out-of-bounds depth value.

[0107] The frame rate threshold is set to 15 frames. When the number of consecutive out-of-bounds depth values ​​within the sliding window exceeds 15 frames, the orientation calculation process is activated.

[0108] In S5, during the orientation calculation, the recent variance of the major axis deviation angle is detected. If the variance is below a threshold, the actual centroid offset relative to the predicted centroid is designated as the main propagation direction. Based on the duration of the average deviation amplitude and the decay rate of the eccentricity, attention, alarm, or emergency warning signals are generated in stages, including:

[0109] The threshold is set based on the following: the threshold is a directional consistency discrimination threshold, which is set according to the upper limit of the statistical variance of the random fluctuation of the long axis deviation angle of the plume caused by turbulent disturbance under typical fire conditions. When the recent variance is lower than the threshold, it is determined that the direction of the plume spread tends to be stable and has statistical reliability in calibrating the main direction of spread.

[0110] Starting from the current frame, backtrack along the time axis to traverse each frame within the sliding window, and determine whether the average deviation magnitude of each frame is greater than the dynamic envelope boundary scalar value corresponding to that frame. Accumulate the number of frames that meet the condition and use the accumulated number of frames as the overrun duration.

[0111] Read the current eccentricity from the current frame morphological tensor. Take 60 consecutive frames backward from the current frame. Read the eccentricity value of the previous frame. Subtract the eccentricity of the previous frame from the current eccentricity to get the first frame's decrease rate. Then read the eccentricity value of the second frame before the current frame and subtract the eccentricity of the second frame before the current frame to get the second frame's decrease rate. Continue in this manner until the eccentricity value of the sixtieth frame before the current frame is read and the eccentricity of the sixtieth frame before the current frame is subtracted to get the sixtieth frame's decrease rate. Add up the above 60 decrease rates to get the total decrease rate. Divide the total decrease rate by 60 to get the average decrease rate per frame. Use the average decrease rate per frame as the decay rate.

[0112] Using the duration of the overrun as the first dimension and the decay rate as the second dimension, a preset frame threshold of 40 frames is set. The intervals with an overrun duration of less than or equal to 40 frames are defined as short-term intervals, and the intervals with an overrun duration of more than 40 frames are defined as long-term intervals. A preset slope threshold of 0.005 is set. The intervals with a decay rate of less than or equal to 0.005 are defined as gradually changing intervals, and the intervals with a decay rate of more than 0.005 are defined as steeply changing intervals.

[0113] The intersection of the short-term interval and the gradually changing interval is taken as the low-value interval, and the intersection of the long-term interval and the abruptly changing interval is taken as the high-value interval.

[0114] The portion of the high-value range that exceeds the 40-frame limit in duration, i.e., the entire range of values ​​with a duration greater than 40 frames and a decay rate greater than 0.005, is defined as the extremely high-value range.

[0115] The calculated actual overrun duration and decay rate are paired, and the interval combination in which the pairing falls is determined: if the actual overrun duration falls into the short time interval and the actual decay rate falls into the slowly changing interval, then both are determined to be in the low value interval, and a signal of interest is generated.

[0116] If the actual duration of the exceedance falls into the long-term range and the actual decay rate falls into the steep change range, then both are determined to be in the high-value range, and an alarm signal is generated.

[0117] If the actual duration of the overrun falls into the extremely high range, that is, the duration of the overrun is greater than 40 frames and the decay rate is greater than 0.005, an emergency warning signal will be generated.

[0118] Please see Figure 4 As shown, a forest fire monitoring and early warning system using unmanned aerial vehicles (UAVs) includes:

[0119] The data acquisition module acquires continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data.

[0120] The tensor construction and wind field restoration module extracts the plume contour from a continuous image sequence and fits it into an ellipse. It calculates the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a morphological tensor. Based on the UAV attitude data, it performs rotor downwash stripping on the wind speed and direction values ​​to restore the background wind field vector.

[0121] The recursive prediction module performs recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and its dynamic envelope boundary of the plume under pure wind drive in the next frame.

[0122] The deviation fusion and orientation calculation module performs deviation fusion between the actual measured morphological tensor and the predicted tensor of the current frame, calculates the average deviation magnitude, and activates orientation calculation when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window.

[0123] The graded early warning generation module detects the recent variance of the major axis deviation angle during the orientation calculation. If the variance is lower than the threshold, the actual center of gravity offset from the predicted center of gravity is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency early warning signals are generated.

[0124] The working principle of this invention is as follows: A continuous image sequence of plumes is acquired via an airborne dual-light pod; environmental wind speed and direction values ​​are collected by an airborne ultrasonic anemometer; and UAV attitude data is collected by the inertial measurement unit inside the flight control computer. The plume contour is extracted from the continuous image sequence and fitted as an ellipse. The equivalent centroid coordinates, major axis skew angle, and eccentricity are calculated to form a shape tensor. Simultaneously, the background wind field vector is reconstructed by removing rotor downwash interference based on the pitch, roll, and yaw angles in the attitude data. The background wind field vector of the current frame is then compared with the shape tensor of the previous frame. The system uses a confidence prediction approach to generate a prediction tensor and its dynamic envelope boundary under pure wind power. It then fuses the measured morphological tensor with the prediction tensor by deviation calculation and calculates the average deviation amplitude. When the average deviation amplitude continuously exceeds the dynamic envelope boundary within a sliding window, the azimuth calculation is activated. In the azimuth calculation, the variance of the major axis deviation angle is detected. If the variance is lower than the threshold, the azimuth offset of the actual centroid relative to the predicted centroid is marked as the main propagation direction. Based on the duration of the deviation amplitude exceeding the threshold and the decay rate of the eccentricity, attention, alarm, or emergency warning signals are generated in stages.

[0125] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1, acquire continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data; S2, extract the plume contour from the continuous image sequence and fit it into an ellipse, solve the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a shape tensor, and perform rotor downwash stripping on the wind speed and wind direction values ​​based on the UAV attitude data to restore the background wind field vector. S3, perform recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and dynamic envelope boundary of the plume under pure wind drive in the next frame. S4, perform deviation fusion between the actual measured morphological tensor and the predicted tensor in the current frame, calculate the average deviation magnitude, and activate the orientation solution when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window. S5, in the orientation calculation, detect the recent variance of the major axis deviation angle. If the variance is lower than the threshold, the actual centroid offset orientation relative to the predicted centroid is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency warning signals are generated.

2. The method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The constituent morphological tensor specifically includes: The dynamic plume region is obtained by extracting adjacent frames from a continuous image sequence and performing a difference operation. Edge tracing is performed on the contour of the dynamic plume region to obtain the complete plume boundary; Using the plume boundary as the fitting object, the least squares fitting algorithm is used to solve the equivalent centroid coordinates, major axis skew angle and eccentricity of the plume ellipse; The equivalent centroid coordinates, major axis skew angle, and eccentricity are combined into a morphological tensor.

3. The method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The reconstructed background wind field vector specifically includes: Based on the pitch and roll angles in the UAV attitude data, the speed interference caused by the fuselage translational motion is separated from the airborne wind speed measurement. Based on the yaw angle in the UAV attitude data, the vortex deflection component generated by the rotor rotation is extracted from the airborne wind direction measurement. The wind speed and wind direction values ​​after separation and stripping are combined to form the background wind field vector.

4. The method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, S3 specifically includes: Apply the background wind field vector of the current frame to the equivalent barycenter coordinates in the morphological tensor of the previous frame, and generate the predicted barycenter coordinates after vector translation mapping. The direction of the background wind field vector in the current frame is conditionally balanced with the major axis offset angle in the morphological tensor of the previous frame to generate the predicted major axis offset angle. The eccentricity in the previous frame's morphological tensor is combined with the magnitude of the current frame's background wind field vector to perform trend-preserving extension, generating the predicted eccentricity. The predicted centroid coordinates, predicted major axis skew angle, and predicted eccentricity are combined into a prediction tensor. The horizontal and vertical envelope boundary values ​​are set according to the translation distance of the predicted centroid coordinates and the approximation range of the predicted major axis skew angle, respectively. The horizontal and vertical envelope boundary values ​​constitute the dynamic envelope boundary.

5. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The generation of the predicted eccentricity specifically includes: The eccentricity in the morphological tensor of the previous frame is compared with the eccentricity of the previous frame and the eccentricity of the consecutive frames before the previous frame to obtain the tilt angle of the eccentricity change. The magnitude of the current frame background wind field vector is used as the perturbation intensity, and it is coupled with the eccentricity change angle to obtain the eccentricity drift under wind field perturbation. The eccentricity drift is superimposed on the eccentricity in the previous frame's morphological tensor, and then the predicted eccentricity is generated after amplitude limiting.

6. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The calculation of the average magnitude of the deviation specifically includes: The centroid coordinates and major axis skew angles are solved from the predicted tensor and the measured tensor respectively. The absolute differences between the Euclidean spacing of the centroid coordinates and the major axis skew angle are normalized in the same direction and then weighted and synthesized to obtain the single-frame deviation amplitude. The single-frame deviation amplitude is accumulated frame by frame according to the acquisition time sequence, and the total accumulated value is divided by the number of frames in the sliding window to obtain the average deviation amplitude. Within the sliding window, a traversal scan is performed on the average amplitude of the deviation, and the out-of-bounds depth value that crosses the dynamic envelope boundary is captured. When the number of consecutive frames of the out-of-bounds depth value exceeds a preset frame threshold, the azimuth calculation is activated.

7. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The out-of-bounds depth value specifically includes: The average magnitude of the deviation in each frame within the sliding window is extracted one by one, and compared with the dynamic envelope boundary value of the same frame to obtain the difference sequence. Extract positive differences from the difference sequence as positive differences, and record the frame position corresponding to each positive difference. After sorting the positive differences according to their frame positions, the consecutive positive differences are accumulated and summed. The sum is then divided by the number of consecutive frames to obtain the out-of-bounds depth value.

8. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, S5 specifically includes: The cumulative number of frames in which the deviation product exceeds the dynamic envelope boundary is calculated from the current frame and consecutive frames preceding the current frame. The cumulative number of frames is used as the duration of the exceedance. At the same time, the ratio of the decrease in eccentricity from the current frame to the preset number of frames is calculated and used as the decay rate. The duration of the overrun is compared in pairs with the decay rate: If both the duration and decay rate exceed the low range, a watch signal is generated; If both the duration and decay rate exceed the high value range, an alarm signal is generated; An emergency warning signal is generated when the decay rate is in the high range and the duration exceeds the extremely high range.

9. A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to claim 8, characterized in that, The specific value ranges of the low value range, the high value range, and the extremely high value range are as follows: Using the duration of the overrun as the first dimension and the decay rate as the second dimension, the duration of the overrun in the first dimension is divided into short-term intervals and long-term intervals according to a preset frame number threshold, and the decay rate in the second dimension is divided into gradually changing intervals and steeply changing intervals according to a preset slope threshold. The low-value range is the intersection of the short-term range and the gradually changing range; the high-value range is the intersection of the long-term range and the steeply changing range; and the extremely high-value range is the portion of the long-term range and the steeply changing range that exceeds the upper limit of the long-term range in terms of duration.

10. A forest fire monitoring and early warning system using unmanned aerial vehicles (UAVs), characterized in that, A method for monitoring and early warning of forest fires using unmanned aerial vehicles (UAVs) according to any one of claims 1-9 includes: The data acquisition module acquires continuous image sequences of fire plumes, environmental wind speed and direction values, and UAV attitude data. The tensor construction and wind field restoration module extracts the plume contour from a continuous image sequence and fits it into an ellipse. It calculates the equivalent centroid coordinates, major axis skew angle and eccentricity of the ellipse to form a morphological tensor. Based on the UAV attitude data, it performs rotor downwash stripping on the wind speed and direction values ​​to restore the background wind field vector. The recursive prediction module performs recursive confidence prediction on the background wind field vector of the current frame and the morphological tensor of the previous frame to obtain the predicted tensor and its dynamic envelope boundary of the plume under pure wind drive in the next frame. The deviation fusion and orientation calculation module performs deviation fusion between the actual measured morphological tensor and the predicted tensor of the current frame, calculates the average deviation magnitude, and activates orientation calculation when the average deviation magnitude continuously exceeds the dynamic envelope boundary within the sliding window. The graded early warning generation module detects the recent variance of the major axis deviation angle during the orientation calculation. If the variance is lower than the threshold, the actual center of gravity offset from the predicted center of gravity is marked as the main direction of spread. Based on the duration of the deviation average amplitude and the decay rate of the eccentricity, graded attention, alarm or emergency early warning signals are generated.

Images