Forest fire early warning method based on quadrotor unmanned aerial vehicle

Abstract

The invention discloses a forest fire early warning method based on a quad-rotor unmanned aerial vehicle. The method belongs to the technical field of forest fire prevention monitoring, and comprises the following implementation processes: after an unmanned aerial vehicle takes off, establishing a terrain model for a monitored forest area through an airborne laser radar, and then performing optimal flight path planning on a patrol path for forest fire point monitoring by adopting a PKO algorithm; the unmanned aerial vehicle carries a thermal imaging camera to carry out forest image data acquisition; collected forest fire point images are classified, identified and marked by adopting a YOLOv8 algorithm in combination with a visual trunk of RMT. According to the method for autonomously completing forest fire early warning by the unmanned aerial vehicle provided by the invention, the patrol efficiency of the unmanned aerial vehicle is improved, the time required for fire response is reduced, and the labor cost is saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of forest fire prevention monitoring and relates to a fire early warning method based on an unmanned aerial vehicle (UAV); specifically, it relates to a forest fire early warning method based on a quad-rotor UAV. Background Art

[0002] Forest fire is one of the most serious natural disasters, characterized by suddenness, great destructiveness, and difficulty in extinguishing. Traditional forest fire monitoring mainly relies on manual patrols and observations from watchtowers, which have the disadvantages of low efficiency, high cost, and limited coverage.

[0003] Therefore, it is necessary to improve the existing technology to solve the shortcomings of the existing technology. Summary of the invention

[0004] In response to the above problems, the purpose of the present invention is to propose a forest fire early warning method based on a quadrotor drone to solve the problems of low efficiency, high cost, and insufficient accuracy of traditional manual inspections; this method uses drones equipped with lidar, thermal imaging cameras, and deep learning algorithms to autonomously complete forest fire early warning tasks, improve patrol efficiency, reduce fire response time, save labor costs, and improve detection accuracy and reliability, thereby effectively preventing and controlling the occurrence of forest fires.

[0005] The technical solution of the present invention is: a forest fire early warning method based on a quad-rotor drone is realized based on the drone equipped with a laser radar and a drone thermal imaging camera;

[0006] It uses drones equipped with lidar, thermal imaging cameras and deep learning algorithms to achieve forest fire early warning. The drone first uses lidar to collect forest terrain data and uses the PKO algorithm to plan the optimal patrol track. During the patrol, the thermal imaging camera collects forest images and uses the YOLOv8 algorithm and RMT network to identify and locate fire points. Finally, the fire source information is sent to relevant personnel and monitoring centers to achieve efficient and accurate forest fire early warning. The specific implementation steps are as follows:

[0007] S1: Start the drone. After the drone takes off, the onboard laser radar actively emits laser pulses to obtain the signal reflected by the detection target and process it to obtain point cloud data. The point cloud data is filtered and classified to remove noise and interference information such as ground points. The digital model is constructed using the processed point cloud data.

[0008] S2: Complete the trajectory planning for fire warning patrol according to the PKO algorithm; there are three stages in the PKO algorithm:

[0009] Phase 1, initialization phase: PKO starts the search process by randomly generating a set of starting solutions from the search space as the first attempt:

[0010] X i.j =LB+(UB-LB)×rand

[0011] i=1,2,...,N and j=1,2,...,Dim

[0012] In the formula, X i.j represents the position of the i-th individual in the j-th dimension, rand represents a random value between 0 and 1, UB and LB represent the upper and lower bounds of the search range;

[0013] Phase 2, Exploration Phase: The drone switches between perching and hovering modes; the drone's position is updated according to the following formula:

[0014] X i (t+1)=X i (t)+∝×T(X j (t)-X i (t)),

[0015] i,j=1,2,...,N and j≠i

[0016] In the iteration process, the solution of the next iteration is represented by Xi(t+1), and the position of the current iteration is represented by Xit; the parameter α is calculated as 2*randn(1,dim), where randn represents a random number in a normal distribution; N represents the overall size, and Dim represents the dimension of the problem under consideration;

[0017] The parameter T plays a key role. Its value is determined dynamically according to the current strategy, which can be "perched" or "hovering". The calculation of T is tailored for each strategy to ensure the best performance in different operating modes.

[0018] In the roosting strategy, the parameter T is calculated as follows:

[0019]

[0020] Crest_angles = 2×pi×rand

[0021] The maximum number of iterations is specified by max Iter, the constant value BF (beat Factor) is set to 8, and rand is a random value between 0 and 1;

[0022] In the hover strategy, the parameter T is calculated as follows:

[0023]

[0024] The fitness of the i-th and j-th UAVs are denoted by PKO_Fitness(i) and PKO_Fitness(j), respectively; in addition, a constant value, BF (beating factor), is set to 8;

[0025] Phase 3, Capture Phase: When the drone approaches the target location in the constraints, it will continue to make more precise adjustments. The mathematical model is as follows:

[0026] X i (t+1)=X i (t)+HA×o×α×(bX best (t)),i=1,2,...,N

[0027] Wherein, the fitness value of the ith UAV is recorded as PKO_Fitness(1), Best_Fitness represents the best fitness value obtained from all iterations; α is a control parameter, calculated as 2*randn(1,dim)-1, and o and HA represent hunting capabilities, which can be calculated by the following two formulas:

[0028]

[0029] b=X i (t)+o 2 ×randn×X best (t)

[0030] The PKO algorithm first creates a population of potential solutions by randomly generating them; during multiple iterations, the positions of the drones are continuously updated using Eqs; if any drone moves outside the search area, the steps of the PKO simulation push them back to the boundary; subsequently, the current position is updated only if the new position provides a better solution quality than the current position; except for the initialization, all steps in the algorithm are performed in each iteration until the maximum number of iterations is reached;

[0031] Step 1: Construct forest terrain model;

[0032] Step 2: Determine the constraints for path planning based on the performance indicators of the drone and the performance of the link and payload;

[0033] Step 3: Initialize the population and randomly generate a set of initial solutions in the search space, each solution represents a possible path;

[0034] Step 4: Evaluate the fitness of each solution based on the objective function and constraints. The higher the fitness, the better the path.

[0035] Step 5: According to the three stages of the PKO algorithm (initialization, exploration, and capture), update the solutions in the population and find a better path; until the maximum number of iterations is reached or the fitness reaches the preset threshold, terminate the algorithm and output the optimal path;

[0036] Step 6: Perform interpolation and smoothing based on the obtained track, adjust the track space model, and improve or re-plan the route;

[0037] Step 7: Output the final trajectory planning data and issue instructions to the drone to perform flight operations;

[0038] S3: During the flight mission, the thermal imaging camera of the drone continuously collects forest fire point data to ensure that the collected fire point images are clear. The forest fire point detection method has two steps:

[0039] Step 1: Comprehensive detection of forest fire spots; increase the number of camera shots during the drone tracking process to ensure that there are overlapping parts in the captured pictures and conduct a large-scale fire spot search; the drone conducts a comprehensive inspection of the forest; the forest is divided into regions using a monomer method, the divided regions are numbered, and the GIS function is used to mark the areas involved in the information of each fire spot;

[0040] Step 2: Fixed-point detection of forest fire points: For the determined fire point area, add a target point close to the fire point in the path planning, so that the drone arrives at the fire point area for fixed-point data collection, and further fire point detection and evaluation are carried out based on the data collected at the fixed point;

[0041] S4: YOLOv8 algorithm is used to classify and identify fire spots in forest images, and RMT visual backbone is used as feature extractor to improve the performance of YOLOv8 in thermal imaging fire spot detection task;

[0042] Step 1: During the drone’s flight path, the drone’s thermal imaging camera collects forest images;

[0043] Step 2: Preprocess the image to make it more consistent with the model’s input requirements; perform noise removal and image enhancement to improve image quality;

[0044] Step 3: Divide the image into multiple grids of equal size;

[0045] Step 4: Select ResNet as the basic network, introduce the Manhattan self-attention mechanism, build the RMT network, input the image into the RMT model, and extract image features;

[0046] Step 5: Input the features extracted by RMT into the subsequent network of YOLOv8 to predict the bounding box and category of each grid;

[0047] Step 6: Predict the confidence value score for each bounding box;

[0048] Step 7: Predict the conditional category probability in each bounding box;

[0049] Step 8: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images;

[0050] Step 9: Calculate the loss function for the target object in the grid;

[0051] Step 10: Output feature image containing feature information;

[0052] The time decay of RetNet proposes a retention mechanism for sequence modeling; the retention mechanism brings time decay to the language model; first consider the sequence modeling problem in a cyclic way, which can be written as follows:

[0053]

[0054] For the parallel training process, it can be expressed as:

[0055]

[0056] Retention(X)=(QK T ⊙D)V

[0057] Where Θ is the complex conjugate of Θ, D∈R |x|x|x| It incorporates causal masking and exponential decay, represents relative distances in one-dimensional sequences, and introduces explicit temporal priors for textual data.

[0058] Starting from retention in RetNet, we evolve it into Manhattan Self-Attention (MaSA), where the unidirectional and one-dimensional temporal decay is transformed into a bidirectional and two-dimensional spatial decay; this spatial decay introduces an explicit spatial prior related to the Manhattan distance into the visual backbone, and simultaneously decomposes the self-attention and spatial decomposition matrices along two axes of the image.

[0059] From unidirectional decay to bidirectional decay: In RetNet, due to the causal nature of text data, the retention is unidirectional, allowing each token to only focus on the previous tokens, not the following tokens;

[0060] This property is not applicable to tasks that lack causal properties, such as image recognition; therefore, the preservation was initially extended to a bidirectional form, expressed as follows:

[0061] BiRetention(X)=(QK T ⊙D Bi )V

[0062]

[0063] In the formula, BiRetention represents bidirectional modeling;

[0064] Attenuation from one dimension to two dimensions: Although Retention now supports bidirectional modeling, this capability is still limited to one dimension and is not applicable to two-dimensional images; to address this limitation, the one-dimensional Retention is extended to two dimensions.

[0065] In the image context, each marker has a unique two-dimensional coordinate in the plane, and for the nth marker, it is represented by (x n ,y n ); To accommodate this, each element in the matrix D is adjusted to represent the Manhattan distance between the 2D coordinates of the respective labeled pair; the matrix D is redefined as follows:

[0066]

[0067] In the early stages of the visual backbone, when trying to model global information, the large number of tokens leads to high self-attention costs; MaSA also encounters such challenges; using existing sparse attention mechanisms, or directly using the recursive / block-recursive form of the original Ret-Net, will destroy the Manhattan distance-based spatial attenuation matrix, resulting in the loss of explicit spatial priors;

[0068] To solve this problem, a simple decomposition method is introduced to decompose not only the self-attention but also the spatial attenuation matrix; the decomposed MaSA is shown as follows:

[0069]

[0070] MaSA(X)=Attn H (Attn W V) T

[0071] In order to further enhance the local expression ability of MaSA, a local context enhancement module using DWConv is introduced:

[0072] X out =MaSA(X)+LCE(⊙);

[0073] S5: Output the fire point image identified by the YOLOv8 algorithm;

[0074] S6: The specific steps of the YOLO-v8 algorithm combined with the RMT network are:

[0075] Step 1: During the drone’s flight path, the drone’s thermal imaging camera collects forest images;

[0076] Step 2: Preprocess the image to make it more consistent with the model’s input requirements; perform noise removal and image enhancement to improve image quality;

[0077] Step 3: Divide the image into multiple grids of equal size;

[0078] Step 4: Select ResNet as the basic network, introduce the Manhattan self-attention mechanism, build the RMT network, input the image into the RMT model, and extract image features;

[0079] Step 5: Input the features extracted by RMT into the subsequent network of YOLOv8 to predict the bounding box and category of each grid;

[0080] Step 6: Predict the confidence value score for each bounding box;

[0081] Step 7: Predict the conditional category probability in each bounding box;

[0082] Step 8: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images;

[0083] Step 9: Calculate the loss function for the target object in the grid;

[0084] Step 10: Output feature image containing feature information;

[0085] S7: Further collect a panoramic view of the scene, locate the (potential) fire source, and send the information to relevant personnel and the monitoring center.

[0086] The beneficial effects of the present invention are as follows: the UAV has the advantages of maneuverability, wide field of view, and diversified equipment, and can quickly cover large forest areas for real-time monitoring and data collection; by being equipped with laser radar, thermal imaging cameras and other equipment, the UAV can obtain forest terrain, temperature, images and other information, providing rich data support for early warning of fires; using thermal imaging cameras to scan large tracts of forests and detect possible fire sources or areas with abnormally high temperatures; this enables fire prevention personnel to respond quickly before a fire occurs or spreads, reducing the probability of fire and improving the efficiency and success rate of fire extinguishing; the present invention takes fire warning as the research object, proposes improvements to the flight algorithm of the flying subject UAV to enhance its flight effect, and proposes an improved image training algorithm for thermal imaging image data of forest fire points to improve training speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 is a system flow chart of the present invention;

[0088] Figure 2It is a flow chart of path planning of the UAV trajectory system in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] The specific technical scheme of the present invention is further described in detail below with reference to specific examples.

[0090] As shown in the figure, the forest fire early warning method based on a quad-rotor drone described in the present invention is realized based on the drone equipped with a laser radar and a drone thermal imaging camera;

[0091] This method uses a UAV equipped with a laser radar, a thermal imaging camera, and a deep learning algorithm to achieve forest fire early warning. The UAV first uses the laser radar to collect forest terrain data and uses the PKO algorithm to plan the optimal patrol track. During the patrol process, the thermal imaging camera collects forest images and uses the YOLOv8 algorithm and the RMT network to identify and locate fire points. Finally, the fire source information is sent to relevant personnel and the monitoring center to achieve efficient and accurate forest fire early warning. The specific implementation steps are as follows:

[0092] S1: Start the drone. After the drone takes off, the onboard laser radar actively emits laser pulses to obtain the signal reflected by the detection target and process it to obtain point cloud data. The point cloud data is filtered and classified to remove noise and interference information such as ground points. The digital model is constructed using the processed point cloud data.

[0093] S2: Complete the trajectory planning for fire warning patrols based on the PKO algorithm;

[0094] There are three stages of drones in the PKO algorithm:

[0095] Phase 1, initialization phase: PKO starts the search process by randomly generating a set of starting solutions from the search space as the first attempt:

[0096] X i.j =LB+(UB-LB)×rand

[0097] i=1,2,...,N and j=1,2,...,Dim

[0098] In the formula, X i.j represents the position of the i-th individual in the j-th dimension, rand represents a random value between 0 and 1, UB and LB represent the upper and lower bounds of the search range;

[0099] Phase 2, Exploration Phase: The drone switches between perching and hovering modes; the drone's position is updated according to the following formula:

[0100] X i (t+1)=X i(t)+∝×T(X j (t)-X i (t)),

[0101] i,j=1,2,...,N and j≠i

[0102] In the iteration process, the solution of the next iteration is represented by Xi(t+1), and the position of the current iteration is represented by Xit; the parameter α is calculated as 2*randn(1,dim), where randn represents a random number in a normal distribution; N represents the overall size, and Dim represents the dimension of the problem under consideration;

[0103] The parameter T plays a key role. Its value is determined dynamically based on the current strategy, which can be "perched" or "hovering". The calculation of T is tailored for each strategy to ensure the best performance in different operating modes.

[0104] In the roosting strategy, the parameter T is calculated as follows:

[0105]

[0106] Crest_angles = 2×pi×rand

[0107] The maximum number of iterations is specified by max Iter, the constant value BF (beat Factor) is set to 8, and rand is a random value between 0 and 1;

[0108] In the hover strategy, the parameter T is calculated as follows:

[0109]

[0110] The fitness of the i-th and j-th UAVs are denoted by PKO_Fitness(i) and PKO_Fitness(j), respectively; in addition, a constant value, BF (beating factor), is set to 8;

[0111] Phase 3, Capture Phase: When the drone approaches the target location in the constraints, it will continue to make more precise adjustments. The mathematical model is as follows:

[0112] X i (t+1)=X i (t)+HA×o×α×(bX best (t)),i=1,2,...,N

[0113] Wherein, the fitness value of the ith UAV is recorded as PKO_Fitness(1), Best_Fitness represents the best fitness value obtained from all iterations; α is a control parameter, calculated as 2*randn(1,dim)-1, and o and HA represent hunting capabilities, which can be calculated by the following two formulas:

[0114]

[0115] b=X i (t)+o 2 ×randn×X best (t)

[0116] The PKO algorithm first creates a population of potential solutions by randomly generating them; during multiple iterations, the positions of the drones are continuously updated using Eqs; if any drone moves outside the search area, the steps of the PKO simulation push it back to the boundary; subsequently, the current position is updated only if the new position provides a better solution quality than the current position; except for the initialization, all steps in the algorithm are performed in each iteration until the maximum number of iterations is reached;

[0117] Step 1: Construct forest terrain model;

[0118] Step 2: Determine the constraints for path planning based on the performance indicators of the drone and the performance of the link and payload;

[0119] Step 3: Initialize the population and randomly generate a set of initial solutions in the search space, each solution represents a possible path;

[0120] Step 4: Evaluate the fitness of each solution based on the objective function and constraints. The higher the fitness, the better the path.

[0121] Step 5: According to the three stages of the PKO algorithm (initialization, exploration, and capture), update the solutions in the population and find a better path; until the maximum number of iterations is reached or the fitness reaches the preset threshold, terminate the algorithm and output the optimal path;

[0122] Step 6: Perform interpolation and smoothing based on the obtained track, adjust the track space model, and improve or re-plan the route;

[0123] Step 7: Output the final trajectory planning data and issue instructions to the drone to perform flight operations;

[0124] The positions of the initial population in the PKO algorithm are generated by random sampling, but random sampling may cause uneven distribution of sample points and affect the convergence speed and accuracy. Therefore, the optimal Latin hypercube sampling is introduced in the initialization stage to generate sample points with more uniform and reasonable distribution, thereby improving the diversity of the initial population.

[0125] Step 1: Determine the dimensions to be sampled and the number of samples required, generate a set of equally spaced randomly distributed sample points in each dimension, ensure that the sample points in each dimension are evenly distributed throughout the range, and the number of sample points in each dimension is equal;

[0126] Step 2: Calculate the distance between sample points in each dimension to avoid correlation between sample points;

[0127] Step 3: In each iteration, a new position is randomly selected for each sample point. The new position can be randomly generated within the value range of the sample point, thereby achieving random redistribution of the sample point position;

[0128] Step 4: Calculate the minimum distance between the new position sample points;

[0129] Step 5: If the minimum distance of the new position is better than the previous position, accept the position as the current optimal position; otherwise, keep the previous position;

[0130] Step 6: Repeat steps 3 to 5 until the specified number of iterations is reached or the stop condition is met;

[0131] In the later stage of the algorithm, the individual kingfishers quickly assimilate, which will lead to a decrease in population diversity and search ability. The Cauchy mutation strategy is introduced to set a mutation probability for each point of the optimal solution, so that it can generate a new position and a new path. The fitness values ​​of the two paths are compared, and the better path is selected to enter the next iteration to jump out of the local optimum and improve the global optimization ability. The new position is updated as follows:

[0132]

[0133] Where cauchy(0,1) is the standard Cauchy distribution function, It means multiplication;

[0134] Cauchy mutation is used to perturb the solution in the kingfisher position update, the fitness values ​​of the previous and next solutions are compared, and the better solution is selected to be substituted into the next iteration, thereby expanding the search scale of the kingfisher algorithm and improving the algorithm's ability to jump out of the local optimum.

[0135] S3: During the flight mission, the drone's thermal imaging camera continuously collects forest fire point data to ensure that the collected fire point images are clear. The forest fire point detection method has two steps:

[0136] Step 1: Comprehensive detection of forest fire spots; increase the number of camera shots during the drone tracking process to ensure that there are overlapping parts in the captured pictures and conduct a large-scale fire spot search; the drone conducts a comprehensive inspection of the forest; the forest is divided into regions using a monomer method, the divided regions are numbered, and the GIS function is used to mark the areas involved in the information of each fire spot;

[0137] Step 2: Spot detection of forest fire points: For the determined fire point area, a target point close to the fire point is added to the path planning, so that the drone can reach the fire point area for spot data collection, and further fire point detection and evaluation are carried out based on the data collected at the spot.

[0138] S4: YOLOv8 algorithm is used to classify and identify fire spots in forest images, and RMT visual backbone is used as feature extractor to improve the performance of YOLOv8 in thermal imaging fire spot detection task;

[0139] The time decay of RetNet proposes a retention mechanism for sequence modeling; the retention mechanism brings time decay to the language model; first consider the sequence modeling problem in a cyclic way; it can be written as follows:

[0140]

[0141] For the parallel training process, it can be expressed as follows:

[0142]

[0143] Retention(X)=(QK T ⊙D)V

[0144] Where Θ is the complex conjugate of Θ, D∈R |x|x|x| It includes causal masking and exponential decay, represents relative distances in one-dimensional sequences, and introduces explicit temporal priors for text data.

[0145] Starting from the retention in RetNet, it evolves into Manhattan Self-Attention (MaSA); in MaSA, the unidirectional and one-dimensional temporal decay is transformed into a bidirectional and two-dimensional spatial decay; this spatial decay introduces an explicit spatial prior related to the Manhattan distance into the visual backbone, and decomposes the self-attention and spatial decomposition matrices simultaneously along the two axes of the image;

[0146] From unidirectional decay to bidirectional decay: In RetNet, due to the causal nature of text data, the retention is unidirectional, allowing each token to only focus on the previous token, not the following token; this feature is not suitable for tasks that lack causal properties, such as image recognition; therefore, the retention was initially extended to a bidirectional form, expressed as follows:

[0147] BiRetention(X)=(QK T ⊙D Bi )V

[0148]

[0149] In the formula, BiRetention represents bidirectional modeling;

[0150] Attenuation from one dimension to two dimensions: Although Retention now supports bidirectional modeling, this capability is still limited to one dimension and is not applicable to two-dimensional images. To address this limitation, the one-dimensional Retention is extended to two dimensions.

[0151] In the image context, each marker has a unique two-dimensional coordinate in the plane, and for the nth marker, it is represented by (x n ,y n ); To accommodate this, adjust each element in the matrix D to represent the Manhattan distance between the two-dimensional coordinates of the respective labeled pairs; redefine the matrix D as follows:

[0152]

[0153] In the early stages of the visual backbone, when trying to model global information, a large number of tokens leads to a high cost of self-attention; MaSA also encounters such a challenge; using the existing sparse attention mechanism, or directly using the recursive / block recursive form of the original Ret-Net, will destroy the Manhattan distance-based spatial attenuation matrix, resulting in the loss of explicit spatial priors; to solve this problem, a simple decomposition method is introduced to decompose not only self-attention but also the spatial attenuation matrix; the decomposed MaSA is shown as follows:

[0154]

[0155] MaSA(X)=Attn H (Attn W V) T

[0156] In order to further enhance the local expression ability of MaSA, a local context enhancement module using DWConv is introduced:

[0157] X out=MaSA(X)+LCE(⊙);

[0158] S5: Output the fire point image identified by the YOLOv8 algorithm;

[0159] S6: The specific steps of the YOLO-v8 algorithm combined with the RMT network are:

[0160] Step 1: During the drone’s flight path, the drone’s thermal imaging camera collects forest images;

[0161] Step 2: Preprocess the image to make it more consistent with the model’s input requirements; perform noise removal and image enhancement to improve image quality;

[0162] Step 3: Divide the image into multiple grids of equal size;

[0163] Step 4: Select ResNet as the basic network, introduce the Manhattan self-attention mechanism, build the RMT network, input the image into the RMT model, and extract image features;

[0164] Step 5: Input the features extracted by RMT into the subsequent network of YOLOv8 to predict the bounding box and category of each grid;

[0165] Step 6: Predict the confidence value score for each bounding box;

[0166] Step 7: Predict the conditional category probability in each bounding box;

[0167] Step 8: Use the neural network structure of the YOLO-v8 algorithm to train the input and output images;

[0168] Step 9: Calculate the loss function for the target object in the grid;

[0169] Step 10: Output feature image containing feature information;

[0170] S7: Further collect a panoramic view of the scene, locate the (potential) fire source, and send the information to relevant personnel and the monitoring center.

Claims

1. A forest fire early warning method based on a quad-rotor drone, characterized in that: The implementation steps are as follows: Step (1): After the drone takes off, the onboard laser radar collects and monitors the full terrain data of the forest area, generates a mathematical model of the overall forest terrain, and completes the track planning for fire warning patrols based on the PKO algorithm; Step (2): During the UAV flight process, the onboard thermal imaging camera collects forest thermal imaging data and sets abnormal temperature alarm thresholds; Step (3): Use the YOLOv8 algorithm to classify and identify the fire points in the forest image, and use the RMT visual backbone as a feature extractor to improve the performance of YOLOv8 in the thermal imaging fire point detection task; Step (4): outputting a feature image containing feature information; Step (5): Draw a panoramic view of the scene and locate the fire source; Step (6): Send the information to relevant personnel and monitoring center.

2. A forest fire early warning method based on a quad-rotor drone according to claim 1, characterized in that: In step (1), the method of collecting all-terrain data of the forest area by using laser radar is: The airborne laser radar actively emits laser pulses, obtains the signals reflected by the detection target and processes them to obtain point cloud data, which is then filtered and classified to remove noise and interference information from ground points. The processed point cloud data is used to build a digital model, and then the trajectory planning of the fire warning patrol is completed according to the PKO algorithm.

3. The forest fire early warning method based on a quad-rotor drone according to claim 1 is characterized in that: In step (1), the implementation steps of completing the track planning for fire warning patrol according to the PKO algorithm are: Step (1.1): Construct forest terrain model; Step (1.2): Determine the cost and constraints of path planning based on the performance indicators of the UAV and the performance of the link and payload; Step (1.3): Initialize the population and randomly generate a set of initial solutions in the search space, each solution represents a possible path; Step (1.4): Evaluate the fitness of each solution based on the objective function and constraints. The higher the fitness, the better the path. Step (1.5): Update the position of the drone according to the rules of the PKO algorithm; Step (1.6): interpolate and smooth the obtained track, adjust the track space model, and improve or re-plan the route; Step (1.7): Output the final trajectory planning data and issue instructions to the drone to perform flight operations; Step (1.8): The drone arrives at the target mission point to perform the mission.

4. The forest fire early warning method based on a quad-rotor drone according to claim 1 is characterized in that: In step (1.5), the updating of the position of the drone according to the rules of the PKO algorithm is specifically: Search phase: adjust the search direction based on the optimal position of the drone and its own experience; Capture phase: When approaching the target, the drone will make adjustments; If the current drone path is better than the previous optimal path, the optimal path is updated.

5. The forest fire early warning method based on a quad-rotor drone according to claim 3 is characterized in that: Regarding the objective function of trajectory planning, three aspects need to be considered: flight length, terrain collision avoidance, and flight altitude; In summary, the objective function is expressed as: f=p1f1+p2f2 In the formula, f1 and f2 are the flight length cost and terrain cost respectively, and the weights of the two optimization objectives are adjusted by changing p1 and p2; In trajectory planning, considering the power cost and time cost, the shorter the flight length, the better, and a shorter flight length means less risk; the expression of the flight length cost is: In the formula, the smaller f1 is, the smaller the flight length is and the smaller the corresponding cost is. This representation method will not affect the track length cost value due to changes in map size or the starting and ending points.

6. A forest fire early warning method based on a quad-rotor drone according to claim 5, characterized in that: To ensure safety, the drone must not collide with the terrain and must maintain a minimum safety distance. Divide adjacent track points into M n To reduce the complexity of the algorithm, the coordinates between the track points are obtained by linear interpolation; therefore, the terrain cost f2 is expressed as: In the formula, A m,n =1 means w m,n The vertical distance between the topographic map coordinates is less than the minimum safety distance d safe ; A m,n =0 means w m,n The vertical distance between the topographic map coordinates is greater than the minimum safety distance d safe .

7. The forest fire early warning method based on a quad-rotor drone according to claim 3 is characterized in that: There are three stages in the PKO algorithm for Kingfisher: Phase 1, initialization phase: PKO starts the search process by randomly generating a set of starting solutions from the search space as the first attempt: X i.j =LB+(UB-LB)×rand i=1,2,...,N and j=1,2,...,Dim Where, X i.j represents the position of the i-th individual in the j-th dimension, rand represents a random value between 0 and 1, UB and LB represent the upper and lower bounds of the search range; Phase 2, Exploration Phase: The drone switches between perching and hovering modes; the drone's position is updated according to the following formula: X i (t+1)=X i (t)+∝×T(X j (t)-X i (t)), i,j=1,2,...,N and j≠i In the iteration process, the solution of the next iteration is represented by Xi(t+1), and the position of the current iteration is represented by Xit; the parameter α is calculated as 2*randn(1,dim), where randn represents a random number in a normal distribution; N represents the overall size, and Dim represents the dimension of the problem under consideration; Phase 3, Capture Phase: When the drone approaches the target location in the constraints, it will make local adjustments. Its mathematical model is expressed as follows: X i (t+1)=X i (t)+HA×o×α×(b-X best (t)),i=1,2,...,N Wherein, the fitness value of the ith UAV is recorded as PKO_Fitness(1), Best_Fitness represents the best fitness value obtained from all iterations; α is a control parameter, calculated as 2*randn(1,dim)-1, and o and HA represent hunting capabilities, calculated by the following two formulas: b=X i (t)+o 2 ×randn×X best (t); The PKO algorithm first creates a population of potential solutions by randomly generating them. During multiple iterations, the positions of the drones are continuously updated using Eqs. If any drone moves outside the search area, the PKO simulation steps push it back to the boundary. Subsequently, the current position is updated only if the new position provides a better solution quality than the current position. Except for initialization, all steps in the algorithm are executed in each iteration until the maximum number of iterations is reached.

8. The forest fire early warning method based on a quad-rotor drone according to claim 7, characterized in that: In phase 2, the value of parameter T is dynamically determined based on the current strategy, including perching or hovering; the calculation of T is tailored for each strategy; Among them, in the habitat strategy, the parameter T is calculated as follows: Crest_angles = 2×pi×rand Where, the maximum number of iterations is specified by maxIter; the constant value BF is set to 8, and rand is a random value between 0 and 1; In the hover strategy, the parameter T is calculated as follows: The fitness of the i-th and j-th UAVs are denoted by PKO_Fitness(i) and PKO_Fitness(j), respectively; in addition, a constant value, BF, is set to 8.

9. The forest fire early warning method based on a quad-rotor drone according to claim 1, characterized in that: In step (3), the fire spots in the forest image are classified and identified using the YOLOv8 algorithm. Since the fire spots are small and irregular in shape, and RMT has strong feature extraction capabilities, it can extract the features of the target while improving robustness and detection accuracy. The steps are as follows: Step (3.1): During the UAV flight process, the thermal imaging camera of the UAV collects forest thermal imaging data; Step (3.2): Preprocess the image to make it meet the input requirements of the model; Perform noise removal and image enhancement to improve image quality; Step (3.3): Divide the image into multiple grids of equal size; Step (3.4): Select ResNet as the basic network, introduce the Manhattan self-attention mechanism, build the RMT network, input the image into the RMT model, and extract image features; Step (3.5): Input the features extracted by RMT into the subsequent network of YOLOv8 to predict the bounding box and category of each grid; Step (3.6): predict the confidence value score for each bounding box; Step (3.7): Predict the conditional category probability in each bounding box; Step (3.8): Use the neural network structure of the YOLO-v8 algorithm to train the input and output images; Step (3.9): Calculate the loss function for the target object in the grid; Step (3.10): Output a feature image containing feature information.

10. A forest fire early warning method based on a quad-rotor drone according to claim 9, characterized in that: RetNet’s time decay proposes a retention mechanism for sequence modeling, which brings time decay to the language model; First, consider the sequence modeling problem in a cyclic way, written as follows: For the parallel training process, it is expressed as follows: Retention(X)=(QK T ⊙D)V Where Θ is the complex conjugate of Θ, D∈R |x|x|x| Contains causal masking and exponential decay, representing relative distances in one-dimensional sequences; Starting from the retention in RetNet, it evolved into MaSA; in MaSA, the unidirectional and one-dimensional time decay is transformed into a bidirectional and two-dimensional spatial decay; From unidirectional decay to bidirectional decay: In RetNet, due to the causal nature of text data, retention is unidirectional, allowing each token to only focus on the previous token; This property is not applicable to tasks that lack causal properties; therefore, the preservation was initially extended to a bidirectional form, expressed as follows: BiRetention(X)=(QK T ⊙D Bi )V In the formula, BiRetention represents bidirectional modeling; Attenuation from one dimension to two dimensions: Although Retention supports bidirectional modeling, this capability is still limited to one dimension. To address this limitation, the one-dimensional Retention is extended to two dimensions. In the image context, each marker has a unique two-dimensional coordinate in the plane, and for the nth marker, it is represented by (x n ,y n ); To accommodate this, adjust each element in the matrix D to represent the Manhattan distance between the two-dimensional coordinates of the respective labeled pairs; redefine the matrix D as follows: In the early stages of the visual backbone, tokens lead to high self-attention costs when trying to model global information; using existing sparse attention mechanisms, or directly using the recursive / block-recursive form of the original Ret-Net, will destroy the Manhattan distance-based spatial attenuation matrix, leading to the loss of explicit spatial priors; To solve this problem, a decomposition method is introduced, and the decomposed MaSA is shown as follows: MaSA(X)=Attn H (Attn W V) T To enhance the local expression ability of MaSA, a local context enhancement module using DWConv is introduced: X out =MaSA(X)+LCE(⊙)。

Images